Knowledge Hut

APRIL 25, 2023

In this article, we will look at how a scrum master facilitates events such as daily scrum meetings, sprint planning, sprint review, and sprint retrospective meetings. They lead the team through the Scrum process, making sure that everyone is involved and that all events are carried out successfully.

Utilities 70

Utilities Project Certification Process 70

Lyft Engineering

MARCH 27, 2023

However, at Lyft we have come to realize that load testing in production is a powerful tool to prepare systems for unexpected bursty traffic and peak events. Lyft must operate seamlessly even when demand spikes It’s imperative for Lyft to be highly available during peak events like Halloween and New Year’s Eve.

Coding 131

Coding Database Systems Engineering 131

Knowledge Hut

JUNE 20, 2023

Whether you are a student, researcher, or industry professional, we recommend taking our Database Certification Courses to stay current with the latest research topics in DBMS. In this blog post, we will introduce you to 10 current database research topic ideas that are likely to be at the forefront of the field in 2023.

Database 52

Database Java Education Data Collection 52

Knowledge Hut

NOVEMBER 27, 2023

To successfully transition, it is crucial to select a data science role and assess your current skills. What are your current skills, areas you need to improve on, interest and the expectation that you must cater to. Attend networking events, reach out to professionals on platforms like LinkedIn, and explore mentorship programs.

Data Science 52

Data Science Datasets Machine Learning Algorithm 52

Confluent

APRIL 25, 2022

Data streaming is a new category of technology that is reshaping the way businesses operate, but there hasn’t been a place for everyone in the ecosystem to come together and […].

Kafka 104

Kafka Technology Data 104

Confluent

AUGUST 30, 2023

Get a sneak peak into what awaits you at Current 2023—featuring captivating keynotes, must-attend sessions, networking opportunities, and much more.

70

70

Data Engineering Podcast

NOVEMBER 26, 2019

Summary The team at Sentry has built a platform for anyone in the world to send software errors and events. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Confluent

MARCH 19, 2019

When should you use an event? I will attempt to articulate in layman’s terms what an event-driven architecture (EDA) is and contrast it with service-oriented architecture (SOA). In essence, this is an attempt to differentiate and/or associate APIs with events. Is not life simply a series of events? If it can, should it?

Architecture 110

Architecture Telecommunication Transportation Database 110

Rockset

FEBRUARY 24, 2022

Event-based architectures have been gaining popularity for some time. With increased adoption has come a flood of options for aggregating and analyzing events. Which databases are optimized for ingesting streaming events and analyzing them in real time? Supporting separation of data access (read/write) mechanisms like CQRS.

AWS 52

AWS Amazon Web Services Kafka Business Intelligence 52

The Pragmatic Engineer

APRIL 11, 2024

Confirming this suspicion is Fabrick’s status page that says Fabrick’s tech staff are involved: “We would like to inform you that the event is currently still ongoing, continuous monitoring is in place, and any slowdowns are affecting only the services provided jointly with the Sella group.

Banking 199

Banking Utilities Database Engineering 199

Data Engineering Podcast

JUNE 25, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. You have some built-in integration with/awareness of orchestrators (currently Airflow). RudderStack also supports real-time use cases.

Data Engineering 225

Data Engineering Data Engineer Python Engineering 225

Zalando Engineering

APRIL 22, 2024

But our system is event driven, all requests we process are delivered as events via Nakadi. Communication Platform Overview Nakadi is a distributed event bus that offers a RESTful API on top of Kafka-like queues. Out of those more than 1000 different event types trigger customer communication.

Algorithm 79

Algorithm Systems Kafka Designing 79

DoorDash Engineering

MARCH 27, 2024

Real-time event processing is a critical component of a distributed system’s scalability. At DoorDash, we rely on message queue systems based on Kafka to handle billions of real-time events. In Kafka, a test tenant processing production event can cause data inconsistencies, including outages and other incidents.

Kafka 103

Kafka Architecture Algorithm Systems 103

Confluent

FEBRUARY 21, 2019

During a recent talk titled Hunters ATT&CKing with the Right Data , which I presented with my brother Jose Luis Rodriguez at ATT&CKcon, we talked about the importance of documenting and modeling security event logs before developing any data analytics while preparing for a threat hunting engagement. Yeah…I can do that already!

Process 81

Process Kafka Datasets SQL 81

Christophe Blefari

FEBRUARY 16, 2024

Disclaimer, the 2 events below will be in French. Before jumping to the news there are a few events I want to write about. You can see a few videos on their landing page as well as the current limitations. Mainly it shows that Mistral is currently replacing LLaMa. ByteDance is the company behinds TikTok.

Food 130

Food Data SQL Government 130

Data Engineering Podcast

APRIL 9, 2023

Summary The customer data platform is a category of services that was developed early in the evolution of the current era of cloud services for data processing. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools.

Data Lake 147

Data Lake Data Warehouse Machine Learning Data 147

Snowflake

APRIL 9, 2024

For Snowflake sales reps, Outreach is the central repository for almost all inbound and outbound communications with current and potential customers. To improve go-to-market (GTM) efficiency, Snowflake created a bi-directional data share with Outreach that provides consistent access to the current version of all our customer engagement data.

BI 73

BI Data Ingestion Data Aggregated Data 73

Data Engineering Podcast

MAY 21, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. How would you characterize the current state of the ecosystem? How would you characterize the current state of the ecosystem?

Data Lake 162

Data Lake Machine Learning Kafka Data Warehouse 162

Data Engineering Podcast

MARCH 24, 2024

How is it different from the current Dagster Cloud product? How is it different from the current Dagster Cloud product? What problems are you trying to solve with Dagster+? What are the notable enhancements beyond the Dagster Core project that this updated platform provides? Email hosts@dataengineeringpodcast.com ) with your story.

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Robinhood

FEBRUARY 27, 2024

Money Drills will kick off at each university this week with events and speakers sharing more on how the program will come to life on campus. “By This event will celebrate our new relationship and commitment to financial education.”

Education 115

Education Finance Programming Government 115

Rockset

NOVEMBER 6, 2019

Events are messages that are sent by a system to notify operators or other systems about a change in its domain. With event-driven architectures powered by systems like Apache Kafka becoming more prominent, there are now many applications in the modern software stack that make use of events and messages to operate effectively.

Kafka 40

Kafka Data Lake SQL Hadoop 40

Zalando Engineering

NOVEMBER 8, 2023

Postgres Logical Replication at Zalando Builders at Zalando have access to a low-code solution that allows them to declare event streams that source from Postgres databases. At the time of writing, there are hundreds of these Postgres-sourced event streams out in the wild at Zalando. Simple, right?

PostgreSQL 139

PostgreSQL Java Database Bytes 139

Rockset

SEPTEMBER 15, 2020

However, one difference between a real-time database and a streaming logging system is that the database provides a query API that can perform random queries on the event stream. The Data Generator simulates a real-life event workload, where every generated event is in JSON format and schemas can change frequently.

Bytes 40

Bytes Database Data Warehouse Data Pipeline 40

The Pragmatic Engineer

OCTOBER 31, 2023

There is no current ETA for recovery of operations in europe-west9-a, but it is expected to be an extended outage. We will provide an update by Wednesday, 2023-04-26 00:30 US/Pacific with current details. I wrote that such an event could tear cables. Customers are advised to fail over to other zones if they are impacted.

AWS 213

AWS Google Cloud Cloud Food 213

Towards Data Science

FEBRUARY 18, 2024

An event is a small, self-contained object that contains the details of something happened at some point in time e.g. user interaction. An event is generated by a producer (e.g. Traditional databases are ill-suited for storing events in high throughput event streams. online dashboard).

Kafka 79

Kafka Python Process Google Cloud 79

Data Engineering Weekly

APRIL 21, 2024

The blog narrates the common challenges with enterprise analytical platforms such as event-centric, schema on-read, and nested json structures. link] Autodesk: Improving confidence in our tracking with auto-generated Snowplow types Autodesk writes a classic data ingestion problem: What can go wrong with event collection?

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

Snowflake

FEBRUARY 27, 2024

Please note: Snowflake currently only supports USD-based pricing for Marketplace Monetization.) Snowflake Native Apps also have an exclusive usage-based pricing model: custom event billing (currently in public preview). Examples here include pricing per query or per month when usage occurs.

Bytes 90

Bytes Electronics Banking Data 90

Engineering at Meta

MAY 16, 2023

We’re rolling out MySQL Raft with the aim to eventually replace our current MySQL semisynchronous databases. We have migrated a large portion of our deployment to MySQL Raft and plan to fully replace the current MySQL semisynchronous databases with it. The MySQL transactions would be encoded as a series of events (e.g.,

MySQL 136

MySQL Building Engineering Database 136

Data Engineering Podcast

JANUARY 28, 2024

Summary Monitoring and auditing IT systems for security events requires the ability to quickly analyze massive volumes of unstructured log data. What were the motivating constraints that led you to your current implementation? What were the motivating constraints that led you to your current implementation?

Data Lake 147

Data Lake Building High Quality Data AWS 147

The Pragmatic Engineer

AUGUST 3, 2023

While retiring FDL will be painful for current customers, it’s at least nice to see Google closing this product in a considerate way. Events tech company Hopin selling its events tech business. I talked with current and former Hopin engineers to get more details on the sale, including the rumored price tag.

Metadata 157

Metadata Engineering Building Technology 157

Rockset

OCTOBER 1, 2019

In this blog post, we will explore a tool we built internally that gives us visibility into Kubernetes events, an excellent source of information about the state of the system, which we find useful in troubleshooting the system and understanding its long-term health. By default, only the past hour of events is preserved by etcd.

SQL 40

SQL Systems Metadata Accessible 40

Snowflake

APRIL 11, 2024

Please note: Public preview access for Snowflake Copilot is currently rolling out for AWS accounts. Or, experience Copilot firsthand at our free Dev Day event on June 6th in the Demo Zone! To gain access, contact your Snowflake account team. We can’t wait to see you there.

SQL 134

SQL AWS Data Analysis High Quality Data 134

Robinhood

MARCH 26, 2024

The event is available to livestream on go.robinhood.com/presents at 7:15 pm ET on March 26 and will be available for replay. Because some of these risks and uncertainties cannot be predicted or quantified and some are beyond our control, you should not rely on our forward-looking statements as predictions of future events.

Banking 115

Banking Insurance Finance Programming 115

Rockset

JUNE 23, 2020

Recommendations API for an Online Event Ticketing System To explore the benefits of replicating a MongoDB database into an analytics platform like Rockset, I’ll be using a simulated event ticketing website. Connecting MongoDB to Rockset The MongoDB connector is currently available for use with a MongoDB Atlas cluster.

MongoDB 40

MongoDB SQL Database Datasets 40

Scott Logic

APRIL 18, 2024

EventBridge A vast majority of AWS services generate events when there is a change in the service or application. AWS EventBridge can have rules that listen for certain events; for example, a rule could listen for when an EC2 instance changes from ‘Pending’ to ‘Running’. Side note, how many TLAs can I fit into this blog post?)

AWS 52

AWS Building Consulting Architecture 52

Lyft Engineering

OCTOBER 3, 2023

For example, we can almost instantly validate that each record is well-formed and complete during event generation. Our Analytic Event Lifecycle below demonstrates the workflow of how much of our data gets to Hive. We log these events asynchronously at the order of millions per second.

Big Data 84

Big Data Metadata Data Warehouse Data 84

DoorDash Engineering

APRIL 4, 2023

The primary purpose of Archiver is to pick up events containing financial transaction data sent by different upstream teams and store this data in a data lake. This data is currently persisted for audit and to be used in case of correcting any accounting data. The event data is stored in AWS S3 in an encoded format in JSON files.

Project 74

Project Banking Kafka Database 74