Confluent

NOVEMBER 30, 2022

What is stream processing, or complex event processing (CEP), and how does it work? Learn about real-time data and event stream analytics in this tutorial.

Process 125

Process Data IT 125

Lyft Engineering

MARCH 27, 2023

However, at Lyft we have come to realize that load testing in production is a powerful tool to prepare systems for unexpected bursty traffic and peak events. Lyft must operate seamlessly even when demand spikes It’s imperative for Lyft to be highly available during peak events like Halloween and New Year’s Eve.

Coding 131

Coding Database Systems Engineering 131

Cloudyard

JULY 16, 2023

Read Time: 2 Minute, 13 Second During the last post we discussed about the Event tables and how these are used for Error logging in your process. The UDTF was responsible of parsing JSON data in a tabular format and identify any erroneous records and log into an Event table. Event tables.

Metadata 52

Metadata Python Utilities Coding 52

Workfall

SEPTEMBER 26, 2023

Reading Time: 9 minutes In this blog, we will cover: What are Server-Sent Events? Why Stream Data Using Server-Sent Events (SSE)? Hands-On Conclusion What are Server-Sent Events? These events can carry information like notifications, updates, or alerts. What is FastAPI?

Python 85

Python Bytes Coding Data 85

dbt Developer Hub

NOVEMBER 3, 2022

Let’s discuss how to convert events from an event-driven microservice architecture into relational tables in a warehouse like Snowflake. This rule empowers microservices to change their database schemas however they like without worrying about breaking other systems. What are Events?

Kafka 52

Kafka ETL Tools BI Database 52

Striim

NOVEMBER 8, 2023

Data Mesh is revolutionizing event streaming architecture by enabling organizations to quickly and easily integrate real-time data, streaming analytics, and more. By exploring the benefits of using Data Mesh for your event streaming architecture, this article will help you decide if it’s the right solution for your organization.

Architecture 52

Architecture Generalist Government Datasets 52

Workfall

JUNE 27, 2023

In this blog, we’ll explore how to harness the power of Kafka to streamline event streaming within a microservices architecture and unlock its full potential for building scalable and responsive systems. Think about it this way, traditionally, all the data needs were being handled by the backend server. Let’s get started!

Kafka 75

Kafka Architecture AWS Transportation 75

Data Engineering Podcast

AUGUST 10, 2020

Summary Event based data is a rich source of information for analytics, unless none of the event structures are consistent. The team at Iteratively are building a platform to manage the end to end flow of collaboration around what events are needed, how to structure the attributes, and how they are captured.

Data Collection 100

Data Collection Data Engineering Data Engineer Education 100

Data Engineering Podcast

DECEMBER 11, 2022

Planetscale is a serverless option for your MySQL workloads that lets you focus on your applications without having to worry about managing the database or fight with differences between development and production. That way data engineers and data users can process to their heart’s content without worrying about their cloud bill.

Database 100

Database MySQL Data Lake MongoDB 100

Confluent

FEBRUARY 2, 2021

Since I first started using Apache Kafka® eight years ago, I went from being a student who had just heard about event streaming to contributing to the transformational, company-wide event […].

Kafka 96

Kafka Data Schemas Cloud Data 96

Data Engineering Podcast

NOVEMBER 26, 2019

Summary The team at Sentry has built a platform for anyone in the world to send software errors and events. You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management.

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Knowledge Hut

JANUARY 30, 2024

It focuses on service operation activities including event planning, access management, problem management, the application lifespan, and helpline assistance. Learn more about Service Management best practices through knowledge hut IT Service Management Certification training. Complete an ITIL assessment panel interview successfully.

Certification 52

Certification Telecommunication Government Designing 52

Data Engineering Podcast

JULY 17, 2022

In this episode Joe Reis, founder of Ternary Data and co-author of "Fundamentals of Data Engineering", turns the tables and interviews the host, Tobias Macey, about his journey into podcasting, how he runs the show behind the scenes, and the other things that occupy his time.

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Knowledge Hut

JANUARY 9, 2024

This includes making decisions about the project scope, schedule, budget, and resources. Provide guidance and solutions so as to manage project crises, challenges, and unexpected events that may impact project success. This includes working with the project team to identify and implement solutions to problems.

Project 52

Project Certification Portfolio Education 52

Confluent

MARCH 19, 2019

When should you use an event? I will attempt to articulate in layman’s terms what an event-driven architecture (EDA) is and contrast it with service-oriented architecture (SOA). In essence, this is an attempt to differentiate and/or associate APIs with events. Is not life simply a series of events?

Architecture 110

Architecture Telecommunication Transportation Database 110

Data Engineering Weekly

MAY 27, 2023

DoorDash writes about its discovery of Generative AI and its application to boost its business. link] Expedia: How Expedia Reviews Engineering Is Using Event Streams as a Source Of Truth “Events as a source of truth” is a simple but powerful idea to persist the state of the business entity as a sequence of state-changing events.

Data Validation 52

Data Validation Data Data Engineering Data Engineer 52

Data Engineering Podcast

FEBRUARY 18, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Join in with the event for the global data community, Data Council Austin. Don't miss out on their only event this year!

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

U-Next

NOVEMBER 17, 2022

Databases are a common type of data store that contains information about a given topic. They can be used to store information about anything, from customer records to inventory levels. . Reports are a common type of output that contains information about a given topic. Applications of AWK Advanced Filter .

Programming Language 52

Programming Language Datasets Programming Database 52

Zalando Engineering

FEBRUARY 2, 2022

In our Microservices Architecture, services communicate both asynchronous via events and synchronous via REST calls. Frequently, a synchronous REST call modifies data in a data store and emits an event based on the changes made. The pattern we are describing here is known as Transactional Outbox. This is depicted in step 1.5.

Utilities 52

Utilities AWS PostgreSQL Datasets 52

Snowflake

MAY 3, 2023

The free two-day event, taking place online May 17 and 18, will explore industry trends, data and technology innovations, and data strategy case studies in financial services. The post Accelerate: Why You Should Attend and Who Will Be at This Virtual Financial Services and Insurance Event appeared first on Snowflake.

Insurance 52

Insurance Banking Finance Cloud 52

Knowledge Hut

JANUARY 29, 2024

Service Operation includes five processes: Event Management Incident Management Request Fulfillment Problem Management Access Management ITIL ® Continual Service Improvement In this stage of ITIL®, stringent quality checks are required to ensure the service process is improved consistently.

Certification 52

Certification Portfolio Designing Telecommunication 52

Confluent

FEBRUARY 11, 2020

Osquery (developed by Facebook) is an open source tool used to gather audit log events from an operating system (OS). What’s unique about osquery is that it uses basic SQL […].

SQL 90

SQL Systems IT Kafka 90

Confluent

FEBRUARY 28, 2019

So you can stream your events from Apache Kafka ® directly into Neo4j to create, update and enrich your graph structures. Then it is really up to you what you want to with the event data. The events can come from frontend systems, API notifications, other databases or streaming systems like Apache Spark and Apache Flink ®.

Kafka 87

Kafka Java Programming Language Data Pipeline 87

Monte Carlo

OCTOBER 31, 2023

tk4,There’s plenty of hype – don’t get me wrong – but there’s also a lot to be excited about, and our speakers will address these opportunities, challenges, and forward-looking predictions for data and AI teams. 2023 has been a crazy year in data, with the rise of GenAI and LLMs eclipsing nearly everything else in tech.

Data Engineering 72

Data Engineering Data Engineer Data Engineering 72

Data Engineering Podcast

JULY 14, 2019

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Machine Learning 100

Machine Learning Media Cloud Data 100

Data Engineering Podcast

MARCH 31, 2019

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Project 100

Project IT Media Data Warehouse 100

Rockset

FEBRUARY 24, 2022

Event-based architectures have been gaining popularity for some time. With increased adoption has come a flood of options for aggregating and analyzing events. Which databases are optimized for ingesting streaming events and analyzing them in real time? Supporting separation of data access (read/write) mechanisms like CQRS.

AWS 52

AWS Amazon Web Services Kafka Business Intelligence 52

Confluent

DECEMBER 14, 2021

This oft-talked-about architecture has no shortage of blog posts, conference talks, podcasts, and discussions. One thing that you may have found lacking is a concrete guide on precisely […].

Building 128

Building Architecture Data 128

Zalando Engineering

APRIL 22, 2024

Introduction Our team is responsible for sending out communications to all our customers at Zalando - e.g. confirming a placed order, informing about new content from a favourite brand or announcing sales campaigns. But our system is event driven, all requests we process are delivered as events via Nakadi.

Algorithm 79

Algorithm Systems Kafka Designing 79

Data Engineering Podcast

APRIL 23, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. How are organizations thinking about the potential value that those types of apps/services can provide?

Data Lake 277

Data Lake Kafka Machine Learning Data Warehouse 277

Confluent

NOVEMBER 30, 2022

This guide will contain specific advice on how to go about building an event streaming platform in your organization. This is the second part of our guide on streaming data and Apache Kafka.

Kafka 52

Kafka Building Data Process 52

Propel Data

OCTOBER 13, 2022

Developers looking to learn about event-driven architectures.

Architecture 52

Architecture Data Analytics Data 52

The Pragmatic Engineer

DECEMBER 2, 2023

A year ago, I spent months doing an investigative report on how UK events tech company Pollen had its staff work for free, as it had run out of money but still kept operating. A year later, the BBC made a documentary about the company called Crashed: $800M Festival Fail.  These people do not exist. No matches.

Software Engineer 301

Software Engineer Software Engineering Insurance Coding 301

Confluent

FEBRUARY 21, 2019

During a recent talk titled Hunters ATT&CKing with the Right Data , which I presented with my brother Jose Luis Rodriguez at ATT&CKcon, we talked about the importance of documenting and modeling security event logs before developing any data analytics while preparing for a threat hunting engagement. Why KSQL and HELK?

Process 81

Process Kafka Datasets SQL 81

Data Engineering Podcast

MARCH 3, 2024

Colleen Tartow has worked across all stages of the data lifecycle, and in this episode she shares her hard-earned wisdom about how to conduct an AI program for your organization. Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year! Data Council : ![Data

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

MARCH 17, 2024

In this episode Gleb Mezhanskiy, founder and CEO of Datafold, discusses the different error conditions and solutions that you need to know about to ensure the accuracy of your data. Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year! Data Council : ![Data

Database 147

Database Data Lake High Quality Data Data Workflow 147

Data Engineering Podcast

JUNE 25, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com ) with your story.

Data Engineering 225

Data Engineering Data Engineer Python Engineering 225