Workfall

SEPTEMBER 26, 2023

Reading Time: 9 minutes In this blog, we will cover: What are Server-Sent Events? Why Stream Data Using Server-Sent Events (SSE)? Hands-On Conclusion What are Server-Sent Events? These events can carry information like notifications, updates, or alerts. What is FastAPI?

Python 85

Python Bytes Coding Data 85

Workfall

JUNE 27, 2023

In this blog, we’ll explore how to harness the power of Kafka to streamline event streaming within a microservices architecture and unlock its full potential for building scalable and responsive systems. In the event of server failure, Kafka can autonomously restart itself, ensuring uninterrupted data flow. Let’s get started!

Kafka 75

Kafka Architecture AWS Transportation 75

Precisely

DECEMBER 15, 2023

Let’s take a look at the Top 5 SAP process automation blog posts of 2023. #5. With shifting customer preferences and disruptive world events shaking up the global supply chain market, many business leaders are left wondering whether they’ll be able to stay competitive. The answer lies in hyperautomation.

Process 59

Process Government Technology Cloud 59

Lyft Engineering

MARCH 27, 2023

However, at Lyft we have come to realize that load testing in production is a powerful tool to prepare systems for unexpected bursty traffic and peak events. Lyft must operate seamlessly even when demand spikes It’s imperative for Lyft to be highly available during peak events like Halloween and New Year’s Eve.

Coding 131

Coding Database Systems Engineering 131

Knowledge Hut

DECEMBER 6, 2023

Voice technologies have played a very important role in making websites and blogs more accessible to people with disabilities and will continue to do so in the coming years. Let’s explore how we can add voice to a blog or a webpage. How can you make a web page talk or add voice to your blog? utterance.volume = 0.5

Telecommunication 52

Telecommunication Education Coding Technology 52

Propel Data

OCTOBER 13, 2022

Developers looking to learn about event-driven architectures.

Architecture 52

Architecture Data Analytics Data 52

Uber Engineering

SEPTEMBER 23, 2021

This article focuses on how we … The post Real-Time Exactly-Once Ad Event Processing with Apache Flink, Kafka, and Pinot appeared first on Uber Engineering Blog. With this new ability came new challenges that needed to be solved at Uber, such as systems for ad auctions, bidding, attribution, reporting, and more.

Kafka 142

Kafka Process Systems Engineering 142

dbt Developer Hub

NOVEMBER 3, 2022

Let’s discuss how to convert events from an event-driven microservice architecture into relational tables in a warehouse like Snowflake. In this blog post we’ll dive into how we tackled one source of quality issues: directly relying on upstream database schemas. So our solution was to start using an intentional contract: Events.

Kafka 52

Kafka ETL Tools BI Database 52

Propel Data

NOVEMBER 14, 2022

Learn the key AWS services to implement a performant and cost-effective serverless event-driven architecture.

AWS 52

AWS Architecture Data Analytics Data 52

The Modern Data Company

JANUARY 31, 2023

These were our most popular blog posts in 2022 according to reader statistics. “ Thus begins another popular Modern blog post from 2022. And our blog post shows the first steps towards unifying data once and for all. If you haven’t been with us long, we had some incredible pieces in the past few years.

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

databricks

NOVEMBER 3, 2023

In this four-part blog series "Lessons learned from building Cybersecurity Lakehouses," we will discuss a number of challenges organizations face with data engineering.

Data Engineering 80

Data Engineering Data Engineer Engineering Building 80

Propel Data

OCTOBER 25, 2022

This article will demonstrate how to deduplicate events in Snowflake using dbt

Data Analytics 52

Data Analytics Data 52

Striim

NOVEMBER 8, 2023

Data Mesh is revolutionizing event streaming architecture by enabling organizations to quickly and easily integrate real-time data, streaming analytics, and more. By exploring the benefits of using Data Mesh for your event streaming architecture, this article will help you decide if it’s the right solution for your organization.

Architecture 52

Architecture Generalist Government Datasets 52

Confluent

APRIL 4, 2019

Event-driven architecture means just that: It’s all about the events. In a microservices architecture, events drive microservice actions. No event, no shoes, no service. In the most basic scenario, microservices that need to take action on a common stream of events all listen to that stream.

Kafka 108

Kafka Data Cleanse Retail Finance 108

Confluent

FEBRUARY 27, 2019

What is more, as the world adopts the event-driven streaming architecture, how does it fit with serverless? Serverless functions provide a synergistic relationship with event streaming applications; they behave differently with respect to streaming workloads but are both event driven. Event-first FaaS? Event-first FaaS.

Kafka 109

Kafka AWS Architecture Cloud 109

Data Engineering Podcast

NOVEMBER 26, 2019

Summary The team at Sentry has built a platform for anyone in the world to send software errors and events. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Confluent

MAY 9, 2019

Event-first thinking enables us to build a new atomic unit: the event. Four pillars of event streaming. Pillar 4 – Operational plane: Event logging, DLQs and automation. To read the other articles in this series, see: Journey to Event Driven – Part 1: Why Event-First Thinking Changes Everything.

Kafka 93

Kafka Pipeline-centric Architecture Database-centric 93

Confluent

FEBRUARY 13, 2019

Part 1 of this series discussed why you need to embrace event-first thinking, while this article builds a rationale for different styles of event-driven architectures and compares and contrasts scaling, persistence and runtime models. Event-driven architecture. Event-driven, reactive architecture. Being event first.

Architecture 90

Architecture Programming Kafka Database-centric 90

DataKitchen

DECEMBER 9, 2022

Query> An AI, Chat GPT wrote this blog post, why should I read it? . While I can provide information on a wide range of topics, I do not have personal experiences or knowledge of specific events or situations. Query> Why are the authors of this blog so lazy that they could not write this themselves? .

Machine Learning 52

Machine Learning Data Preparation Government Data Analytics 52

Confluent

DECEMBER 14, 2021

This oft-talked-about architecture has no shortage of blog posts, conference talks, podcasts, and discussions. One thing that you may have found lacking is a concrete guide on precisely […].

Building 128

Building Architecture Data 128

Confluent

JUNE 11, 2019

For event streaming application developers, it is important to continuously update the streaming pipeline based on the need for changes in the individual applications in the pipeline. It is also important to understand some of the common streaming topologies that streaming developers use to build an event streaming pipeline.

Kafka 85

Kafka Cloud Java MongoDB 85

Rockset

FEBRUARY 24, 2022

Event-based architectures have been gaining popularity for some time. With increased adoption has come a flood of options for aggregating and analyzing events. Which databases are optimized for ingesting streaming events and analyzing them in real time? Supporting separation of data access (read/write) mechanisms like CQRS.

AWS 52

AWS Amazon Web Services Kafka SQL 52

Cloudera

MAY 10, 2024

The event, taking place virtually from May 14-15, features a variety of speaking sessions from experts, community members, and practitioners who will share insights and best practices for leveraging the full power of Iceberg. The post Join us at the Iceberg Summit 2024 appeared first on Cloudera Blog.

Government 101

Government Data Governance Data Pipeline Big Data 101

Zalando Engineering

NOVEMBER 8, 2023

Introduction This blog post describes a recent contribution from Zalando to the Postgres JDBC driver to address a long-standing issue with the driver’s integration with Postgres’ logical replication that resulted in runaway Write-Ahead Log (WAL) growth. So what is happening at a low level during this event-streaming process?

PostgreSQL 139

PostgreSQL Java Database Bytes 139

Propel Data

OCTOBER 23, 2022

Learn how to use the incremental model in dbt to manage data streams in your Snowflake warehouse.

Data Analytics 52

Data Analytics Building Data Management 52

The Pragmatic Engineer

APRIL 9, 2024

Context My initial motivation with the job board was to add a kind of "helpful monetization" to both my blog, and The Pragmatic Engineer Newsletter. In the blog – and at the end of all free newsletter issues – jobs posted by customers of the job board were shown.

Software Engineer 256

Software Engineer Software Engineering Engineering Recruitment 256

Robinhood

MARCH 26, 2024

The event is available to livestream on go.robinhood.com/presents at 7:15 pm ET on March 26 and will be available for replay. Because some of these risks and uncertainties cannot be predicted or quantified and some are beyond our control, you should not rely on our forward-looking statements as predictions of future events.

Banking 121

Banking Insurance Finance Programming 121

Data Engineering Podcast

FEBRUARY 25, 2024

Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year! Data Council Logo]([link] Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year! Promo Code: dataengpod20 Dagster : ![Dagster

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

MARCH 10, 2024

Join us at the top event for the global data community, Data Council Austin. Don't miss out on our only event this year! Contact Info LinkedIn Twitter Alex's Article on Dremio's Blog Alex's Substack Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

The Pragmatic Engineer

JUNE 1, 2023

for the simulation engine Go on the backend PostgreSQL for the data layer React and TypeScript on the frontend Prometheus and Grafana for monitoring and observability And if you were wondering how all of this was built, Juraj documented his process in an incredible, 34-part blog series. You can read this here. Serving a web page. Persistence.

Education 363

Education Project PostgreSQL Software Engineer 363

Christophe Blefari

MARCH 8, 2024

This is mainly due to the fact that I've done my annual University lectures and talked at 7 events since the beginning of the year. At every event I talk, I decide to do a new presentation. I mean, there is a big difference between podcast for instance and news blogging like I'm doing.

Data 130

Data Algorithm Database Education 130

Towards Data Science

APRIL 30, 2024

This blog post is my note after reading the paper: The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing. Spark Streaming ) Most that provide event-time-based windowing rely on ordering or have limited window triggering.

Google Cloud 63

Google Cloud Process Cloud Lambda Architecture 63

Data Engineering Weekly

APRIL 21, 2024

The blog narrates how Chronon fits into Stripe’s online and offline requirements. The blog narrates using Apache Arrow Flight RPC to build data querying, post-processing, and caching layers. The blog highlights the lack of control in schema, schema drifting, and scalability as the key challenges.

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

Rockset

MAY 31, 2019

Live dashboards can help organizations make sense of their event data and understand what's happening in their businesses in real time. In many situations, it is important to be able to take immediate action based on real-time event data, as in the case of limited-time sales in e-commerce or managing contact center service levels.

BI 40

BI Java Data SQL 40

DoorDash Engineering

MARCH 27, 2024

Real-time event processing is a critical component of a distributed system’s scalability. At DoorDash, we rely on message queue systems based on Kafka to handle billions of real-time events. In Kafka, a test tenant processing production event can cause data inconsistencies, including outages and other incidents.

Kafka 103

Kafka Architecture Algorithm Systems 103

Christophe Blefari

MARCH 17, 2023

Airflow alternatives meetup Click on the image to go the the LinkedIn event. We are organising next week with the Paris Apache Airflow Meetup group an online event to discuss about Airflow alternatives. We will host another event soon after with others. You can either register on LinkedIn either join the meetup event.

Data 130

Data SQL Deep Learning Kafka 130

Cloudera

MARCH 6, 2024

As we enter into a new month, the Cloudera team is getting ready to head off to the Gartner Data & Analytics Summit in Orlando, Florida for one of the most important events of the year for Chief Data Analytics Officers (CDAOs) and the field of data and analytics.

Data Analytics 106

Data Analytics Data Technology Data Engineering 106