Confluent

MAY 23, 2019

Let us cut to the chase: Kafka Summit London session videos are available! This year’s London event was almost double the size of last year, so I’m especially happy to see these numbers on the page: 2 days of Apache Kafka ®. World-class Kafka keynotes. And if you weren’t there? Well, dig in and start learning! 3 keynotes.

Kafka 109

Kafka Cloud Database Engineering 109

Cloudera

OCTOBER 4, 2022

This blog post will provide guidance to administrators currently using or interested in using Kafka nodes to maintain cluster changes as they scale up or down to balance performance and cloud costs in production deployments. Kafka brokers contained within host groups enable the administrators to more easily add and remove nodes.

Kafka 78

Kafka Data Cloud Big Data 78

Knowledge Hut

NOVEMBER 19, 2023

With advances in technology, wearable devices do provide some trace of your health. Importance of Data Science in IoT IoT and Data Science are amongst the most defining technologies of our age. Challenges of Data Science for IoT Any great technology brings along its own unique set of challenges. Here is the broad.

Data Science 52

Data Science Technology Machine Learning Big Data 52

Confluent

AUGUST 4, 2021

Today, I’m very excited to announce an all-new website dedicated to Apache Kafka®, event streaming, and associated cloud technologies. The site is called Confluent Developer, and it represents a significant […].

Kafka 84

Kafka Cloud Technology IT 84

Rockset

AUGUST 16, 2022

Streams for Everyone If you have come this far it means you have already considered or are considering using event streaming in your data architecture for the wide variety of benefits it can offer. In either case, both Amazon Kinesis and Apache Kafka can help but which one is the right fit for you and your goals.

Kafka 52

Kafka AWS Cloud Java 52

Data Engineering Podcast

NOVEMBER 26, 2019

Summary The team at Sentry has built a platform for anyone in the world to send software errors and events. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today. Closing Announcements Thank you for listening!

Data Warehouse 100

Data Warehouse Building PostgreSQL Kafka 100

Data Engineering Podcast

JULY 1, 2019

Scaling the volume of events that can be processed in real-time can be challenging, so Paul Brebner from Instaclustr set out to see how far he could push Kafka and Cassandra for this use case. Coming up this fall is the combined events of Graphorum and the Data Architecture Summit.

Kafka 100

Kafka Finance Media Architecture 100

Confluent

FEBRUARY 28, 2019

Only a little more than one month after the first release, we are happy to announce another milestone for our Kafka integration. Today, you can grab the Kafka Connect Neo4j Sink from Confluent Hub. . Neo4j extension – Kafka sink refresher. Then it is really up to you what you want to with the event data.

Kafka 86

Kafka Java Programming Language Data Pipeline 86

Confluent

MAY 13, 2019

In the last year, we’ve experienced enormous growth on Confluent Cloud, our fully managed Apache Kafka ® service. As Confluent Cloud has grown, we’ve noticed two gaps that very clearly remain to be filled in managed Apache Kafka services. Five seconds to Kafka (or, never make another cluster again!).

Kafka 100

Kafka Cloud Management Building 100

Confluent

JUNE 27, 2019

Confluent’s clients for Apache Kafka ® recently passed a major milestone—the release of version 1.0. Magnus Edenhill first started developing librdkafka about seven years ago, later joining Confluent in the very early days to help foster the community of Kafka users outside the Java ecosystem. In the v0.11.6

Kafka 106

Kafka Designing Java Coding 106

Precisely

DECEMBER 28, 2023

Enterprise technology is having a watershed moment; no longer do we access information once a week, or even once a day. Streaming data pipelines, by extension, offer an architecture capable of handling large volumes of data, accommodating millions of events in near real time. But insights derived from day-old data don’t cut it.

Data Pipeline 91

Data Pipeline Building Kafka Big Data 91

Lyft Engineering

NOVEMBER 29, 2023

It also allowed us to optimize for handling time-series data and event data at scale. Real-time Ingestion Events from our real-time analytics pipeline were configured to be sent into our internal Flink application, streamed to Kafka, and written into Druid. ioConfig: Kafka server info, topic names, etc. (ex.

Kafka 104

Kafka Data Ingestion Datasets Architecture 104

Confluent

FEBRUARY 20, 2019

How many Kafka Summits should there be in a year? Some say there should be one giant event where everybody gathers at once. Others say you should live every day like it’s Kafka Summit. Anyway, let me remind you of the dates and places: Kafka Summit New York – April 2, 2019. Kafka Summit London – May 13-14, 2019.

Kafka 66

Kafka Programming Architecture Technology 66

Rockset

APRIL 26, 2022

As Kafka Summit is in full swing in London this week and the topic of event streaming is all over my Linkedin feed, I saw a post asking " Is streaming dead? That streaming is rocking and with Kafka Summit this week, I thought it a good time to emphasize the importance of streaming data in today’s modern real-time data stack.

Kafka 52

Kafka Data Warehouse Database Data 52

Jesse Anderson

DECEMBER 12, 2022

They were the first companies to commercialize open source big data technologies and pushed the marketing and commercialization of Hadoop. Apache Kafka came in 2011 and gave the industry a much better way to move real-time data. Apache Kafka has its architectural limitations, and Apache Pulsar was released in 2016.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Netflix Tech

JUNE 1, 2023

In this blog post, we’ll discuss the methods we used to ensure a successful launch, including: How we tested the system Netflix technologies involved Best practices we developed Realistic Test Traffic Netflix traffic ebbs and flows throughout the day in a sinusoidal pattern. We used Elasticsearch dashboards to analyze results.

Algorithm 136

Algorithm Metadata Kafka Systems 136

Confluent

MAY 2, 2019

When managing Apache Kafka ® clusters at scale, tasks that are simple on small clusters turn into significant burdens. Relatedly, KIP-226 enabled dynamic broker reconfiguration since Apache Kafka 1.1. See the documentation (or, if you please, the Apache Kafka wiki ) for a complete list of which parameters this applies to.

Kafka 87

Kafka Management Food Consulting 87

Data Engineering Podcast

OCTOBER 15, 2023

As you evolve the technical and product direction, what is your heuristic for balancing the unification of interfaces and system integration against the ability to swap different components or interfaces as new technologies are introduced? What are the most interesting, innovative, or unexpected ways that you have seen Decodable used?

Process 182

Process Building SQL BI 182

Confluent

AUGUST 29, 2019

We know that Apache Kafka ® is great when you’re dealing with streams, allowing you to conveniently look at streams as tables. Kafka already allows you to look at data as streams or tables; graphs are a third option, a more natural representation with a lot of grounding in theory for some use cases. 8, and so on. Here we go!

Kafka 55

Kafka Process Algorithm Cloud 55

Confluent

MAY 2, 2019

When managing Apache Kafka ® clusters at scale, tasks that are simple on small clusters turn into significant burdens. Relatedly, KIP-226 enabled dynamic broker reconfiguration since Apache Kafka 1.1. See the documentation (or, if you please, the Apache Kafka wiki ) for a complete list of which parameters this applies to.

Kafka 57

Kafka Management Food Consulting 57

Data Engineering Podcast

DECEMBER 24, 2023

Summary Kafka has become a ubiquitous technology, offering a simple method for coordinating events and data across different systems. Can you describe your experiences with Kafka? What are the operational challenges that you have had to overcome while working with Kafka?

Kafka 245

Kafka Data Lake High Quality Data SQL 245

Knowledge Hut

MAY 2, 2024

Since then, Apache Spark has seen a very high adoption rate from top-notch technology companies like Google, Facebook, Apple, Netflix etc. Spark streaming also has in-built connectors for Apache Kafka which comes very handy while developing Streaming applications. Apache Spark was developed by a team at UC Berkeley in 2009.

Scala 52

Scala Hospitality Healthcare Retail 52

Data Engineering Podcast

JULY 27, 2021

The technology to make that possible has been around for a number of years, but the barriers to adoption have still been high due to the level of technical understanding and operational capacity that have been required to run at scale. How do you handle migrating existing projects, particularly if they are using Kafka currently?

Building 100

Building Management Kafka Data Warehouse 100

Data Engineering Podcast

SEPTEMBER 28, 2020

Summary Kafka has become a de facto standard interface for building decoupled systems and working with streaming data. To make the benefits of the Kafka ecosystem more accessible and reduce the operational burden, Alexander Gallego and his team at Vectorized created the Red Panda engine.

Kafka 100

Kafka BI Big Data Data Engineering 100

Data Engineering Podcast

MAY 11, 2020

His most recent endeavor at StreamNative is focused on combining the capabilities of Pulsar with the cloud native movement to make it easier to build and scale real time messaging systems with built in event processing capabilities. One of the critical elements in the success of a piece of technology is the ecosystem that grows around it.

Lambda Architecture 100

Lambda Architecture Cloud Kafka Hadoop 100

Data Engineering Weekly

SEPTEMBER 24, 2023

Learn from today's top data leaders and architects at The Data Observability Summit on how to build more trustworthy and reliable data & AI products with the latest technologies, processes, and strategies shaping our industry (yes, LLMs will be on the table). RSVP NOW Microsoft: Why is it so hard to ship a simple LLM feature?

Data Engineering 67

Data Engineering Data Engineer Engineering Kafka 67

Knowledge Hut

JUNE 26, 2023

Before working on these initiatives, you should be conversant with topics and technologies. Top 4 Data Engineering Project Ideas: Beginner & Final Year Students Becoming an expert data engineer necessitates familiarity with the best practices and cutting-edge technologies in your field. Source Code: Yelp Review Analysis 2.

Data Engineering 98

Data Engineering Data Engineer Coding Project 98

Precisely

SEPTEMBER 12, 2023

Precisely data integrity solutions fuel your Confluent and Apache Kafka streaming data pipelines with trusted data that has maximum accuracy, consistency, and context and we’re ready to share more with you at the upcoming Current 2023. Let’s cover some additional information to know before attending.

Kafka 52

Kafka Data Integration Data Pipeline Datasets 52

Cloudera

FEBRUARY 17, 2021

As a matter of fact, these new challenges gave way to tremendous opportunities for businesses to optimize their key success metrics by leveraging technology. Streaming data (or data-in-motion) is one such technology space that thrived during these times. Click here for a quick overview video or download our eBook to get more details.

Kafka 60

Kafka Food Manufacturing Healthcare 60

Lyft Engineering

JUNE 28, 2023

The Event Driven Decisions capability in particular turned out to be general enough as to be applicable to a wide range of use cases. One key component is the Analytics Event Abstraction layer. as well as multiple technologies (JVM, Python Kernel, etc.) Note that in the code block we defined a RealtimeMLPipline as pipe.

Machine Learning 115

Machine Learning Building Metadata Kafka 115

Confluent

APRIL 25, 2022

Data streaming is a new category of technology that is reshaping the way businesses operate, but there hasn’t been a place for everyone in the ecosystem to come together and […].

Kafka 103

Kafka Technology Data 103

Data Engineering Podcast

AUGUST 3, 2021

Their SDKs and plugins make event streaming easy, and their integrations with cloud applications like Salesforce and ZenDesk help you go beyond event streaming. Contact Info Linkedin Twitter Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?

Data Lake 130

Data Lake Data Warehouse Hadoop Architecture 130

Data Engineering Weekly

JANUARY 21, 2024

Sebastian Raschka: Ten Noteworthy AI Research Papers of 2023 It is an eventful 2023 in machine learning and AI research. The approach revolves around empowering all team members to make data-informed decisions independently, leveraging a suite of tools built on open-source technologies. Visit rudderstack.com to learn more.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Data Engineering Podcast

JULY 6, 2020

For more opportunities to stay up to date, gain new skills, and learn from your peers there are a growing number of virtual events that you can attend from the comfort and safety of your home. Go to dataengineeringpodcast.com/conferences to check out the upcoming events being offered by our partners and get registered today!

Systems 100

Systems Kafka SQL Government 100

Rockset

MARCH 19, 2020

Intro In recent years, Kafka has become synonymous with “streaming,” and with features like Kafka Streams, KSQL, joins, and integrations into sinks like Elasticsearch and Druid, there are more ways than ever to build a real-time analytics application around streaming data in Kafka.

Kafka 40

Kafka Database Process SQL 40

Data Engineering Podcast

MARCH 8, 2021

Their SDKs and plugins make event streaming easy, and their integrations with cloud applications like Salesforce and ZenDesk help you go beyond event streaming. What are some of the technologies or architectural patterns that Molecula might replace in a companies data platform? Closing Announcements Thank you for listening!

IT 100

IT Data Warehouse MongoDB Kafka 100

Big Data Tools

JUNE 8, 2022

News A lot of engineering is about learning new things and keeping a finger on the pulse of new technologies. This section is about updates that are in the works for technologies and that you may want to keep an eye on. Big Data Event: London – Thousands of attendees are expected to participate in this big data event in London.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52