Data Engineering Podcast

SEPTEMBER 28, 2020

Summary Kafka has become a de facto standard interface for building decoupled systems and working with streaming data. This was a fascinating conversation with an energetic and enthusiastic engineer and founder about the challenges and opportunities in the realm of streaming data.

Kafka 100

Kafka BI Big Data Data Engineering 100

Cloudera

OCTOBER 20, 2022

Information technology has been at the heart of governments around the world, enabling them to deliver vital citizen services, such as healthcare, transportation, employment, and national security. All of these functions rest on technology and share a valuable commodity: data. .

Big Data 74

Big Data Transportation Healthcare Government 74

Data Engineering Podcast

OCTOBER 12, 2020

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management What are the pieces of advice that you wish you had received early in your career of data engineering? If you hand a book to a new data engineer, what wisdom would you add to it?

Business Intelligence 100

Business Intelligence BI Consulting Data Ingestion 100

Data Engineering Weekly

MARCH 12, 2023

Contribute to the Rudderstack Transformations Library, Win $1000 RudderStack Transformations lets you customize event data in real time with your own JavaScript or Python code. link] Editor’s Note: Data Engineering Radio At Data Engineering Weekly, We strive to bring the best thought process around building and operating data.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

ThoughtSpot

JANUARY 10, 2023

As I reflect back on season three of The Data Chief, I once again am so grateful for our guests who share their insights and inspire us all. In fact, The Data Chief podcast was downloaded in over 190 countries and was among Spotify’s top 5% of most shared content globally. THANK YOU!

Recruitment 52

Recruitment Food Data Data Science 52

Monte Carlo

JULY 7, 2022

At the time, he was one of a very few people using the term “data contract.” Data contracts have since became one of the most discussed topics in data engineering. For posterity, we have preserved Barr’s forward that examines what was then a very nascent trend, but we have also added an updated data contract FAQ as an addendum.

Data Warehouse 52

Data Warehouse Software Engineer Software Engineering Data 52

Data Engineering Podcast

DECEMBER 18, 2022

Summary The data ecosystem has seen a constant flurry of activity for the past several years, and it shows no signs of slowing down. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. RudderStack helps you build a customer data platform on your warehouse or data lake.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

Monte Carlo

MARCH 28, 2024

In recent years, we’ve seen all sorts of new job titles emerge that would have been inscrutable just a decade or two ago – cloud architect, data reliability engineer , data product manager , director of hybrid working, and yes, DataOps engineer. So what exactly IS a DataOps engineer? What does a DataOps engineer do?

Pipeline-centric 75

Pipeline-centric Engineering BI Google Cloud 75

Scott Logic

MAY 4, 2023

This blog doesn’t intend to provide an overview of Generative AI or Large Language Models (LLMs), partly as this has been covered in other posts on this blog and our podcast. My career as a technology strategist and architect has tended to require looking further out – considering not just the opportunities from technology but also the risks.

Architecture 78

Architecture Machine Learning Data Science Government 78

Data Engineering Podcast

JANUARY 28, 2018

Summary Sharing data across multiple computers, particularly when it is large and changing, is a difficult problem to solve. In order to provide a simpler way to distribute and version data sets among collaborators the Dat Project was created. You can help support the show by checking out the Patreon page which is linked from the site.

Data 100

Data Project Electronics Data Management 100

Confluent

MARCH 20, 2019

By continuous refinement , I mean that Kafka Streams emits new results whenever records are updated. By continuous refinement , I mean that Kafka Streams emits new results whenever records are updated. You get to focus on the logic of your data processing pipeline. It is important for the operational characteristics, though.

Kafka 106

Kafka Programming Process Designing 106

Confluent

FEBRUARY 27, 2019

Given that it is at a relatively early stage, developers are still trying to grok the best approach for each cloud vendor and often face the following question: Should I go cloud native with AWS Lambda, GCP functions, etc., FaaS functions only solve the compute part, but where is data stored and managed, and how is it accessed?

Kafka 109

Kafka AWS Architecture Cloud 109

Monte Carlo

APRIL 1, 2021

We sat down with C indi Howson , Chief Data Strategy Officer at T houghtSpot , the leading search and AI-driven analytics platform, for a wide-ranging conversation about her daily work, common challenges organizations face on the road to data democratization, and diversity in data science. And for good reason.

Building 52

Building BI Business Intelligence Data 52

Monte Carlo

AUGUST 11, 2022

As companies increasingly rely on data to drive decision making and power digital products, it’s critical that this data is accessible, discoverable, and reliable. For decades, organizations have relied on data catalogs to drive these initiatives. Adoption was slow, which for data catalogs can be a time-to-value death spiral.

Government 52

Government Data Datasets Machine Learning 52

Monte Carlo

FEBRUARY 19, 2024

And there are few data industry buzzwords that have been more hyped over the last four years than the self-service darling, “the data mesh.” Now, let me just say on the front end, the data mesh is not an Avatar 2 situation. (14 Now, let me just say on the front end, the data mesh is not an Avatar 2 situation. (14

Architecture 59

Architecture Government Data Data Architecture 59