Data Engineering Podcast

JULY 30, 2023

This makes it impossible to select a truly future-proof technology stack for your data platform, making an eventual migration inevitable. In this episode Gleb Mezhanskiy and Rob Goretsky share their experiences leading various data platform migrations, and the hard-won lessons that they learned so that you don't have to.

SQL 130

SQL Machine Learning Python Data 130

KDnuggets

APRIL 25, 2024

List of top MLOPs platforms that will help you with integration, training, tracking, deployment, monitoring, CI/CD, and optimizing the infrastructure.

91

91

Lyft Engineering

MARCH 12, 2024

Summary At Lyft we have built a platform for developing, training and serving Reinforcement Learning models for typical internet industry applications with a focus on Contextual Bandits. The big advantage of this approach is that it allows us to leverage lots of proven platform components.

Algorithm 66

Algorithm Machine Learning Datasets Food 66

Data Engineering Podcast

APRIL 9, 2023

Summary The customer data platform is a category of services that was developed early in the evolution of the current era of cloud services for data processing. Now that the data warehouse has taken center stage a new approach of composable customer data platforms is emerging.

Data Lake 147

Data Lake Data Warehouse Machine Learning Data 147

databricks

NOVEMBER 15, 2023

The observation that "software is eating the world" has shaped the modern tech industry. Today, software is ubiquitous in our lives, from the.

Data 142

Data 142

Data Engineering Podcast

FEBRUARY 26, 2023

Jean-Georges Perrin was tasked with designing a new data platform implementation at PayPal and wound up building a data mesh. Jean-Georges Perrin was tasked with designing a new data platform implementation at PayPal and wound up building a data mesh.

Building 147

Building Metadata Machine Learning Data Integration 147

Towards Data Science

FEBRUARY 9, 2024

How to build a modern, scalable data platform to power your analytics and data science projects (updated) Table of Contents: What’s changed? The Platform Integration Data Store Transformation Orchestration Presentation Transportation Observability Closing What’s changed? Since 2021, maybe a better question is what HASN’T changed?

Building 81

Building Transportation Data Lake Metadata 81

Data Engineering Podcast

AUGUST 27, 2023

Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Go to dataengineeringpodcast.com/materialize today to get 2 weeks free!

Database 130

Database Building PostgreSQL BI 130

KDnuggets

OCTOBER 25, 2023

What are the top 5 cloud machine learning platforms in the market today. Our list will help provide some vital insights into which platform might best cater to your specific machine learning needs. See what KDnuggets recommends.

Machine Learning 118

Machine Learning Cloud 118

Start Data Engineering

JUNE 30, 2023

Components of a self-serve platform 3. Building a self-serve data platform 3.1. Introduction Most companies want to build a self-serve data platform. Creating dataset(s) 3.1.1. Gather requirements 3.1.2. Get data foundations right 3.2. Accessing data 3.3. Identify and remove dependencies 4. Conclusion 5. Further reading 6.

Building 130

Building Datasets Data Accessible 130

Knowledge Hut

DECEMBER 27, 2023

Due to its rising demand, businesses have started realizing the potential of blockchain platforms. The demand for blockchain development platforms has skyrocketed as enterprises have begun to build blockchain apps to test the capabilities of the technology. What is a Blockchain Platform? What Languages does the Platform Support?

Programming Language 77

Programming Language Banking Utilities Algorithm 77

Confluent

MAY 4, 2023

Hardening the innovative feature set introduced in recent releases, Confluent Platform 7.4 enables you to enhance scalability and simplify your architecture, accelerate time to market, and improve data quality.

Architecture 104

Architecture Data 104

Zalando Engineering

AUGUST 2, 2023

The portal is based on Spotify's Backstage platform with additional extensions built internally. As the official product manager, I helped solidify the vision with a platform mindset and an experience strategically focused on interoperability and usability. Arthur, how about you? How are you involved here? Arthur: Hello!

Software Engineer 98

Software Engineer Software Engineering Engineering Data Pipeline 98

Data Engineering Podcast

JULY 2, 2023

What are the most interesting, innovative, or unexpected ways that you have seen feature platforms used? What are the resources that you find most helpful in understanding and designing feature platforms? What are the most interesting, innovative, or unexpected ways that you have seen feature platforms used?

Machine Learning 130

Machine Learning Data Engineering Data Engineer Engineering 130

Robinhood

FEBRUARY 7, 2024

Hadoop-Based Batch Processing Platform (V1) Initial Architecture In our early days of batch processing, we set out to optimize data handling for speed and enhance developer efficiency. Credits All of this is just one chapter in our batch processing platform journey. This helps with overall cost efficiency. Tiecheng S.,

Process 74

Process Hadoop Architecture Accessible 74

Ripple Engineering

JANUARY 29, 2024

Why do we need a Data Platform? At Ripple, different business units require a platform to quickly and easily turn their data into insights and compelling customer experiences. Why is it difficult to build a centralized platform? How does Data Platform provide APIs for Ingestion and Lookup?

Database-centric 52

Database-centric Pipeline-centric NoSQL High Quality Data 52

KDnuggets

OCTOBER 1, 2023

Explore the essentials of Google Cloud Platform for data science and ML, from account setup to model deployment, with hands-on project examples.

Google Cloud 136

Google Cloud Cloud Data Science Project 136

KDnuggets

JANUARY 13, 2023

Here's a list of the best platforms to practice SQL, ace your SQL interviews, and land your dream data role. Looking to level up your SQL skills?

SQL 139

SQL Data 139

Confluent

FEBRUARY 14, 2024

Confluent Platform 7.6 brings upgrading for existing clusters from ZooKeeper to KRaft, compaction support for Tiered Storage, OAuth (early access), improvements to the Oracle CDC premium connector, and more.

Accessible 119

Accessible Accessibility 119

LinkedIn Engineering

DECEMBER 20, 2023

Our members rely on us to create an environment on our platform where they can safely learn and grow in their careers. In this blog post, we discuss how we are harnessing AI to help us with abuse prevention and share an overview of our infrastructure and the role it plays in identifying and mitigating abusive behavior on our platform.

Building 91

Building Algorithm Kafka Machine Learning 91

Data Engineering Podcast

JUNE 19, 2022

Summary Metadata is the lifeblood of your data platform, providing information about what is happening in your systems. A variety of platforms have been developed to capture and analyze that information to great effect, but they are inherently limited in their utility due to their nature as storage systems. In fact, while only 3.5%

Metadata 130

Metadata MongoDB Scala MySQL 130

Confluent

FEBRUARY 10, 2021

We are pleased to announce the release of Confluent Platform 6.1. With this release, we are further simplifying management tasks for Apache Kafka® operators and providing even higher availability for […].

Kafka 142

Kafka Management Cloud 142

Confluent

AUGUST 30, 2023

Confluent Platform 7.5 brings SSO for Control Center, simplified interface with Confluent using v3 of the REST proxy API, and bidirectional Cluster Linking.

78

78

Cloudera

FEBRUARY 2, 2024

Forrester Research recently released the Forester Wave for Streaming Platforms, Q4 2023. This year they broadened their criteria to streaming platforms in recognition of market needs for complete capabilities, development tools, and scalability—all capabilities Cloudera has invested in over several years.

Kafka 97

Kafka Architecture Data Architecture Cloud 97

Engineering at Meta

JANUARY 26, 2023

The technical journey discusses the motivations, challenges, and technical solutions employed for warehouse schematization, especially a change to the wire serialization format employed in Meta’s data platform for data interchange related to Warehouse Analytics Logging. Logger is at the heart of the data platform.

Bytes 102

Bytes Data Engineering Coding 102

LinkedIn Engineering

MARCH 27, 2023

It can be complex and difficult to navigate, which makes the LinkedIn Business Platform such an important tool for delivering high-quality experiences to LinkedIn sales representatives (our internal consumers) and also customers of our products and services. This only applies to geographic region H when customer types are I or J.”

Architecture 78

Architecture Coding Project Systems 78

KDnuggets

OCTOBER 18, 2023

Cloud databases have made it easier and cheaper to develop enterprise-level applications, offering flexibility, convenience, and standard database functionality. See what KDnuggets recommends.

Database 133

Database Cloud IT Data Engineering 133

Knowledge Hut

DECEMBER 28, 2023

During my research on emerging technologies a few years ago, I was drawn in by the capabilities of AI platforms. Many sectors have adopted it, leading to the emergence of AI platforms that offer tools and services for constructing intelligent applications. Artificial intelligence has progressed dramatically in the past decade.

Amazon Web Services 64

Amazon Web Services Machine Learning Medical Deep Learning 64

KDnuggets

MAY 10, 2023

Chatbot Arena is a benchmark platform for large language models, where the community can contribute new models and evaluate them.

Process 110

Process 110

Pinterest Engineering

OCTOBER 31, 2023

Harry Zhang, Jiajun Wang, Yi Li, Shunyao Li, Ming Zong, Haniel Martino, Cathy Lu, Quentin Miao, Hao Jiang, James Wen, David Westbrook | Cloud Runtime Team Image Source: [link] Overview Modern compute platforms are foundational to accelerating innovation and running applications more efficiently.

Architecture 95

Architecture Pipeline-centric Accessible Accessibility 95