Data Engineering Podcast

JANUARY 8, 2023

Join your peers at Gartner Data & Analytics Summit, March 20 – 22 in Orlando, FL for 3 days of expert guidance, peer networking and collaboration. What are the opportunities for using utilities such as dbt for managing logical complexity as the number of pipelines scales? SQL has traditionally been challenging to compose.

PostgreSQL 130

PostgreSQL Data Lake Data Warehouse BI 130

dbt Developer Hub

DECEMBER 19, 2023

Overview ​ Over the course of my three years running the Partner Engineering team at dbt Labs, the most common question I've been asked is, How do we integrate with dbt? This guide doesn't include how to integrate with dbt Core. Instead, we're going to focus on integrating with dbt Cloud. New to dbt Cloud? ​

Metadata 52

Metadata Cloud Accessible Accessibility 52

Data Engineering Podcast

JANUARY 15, 2023

With a PostgreSQL-compatible interface, you can now work with real-time data using ANSI SQL including the ability to perform multi-way complex joins, which support stream-to-stream, stream-to-table, table-to-table, and more, all in standard SQL. If you've learned something or tried out a project from the show then tell us about it!

Coding 130

Coding Building PostgreSQL Data Lake 130

Data Engineering Weekly

MAY 28, 2023

I'm more than happy to collaborate and help the community. Treat Kafka as immutable infra Adopt multi-instance over multi-tenant We discussed Murron's design in detail here if anyone wants to know more about it. It is exciting to see the case study of dbt coming out from Zendesk, focusing on foundations and scalability.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Data Engineering Podcast

MARCH 13, 2022

In this episode Tristan Spaulding, head of product at Acceldata, explains the multi-dimensional nature of gaining visibility into your running data platform and how they have architected their platform to assist in that endeavor. Atlan is a collaborative workspace for data-driven teams, like Github for engineering or Figma for design teams.

Data Lake 100

Data Lake Relational Database Data Data Engineering 100

AltexSoft

JANUARY 26, 2022

If we take the more traditional approach to data-related jobs used by larger companies, there are different specialists doing narrowly-focused tasks on different sides of the project. Get acquainted with how data is prepared for machine learning projects in our dedicated video. Close collaboration with other team members.

Engineering 59

Engineering Software Engineer Software Engineering Data Warehouse 59

Data Engineering Podcast

APRIL 10, 2022

Summary Data engineering is a practice that is multi-faceted and requires integration with a large number of systems. Atlan is a collaborative workspace for data-driven teams, like Github for engineering or Figma for design teams. If you’ve learned something or tried out a project from the show then tell us about it!

Data Integration 100

Data Integration SQL Data Engineering Data Engineer 100

Data Engineering Podcast

JANUARY 7, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

ETL System 100

ETL System Data Pipeline Data Warehouse Architecture 100

DataKitchen

APRIL 13, 2021

Companies that implement DataOps find that they are able to reduce cycle times from weeks (or months) to days, virtually eliminate data errors, increase collaboration, and dramatically improve productivity. Collaboration and Sharing. Lentiq — Lentiq is the data science environment that brings your projects to life. .

Consulting 123

Consulting Machine Learning Data Science Data Pipeline 123

AltexSoft

JULY 18, 2023

Its flexibility allows it to operate on single-node machines and large clusters, serving as a multi-language platform for executing data engineering , data science , and machine learning tasks. The Spark project on GitHub , where the bulk of development occurs, has garnered over 36,3k stars and counts more than 1,9k contributors.

Big Data 52

Big Data Data Process Process Hadoop 52

Ascend.io

MAY 19, 2023

Stage 3 begins as these early adopters collaborate formally and informally, identifying and documenting best practices and patterns in the form of “reference architectures”. The automation frees engineers to focus on strategic, high-impact projects and business engagement. We see this pattern play out every day. The result?

Data Pipeline 52

Data Pipeline Data Engineering Data Engineer Media 52