Data Engineering Podcast

JUNE 12, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Open Source DataHub is running in production at several companies like Peloton, Optum, Udemy, Zynga and others.

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Data Engineering Podcast

SEPTEMBER 25, 2022

Summary Data integration from source systems to their downstream destinations is the foundational step for any data product. In this episode Mark Van de Wiel explains how they integrated CDC functionality into their existing product, discusses the nuances of different approaches to change data capture from various sources.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

NOVEMBER 13, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. For more information on Prefect, visit dataengineeringpodcast.com/prefect.

Building 130

Building Metadata MongoDB MySQL 130

Monte Carlo

MARCH 25, 2024

.” Justin Swenson Data, Analytics, and AI Governance Lead, Sub-Zero That means Monte Carlo needs to support and integrate across those same solutions and architectures. Deepening our integrations with modern orchestration tools like dbt and Airflow. More pipelines, more data sources, and more consumers.

MySQL 64

MySQL Government Data Governance Kafka 64

Data Engineering Podcast

JULY 3, 2022

Summary The perennial challenge of data engineers is ensuring that information is integrated reliably. In order to quickly identify if and how two data systems are out of sync Gleb Mezhanskiy and Simon Eskildsen partnered to create the open source data-diff utility. Go to dataengineeringpodcast.com/ascend and sign up for a free trial.

Data Integration 100

Data Integration MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 2, 2022

Roambee is a platform that collects, integrates, and analyzes all of that information to provide companies with the critical insights that businesses need to stay running, especially in a time of such constant change. Can you describe the types of information sources and the format of the data that you are working with?

Metadata 100

Metadata Electronics MongoDB MySQL 100

Data Engineering Podcast

OCTOBER 30, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. What are the options for integrating Sisense with an organization’s data platform?

Business Intelligence 130

Business Intelligence Metadata MongoDB MySQL 130

Data Engineering Podcast

DECEMBER 4, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. No more shipping and praying, you can now know exactly what will change in your database!

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

Data Engineering Podcast

NOVEMBER 13, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. No more shipping and praying, you can now know exactly what will change in your database!

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

OCTOBER 23, 2022

In this episode field CTO Manjot Singh shares his experiences as an early user of MySQL and MariaDB and explains how the suite of products being built on top of the open source foundation address the growing needs for advanced storage and analytical capabilities. Go to dataengineeringpodcast.com/ascend and sign up for a free trial.

Database 100

Database MySQL Cloud MongoDB 100

Christophe Blefari

MAY 6, 2023

They come up with a lot of news: they will implement Delta Sharing protocol, lower the storage for $118/TB to $25, partner with AWS and propose low-code data integration tool. Data projects on personal data — Petrica dive into her Medium data with DuckDB and Plotly and Stefen analysed his Uber spends with dbt and Postgres.

Programming Language 100

Programming Language Data Python Data Warehouse 100

Data Engineering Podcast

JULY 10, 2022

Beyond the work of assembling a set of technologies and building integrations across them, there is also the work of growing and organizing a team that can support and benefit from that platform. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows.

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

SEPTEMBER 11, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. There are numerous commercial and open source options for collecting, transforming, and integrating data.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

OCTOBER 30, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. No more shipping and praying, you can now know exactly what will change in your database!

Engineering 100

Engineering MongoDB Scala MySQL 100

Rockset

NOVEMBER 9, 2021

There are multiple change data capture methods available when using a MySQL or Postgres database. In this post, we’re going to dive deeper into the different ways you can implement CDC if you have either a MySQL and Postgres database and compare the approaches.

MySQL 52

MySQL Kafka AWS Database 52

Data Engineering Podcast

SEPTEMBER 25, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. What are the available integration/extension points for teams who are using Soda Core?

Building 100

Building Metadata MongoDB MySQL 100

Data Engineering Podcast

JUNE 19, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. No more shipping and praying, you can now know exactly what will change in your database!

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

SEPTEMBER 18, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. What are the sources of information that are needed to be able to answer these questions?

Hospitality 100

Hospitality Food MongoDB Scala 100

Christophe Blefari

APRIL 28, 2023

This is mainly a roadmap of the MeticFlow integration within dbt ecosystem. At the moment we have a dbt Semantic Layer that correspond to YAML definitions and MetricFlow—which was Transform open-source project—that is able to understand the semantics to generates SQL. A lot of changes will happen to MetricFlow incl.

SQL 100

SQL Food PostgreSQL Data 100

Data Engineering Podcast

OCTOBER 9, 2022

In this episode Purvi Shah, the VP of Enterprise Big Data Platforms at American Express, explains how they have invested in the cloud to power this visibility and the complex suite of integrations they have built and maintained across legacy and modern systems to make it possible.

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 9, 2022

In order to make the data lakehouse available to a wider audience the team at Iomete built an all-in-one service that handles management and integration of the various technologies so that you can worry about answering important business questions. For more information on Prefect, visit dataengineeringpodcast.com/prefect.

Metadata 100

Metadata AWS MongoDB MySQL 100

Data Engineering Podcast

OCTOBER 16, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. What are some of the ways that Pathway can be integrated into an analytical system?

Database 100

Database Metadata MongoDB MySQL 100

Data Engineering Podcast

AUGUST 12, 2019

Because of the myriad sources and destinations that are available, it is also among the most difficult tasks that we encounter. Fivetran is a platform that does the hard work for you and replicates information from your source systems into whichever data warehouse you use. Integration of multiple data sources (e.g.

Media 100

Media Data Warehouse Data Big Data 100

Data Engineering Podcast

OCTOBER 16, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. What are some of the investments that Dremio has made to the broader open source/open lakehouse ecosystem?

Data Lake 100

Data Lake Food MongoDB Scala 100

Data Engineering Podcast

JUNE 5, 2022

The team at Skyflow decided that the second best way is to build a storage system dedicated to securely managing your sensitive information and making it easy to integrate with your applications and data systems. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP.

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

JULY 31, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. What is involved in integrating Manta with an organization’s data systems?

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

SEPTEMBER 4, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. What are the data sources that you are working with to derive climate information?

MongoDB 100

MongoDB Scala MySQL Data Ingestion 100

Grouparoo

FEBRUARY 25, 2021

While having these integrations already written for them saved plenty of time, there was something they missed: their normal developer workflow. Grouparoo now supports declarative data models and integrations to continuously sync your data to all of your cloud-based tools. Here is the example app from the video.

Data 52

Data Database Cloud Engineering 52

Data Engineering Podcast

OCTOBER 2, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. What are some of the other sources/pieces of information that you integrate into the lineage graph?

IT 100

IT Food PostgreSQL MongoDB 100

Data Engineering Podcast

JULY 3, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. No more shipping and praying, you can now know exactly what will change in your database!

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

JUNE 12, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. No more shipping and praying, you can now know exactly what will change in your database!

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

JULY 24, 2022

This complicates the work involved in making end-to-end workflows visible and integrated. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs.

MongoDB 100

MongoDB Scala MySQL Data Lake 100

Data Engineering Weekly

FEBRUARY 19, 2023

Plum Living writes about integrating the dbt semantic layer with Superset and the developer workflow. link] Antons Tocilins-Ruberts: End-to-End ML Pipelines with MLflow: Tracking, Projects & Serving MLFlow is an open-source platform for managing the machine learning lifecycle. All rights reserved ProtoGrowth Inc, India.

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

Striim

NOVEMBER 17, 2023

Striim serves as a real-time data integration platform that seamlessly and continuously moves data from diverse data sources to destinations such as cloud databases, messaging systems, and data warehouses, making it a vital component in modern data architectures. In this blog, we will focus on a PostgreSQL database.

Machine Learning 52

Machine Learning Data Process PostgreSQL Process 52

Workfall

AUGUST 14, 2023

These pipelines ensure that data flows smoothly from various sources to its intended destinations, enabling businesses to make informed decisions and gain valuable insights. DBT and Airflow Integration Best Practices Scalability and Flexibility Hands-On Conclusion Understanding DBT and Airflow What is DBT? What is Airflow?

Data Pipeline 62

Data Pipeline Raw Data Data Database 62

Grouparoo

DECEMBER 13, 2021

As part of our goal to make Data Engineer's lives easier by integrating more deeply with the Modern Data Stack , we've been working on new ways to trigger Grouparoo's Schedules to run. This year we've watched dbt take off as the open source data transformation tool of choice and were excited for their 1.0

Metadata 52

Metadata Data Warehouse SQL Utilities 52

Data Engineering Podcast

AUGUST 13, 2022

With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/ascend and sign up for a free trial.

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100