Rockset

NOVEMBER 9, 2021

There are multiple change data capture methods available when using a MySQL or Postgres database. In this post, we’re going to dive deeper into the different ways you can implement CDC if you have either a MySQL and Postgres database and compare the approaches.

MySQL 52

MySQL Kafka AWS Database 52

Monte Carlo

MARCH 6, 2024

At the start of 2023, we extended our data observability solution’s monitoring capabilities to popular transactional databases like Postgres , MySQL , and Microsoft MySQL Server. End-to-end data lineage then helps trace these issues to their point of origin. There are several common reasons for this.

Database 52

Database MySQL Data Warehouse AWS 52

Christophe Blefari

APRIL 19, 2024

Data pipeline, incremental vs. full load — A comprehension comparison between 2 mode of ingestion with a decision tree about which one to pick. ❤️ Scaling to count billions — An awesome retrospective of the Canva OLAP architecture to count marketplace usage, from MySQL to Snowflake + buckets. To watch.

MySQL 130

MySQL Data Datasets SQL 130

Data Engineering Podcast

DECEMBER 11, 2022

Managing the operational concerns for your database can be complex and expensive, especially if you need to scale to large volumes of data, high traffic, or geographically distributed usage. Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows.

Database 100

Database MySQL Data Lake MongoDB 100

Hevo

APRIL 12, 2024

As analytics in your company graduates from a MySQL/PostgreSQL/SQL Server, a pertinent question that you need to answer is which data warehouse is best suited for you. This blog tries to compare Redshift vs BigQuery – two very famous cloud data warehouses today.

PostgreSQL 40

PostgreSQL MySQL Data Warehouse SQL 40

Cloudera

FEBRUARY 12, 2024

For data warehouse: create data warehouses through the Cloudera command line interface with the parameter “privateDNSZoneAKS”: set to”None.” ” For Liftie-based data services: the entitlement “LIFTIE_AKS_DISABLE_PRIVATE_DNS_ZONE” must be set.

MySQL 52

MySQL PostgreSQL Data Warehouse Machine Learning 52

Cloudera

SEPTEMBER 17, 2018

These trends and demands lead to stress for existing data warehouse solutions – scale, efficiency, security integrations, IT budgets, ease of access. Cloudera recently launched Cloudera Data Warehouse, a modern data warehousing solution. Data Discovery and Exploration.

Data Warehouse 53

Data Warehouse SQL Portfolio MySQL 53

Data Engineering Podcast

DECEMBER 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. or any other destination you choose.

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130

Knowledge Hut

DECEMBER 21, 2023

What is Data Engineering? Data engineering is the method to collect, process, validate and store data. It involves building and maintaining data pipelines, databases, and data warehouses. The purpose of data engineering is to analyze data and make decisions easier.

Data Engineering 52

Data Engineering Data Engineer Engineering MongoDB 52

Data Engineering Podcast

DECEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata Business Intelligence Data Lake BI 130

Data Engineering Podcast

JUNE 22, 2021

RudderStack’s smart customer data pipeline is warehouse-first. It builds your customer data warehouse and your identity graph on your data warehouse, with support for Snowflake, Google BigQuery, Amazon Redshift, and more. RudderStack’s smart customer data pipeline is warehouse-first.

Database 100

Database MySQL PostgreSQL Data Warehouse 100

Data Engineering Podcast

DECEMBER 25, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. or any other destination you choose.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Striim

JULY 7, 2023

A guide on when real-time data pipelines are the most reliable way to keep production databases and warehouses in sync. Analytical workloads aren’t meant for transactional databases, which are optimized for high latency reads, writes, and data integrity. Data warehouses optimize for precisely this using OLAP.

PostgreSQL 52

PostgreSQL MySQL Data Warehouse Data Pipeline 52

Data Engineering Podcast

DECEMBER 29, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode.

Management 130

Management Metadata Business Intelligence Data Lake 130

ThoughtSpot

OCTOBER 27, 2023

Every business that analyzes their operational (or transactional) data needs to build a custom data pipeline involving several batch or streaming jobs to extract transactional data from relational databases , transform it, and load it into the data warehouse. For example, “what is the qtysold over period of time?”

Building 70

Building MySQL Data Warehouse SQL 70

Data Engineering Podcast

JANUARY 22, 2023

To help other people find the show please leave a review on Apple Podcasts and tell your friends and co-workers Links Tonic Djinn Django Ruby on Rails C# Entity Framework PostgreSQL MySQL Oracle DB MongoDB Parquet Databricks Mockaroo The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Sponsored By: Materialize : ![Materialize]([link]

PostgreSQL 130

PostgreSQL MongoDB MySQL Machine Learning 130

ThoughtSpot

OCTOBER 27, 2023

Every business that analyzes their operational (or transactional) data needs to build a custom data pipeline involving several batch or streaming jobs to extract transactional data from relational databases , transform it, and load it into the data warehouse. For example, “what is the qtysold over period of time?”

Building 69

Building MySQL Data Warehouse SQL 69

Data Engineering Podcast

DECEMBER 4, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. or any other destination you choose.

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Systems 100

Systems Metadata Building MongoDB 100

Data Engineering Podcast

NOVEMBER 27, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. or any other destination you choose.

Data Lake 100

Data Lake Data Warehouse MongoDB Data Pipeline 100

Data Engineering Podcast

OCTOBER 30, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

NOVEMBER 20, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Lake 130

Data Lake Data Ingestion MongoDB Scala 130

Data Engineering Podcast

NOVEMBER 6, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

MongoDB 130

MongoDB Scala MySQL Data Lake 130

Monte Carlo

DECEMBER 4, 2023

Loading ChatGPT ETL prompts can help write scripts to load data into different databases, data lakes, or data warehouses. I'd like to import this data into my MySQL database into a table called products_table. The data is currently in a pandas DataFrame. Can you assist in writing this script?

PostgreSQL 52

PostgreSQL Data Lake ETL Tools MySQL 52

Data Engineering Podcast

AUGUST 6, 2022

Summary The optimal format for storage and retrieval of data is dependent on how it is going to be used. For analytical systems there are decades of investment in data warehouses and various modeling techniques. For analytical systems there are decades of investment in data warehouses and various modeling techniques.

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

DECEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. or any other destination you choose.

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

Data Engineering Podcast

AUGUST 21, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Data Engineering Podcast

JULY 3, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Tired of deploying bad data?

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

JUNE 26, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

NOVEMBER 27, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Process 100

Data Process Process Metadata Business Intelligence 100

Data Engineering Podcast

DECEMBER 4, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Business Intelligence 100

Business Intelligence Metadata BI MongoDB 100

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

AUGUST 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 100

Metadata MongoDB Scala MySQL 100

Monte Carlo

MARCH 25, 2024

.” Justin Swenson Data, Analytics, and AI Governance Lead, Sub-Zero That means Monte Carlo needs to support and integrate across those same solutions and architectures. Deepening our integrations with modern orchestration tools like dbt and Airflow. Supporting new table formats in Snowflake and Databricks.

MySQL 64

MySQL Government Data Governance Kafka 64

Data Engineering Podcast

OCTOBER 16, 2022

Summary The "data lakehouse" architecture balances the scalability and flexibility of data lakes with the ease of use and transaction support of data warehouses. Dremio is one of the companies leading the development of products and services that support the open lakehouse.

Data Lake 100

Data Lake Food MongoDB Scala 100

Data Engineering Podcast

JULY 17, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Tired of deploying bad data?

Data Management 130

Data Management Management Metadata MongoDB 130