Data Engineering Podcast

AUGUST 21, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Lambda Architecture 100

Lambda Architecture MongoDB Scala MySQL 100

Rockset

AUGUST 30, 2021

It eliminates the cost and complexity around data preparation, performance tuning and operations, helping to accelerate the movement from batch to real-time analytics. The latest Rockset release, SQL-based rollups, has made real-time analytics on streaming data a lot more affordable and accessible.

SQL 52

SQL Kafka MongoDB MySQL 52

Rockset

MAY 17, 2022

Limitations of NoSQL SQL supports complex queries because it is a very expressive, mature language. Complex SQL queries have long been commonplace in business intelligence (BI). And when systems such as Hadoop and Hive arrived, it married complex queries with big data for the first time. This is intentionally not their forte.

SQL 52

SQL NoSQL Hadoop MongoDB 52

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

JUNE 12, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Unstructured Data 130

Unstructured Data MongoDB Scala MySQL 130

Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

OCTOBER 30, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 16, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Lake 100

Data Lake Food MongoDB Scala 100

Data Engineering Podcast

JULY 24, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

MongoDB 100

MongoDB Scala MySQL Data Lake 100

Data Engineering Podcast

JULY 3, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Great Expectations, Soda SQL, etc.)

Data Integration 100

Data Integration MongoDB Scala MySQL 100

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Metadata 130

Metadata MongoDB Scala MySQL 130

Data Engineering Podcast

JUNE 5, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Security 100

Data Security Metadata MongoDB Scala 100

Data Engineering Podcast

JULY 17, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

OCTOBER 2, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

IT 100

IT Food PostgreSQL MongoDB 100

Data Engineering Podcast

AUGUST 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Metadata 100

Metadata MongoDB Scala MySQL 100

DareData

JANUARY 30, 2023

Our goal is to help data scientists better manage their models deployments or work more effectively with their data engineering counterparts, ensuring their models are deployed and maintained in a robust and reliable way. DigDag: An open-source orchestrator for data engineering workflows. Stanford's Relational Databases and SQL.

Data Engineering 52

Data Engineering Data Engineer NoSQL Engineering 52

Data Engineering Podcast

SEPTEMBER 25, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Hospitality 100

Hospitality Food MongoDB Scala 100

Rockset

JANUARY 9, 2023

Technical Challenges Our original data infrastructure was built around an on-premises MongoDB database that ingested and stored all user transaction data. If we didn’t have the right indexes already built, or the SQL query wasn’t optimized, the MySQL query would inevitably drag or time out. It took just one week.

MySQL 52

MySQL MongoDB Recruitment Data Ingestion 52

Data Engineering Podcast

AUGUST 28, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Building 100

Building MongoDB Scala MySQL 100

Data Engineering Podcast

JULY 31, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Analysis 100

Data Analysis MongoDB Scala Algorithm 100

Data Engineering Podcast

JULY 10, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

OCTOBER 9, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

AUGUST 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Data Engineering Podcast

AUGUST 6, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Machine Learning 100

Machine Learning Database MySQL PostgreSQL 100

Data Engineering Podcast

JUNE 26, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

Data Management 100

Data Management Management MongoDB Scala 100

Rockset

NOVEMBER 23, 2021

With clickstream or sensor data, the volume can be incredibly high — many terabytes of data per day — as well as incredibly unpredictable, scaling up and down rapidly. Change data capture streams. It is now possible to continuously capture changes as they happen in your operational database like MongoDB or Amazon DynamoDB.

MongoDB 59

MongoDB Data Ingestion PostgreSQL SQL 59

Data Engineering Podcast

SEPTEMBER 4, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. That’s where our friends at Ascend.io

MongoDB 100

MongoDB Scala MySQL Data Ingestion 100

AltexSoft

JUNE 26, 2023

Read our article on Hotel Data Management to have a full picture of what information can be collected to boost revenue and customer satisfaction in hospitality. While all three are about data acquisition, they have distinct differences. Structured data is modeled to be easily searchable and occupy minimal storage space.

Data Collection 52

Data Collection Machine Learning Unstructured Data Non-relational Database 52

Knowledge Hut

APRIL 25, 2023

It allows data engineers to easily build and customize data models and create interactive dashboards and reports. Looker also provides an SQL-based interface for querying and analyzing data, which makes it easy for data engineers to integrate with existing tools and applications.

Data Engineering 52

Data Engineering Data Engineer Engineering Google Cloud 52

AltexSoft

MAY 12, 2023

A fixed schema means the structure and organization of the data are predetermined and consistent. It is commonly stored in relational database management systems (DBMSs) such as SQL Server, Oracle, and MySQL, and is managed by data analysts and database administrators. Unstructured data has the potential to grow exponentially.

Unstructured Data 59

Unstructured Data NoSQL Hadoop Data Lake 59

Rockset

MARCH 15, 2021

While there’s typically some amount of data engineering required here, there are ways to minimize it. For example, instead of denormalizing the data, you could use a query engine that supports joins. This will avoid unnecessary processing during data ingestion and reduce the storage bloat due to redundant data.

MongoDB 40

MongoDB Data Ingestion Analytics Application Kafka 40

Rockset

NOVEMBER 28, 2022

We’ve already built CDC-based data connectors for many common sources: DynamoDB , MongoDB , and more. For some background, when you ingest data into Rockset you can specify a SQL query, called an ingest transformation , that is evaluated on your source data.

Data Warehouse 52

Data Warehouse PostgreSQL MongoDB Data Pipeline 52

Rockset

MAY 12, 2022

However, these databases tend to sacrifice support for complex SQL queries at any scale. Instead, these database makers have offloaded complex analytics onto application code and their developers, who have neither the skills nor the time to constantly update queries as data sets evolve. One layer processes batches of historic data.

Analytics Application 52

Analytics Application Lambda Architecture Hadoop Electronics 52

Rockset

DECEMBER 9, 2021

Lifting-and-shifting their big data environment into the cloud only made things more complex. The modern data stack introduced a set of cloud-native data solutions such as Fivetran for data ingestion, Snowflake, Redshift or BigQuery for data warehousing , and Looker or Mode for data visualization.

Transportation 52

Transportation BI SQL Data Warehouse 52

AltexSoft

JULY 29, 2022

To facilitate data ingestion, there are Apache Flume aggregating log data from multiple servers and Apache Sqoop designed to transport information between Hadoop and relational (SQL) databases. The most common language for data analysis is SQL but barebone Hadoop doesn’t support it.

Hadoop 59

Hadoop Big Data Google Cloud NoSQL 59

Rockset

OCTOBER 4, 2022

They possess distributed architectures that allow for scalability to handle performance or data volume requirements. Both offer SQL support and are capable of ingesting streaming data from Kafka. This means users need to configure their streams to batch data ahead of loading into ClickHouse.

MySQL 52

MySQL Kafka Aggregated Data Architecture 52