Data Engineering Podcast

NOVEMBER 13, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

MongoDB 100

MongoDB Scala MySQL Google Cloud 100

Data Engineering Podcast

OCTOBER 30, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 25, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Hospitality 100

Hospitality Food MongoDB Scala 100

Data Engineering Podcast

JULY 24, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

MongoDB 100

MongoDB Scala MySQL Data Lake 100

Data Engineering Podcast

JULY 10, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Data Engineering 100

Data Engineering Data Engineer Engineering MongoDB 100

Data Engineering Podcast

JULY 3, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Data Integration 100

Data Integration MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Sifflet also offers a 2-week free trial.

Data Pipeline 130

Data Pipeline Building MongoDB Scala 130

Data Engineering Podcast

OCTOBER 9, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Food 100

Food MongoDB Scala MySQL 100

Data Engineering Podcast

OCTOBER 2, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

IT 100

IT Food PostgreSQL MongoDB 100

Data Engineering Podcast

JUNE 26, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Data Management 100

Data Management Management MongoDB Scala 100

Knowledge Hut

MARCH 19, 2024

It also keeps backups, media files, log data, and static website content. S3 is suitable across several scenarios that utilize S3’s durability, availability, and security features, such as data archiving, content distribution, and data lake implementations, among many others.

AWS 52

AWS Cloud Computing Amazon Web Services Relational Database 52

Rockset

AUGUST 30, 2021

Apache Kafka has made acquiring real-time data more mainstream, but only a small sliver are turning batch analytics, run nightly, into real-time analytical dashboards with alerts and automatic anomaly detection. The majority are still draining streaming data into a data lake or a warehouse and are doing batch analytics.

SQL 52

SQL Kafka MongoDB MySQL 52

Silectis

JANUARY 3, 2021

Data engineers design, build, and maintain data pipelines that transform data from a raw state to a useful one, ready for analysis or data science modeling. Data Integration Combining data from various, disparate sources into one unified view.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

ProjectPro

DECEMBER 7, 2021

Generally, data pipelines are created to store data in a data warehouse or data lake or provide information directly to the machine learning model development. Keeping data in data warehouses or data lakes helps companies centralize the data for several data-driven initiatives.

Data Pipeline 52

Data Pipeline Architecture Kafka AWS 52

Rockset

OCTOBER 11, 2022

In this blog, we’ll compare and contrast how Elasticsearch and Rockset handle data ingestion as well as provide practical techniques for using these systems for real-time analytics. Logstash is an event processing pipeline that ingests and transforms data before sending it to Elasticsearch.

Data Ingestion 52

Data Ingestion Kafka Relational Database PostgreSQL 52

Data Engineering Podcast

NOVEMBER 20, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Data Lake 130

Data Lake Data Ingestion MongoDB Scala 130

DareData

JANUARY 30, 2023

Our goal is to help data scientists better manage their models deployments or work more effectively with their data engineering counterparts, ensuring their models are deployed and maintained in a robust and reliable way. DigDag: An open-source orchestrator for data engineering workflows.

Data Engineering 52

Data Engineering Data Engineer NoSQL Engineering 52

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. In fact, while only 3.5%

Metadata 130

Metadata MongoDB Scala MySQL 130

Data Engineering Podcast

JULY 31, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Sifflet also offers a 2-week free trial.

Data Analysis 100

Data Analysis MongoDB Scala Algorithm 100

ProjectPro

FEBRUARY 8, 2023

It offers a simple and efficient solution for data processing in organizations. It offers users a data integration tool that organizes data from many sources, formats it, and stores it in a single repository, such as data lakes, data warehouses, etc., where it can be used to facilitate business decisions.

AWS 98

AWS Scala Metadata Data Lake 98

AltexSoft

JUNE 26, 2023

Read our article on Hotel Data Management to have a full picture of what information can be collected to boost revenue and customer satisfaction in hospitality. While all three are about data acquisition, they have distinct differences. Non-relational databases , on the other hand, work for data forms and structures other than tables.

Data Collection 52

Data Collection Machine Learning Unstructured Data Non-relational Database 52

ProjectPro

AUGUST 24, 2021

Data Engineering Project for Beginners If you are a newbie in data engineering and are interested in exploring real-world data engineering projects, check out the list of data engineering project examples below. This big data project discusses IoT architecture with a sample use case.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52

Rockset

NOVEMBER 28, 2022

These will help users more easily configure the correct transformations on top of CDC data. The full list of templates and platforms we’re announcing support for includes the following: Template Support Debezium : An open source distributed platform for change data capture.

Data Warehouse 52

Data Warehouse PostgreSQL MongoDB Data Pipeline 52

ProjectPro

JANUARY 31, 2023

There are three steps involved in the deployment of a big data model: Data Ingestion: This is the first step in deploying a big data model - Data ingestion, i.e., extracting data from multiple data sources. What is the difference between SQL and MySQL? MySQL is a non-relational database.

Big Data 52

Big Data Hadoop AWS Relational Database 52

AltexSoft

MAY 12, 2023

A fixed schema means the structure and organization of the data are predetermined and consistent. It is commonly stored in relational database management systems (DBMSs) such as SQL Server, Oracle, and MySQL, and is managed by data analysts and database administrators. Google Cloud Storage can also be used as a data lake system.

Unstructured Data 59

Unstructured Data NoSQL Hadoop Data Lake 59

Cloudera

OCTOBER 11, 2021

Modak’s Nabu is a born in the cloud, cloud-neutral integrated data engineering platform designed to accelerate the journey of enterprises to the cloud. The platform converges data cataloging, data ingestion, data profiling, data tagging, data discovery, and data exploration into a unified platform, driven by metadata.

Data Engineering 89

Data Engineering Data Engineer Cloud Engineering 89

ProjectPro

AUGUST 21, 2015

What are the steps involved in deploying a big data solution? The command to check the list of all tables present in a single database using Sqoop is as follows- Sqoop list-tables –connect jdbc: mysql: //localhost/user; 7. Sqoop provides the capability to store large sized data into a single field based on the type of data.

Hadoop 40

Hadoop Big Data Metadata Java 40