Data Engineering Podcast

DECEMBER 25, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode.

Building 100

Building Metadata Business Intelligence Data Lake 100

Data Engineering Podcast

JUNE 26, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

JUNE 12, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

JULY 3, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Tired of deploying bad data?

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 17, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Tired of deploying bad data?

Data Management 130

Data Management Management Metadata MongoDB 130

Data Engineering Podcast

NOVEMBER 20, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

Data Engineering Podcast

JULY 31, 2022

Summary Data lineage is the roadmap for your data platform, providing visibility into all of the dependencies for any report, machine learning model, or data warehouse table that you are working with.

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

JULY 10, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Tired of deploying bad data?

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Data Engineering Podcast

JUNE 5, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Security 100

Data Security Metadata MongoDB Scala 100

Striim

FEBRUARY 10, 2023

To facilitate the data integration process, it used Striim, which allowed it to: Import historical and real-time on-premise data from its Oracle and DB2 mainframe databases. Process the data in flight, including detecting and transforming mismatched timestamp fields.

Data Analytics 52

Data Analytics Telecommunication Insurance Google Cloud 52

Data Engineering Podcast

OCTOBER 9, 2022

Summary The core of any data platform is the centralized storage and processing layer. For many that is a data warehouse, but in order to support a diverse and constantly changing set of uses and technologies the data lakehouse is a paradigm that offers a useful balance of scale and cost, with performance and ease of use.

Metadata 100

Metadata AWS MongoDB MySQL 100

Knowledge Hut

JANUARY 24, 2024

Power BI takes advantage of Microsoft's business analytics. The business intelligence market has multiplied in recent years and is anticipated to do so going forward. You should be data-driven if you want to pursue your career in Business Intelligence, Analytics, or Data Science.

BI 98

BI Business Intelligence Non-relational Database Data Analysis 98

Knowledge Hut

DECEMBER 26, 2023

Thus, to build a career in Data Science, you need to be familiar with how the business operates, its business model, strategies, problems, and challenges. Data Science Roles As Data Science is a broad field, you will find multiple different roles with different responsibilities.

Data Science 97

Data Science BI Business Intelligence Data Mining 97

Knowledge Hut

JUNE 23, 2023

You should be well-versed with SQL Server, Oracle DB, MySQL, Excel, or any other data storing or processing software. You must develop predictive models to help industries and businesses make data-driven decisions. You can start as a software administrator, a database analyst, or a business intelligence analyst.

Data Engineering 52

Data Engineering Data Engineer Engineering Non-relational Database 52

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data.

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Rockset

FEBRUARY 2, 2022

Options for joining data in MongoDB include denormalization or use of the $lookup operator , but both are less flexible and powerful than a relational join. 2 – Use a Data Virtualization Tool The next approach is to use a data virtualization tool. 3 – Use a Data Warehouse Next, you can replicate your data to a data warehouse.

MongoDB 52

MongoDB NoSQL Data Warehouse BI 52

Silectis

JANUARY 3, 2021

If you’re new to data engineering or are a practitioner of a related field, such as data science, or business intelligence, we thought it might be helpful to have a handy list of commonly used terms available for you to get up to speed. Big Data Large volumes of structured or unstructured data.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

Grouparoo

OCTOBER 21, 2021

Data synchronization is crucial in today’s interconnected world, where data isn’t neatly contained in a single location anymore. The introduction of data warehouses has helped to centralize data into one place, but then there are critical questions of how do you sync data between the data warehouse and other locations.

Data Warehouse 52

Data Warehouse Data MySQL Cloud 52

AltexSoft

JUNE 26, 2023

Data collection is a methodical practice aimed at acquiring meaningful information to build a consistent and complete dataset for a specific business purpose — such as decision-making, answering research questions, or strategic planning. Structured data is modeled to be easily searchable and occupy minimal storage space.

Data Collection 52

Data Collection Machine Learning Unstructured Data Non-relational Database 52

Databand.ai

DECEMBER 13, 2022

We’d be remiss not to share that Joseph was a recent guest on Databand’s MAD Data Podcast , where he discussed ways to keep data systems from becoming unwieldy and shared tips for data teams to manage their data warehouses and keep data pipelines running reliably. You can also watch the video recording.

Data Engineering 52

Data Engineering Data Engineer Engineering AWS 52

ProjectPro

FEBRUARY 18, 2022

Why is ETL used in Data Science? It entails gathering data from numerous sources, converting it, and then storing it in a new single data warehouse. ETL pipelines help data scientists to prepare data for analytics and business intelligence. ETL stands for Extract, Transform, and Load.

Project 52

Project AWS Kafka Healthcare 52

Databand.ai

DECEMBER 20, 2022

Ken spreads this knowledge on his YouTube channel and on his Medium blog , where his content provides insight into the data science community, career advice, and sports related analysis. On LinkedIn, he posts regularly about data analytics and data science. You can also watch both episodes with Maxime (episodes #18 and #19).

Data Analytics 52

Data Analytics Google Cloud Data Science Data Mining 52

ProjectPro

AUGUST 24, 2021

Data Warehousing: Data warehousing utilizes and builds a warehouse for storing data. A data engineer interacts with this warehouse almost on an everyday basis. Data Analytics: A data engineer works with different teams who will leverage that data for business solutions.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52

Knowledge Hut

NOVEMBER 2, 2023

In this, there are options for SQL Server, Oracle, MariaDB, MySQL, PostgreSQL, and Amazon Aurora. If you want to use in-memory data caches for your real-time applications, you can try Amazon DynamoDB and ElastiCache. Amazon Redshift is a data warehouse that makes performing business intelligence tasks easy for data analysts.

Amazon Web Services 52

Amazon Web Services AWS IT Transportation 52

ProjectPro

DECEMBER 7, 2021

Generally, data pipelines are created to store data in a data warehouse or data lake or provide information directly to the machine learning model development. Keeping data in data warehouses or data lakes helps companies centralize the data for several data-driven initiatives.

Data Pipeline 52

Data Pipeline Architecture Kafka AWS 52

AltexSoft

OCTOBER 8, 2021

Data integration defines the process of collecting data from a number of disparate source systems and presenting it in a unified form within a centralized location like a data warehouse. So, why is data integration such a big deal? Connections to both data warehouses and data lakes are possible in any case.

Data Integration 52

Data Integration Hadoop Data Warehouse Data Lake 52

AltexSoft

OCTOBER 21, 2022

A publisher (say, telematics or Internet of Medical Things system) produces data units, also called events or messages , and directs them not to consumers but to a middleware platform — a broker. A subscriber is a receiving program such as an end-user app or business intelligence tool. You can find off-the-shelf links for.

Kafka 93

Kafka Hadoop ETL Tools Big Data 93

ProjectPro

JANUARY 31, 2023

Step 5: Data Validation This is the last step involved in the process of data preparation. In this step, automated procedures are used for the data to verify its accuracy, consistency, and completeness. The prepared data is then stored in a data warehouse or a similar repository. For example – MySQL.

Big Data 52

Big Data Hadoop AWS Relational Database 52

ProjectPro

JULY 27, 2021

Non-relational databases are ideal if you need flexibility for storing the data since you cannot create documents without having a fixed schema. E.g. PostgreSQL, MySQL, Oracle, Microsoft SQL Server. E.g. Redis, MongoDB, Cassandra, HBase , Neo4j, CouchDB What is data modeling? Data is regularly updated.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40