Data Engineering Podcast

MAY 7, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. What are some of the categories of attributes that need to be managed in a prototypical customer profile?

Pipeline-centric 130

Pipeline-centric Data Lake Machine Learning Data Warehouse 130

Data Engineering Podcast

AUGUST 13, 2022

Summary Data engineers have typically left the process of data labeling to data scientists or other roles because of its nature as a manual and process heavy undertaking, focusing instead on building automation and repeatable systems. Data stacks are becoming more and more complex. Sifflet also offers a 2-week free trial.

Machine Learning 100

Machine Learning Pipeline-centric Database-centric MongoDB 100

Knowledge Hut

DECEMBER 26, 2023

Data Engineering is typically a software engineering role that focuses deeply on data – namely, data workflows, data pipelines, and the ETL (Extract, Transform, Load) process. What is Data Science? What are the roles and responsibilities of a Data Engineer? What is the need for Data Science?

Data Engineering 98

Data Engineering Data Engineer Engineering Pipeline-centric 98

Towards Data Science

MAY 19, 2023

Data Entropy — More Data, More Problems? How to navigate and embrace complexity in a modern data organisation. Business users are unable to find and access data assets critical to their workflows. Data engineers spend countless hours troubleshooting broken pipelines. The list goes on.

Pipeline-centric 103

Pipeline-centric Data Software Engineer Software Engineering 103

Monte Carlo

MARCH 28, 2024

In recent years, we’ve seen all sorts of new job titles emerge that would have been inscrutable just a decade or two ago – cloud architect, data reliability engineer , data product manager , director of hybrid working, and yes, DataOps engineer. So what exactly IS a DataOps engineer? What does a DataOps engineer do? It depends!

Pipeline-centric 75

Pipeline-centric Engineering BI Google Cloud 75

Databand.ai

JULY 19, 2023

Data Lineage Tools: Key Capabilities and 5 Notable Solutions Ryan Yackel July 19, 2023 What Are Data Lineage Tools? Data lineage tools provide a visual representation of your data’s journey across multiple systems and transformations. Data lineage tools are not a new concept.

Pipeline-centric 52

Pipeline-centric Data Governance Metadata Government 52

Cloudera

DECEMBER 13, 2023

Implementing a cohesive data vision LGIM’s Global Chief Operating Office is responsible for the company’s technology, data, client servicing and the management of its strategic operating agenda. LGIM’s data ecosystem had become fragmented over time as it had grown.

Pipeline-centric 89

Pipeline-centric Professional Services Datasets BI 89

Knowledge Hut

NOVEMBER 2, 2023

For those aspiring to build a career within the Azure ecosystem, navigating the choices between Azure Data Engineers and Azure DevOps Engineers can be quite challenging. Azure Data Engineers and Azure DevOps Engineers are two critical components of the Azure ecosystem for different but interconnected reasons.

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Ascend.io

OCTOBER 20, 2023

Snowflake has swiftly risen to prominence in the data world, and its consumption-based pricing model is well-known among data professionals. The Nuances of Snowflake Costing Snowflake’s pricing strategy is an exemplification of its user-centric approach: pay for what you use. So Why Do Snowflake Costs Become Prohibitive?

Pipeline-centric 52

Pipeline-centric IT Data Pipeline Bytes 52

Knowledge Hut

JUNE 20, 2023

A novice data scientist prepared to start a rewarding journey may need clarification on the differences between a data scientist and a machine learning engineer. Many people are learning data science for the first time and need help comprehending the two job positions. Facial reorganization, social media optimization, etc.

Machine Learning 52

Machine Learning Data Engineering Data Engineer Engineering 52

Knowledge Hut

NOVEMBER 17, 2023

The demand for data-related professions, including data engineering, has indeed been on the rise due to the increasing importance of data-driven decision-making in various industries. Becoming an Azure Data Engineer in this data-centric landscape is a promising career choice. Who is an Azure Data Engineer?

Data Engineering 52

Data Engineering Data Engineer Engineering Scala 52

Data Engineering Weekly

APRIL 16, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make collecting data from every application, website, and SaaS platform easy, then activating it in your warehouse and business tools. 🍒 We'll send exclusive The Data Stack Show swag just for participating!

Data Engineering 52

Data Engineering Data Engineer Engineering Pipeline-centric 52

Monte Carlo

NOVEMBER 14, 2023

Imagine, if you will, a world where data just… flows. This is the world that data orchestration tools aim to create. Data orchestration tools minimize manual intervention by automating the movement of data within data pipelines. So, what makes a good data orchestration tool? No hiccups.

Pipeline-centric 40

Pipeline-centric Google Cloud Data Workflow Python 40

Ascend.io

JUNE 22, 2023

As we chart the course into the future, it’s evident that the way organizations handle data is undergoing a seismic shift. A new model, known as data mesh, is set to disrupt the traditional centralized data ownership model and chart a transformative path towards decentralization and treating data as a product.

Technology 52

Technology Pipeline-centric Consulting Data 52

Knowledge Hut

FEBRUARY 27, 2023

A data engineer is a key member of an enterprise data analytics team and is responsible for handling, leading, optimizing, evaluating, and monitoring the acquisition, storage, and distribution of data across the enterprise. Data Engineers indulge in the whole data process, from data management to analysis.

Data Engineering 52

Data Engineering Data Engineer Database-centric Pipeline-centric 52

The Modern Data Company

DECEMBER 29, 2022

And much of this involves finally harnessing data and new technologies to the fullest potential. Many of Deloitte’s Predictions Will Require Access to Real-time Data Analysis The report lists several areas where consumer demand will shift banking products. The report mentions several key areas ripe for a data revolution.

Banking 52

Banking Pipeline-centric Building Finance 52

Knowledge Hut

JANUARY 30, 2024

In today's world, where data rules the roost, data extraction is the key to unlocking its hidden treasures. As someone deeply immersed in the world of data science, I know that raw data is the lifeblood of innovation, decision-making, and business progress. What is data extraction?

ETL Tools 52

ETL Tools Database-centric Data Mining Data Cleanse 52

Ascend.io

NOVEMBER 14, 2023

Visual representation of Conway’s Law ( source ) The Parallels Between the AI Revolution and Digital Transformation A dozen years ago, a similar wave of data and compute technologies coalesced to challenge business as usual for many companies. The CAIO is the vanguard of data privacy and security for the new AI-based capabilities.

Pipeline-centric 52

Pipeline-centric Government Data Pipeline Recruitment 52

Knowledge Hut

SEPTEMBER 26, 2023

Organisations are constantly looking for robust and effective platforms to manage and derive value from their data in the constantly changing landscape of data analytics and processing. These platforms provide strong capabilities for data processing, storage, and analytics, enabling companies to fully use their data assets.

Data Lake 52

Data Lake Database-centric Pipeline-centric Machine Learning 52

ProjectPro

DECEMBER 16, 2021

Are you a newbie in the data science domain ready to embark on a rewarding journey but are confused between the roles of a Machine Learning Engineer vs Data Scientist? Data Science is an emerging discipline and so are the roles and job titles pretty much evolving. Consider an AI/ML system as the combination of "Data" and "Code."

Machine Learning 52

Machine Learning Engineering Pipeline-centric Database-centric 52

RandomTrees

NOVEMBER 29, 2023

Here the practice of data warehousing and warehouse system is very important and the use of right modelling techniques has become a very important factor in todays’ competitive world. In this choice, Big Data will play an important role and its choice is also inevitably crucial in the Business Intelligence and related systems.

Data Warehouse 52

Data Warehouse Data Lake Database-centric Data Cleanse 52

AltexSoft

JULY 18, 2023

These seemingly unrelated terms unite within the sphere of big data, representing a processing engine that is both enduring and powerfully effective — Apache Spark. Maintained by the Apache Software Foundation, Apache Spark is an open-source, unified engine designed for large-scale data analytics. What is Apache Spark?

Big Data 52

Big Data Data Process Process Hadoop 52

phData: Data Engineering

JULY 18, 2023

The Ultimate Modern Data Stack Migration Guide phData Marketing July 18, 2023 This guide was co-written by a team of data experts, including Dakota Kelley, Ahmad Aburia, Sam Hall, and Sunny Yan. Imagine a world where all of your data is organized, easily accessible, and routinely leveraged to drive impactful outcomes.

Data Warehouse 52

Data Warehouse Pipeline-centric Government Data 52