article thumbnail

ETL?—?High Quality Data Pipelines

Medium Data Engineering

As you are probably aware, data pipeline is a rather broad term. It’s a collection of tasks that transfer, alter, or provide data. A… Continue reading on Medium »

article thumbnail

Low Code And High Quality Data Engineering For The Whole Organization With Prophecy

Data Engineering Podcast

Summary There is a wealth of tools and systems available for processing data, but the user experience of integrating them and building workflows is still lacking. Raj Bains founded Prophecy to address this need by creating a UI first platform for building and executing data engineering workflows that orchestrates Airflow and Spark.


Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Data Quality Dimensions: Assuring Your Data Quality with Great Expectations


This article highlights the significance of ensuring high-quality data and presents six key dimensions for measuring it. These dimensions include Completeness, Consistency, Integrity, Timelessness, Uniqueness, and Validity.

Data 100
article thumbnail

Data Observability for Analytics and ML teams

Medium Data Engineering

Principles, practices, and examples for ensuring high quality data flows Continue reading on Towards Data Science »

Data 98
article thumbnail

Improving Public Sector Decision Making With Simple, Automated Record Linking


What is data linking and why does it matter? Availability of more, high quality data is a critical enabler for better decision making.

article thumbnail

Want to streamline your data workflows and improve collaboration across teams?

Medium Data Engineering

DataOps combines the best practices of Agile, DevOps, and Lean to manage data-driven workflows and deliver high-quality data products and… Continue reading on Medium »

article thumbnail

AI / ML Survival Guide: Conquer DataOps and Data Composability Challenges and Transform into a Truly Data-Driven Organization

The Modern Data Company

Get to the Future Faster – Modernize Your Manufacturing Data Architecture Without Ripping and Replacing Implementing customer lifetime value as a mission-critical KPI has many challenges. Companies need consistent, high-quality data and a straightforward way to measure CLV.

Data 88