Ascend.io

MARCH 17, 2023

These engineering functions are almost exclusively concerned with data pipelines, spanning ingestion, transformation, orchestration, and observation — all the way to data product delivery to the business tools and downstream applications. Pipelines need to grow faster than the cost to run them.

Data Pipeline 52

Data Pipeline Datasets Data Software Engineer 52

Monte Carlo

JANUARY 16, 2024

Factor in the advertising strategies, media production, partner programming, audience analytics…and you’re looking at an ocean of data that would fill even the deepest trench (we’d like a television show about that too, please!). So how does Fox’s data strategy support these complex data workflows? Image from Castor.

Government 52

Government High Quality Data Data Governance Entertainment 52

Data Engineering Podcast

MAY 20, 2021

Summary Data governance is a phrase that means many different things to many different people. This is because it is actually a concept that encompasses the entire lifecycle of data, across all of the people in an organization who interact with it. RudderStack’s smart customer data pipeline is warehouse-first.

Data Governance 100

Data Governance Government Data Warehouse Data Pipeline 100

Ascend.io

JULY 28, 2023

Data pipelines are integral to business operations, regardless of whether they are meticulously built in-house or assembled using various tools. As companies become more data-driven, the scope and complexity of data pipelines inevitably expand. Ready to fortify your data management practice?

Data Pipeline 52

Data Pipeline Architecture Lambda Architecture Data Architecture 52

Knowledge Hut

APRIL 20, 2023

Data governance refers to the set of policies, procedures, mix of people and standards that organisations put in place to manage their data assets. It involves establishing a framework for data management that ensures data quality, privacy, security, and compliance with regulatory requirements.

Data Governance 59

Data Governance Government Data Cleanse Data Security 59

Cloudera

DECEMBER 20, 2021

But balancing a strong layer of security and governance with easy access to data for all users is no easy task. Another option — a more rewarding one — is to include centralized data management, security, and governance into data projects from the start. …and congratulations to the winner: Bank of the West.

Government 89

Government Data Security Banking Metadata 89

Confluent

MARCH 19, 2024

Check out the latest features on Confluent Cloud including improvements to 80+ connectors, stream governance updates, enterprise cluster savings, and much more.

Data Pipeline 64

Data Pipeline Building Government Cloud 64

Monte Carlo

JUNE 14, 2023

In this post, we will help you quickly level up your overall knowledge of data pipeline architecture by reviewing: Table of Contents What is data pipeline architecture? Why is data pipeline architecture important? What is data pipeline architecture? Why is data pipeline architecture important?

Data Pipeline 52

Data Pipeline Architecture Data Lake Data Warehouse 52

Cloudera

FEBRUARY 8, 2021

Below is the entire set of steps in the data lifecycle, and each step in the lifecycle will be supported by a dedicated blog post(see Fig. 1): Data Collection – data ingestion and monitoring at the edge (whether the edge be industrial sensors or people in a vehicle showroom). 1 The enterprise data lifecycle.

Data Pipeline 116

Data Pipeline Building Manufacturing Data Warehouse 116

Data Engineering Podcast

APRIL 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Cloudera

MARCH 9, 2021

Governance and the sustainable handling of data is a critical success factor in virtually all organizations. While Cloudera Data Platform (CDP) already supports the entire data lifecycle from ‘Edge to AI’, we at Cloudera are fully aware that enterprises have more systems outside of CDP. Assets: Files. HIVE Table.

Data Governance 95

Data Governance Government Metadata Datasets 95

Snowflake

JULY 6, 2023

At Goldman Sachs, the Legend data platform and Snowflake Native Apps Framework are not just helping teams understand all that data, but also transform it, govern it, share it, and model it—improving timely, data-driven insights and collaborative decision making. How do you make sense out of all of it?

Government 52

Government Data Engineering Data Engineer Database 52

Snowflake

AUGUST 24, 2023

Early on, the team was pulling data into PySpark and then pushing the data back into Snowflake, but experienced challenges with pipeline speed and quality. ” The post homegenius Improves Speed and Quality of Data Pipelines with Snowpark for Python appeared first on Snowflake.

Data Pipeline 52

Data Pipeline Python Programming Language Data Validation 52

Data Engineering Podcast

FEBRUARY 18, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 262

Data Lake High Quality Data Data Warehouse Google Cloud 262

Data Engineering Podcast

APRIL 28, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

MAY 5, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Cloudera

SEPTEMBER 17, 2020

To tackle these challenges, we’re thrilled to announce CDP Data Engineering (DE) , the only cloud-native service purpose-built for enterprise data engineering teams. Native Apache Airflow and robust APIs for orchestrating and automating job scheduling and delivering complex data pipelines anywhere.

Data Pipeline 103

Data Pipeline Data Engineering Data Engineer Engineering 103

Data Engineering Podcast

MARCH 24, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 162

Data Lake High Quality Data Hadoop Data Pipeline 162

Data Engineering Podcast

MAY 12, 2024

[Starburst Logo]([link] This episode is brought to you by Starburst - an end-to-end data lakehouse platform for data engineers who are battling to build and scale high quality data pipelines on the data lake. Want to see Starburst in action? Want to see Starburst in action?

Management 147

Management Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

FEBRUARY 11, 2024

Dagster offers a new approach to building and running data platforms and data pipelines. What are the technical capabilities that need to be present for an effective data sharing solution? How does that change as a function of the type of data? How does that change as a function of the type of data?

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Data Engineering Podcast

APRIL 7, 2024

Dagster offers a new approach to building and running data platforms and data pipelines. Starburst Logo]([link] This episode is brought to you by Starburst - a data lake analytics platform for data engineers who are battling to build and scale high quality data pipelines on the data lake.

Data Lake 162

Data Lake High Quality Data BI Data Workflow 162

Data Engineering Podcast

APRIL 14, 2024

Dagster offers a new approach to building and running data platforms and data pipelines. Starburst Logo]([link] This episode is brought to you by Starburst - a data lake analytics platform for data engineers who are battling to build and scale high quality data pipelines on the data lake.

Non-relational Database 147

Non-relational Database Relational Database Database Designing 147

Data Engineering Podcast

FEBRUARY 4, 2024

Dagster offers a new approach to building and running data platforms and data pipelines. Starburst Logo]([link] This episode is brought to you by Starburst - a data lake analytics platform for data engineers who are battling to build and scale high quality data pipelines on the data lake.

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

Databand.ai

NOVEMBER 30, 2022

The Ravit Show Q&A: How More Data Observability Leads to Better Governance Ryan Yackel 2022-11-30 10:18:32 We recently had the opportunity to join an episode of The Ravit Show , a community for data science and AI professionals to upskill, grow, share, and learn from each other.

Government 52

Government Data Governance Architecture Google Cloud 52

Hevo

FEBRUARY 2, 2024

Organizations of any size who face the risk of cyber attacks or want to enhance their data governance practices can benefit […] Therefore, securing it from cyber attacks, malware, and other factors becomes crucial. That is where vulnerability assessment comes into play.

Data Pipeline 40

Data Pipeline Data Governance Government Data 40

Data Engineering Podcast

MARCH 3, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Programming 162

Programming Data Lake High Quality Data Data Pipeline 162

Data Engineering Podcast

MARCH 31, 2024

Dagster offers a new approach to building and running data platforms and data pipelines. Starburst Logo]([link] This episode is brought to you by Starburst - a data lake analytics platform for data engineers who are battling to build and scale high quality data pipelines on the data lake.

Project 130

Project Data Lake High Quality Data Data Workflow 130

Data Engineering Podcast

FEBRUARY 25, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Database 162

Database Technology Data Lake High Quality Data 162

Data Engineering Podcast

MAY 26, 2024

[Starburst Logo]([link] This episode is brought to you by Starburst - an end-to-end data lakehouse platform for data engineers who are battling to build and scale high quality data pipelines on the data lake. Want to see Starburst in action? Want to see Starburst in action?

Systems 130

Systems Data Lake High Quality Data Google Cloud 130

Data Engineering Podcast

MARCH 17, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Database 147

Database Data Lake High Quality Data Data Workflow 147

RudderStack

MAY 12, 2021

Reverse ETL is nothing but just another data pipeline that carries data from one end to another, & Single Pipe Simplifies Your Stack, Security & Data governance

Data Pipeline 40

Data Pipeline Data Governance Government Data 40

Data Engineering Podcast

MARCH 10, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

Christophe Blefari

MARCH 15, 2024

Assisted AI wars are around the corner — I'm only following the French news, but the government is proudly doubling its budget for "AI defense" From what I know, AI is mainly used as an information companion to find signals in the huge amount of data we generate, creating more efficient agents.

Metadata 272

Metadata Datasets Data Data Warehouse 272

Cloudera

MAY 10, 2024

Leveraging Cloudera’s platform, powered by Iceberg, organizations can transform their data and analytics capabilities with open data lakehouses, making the most of data from across the entire enterprise with unique tools and no unnecessary data movement or transformations along the way.

Government 101

Government Data Governance Data Pipeline Big Data 101

Data Engineering Podcast

JANUARY 21, 2024

How have the architectural shifts changed the ways that organizations interact with their customer data? What are the governance policy and enforcement challenges that are added with the expansion of access and responsibility? How have the architectural shifts changed the ways that organizations interact with their customer data?

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Data Engineering Podcast

DECEMBER 24, 2023

[Starburst Logo]([link] This episode is brought to you by Starburst - a data lake analytics platform for data engineers who are battling to build and scale high quality data pipelines on the data lake.

Kafka 245

Kafka Data Lake High Quality Data SQL 245

The Modern Data Company

JANUARY 22, 2024

While traditional approaches involve piecing together disparate data products, DataOS® offers a composable framework that seamlessly integrates with existing technology stacks. Data engineering excellence Modern offers robust solutions for building, managing, and operationalizing data pipelines.

Data Integration 98

Data Integration Metadata Government Unstructured Data 98