Data Engineering Podcast

JANUARY 7, 2024

Summary Data processing technologies have dramatically improved in their sophistication and raw throughput. Unfortunately, the volumes of data that are being generated continue to double, requiring further advancements in the platform capabilities to keep up. Want to see Starburst in action?

Data Process 147

Data Process Process Data Lake High Quality Data 147

Precisely

SEPTEMBER 8, 2023

Across every industry, companies are striving to improve their capacity for making data-driven decisions. For data teams, that often leads to a burgeoning inbox of new projects, as business users throughout the organization strive to discover new insights and find new ways of creating value for the business.

Data Governance 52

Data Governance Government Data Big Data 52

Knowledge Hut

APRIL 20, 2023

Data governance refers to the set of policies, procedures, mix of people and standards that organisations put in place to manage their data assets. It involves establishing a framework for data management that ensures data quality, privacy, security, and compliance with regulatory requirements.

Data Governance 59

Data Governance Government Data Cleanse Data Security 59

Precisely

SEPTEMBER 5, 2023

Data is pivotal for the success of business operations. With cloud computing, the capacity to extract value from data is greater than ever. As this realization grows, businesses are shifting their investments from hardware to technologies that optimize data assets. Nevertheless, MDM is not a silver bullet for data quality.

Data Governance 52

Data Governance Government Data Management Management 52

Precisely

JANUARY 5, 2023

Does your company have a formal data strategy? If so, does that strategy effectively lay out a path toward better business outcomes by helping you optimize your use of data? Data governance plays a critical role in any effective data strategy. Data governance plays a critical role in any effective data strategy.

Data Governance 52

Data Governance Government Data Raw Data 52

Monte Carlo

JANUARY 24, 2024

Data is among your company’s most valuable commodities, but only if you know how to manage it. More data, more access to data, and more regulations mean data governance has become a higher-stakes game. At the same time, data governance technologies are growing more intelligent.

Data Governance 52

Data Governance Government Data Lake Data Architecture 52

Snowflake

SEPTEMBER 27, 2023

At Snowflake, we are dedicated to helping our customers effectively mobilize their data while upholding stringent standards for compliance and data governance. Today, we are thrilled to announce the general availability of the Data Governance Interface in Snowsight.

Data Governance 60

Data Governance Government SQL Database 60

Cloudera

MARCH 3, 2022

Whether the enterprise uses dozens or hundreds of data sources for multi-function analytics, all organizations can run into data governance issues. Bad data governance practices lead to data breaches, lawsuits, and regulatory fines — and no enterprise is immune. . Everyone Fails Data Governance.

Data Governance 100

Data Governance Government Metadata Medical 100

Monte Carlo

JULY 24, 2023

If there’s one thing that’s become abundantly clear in the world of tech, it’s that “data is the new oil.” And, much like oil, data can be a tricky substance – it’s incredibly valuable when refined and put to use, but it’s a flammable mess if not properly managed.

Data Governance 52

Data Governance Government Education Retail 52

The Modern Data Company

MARCH 7, 2023

Data governance can be a powerful agent in scaling the use and distribution of trusted data throughout the company. However, more often than not, it conjures up the idea of a central authority strictly guarding against such access. If you missed it, make sure to catch up on Part 1 – Data Timeliness.

Data Governance 52

Data Governance Government Metadata Datasets 52

Data Engineering Podcast

MARCH 9, 2020

Summary Data governance is a complex endeavor, but scaling it to meet the needs of a complex or globally distributed organization requires a well considered and coherent strategy. We have partnered with organizations such as O’Reilly Media, Corinium Global Intelligence, ODSC, and Data Council.

Data Governance 100

Data Governance Government Architecture Data 100

Monte Carlo

MAY 24, 2024

They rely on internal analytics to make data driven decisions and, as a company in a regulated industry, regularly report to government agencies. “We We take data quality very seriously,” said André Gonzalez, Data Manager, Eureciclo. “If If bad data was sent to a regulatory agency, that would create issues for us.”

Government 52

Government Data Governance Data Datasets 52

Data Engineering Podcast

MAY 26, 2024

When that system is responsible for the data layer the process becomes more challenging. Sriram Panyam has been involved in several projects that required migration of large volumes of data in high traffic environments. Can you start by sharing some of your experiences with data migration projects?

Systems 130

Systems Data Lake High Quality Data Google Cloud 130

Data Engineering Podcast

MAY 12, 2024

Summary Building a data platform is a substrantial engineering endeavor. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is supported by Code Comments, an original podcast from Red Hat. Data lakes are notoriously complex.

Management 147

Management Data Lake High Quality Data Machine Learning 147

Data Engineering Podcast

FEBRUARY 11, 2024

Summary Sharing data is a simple concept, but complicated to implement well. There are also numerous technical considerations to be made, particularly if the producer and consumer of the data aren't using the same platforms. Dagster offers a new approach to building and running data platforms and data pipelines.

Data Lake 147

Data Lake High Quality Data Government Data Pipeline 147

Databand.ai

JULY 19, 2023

Complete Guide to Data Ingestion: Types, Process, and Best Practices Helen Soloveichik July 19, 2023 What Is Data Ingestion? Data Ingestion is the process of obtaining, importing, and processing data for later use or storage in a database. In this article: Why Is Data Ingestion Important?

Data Ingestion 52

Data Ingestion Process Data Cleanse Data Governance 52

Databand.ai

JULY 6, 2023

Data Integrity Testing: Goals, Process, and Best Practices Niv Sluzki July 6, 2023 What Is Data Integrity Testing? Data integrity testing refers to the process of validating the accuracy, consistency, and reliability of data stored in databases, data warehouses, or other data storage systems.

Data Integration 52

Data Integration Process Data Validation Data Governance 52

Data Engineering Podcast

FEBRUARY 4, 2024

Summary Stream processing systems have long been built with a code-first design, adding SQL as a layer on top of the existing framework. RisingWave is a database engine that was created specifically for stream processing, with S3 as the storage layer. Want to see Starburst in action?

SQL 173

SQL Data Lake High Quality Data Data Pipeline 173

phData: Data Engineering

SEPTEMBER 19, 2023

With the amount of data companies are using growing to unprecedented levels, organizations are grappling with the challenge of efficiently managing and deriving insights from these vast volumes of structured and unstructured data. What is a Data Lake?

Data Lake 52

Data Lake Process Metadata Data Warehouse 52

Monte Carlo

JANUARY 17, 2023

With the rise of GDPR and other compliance measures, data security is under more scrutiny than ever before. Yet as organizations invest in more data and more data accessibility, keeping all that data safe and well-maintained has never been more challenging. Table of Contents What is data governance?

Data Governance 40

Data Governance Government Building Data Storage 40

Knowledge Hut

MARCH 7, 2024

The year 2024 saw some enthralling changes in volume and variety of data across businesses worldwide. The surge in data generation is only going to continue. Foresighted enterprises are the ones who will be able to leverage this data for maximum profitability through data processing and handling techniques.

Big Data 98

Big Data Bytes Data Governance Raw Data 98

Data Engineering Podcast

DECEMBER 31, 2023

Summary Working with financial data requires a high degree of rigor due to the numerous regulations and the risks involved in security breaches. In this episode Andrey Korchack, CTO of fintech startup Monite, discusses the complexities of designing and implementing a data platform in that sector. Want to see Starburst in action?

Designing 130

Designing Data Lake High Quality Data SQL 130

Data Engineering Podcast

NOVEMBER 26, 2023

Summary Building a data platform that is enjoyable and accessible for all of its end users is a substantial challenge. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. Data lakes are notoriously complex.

Architecture 147

Architecture Data Lake High Quality Data SQL 147

Precisely

APRIL 11, 2023

It’s a common scenario across industries: a new strategic initiative is launching, and you need trustworthy, relevant data to analyze and support the best outcomes for the business. Let’s explore some of the questions you must ask as you begin the process, then dive into the solutions for common challenges you may face.

Accessible 52

Accessible Accessibility Metadata Data Integration 52

Snowflake

JUNE 6, 2024

You can now use Snowflake Notebooks to simplify the process of connecting to your data and to amplify your data engineering, analytics and machine learning workflows. Faster, easier AI/ML and data engineering workflows Explore, analyze and visualize data using Python and SQL.

SQL 92

SQL Python Data Workflow Machine Learning 92

Data Engineering Podcast

MARCH 17, 2024

Summary A significant portion of data workflows involve storing and processing information in database engines. Validating that the information is stored and processed correctly can be complex and time-consuming, especially when the source and destination speak different dialects of SQL. Data lakes are notoriously complex.

Database 147

Database Data Lake High Quality Data Data Workflow 147

Snowflake

SEPTEMBER 11, 2023

Data privacy regulations are sweeping across the globe, with some 71% of countries worldwide adopting data protection and privacy legislation. The European Union’s General Data Protection Regulation (GDPR) , one of the more well-known and far-reaching of these privacy regulations, went into effect on May 25, 2018.

Cloud 77

Cloud Electronics Government Data Governance 77

Ascend.io

DECEMBER 1, 2022

Data migration is one of the most common undertakings for data teams. Yet, many businesses underestimate the process—resulting in extra time and money spent. A data migration process usually takes longer than it should and requires several teams. What Is Data Migration?

Process 52

Process Data ETL Tools Database 52

Snowflake

JUNE 4, 2024

Snowflake provides a strong data foundation anchored on unified data, optimal TCO and universal governance. The Snowflake platform eliminates silos to enable any architectural pattern, while supporting all data types and workloads.

Government 80

Government Data Ingestion Data PostgreSQL 80

Snowflake

JUNE 5, 2024

With the rapid adoption of generative AI (gen AI) and large language models (LLMs), organizations are racing to unlock as much business value as possible from their content across data, apps and models. Snowflake Horizon continues to innovate, allowing data teams to now collaborate in a governed way within their organization.

Government 97

Government Accessible Accessibility Metadata 97

Data Engineering Podcast

MARCH 10, 2024

Summary Data lakehouse architectures are gaining popularity due to the flexibility and cost effectiveness that they offer. The link that bridges the gap between data lake and warehouse capabilities is the catalog. Data lakes are notoriously complex. Join us at the top event for the global data community, Data Council Austin.

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

Data Engineering Podcast

APRIL 28, 2024

In this episode he explains the data collection and preparation process, the collection of model types and sizes that work together to power the experience, and how to incorporate it into your workflow to act as a second brain. Data lakes are notoriously complex. Your first 30 days are free! Want to see Starburst in action?

Building 147

Building Data Lake High Quality Data Machine Learning 147

Monte Carlo

JUNE 6, 2023

Sure, data quality is everyones’ problem. But who is responsible for data quality? Some organizations will attempt to diffuse the responsibility widely across data stewards, data owners, data engineering and governance committees, each owning a fraction of the data value chain.

Data Governance 98

Data Governance Government Data Data Engineering 98

Snowflake

FEBRUARY 15, 2024

At Snowflake, we believe in empowering our customers to harness the full potential of their data while maintaining robust compliance standards and safeguarding data privacy. You can then protect the data with a row access policy, masking policy or both. Ready to try Data Classification for yourself?

Data 98

Data Data Governance Government Data Analysis 98

Monte Carlo

MARCH 13, 2024

It’s rare to find a data leader, engineer, scientist, or analyst — or business stakeholder — who doesn’t believe data quality is important. When your company relies on data to power products or decision-making processes, everyone agrees that data needs to be accurate and trustworthy. And there’s a huge difference.

Data Governance 52

Data Governance Government Metadata Data 52

Snowflake

JULY 18, 2023

For many businesses, gathering compliance data means manually collecting PDFs and screenshots. That’s a slow and laborious process, but anecdotes AI streamlines compliance and eliminates redundant work with its advanced compliance data infrastructure. Discover how the company built its platform on Snowflake.

Data Process 52

Data Process Process Data Lake BI 52

Towards Data Science

JUNE 8, 2023

We examine which team structures are the best suited for efficiently improving data quality. Sure, data quality is everyones’ problem. Some organizations will attempt to diffuse the responsibility widely across data stewards, data owners, data engineering and governance committees, each owning a fraction of the data value chain.

Data Governance 73

Data Governance Government Generalist Data Engineering 73