Data Engineering Podcast

JANUARY 23, 2022

In this episode he shares his career journey, the challenges related to management of data professionals, and the platform design that he and his team have built to power analytics at a large company. He also provides some excellent insights into the factors that play into the build vs. buy decision at different organizational sizes.

Building 100

Building Management Data Pipeline Metadata 100

Data Engineering Podcast

NOVEMBER 13, 2022

Summary Building data products is an undertaking that has historically required substantial investments of time and talent. With the rise in cloud platforms and self-serve data technologies the barrier of entry is dropping. Shane Gibson co-founded AgileData to make analytics accessible to companies of all sizes.

Building 130

Building Metadata MongoDB MySQL 130

Data Engineering Podcast

JULY 27, 2021

Datastax has recently introduced a new managed offering for Pulsar workloads in the form of Astra Streaming that lowers those barriers and make stremaing workloads accessible to a wider audience. RudderStack’s smart customer data pipeline is warehouse-first. Go to dataengineeringpodcast.com/97things today to get your copy!

Building 100

Building Management Kafka Data Warehouse 100

ProjectPro

FEBRUARY 16, 2023

With so much riding on the efficiency of ETL processes for data engineering teams, it is essential to take a deep dive into the complex world of ETL on AWS to take your data management to the next level. This is particularly useful for companies that need to process data in near-real-time.

AWS 52

AWS Data Management ETL Tools Management 52

Data Engineering Podcast

SEPTEMBER 3, 2018

Using master data management to build a data catalog helps you answer these questions reliably and simplify the process of building your business intelligence reports. Can you start by establishing a definition of data mastering that we can work from?

Data Management 100

Data Management Management Relational Database Business Intelligence 100

Cloudera

FEBRUARY 4, 2021

Financial services institutions need the ability to analyze and act on massive volumes of data from diverse sources in order to monitor, model, and manage risk across the enterprise. They need a comprehensive data and analytics platform to model risk exposures on-demand. End-to-end Data Lifecycle.

Technology 90

Technology Data Management Management Insurance 90

Data Engineering Podcast

MARCH 27, 2022

The growing prominence of cloud and hybrid environments in data management adds additional stress to an already complex endeavor. Privacera is an enterprise grade solution for cloud and hybrid data governance built on top of the robust and battle tested Apache Ranger project. What do you have planned for the future of Privacera?

Data Governance 130

Data Governance Government Cloud Building 130

Data Engineering Podcast

AUGUST 20, 2021

Despite improvements in performance and data architecture they still require significant knowledge and experience to deploy and manage. In this episode Vikrant Dubey discusses his work on the Cuelake project which allows data analysts to build a lakehouse with SQL queries. Closing Announcements Thank you for listening!

Building 100

Building Data Lake Data Warehouse SQL 100

The Modern Data Company

APRIL 15, 2024

With this in mind, let’s explore how to demystify the process of building your data-driven strategy, making it accessible and actionable. We’ll uncover how you can transform data into a strategic asset that propels your organization forward without getting lost in the complexity of its creation.

Building 52

Building Raw Data Food Data 52

Data Engineering Podcast

DECEMBER 7, 2020

Summary Building data products are complicated by the fact that there are so many different stakeholders with competing goals and priorities. Different organizations have tried a multitude of organizational strategies to improve the success rate of these data teams with varying levels of success.

Building 100

Building Big Data Consulting Data Engineering 100

Data Engineering Podcast

FEBRUARY 13, 2022

When that data pertains to your customers it can become even more complex. In this episode CEO and founder Soumyadeb Mitra explains how Rudderstack compares to the various other tools and platforms that share some overlap, how to set it up for your own data needs, and how it is architected to scale to meet demand.

Building 100

Building Hadoop Data Pipeline Metadata 100

Databand.ai

AUGUST 30, 2023

Building a Winning Data Quality Strategy: Step by Step Eitan Chazbani August 30, 2023 What Is a Data Quality Strategy? A data quality strategy details the processes, tools, and techniques employed to ensure your company’s data is accurate, consistent, complete, and up-to-date.

Building 52

Building Data Cleanse Data Governance Datasets 52

Data Engineering Podcast

NOVEMBER 9, 2020

Summary A data catalog is a critical piece of infrastructure for any organization who wants to build analytics products, whether internal or external. While there are a number of platforms available for building that catalog, many of them are either difficult to deploy and integrate, or expensive to use at scale.

Building 100

Building PostgreSQL BI Metadata 100

Data Engineering Podcast

AUGUST 24, 2020

Summary In order to scale the use of data across an organization there are a number of challenges related to discovery, governance, and integration that need to be solved. The key to those solutions is a robust and flexible metadata management system. If you hand a book to a new data engineer, what wisdom would you add to it?

Metadata 100

Metadata Management Kafka Data Engineering 100

Data Engineering Podcast

FEBRUARY 3, 2020

If you are curious about data sharing strategies or data collaboratives, then listen now to learn more! You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management.

Building 100

Building Media AWS Data 100

Towards Data Science

AUGUST 2, 2023

A Step-by-Step Guide to Building an Effective Data Quality Strategy from Scratch How to build an interpretable data quality framework based on user expectations Photo by Rémi Müller on Unsplash As data engineers, we are (or should be) responsible for the quality of the data we provide.

Building 98

Building Data Consolidation Data Datasets 98

Snowflake

APRIL 8, 2024

In 2020, Snowflake announced a new global competition to recognize the work of early-stage startups building their apps — and their businesses — on Snowflake, offering up to $250,000 in investment as the top prize.

Pipeline-centric 131

Pipeline-centric Food Healthcare Unstructured Data 131

Data Engineering Podcast

FEBRUARY 8, 2021

Summary With all of the tools and services available for building a data platform it can be difficult to separate the signal from the noise. In this episode Zeeshan Qureshi and Michelle Ark share their experiences using DBT to manage the data warehouse for Shopify. What kinds of data sources are you working with?

Data Warehouse 100

Data Warehouse Building BI SQL 100

Data Engineering Podcast

JULY 27, 2020

In this episode he shares his approach to testing complex systems, the common challenges that are faced by engineers who build them, and why it is important to understand their limitations. With their open-source foundation, fixed pricing, and unlimited volume, they are enterprise ready, but accessible to everyone.

Systems 100

Systems Building Scala Java 100

Data Engineering Podcast

APRIL 19, 2020

Summary Modern applications frequently require access to real-time data, but building and maintaining the systems that make that possible is a complex and time consuming endeavor. Eventador is a managed platform designed to let you focus on using the data that you collect, without worrying about how to make it reliable.

Building 100

Building PostgreSQL MongoDB SQL 100

Data Engineering Podcast

AUGUST 26, 2019

Summary Data engineers are responsible for building tools and platforms to power the workflows of other members of the business. Each group of users has their own set of requirements for the way that they access and interact with those platforms depending on the insights they are trying to gather.

Building 100

Building Data Analytics Media Data Engineering 100

Data Engineering Podcast

DECEMBER 14, 2020

In this episode Andrew Gross, Bobby Muldoon, and Anup Segu describe the self service data platform that they have built to allow data analysts to own the end-to-end delivery of data projects and how that has allowed them to scale their output. If you hand a book to a new data engineer, what wisdom would you add to it?

Data Analytics 100

Data Analytics Building MySQL Data 100

Robinhood

MAY 30, 2024

Robinhood Crypto customers in the United States can now use our API to view crypto market data, manage portfolios and account information, and place crypto orders programmatically Today, we are excited to announce the Robinhood Crypto trading API , ushering in a new era of convenience, efficiency, and strategy for our most seasoned crypto traders.

Insurance 132

Insurance Portfolio Algorithm Coding 132

Data Engineering Podcast

MAY 19, 2019

Summary In recent years the traditional approach to building data warehouses has shifted from transforming records before loading, to transforming them afterwards. The data build tool (dbt) is designed to bring battle tested engineering practices to your analytics pipelines. When is it the wrong choice?

Data Analytics 100

Data Analytics Building Engineering Data Warehouse 100

Snowflake

FEBRUARY 14, 2024

How do we build trust in AI? I’ve spent over 20 years helping large corporations gain significant market footholds by optimizing data, analytics and AI – and have seen firsthand how a holistic data strategy is foundational to avoiding wasted resource investment and achieving success in a new competitive landscape.

Architecture 113

Architecture Government Data Management Management 113

Data Engineering Podcast

SEPTEMBER 2, 2019

Summary Data professionals are working in a domain that is rapidly evolving. In order to stay current we need access to deeply technical presentations that aren’t burdened by extraneous marketing. What was your original reason for focusing your efforts on fostering a community of data engineers?

Building 100

Building Data Database Design Media 100

Data Engineering Podcast

MARCH 18, 2023

He also explains why data security is distinct from application security and some methods for reducing the challenge of working across different data systems. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Join in with the event for the global data community, Data Council Austin.

Data Security 130

Data Security Machine Learning Data Science Metadata 130

Data Engineering Podcast

FEBRUARY 19, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Hey there podcast listener, are you tired of dealing with the headache that is the 'Modern Data Stack'? It's supposed to make building smarter, faster, and more flexible data infrastructures a breeze.

IT 147

IT Data Lake Metadata Data Warehouse 147

Knowledge Hut

JUNE 20, 2023

However, to succeed in e-commerce, businesses require certain skills and tools to build and run an effective online storefront. Hence, it is recommended that you build your career with professional courses that help you acquire the necessary skills to thrive in the world of e-commerce.

Building 52

Building Media Data Analysis Certification 52

Cloudera

MAY 30, 2024

In this context, Cloudera and TAI Solutions have partnered to help financial services customers accelerate their data-driven transformation, improve customer centricity, ensure compliance with regulations, enhance risk management, and drive innovation.

Data Analytics 83

Data Analytics Banking Insurance Finance 83

The Modern Data Company

DECEMBER 29, 2022

And much of this involves finally harnessing data and new technologies to the fullest potential. Many of Deloitte’s Predictions Will Require Access to Real-time Data Analysis The report lists several areas where consumer demand will shift banking products. A robust understanding of potential new customers.

Banking 52

Banking Pipeline-centric Building Finance 52

Snowflake

APRIL 29, 2024

The second batch of DORA policy products, which aims to strengthen oversight and risk management for third-party ICT providers, is currently under development. Identification of potential threats through data analysis and risk assessment processes to shift toward more proactive risk management. Five Core Pillars of DORA 1.

Transportation 105

Transportation Data Governance Government Consulting 105

Towards Data Science

FEBRUARY 6, 2024

This generalisation makes their data models complex and cryptic and require domain expertise. Even harder to manage, a common setup within large organisations is to have several instances of these systems with some underlaying processes in charge of transmitting data among them, which could lead to duplications, inconsistencies, and opacity.

Systems 78

Systems Raw Data Metadata Data Cleanse 78

U-Next

JUNE 27, 2022

The product manager job description isn’t enough to get to know the job in detail. You should also enlighten yourself on the functions of a product manager, roles and responsibilities. Introduction Product Manager Roles And Responsibilities. What product management is? What a product manager does?

Management 52

Management Finance Designing Engineering 52

The Modern Data Company

FEBRUARY 2, 2024

Ever wondered why building data-driven applications feels like an uphill battle? It’s not just you – turning raw data into something meaningful can be a real challenge. In today’s fast-paced business environment, data-driven insights are the lifeblood of staying ahead.

Raw Data 75

Raw Data Utilities Government Data 75

Knowledge Hut

DECEMBER 26, 2023

A growing amount of information is becoming digital and accessible through wireless and wired digital communication networks in addition to the pervasive internet. Although the Cybersecurity course duration may vary, employees get an opportunity to build expertise in the subject and reduce cyberattack possibilities.

Hospitality 98

Hospitality Finance Electronics Retail 98

The Modern Data Company

FEBRUARY 15, 2023

Maintaining a clear strategy for gathering, storing, and labeling data is challenging. Limited resources: Data management has always been resource intensive, but not all organizations can maintain a full data team. Without suitable resources for company-wide data management, it’s easier to fall behind.

Government 98

Government Data Governance Raw Data Data 98