Snowflake

APRIL 8, 2024

In 2020, Snowflake announced a new global competition to recognize the work of early-stage startups building their apps — and their businesses — on Snowflake, offering up to $250,000 in investment as the top prize. Our sincere thanks go out to everyone who participated in this year’s competition. SignalFlare.ai

Pipeline-centric 127

Pipeline-centric Food Healthcare Unstructured Data 127

Monte Carlo

AUGUST 7, 2023

Prevention: Finally, data observability also provides mechanisms to prevent data issues from happening in the first place, like placing circuit breaking in pipelines and creating visibility around the impact code changes would make on data, among other proactive measures of preventing bad data from entering your pipelines in the first place.

Coding 52

Coding Data Software Engineer Software Engineering 52

Data Engineering Podcast

DECEMBER 18, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. Not Endless DAGs!

Data Lake 100

Data Lake Data Warehouse Data Pipeline MongoDB 100

The Modern Data Company

DECEMBER 29, 2022

If they can streamline their data pipelines to provide actionable insights, they can usher in the next generation of financial services and survive the next global disruption. In addition, business users can leverage drag-and-drop engineering to build pipelines without extensive coding.

Banking 52

Banking Pipeline-centric Building Finance 52

Data Engineering Podcast

JULY 17, 2022

In this episode Crux CTO Mark Etherington discusses the different costs involved in managing external data, how to think about the total return on investment for your data, and how the Crux platform is architected to reduce the toil involved in managing third party data. Tired of deploying bad data?

Data Management 130

Data Management Management Metadata MongoDB 130

Data Engineering Podcast

NOVEMBER 20, 2022

In this episode CEO and founder Salma Bakouk shares her views on the causes and impacts of "data entropy" and how you can tame it before it leads to failures. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Lake 130

Data Lake Data Ingestion MongoDB Scala 130

Data Engineering Podcast

DECEMBER 4, 2022

In this episode Arjun Narayan explains how the technical barriers to adopting real-time data in your analytics and applications have become surmountable by organizations of all sizes. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Lake 100

Data Lake MongoDB MySQL Data Warehouse 100

Data Engineering Podcast

OCTOBER 30, 2022

In this episode he shares his experiences working with organizations to adopt analytics engineering patterns and the ways that Optimus and dbt were combined to let data analysts deliver insights without the roadblocks of complex pipeline management. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

DECEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Database 100

Database MySQL Data Lake MongoDB 100

Snowflake

DECEMBER 21, 2023

Welcome to Snowflake’s Startup Spotlight, where we feature awesome companies building businesses on Snowflake. Patch enables software engineers to build highly scalable and performant production apps using Snowflake data, without ETL or changes to your existing infrastructure, and with no boilerplate code in the application.

Software Engineer 90

Software Engineer Software Engineering Database Data Pipeline 90

Data Engineering Podcast

NOVEMBER 27, 2022

He also discusses the improvements that have been incorporated into FeatureBase to simplify integration with the rest of your data stack, and the SQL interface that was added to make working with the product easier. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Lake 100

Data Lake Data Warehouse MongoDB Data Pipeline 100

Data Engineering Podcast

NOVEMBER 6, 2022

In this episode she shares the story behind the project, the details of how it is implemented, and how you can use it for your own data projects. RudderStack helps you build a customer data platform on your warehouse or data lake. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

MongoDB 130

MongoDB Scala MySQL Data Lake 130

Data Engineering Podcast

JULY 10, 2022

Summary Building a data platform is a journey, not a destination. Beyond the work of assembling a set of technologies and building integrations across them, there is also the work of growing and organizing a team that can support and benefit from that platform. Tired of deploying bad data?

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Snowflake

NOVEMBER 1, 2023

That’s why we created Snowflake Cortex (in private preview), Snowflake’s new, intelligent, fully managed service that enables organizations to quickly analyze data and build AI applications — all within Snowflake. See demo here. Analysts can ask Snowflake Copilot a question, and it will write a SQL query using relevant tables.

Unstructured Data 106

Unstructured Data SQL Python Accessible 106

The Modern Data Company

JANUARY 22, 2024

Transforming Data Management and Gaining Recognition Today, enterprises are investing heavily in varied data products, often necessitating intricate integration and customization efforts, Modern has identified a crucial need for a humanized and flexible data platform.

Data Integration 98

Data Integration Metadata Government Unstructured Data 98

Data Engineering Podcast

JUNE 26, 2022

In this episode Isaac Brodsky explains how the Unfolded platform is architected, their experience joining the team at Foursquare, and how you can start using it for analyzing your spatial data today. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

The Modern Data Company

JANUARY 22, 2024

Transforming Data Management and Gaining Recognition Today, enterprises are investing heavily in varied data products, often necessitating intricate integration and customization efforts, Modern has identified a crucial need for a humanized and flexible data platform.

Data Integration 87

Data Integration Metadata Government Unstructured Data 87

Data Engineering Podcast

MAY 1, 2022

He describes how the platform is architected, the challenges related to selling cloud technologies into enterprise organizations, and how you can adopt Matillion for your own workflows to reduce the maintenance burden of data integration workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Warehouse 100

Data Warehouse Data Integration Cloud Google Cloud 100

Data Engineering Podcast

JULY 3, 2022

She also discusses her views on the role of the data lakehouse as a building block for these architectures and the ongoing influence that it will have as the technology matures. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Tired of deploying bad data?

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

APRIL 17, 2022

In this episode Prineha Narang, co-founder and CTO of Aliro, explains how these systems work, the capabilities that they can offer, and how you can start preparing for a post-quantum future for your data systems. Atlan is a collaborative workspace for data-driven teams, like Github for engineering or Figma for design teams.

Data Warehouse 100

Data Warehouse SQL Data Engineering Data Engineer 100

Meltano

JULY 19, 2022

Use Cases of Custom Extractors The use cases of custom extractors fall into one of the following three categories: Unconventional data sources: You may need to build a custom extractor to deal with custom databases, queues, files, internal APIs, etc. You can check out the complete custom extractor code on this GitHub repo.

Building 52

Building Python Database Project 52

Data Engineering Podcast

NOVEMBER 13, 2022

Summary CreditKarma builds data products that help consumers take advantage of their credit and financial capabilities. To make that possible they need a reliable data platform that empowers all of the organization’s stakeholders. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

MongoDB 100

MongoDB Scala Google Cloud MySQL 100

Data Engineering Podcast

APRIL 24, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. Missing data?

Data Warehouse 100

Data Warehouse Data Lake Data Pipeline BI 100

Data Engineering Podcast

JUNE 19, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Monte Carlo

JUNE 6, 2023

But is there more to generative AI than a fancy demo on Twitter? And how will it impact data? How generative AI will disrupt data With the advent of OpenAI’s DALL-E and ChatGPT, large language models became much more useful to the vast majority of humans. Let’s assess.

Data Engineering 84

Data Engineering Data Engineer Engineering Business Intelligence 84

Data Engineering Podcast

APRIL 3, 2022

In this episode Satish Jayanthi explains how he is building a framework to allow enterprises to move quickly while maintaining guardrails for data workflows. This allows everyone in the business to participate in data analysis in a sustainable manner. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.

Data Warehouse 100

Data Warehouse Data Workflow Data Architecture SQL 100

Monte Carlo

NOVEMBER 9, 2023

And in many ways, LLMs are going to make data engineers more valuable – and that’s exciting! Still, it’s one thing to show your boss a cool demo of a data discovery tool or text-to-SQL generator – it’s another thing to use it with your company’s proprietary data, or even more concerning, customer data.

Unstructured Data 86

Unstructured Data Database Data Pipeline Architecture 86

Data Engineering Podcast

MARCH 27, 2022

As these systems are scaled to larger volumes of data and higher throughputs the RocksDB engine can become a bottleneck for performance. In this episode Adi Gelvan shares the work that he and his team at SpeeDB have put into building a drop-in replacement for RocksDB that eliminates that bottleneck.

Relational Database 100

Relational Database Database SQL Data Warehouse 100

Data Engineering Podcast

APRIL 10, 2022

In this episode CEO and founder Itamar Ben hemo explains how his experiences in the industry led to his vision for the Rivery platform as a single place to build end-to-end analytical workflows, including how it is architected and how you can start using it today for your own work.

Data Integration 100

Data Integration SQL Data Engineering Data Engineer 100

Data Engineering Podcast

DECEMBER 21, 2020

I’m working with O’Reilly on a project to collect the 97 things that every data engineer should know, and I need your help. When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Data Governance 130

Data Governance Government Data Lake Banking 130

Data Engineering Podcast

JUNE 5, 2022

Summary The best way to make sure that you don’t leak sensitive data is to never have it in the first place. The team at Skyflow decided that the second best way is to build a storage system dedicated to securely managing your sensitive information and making it easy to integrate with your applications and data systems.

Data Security 100

Data Security Metadata MongoDB Scala 100

Snowflake

JUNE 27, 2023

Containers have emerged as the modern approach to package code in any language to ensure portability and consistency across environments, especially for sophisticated AI/ML models and full-stack data-intensive apps. These types of modern data products frequently deal with massive amounts of proprietary data.

Scala 61

Scala Programming Language Data Science Python 61

Grouparoo

MAY 11, 2021

In practice, one does not simply connect Sheets and Mailchimp: There's a disconnect here; it's not so point-and-shoot to get data flowing across this pipeline. Let's build him a pipeline to get data from Google Sheets to flow automatically into Mailchimp. Why don't we help Dylan out a bit?

Data Pipeline 52

Data Pipeline Data Project Utilities 52

Data Engineering Podcast

JUNE 12, 2022

Summary Building a well rounded and effective data team is an iterative process, and the first hire can set the stage for future success or failure. Trupti Natu has been the first data hire multiple times and gone through the process of building teams across the different stages of growth.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

MAY 18, 2021

Summary Data lineage is the common thread that ties together all of your data pipelines, workflows, and systems. In order to get a holistic understanding of your data quality, where errors are occurring, or how a report was constructed you need to track the lineage of the data from beginning to end.

Metadata 100

Metadata Kafka Data Warehouse Hadoop 100

Monte Carlo

MAY 26, 2023

A better measurement is the data downtime formula (above), which more comprehensively measures the amount of time the data was inaccurate, missing, or otherwise erroneous. A proactive approach for measuring data freshness is to create service level agreements or SLAs for specific data pipelines.

Data Pipeline 98

Data Pipeline Data Data Warehouse Machine Learning 98

Monte Carlo

JUNE 27, 2023

Notably, the data and application announcements are expanded functionality to build critical mass around some of last year’s major announcements , but Snowpark Container Services sticks out right away as exciting new capabilities that will have an immediate impact on the data world.

Data Lake 59

Data Lake Government SQL Unstructured Data 59