Christophe Blefari

SEPTEMBER 28, 2023

Over the years Cloudera logo has been replaced by Snowflake and Databricks ones. We jumped from HDFS to Cloud Storage (S3, GCS) for storage and from Hadoop, Spark to Cloud warehouses (Redshift, BigQuery, Snowflake) for processing. For downstream data quality there are also a lot of tools. Cloud-first.

Cloud Storage 147

Cloud Storage Big Data Hadoop SQL 147

Data Engineering Podcast

JUNE 4, 2023

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. What are some of the tools and architectures that an organization might be able to replace with Agile Data Engine? When is Agile Data Engine the wrong choice?

Data Engineering 130

Data Engineering Data Engineer Engineering Data Lake 130

Data Engineering Podcast

JANUARY 21, 2024

Recent years have seen the adoption of reverse ETL, cloud data warehouses, and sophisticated product analytics suites. What are the most interesting, unexpected, or challenging lessons that you have learned while working on CDPs? The Machine Learning Podcast helps you go from idea to production with machine learning.

Data Lake 147

Data Lake High Quality Data NoSQL Data Warehouse 147

Snowflake

FEBRUARY 28, 2024

Welcome to Snowflake’s Startup Spotlight, where we learn about awesome companies building businesses on Snowflake. Chabi is your all-in-one data stack with state-of-the-art, built-in data warehouse, ETL, data modeling and personalized analytics that are tailored to meet your unique data and BI needs.

BI 83

BI Data Warehouse Accessible Accessibility 83

Snowflake

DECEMBER 21, 2023

Welcome to Snowflake’s Startup Spotlight, where we feature awesome companies building businesses on Snowflake. At Optimizely, we computed our billing and statistics pipelines in Snowflake. These interfaces are designed to make using Snowflake data in production as easy as importing a code library.

Software Engineer 86

Software Engineer Software Engineering Database Data Pipeline 86

Snowflake

SEPTEMBER 7, 2023

As part of running these workloads, customers sometimes require the ability to reach external network locations on the public internet outside of Snowflake. And with the advancement of large language models (LLMs), customers may want to call an external LLM (such as Amazon Bedrock and Azure OpenAI) from Snowflake.

Amazon Web Services 114

Amazon Web Services AWS Government Python 114

Data Engineering Podcast

APRIL 2, 2023

The rapid growth and proliferation of data tools helped establish the "Modern Data Stack" as a de-facto architectural paradigm. What are the most interesting, unexpected, or challenging lessons that you have learned while working on the MAD landscape over the years? Email hosts@dataengineeringpodcast.com ) with your story.

Hadoop 130

Hadoop Machine Learning Python Architecture 130

Snowflake

OCTOBER 9, 2023

Easily collect and store digital events directly to create a complete composable customer data platform (CDP) Marketers are increasingly leveraging the Snowflake Data Cloud as the foundation for all of their customer data analytics and activation. The composable CDP is modular and unbundled.

Cloud 93

Cloud Building SQL Data 93

Snowflake

MAY 31, 2023

Welcome to Snowflake’s Startup Spotlight, where we highlight the people and companies building businesses on Snowflake. Things I’ve worked on include co-creation of Apache Arrow (used inside Snowflake, with over 50 million downloads/month), and I am Co-Founder and former CTO of Dremio. Tell us about yourself. Empowerment.

SQL 93

SQL Database Data Engineering Data Engineer 93

Snowflake

JANUARY 29, 2024

Imagine seamlessly integrating your Google Analytics data with Snowflake, allowing you to combine it effortlessly with other key sources like CRM, ERP, social media metrics, email campaign data, and whatever data sources compose the full scope of your data estate. Here’s a quick guide to get started: 1.

Raw Data 107

Raw Data Aggregated Data Data Government 107

Towards Data Science

NOVEMBER 4, 2023

Platform Specific Tools and Advanced Techniques Photo by Christopher Burns on Unsplash The modern data ecosystem keeps evolving and new data tools emerge now and then. Image by author It also might be a datalake in the center and it depends on the type of our data platform and tools we use.

Data Engineering 85

Data Engineering Data Engineer Engineering BI 85

Data Engineering Podcast

NOVEMBER 13, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Building 130

Building Metadata MongoDB MySQL 130

Data Engineering Podcast

JANUARY 7, 2022

Summary Reverse ETL is a product category that evolved from the landscape of customer data platforms with a number of companies offering their own implementation of it. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm.

ETL System 100

ETL System Data Pipeline Data Warehouse Architecture 100

Data Engineering Podcast

DECEMBER 17, 2023

Summary The "modern data stack" promised a scalable, composable data platform that gave everyone the flexibility to use the best tools for every job. The team at 5X understand the pain involved and the barriers to productivity and set out to solve it by pre-integrating the best tools from each layer of the stack.

Data Lake 130

Data Lake High Quality Data SQL Architecture 130

Snowflake

SEPTEMBER 21, 2023

With over 11,000 available solutions , an increase of 7,258% over the last 12 years, marketing organizations have never had more tool options to choose from. The data driving the provider’s application is stored and processed in the provider’s own Snowflake account. No ETL, FTP, or APIs are required.

Google Cloud 82

Google Cloud Unstructured Data Technology Data 82

Towards Data Science

FEBRUARY 9, 2024

The Platform To my surprise, there is still no single vendor solution that has domain over the entire data vista, although Snowflake has been trying their best through acquisition and development efforts (Snowpipe, Snowpark, Snowplow). Transformation Few tools have revolutionized data engineering like dbt.

Building 81

Building Transportation Data Lake Metadata 81

Data Engineering Podcast

DECEMBER 28, 2022

Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Output data can be streamed into a data lake for query engines like Presto, Trino or Spark SQL, a data warehouse like Snowflake or Redshift., Upsolver : ![Upsolver]([link]

Data Lake 130

Data Lake Data Warehouse Data Pipeline MongoDB 130

Data Engineering Podcast

DECEMBER 26, 2021

In this episode Zhamak re-joins the show to discuss the real world benefits that have been seen, the lessons that she has learned while working with her clients and the community, and her vision for the future of the data mesh. Go to dataengineeringpodcast.com/atlan today and sign up for a free trial. When is a data mesh the wrong approach?

BI 130

BI Data Warehouse Data Engineering Data Engineer 130

ThoughtSpot

JUNE 23, 2023

ThoughtSpot Sage supplements GPT’s algorithm with information on attribute columns, synonyms, indexed values, formulas, join paths, and analytical keywords—generating accurate, business-ready SQL functions that are executed on complex relational schemas housed in Snowflake.

Cloud 79

Cloud Programming Language Government Data Security 79

Data Engineering Podcast

OCTOBER 29, 2021

This has given rise to methods as varied as star and snowflake schemas, data vault modeling, and wide tables. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. When is Narrator the wrong choice?

Data Warehouse 100

Data Warehouse BI Data Workflow Data Pipeline 100

Data Engineering Podcast

OCTOBER 30, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Business Intelligence 130

Business Intelligence Metadata MongoDB MySQL 130

Data Engineering Podcast

DECEMBER 25, 2022

Summary Encryption and security are critical elements in data analytics and machine learning applications. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. or any other destination you choose.

Machine Learning 130

Machine Learning Systems Data Lake Data Warehouse 130

Data Engineering Podcast

DECEMBER 12, 2021

With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. What are the most interesting, unexpected, or challenging lessons that you have learned while working on data engineering at Capital One?

Building 130

Building Data Pipeline BI Data Warehouse 130

Snowflake

FEBRUARY 21, 2023

Welcome to Snowflake’s Startup Spotlight, where we learn about awesome companies building businesses on Snowflake. Propel is an API platform for developers to build web and mobile apps with their Snowflake data. It allows developers to build web and mobile applications with their Snowflake data.

AWS 117

AWS Raw Data Building Architecture 117

Data Engineering Podcast

JUNE 19, 2022

Their SDKs make event streaming from any app or website easy, and their state-of-the-art reverse ETL pipelines enable you to send enriched data to any cloud tool. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP.

Metadata 130

Metadata MongoDB Scala MySQL 130

Data Engineering Podcast

DECEMBER 21, 2021

With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. The perennial problem of embedding SQL into another host language or DSL is how to deal with validation and developer tooling. When is Cube the wrong choice?

Data Analytics 130

Data Analytics BI Computer Science SQL 130

Snowflake

FEBRUARY 7, 2023

First-generation tools have integrated product instrumentation libraries that collect and hold data in black box stores within their closed analytic platforms. At best, they can bring in a limited set of properties from an enterprise data warehouse using reverse ETL tools. Every decision could be justified with data.

BI 80

BI Building ETL Tools Data Warehouse 80

Data Engineering Podcast

OCTOBER 23, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Project 130

Project Metadata MongoDB MySQL 130

Data Engineering Podcast

JANUARY 1, 2022

With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. star/snowflake schema, data vault, etc.) What are the most interesting, unexpected, or challenging lessons that you have learned while working on dbtplyr?

Data Warehouse 100

Data Warehouse BI Data Workflow Coding 100

Data Engineering Podcast

MAY 29, 2022

In this episode they explain how their platform optimizes your Snowflake warehouses to reduce cost, as well as identifying improvements that you can make in your queries to reduce their contribution to your bill. What are the most interesting, unexpected, or challenging lessons that you have learned while working on Bluesky?

Cloud 100

Cloud Data Warehouse Data Lake Data 100

Monte Carlo

MARCH 4, 2024

And as crazy as it sounds at first blush, innovations in table formats got prime air-time at both Snowflake and Databricks annual conferences in 2023. At a broad feature level this translates to machine learning monitors that understand how these aspects of your data platform normally behave and alert you when anomalies occur.

Metadata 52

Metadata Data Lake Data Warehouse Data Engineering 52

Data Engineering Podcast

JULY 2, 2021

With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads? No more scripts, just SQL.

Data Integration 100

Data Integration SQL BI Data Warehouse 100

Data Engineering Podcast

NOVEMBER 27, 2021

With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. What are the most interesting, unexpected, or challenging lessons that you have learned while working on Mozart? When is Mozart the wrong choice?

BI 100

BI Data Data Warehouse Coding 100

Monte Carlo

JANUARY 23, 2024

One of the first data engineers at Facebook and Airbnb, he wrote and open sourced the wildly popular orchestrator, Apache Airflow , followed shortly thereafter by Apache Superset , a data exploration tool that’s taking the data viz landscape by storm. Enter, the data engineer. All too often, 5-10 minutes of work at 11:30 p.m.

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Christophe Blefari

JANUARY 20, 2024

Learn data engineering, all the references ( credits ) This is a special edition of the Data News. But right now I'm in holidays finishing a hiking week in Corsica 🥾 So I wrote this special edition about: how to learn data engineering in 2024. This is close to what we also call ETL or ELT. On a regular schedule.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Data Engineering Podcast

FEBRUARY 27, 2022

In this episode Tobias Macey, the host of the show, reflects on his plans for building a data platform and what he has learned from running the podcast that is influencing his choices. What other tools/systems will need to integrate with it? Go to dataengineeringpodcast.com/atlan today and sign up for a free trial.

Designing 100

Designing Metadata Data Lake Relational Database 100

Workfall

SEPTEMBER 18, 2023

In this digital era, businesses thrive on data, and making this data dance harmoniously with your analytics tools is crucial. Pair this with Snowflake , the cloud data warehouse that acts as a vault for your insights, and you have a recipe for data-driven success.

Data Pipeline 93

Data Pipeline Raw Data Data Schemas Healthcare 93