Data Engineering Podcast

AUGUST 27, 2021

Your host is Tobias Macey and today I’m interviewing Dan DeMers about Cinchy, a dataware platform aiming to simplify the work of data integration by eliminating ETL/ELT Interview Introduction How did you get involved in the area of data management? Can you describe what Cinchy is and the story behind it?

Data Integration 100

Data Integration Architecture Data Warehouse Data Lake 100

Knowledge Hut

NOVEMBER 2, 2023

Learn how to aggregate real-time data using several big data tools like Kafka, Zookeeper, Spark, HBase, and Hadoop. ETL and ELT Operations Study the Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) methods for data integration on AWS. Aptitude for learning new big data techniques and technologies.

Data Engineering 52

Data Engineering Data Engineer Project Coding 52

Data Engineering Podcast

NOVEMBER 6, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Building 130

Building IT Metadata MongoDB 130

Data Engineering Podcast

OCTOBER 30, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Business Intelligence 130

Business Intelligence Metadata MongoDB MySQL 130

KDnuggets

AUGUST 3, 2022

10 Most Used Tableau Functions • Is Domain Knowledge Important for Machine Learning? • ETL vs ELT: Data Integration Showdown • Free MLOps Crash Course for Beginners • 90% of Today’s Code is Written to Prevent Failure, and That’s a Problem.

Machine Learning 63

Machine Learning Data Integration Coding Data 63

Data Engineering Podcast

DECEMBER 8, 2019

In this episode Kent Graziano, chief technical evangelist for SnowflakeDB, explains how it is differentiated from other managed platforms and traditional data warehouse engines, the features that allow you to scale your usage dynamically, and how it allows for a shift in your workflow from ETL to ELT.

Data Warehouse 100

Data Warehouse Cloud AWS Relational Database 100

Meltano

JULY 19, 2022

Combined, these three stages are often referred to as ELT (extract, load, transform). In this article, you will learn about custom extractors in ELT pipelines and how you can implement a custom extractor for extracting data from a JSON placeholder API to a JSONL file using Meltano. What Are Custom Extractors?

Building 52

Building Python Database Project 52

Data Engineering Podcast

DECEMBER 4, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Visit dataengineeringpodcast.com/montecarlo to learn more. When is Zing the wrong choice?

Business Intelligence 100

Business Intelligence Metadata BI MongoDB 100

Data Engineering Podcast

OCTOBER 16, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Database 100

Database Metadata MongoDB MySQL 100

KDnuggets

NOVEMBER 17, 2021

Don’t Waste Time Building Your Data Science Network; 19 Data Science Project Ideas for Beginners; How I Redesigned over 100 ETL into ELT Data Pipelines; Anecdotes from 11 Role Models in Machine Learning; The Ultimate Guide To Different Word Embedding Techniques In NLP.

Data Science 64

Data Science Project Building Machine Learning 64

Data Engineering Podcast

OCTOBER 2, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Metadata 100

Metadata Electronics MongoDB MySQL 100

Data Engineering Podcast

OCTOBER 30, 2022

An ELT workflow with dbt as the transformation utility has become a popular pattern for building analytics systems. What are the most interesting, unexpected, or challenging lessons that you have learned while working on dbt/Optimus projects? The Machine Learning Podcast helps you go from idea to production with machine learning.

Engineering 100

Engineering MongoDB Scala MySQL 100

Data Engineering Podcast

SEPTEMBER 25, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Building 100

Building Metadata MongoDB MySQL 100

Data Engineering Podcast

SEPTEMBER 23, 2018

You work hard to make sure that your data is reliable and accurate, but can you say the same about the deployment of your machine learning models? The Skafos platform from Metis Machine was built to give your data scientists the end-to-end support that they need throughout the machine learning lifecycle.

Data Lake 100

Data Lake Data Warehouse Data Architecture Architecture 100

Monte Carlo

DECEMBER 29, 2022

Data Observability: How to Build Your Own Data Anomaly Detectors Using SQL The same author, Monte Carlo data scientist Ryan Kearns, dives into how to build machine learning driven data anomaly detectors using SQL. Building Spark Lineage For Data Lakes Everyone writes about the successful projects, but you learn more from the failures.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

Data Engineering Podcast

AUGUST 20, 2021

Your host is Tobias Macey and today I’m interviewing Vikrant Dubey about Cuebook and their Cuelake project for building ELT pipelines for your data lakehouse entirely in SQL Interview Introduction How did you get involved in the area of data management? Can you describe what Cuelake is and the story behind it?

Building 100

Building Data Lake Data Warehouse SQL 100

Data Engineering Podcast

OCTOBER 9, 2022

Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. The Machine Learning Podcast helps you go from idea to production with machine learning.

Metadata 100

Metadata AWS MongoDB MySQL 100

Pipeline Data Engineering

DECEMBER 14, 2021

The 2021 Pipies go to… Data Acquisition: Airbyte Self-definition: Open-Source Data Integration Pipelines | ELT. ETL/ELT: Prefect Self-definition: The New Standard in Dataflow Automation. The whale that rules the T in ELT that coined the term Analytics Engineering. A European player with McLaren expertise, supported by Project A.

Data Architect 52

Data Architect Python BI Kafka 52

KDnuggets

NOVEMBER 17, 2021

Don’t Waste Time Building Your Data Science Network; 19 Data Science Project Ideas for Beginners; How I Redesigned over 100 ETL into ELT Data Pipelines; Anecdotes from 11 Role Models in Machine Learning; The Ultimate Guide To Different Word Embedding Techniques In NLP.

Data Science 51

Data Science Project Building Machine Learning 51

AltexSoft

AUGUST 29, 2023

If your organization fits into one of these categories and you’re considering implementing advanced data management and analytics solutions, keep reading to learn how data lakes work and how they can benefit your business. This makes them ideal for more advanced analytics activities, including real-time analytics and machine learning.

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Rockset

OCTOBER 20, 2021

These solutions have worked well for the batch ELT world over the past decade, where data teams are used to dealing with data that is only occasionally refreshed and analytics queries that can take minutes or even hours to complete. You can learn more about configuring incremental models in dbt here.

SQL 52

SQL PostgreSQL MongoDB NoSQL 52

Christophe Blefari

MAY 19, 2023

The component I'm most proud of is the ELT framework we built on top of Airflow to give total autonomy to analysts and scientists on the data loading and transformation processes. At least I learn from my mistakes. During my time at Kapten, we built a data stack with Airflow, BigQuery and Metabase + Tableau. ❤️

Data 130

Data Data Engineering Data Engineer Hadoop 130

ProjectPro

JANUARY 24, 2023

They are PaaS-based ETL and ELT solutions. Learn more about Big Data Tools and Technologies with Innovative and Exciting Big Data Projects Examples. Explore ProjectPro Repository to work on hands-on data engineering projects that leverage both Azure Data Factory and AWS Glue to help you learn more about the differences between them.

AWS 52

AWS Cloud Amazon Web Services Scala 52

Data Engineering Podcast

JANUARY 20, 2020

And for your machine learning workloads, they just announced dedicated CPU instances. You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management. Closing Announcements Thank you for listening!

Kafka 100

Kafka Media Architecture Data Pipeline 100

Cloudera

JULY 13, 2021

Make sure to take the Airflow tour to learn more. A key aspect of ETL or ELT pipelines is automation. This will allow defining of custom DAGs and scheduling of jobs based on certain event triggers like an input file showing up in an S3 or ADLS bucket. This is what makes Airflow so powerful and flexible. Automation APIs.

Data Engineering 126

Data Engineering Data Engineer Engineering Pipeline-centric 126

Data Engineering Podcast

MAY 3, 2021

What are some of the most valuable lessons that you have learned from working with your customers about how to apply DataOps principles? What are some of the most interesting, unexpected, or challenging lessons that you have learned while building your respective platforms and businesses? data scientist or data analyst).

Data Warehouse 100

Data Warehouse Data Pipeline BI Metadata 100

Data Engineering Podcast

JULY 28, 2019

And for your machine learning workloads, they just announced dedicated CPU instances. Go to dataengineeringpodcast.com/conferences to learn more and take advantage of our partner discounts when you register. What resources do you recommend for someone who wants to learn more about this approach and start using it in their organization?

Data Integration 100

Data Integration Metadata Media Architecture 100

Data Engineering Podcast

SEPTEMBER 21, 2020

There are always new tools, systems, and design patterns to learn, which leads to a great deal of confusion for newcomers. Learn how we streamline and accelerate manual processes to help you derive real results from your data at dataengineeringpodcast.com/immuta. Summary Data engineering is a constantly growing and evolving discipline.

Data Engineering 100

Data Engineering Data Engineer Engineering AWS 100

Data Engineering Podcast

FEBRUARY 22, 2021

motivation for creating open source ELT as a business Can you describe how the Airbyte platform is implemented? What are the most interesting, unexpected, or challenging lessons that you have learned while building Airbyte? motivation for creating open source ELT as a business Can you describe how the Airbyte platform is implemented?

Data Integration 100

Data Integration Data Warehouse Data Pipeline BI 100

Data Engineering Podcast

MARCH 22, 2021

ELT – keep all source tables and use DBT for converting relevant tables into star/snowflake/data vault/wide tables) What are your thoughts on the viability of a data lake as the destination system? What are the most interesting, unexpected, or challenging lessons that you have learned from digging deep into CDC implementation?

Data Warehouse 100

Data Warehouse Metadata Data Lake Hadoop 100

Snowflake

MAY 17, 2023

Spresso solutions leverage machine learning, advanced analytics, and AI to deliver data-driven insights and automated prescriptive analytics that empower retailers to take meaningful action. The post Spresso Optimizes Billions of Impressions, Powered by Snowflake appeared first on Snowflake.

Retail 52

Retail Machine Learning Data Warehouse Architecture 52

Cloudera

JANUARY 6, 2021

Python is used extensively among Data Engineers and Data Scientists to solve all sorts of problems from ETL/ELT pipelines to building machine learning models. Introduction. Apache HBase is an effective data storage system for many workflows but accessing this data specifically through Python can be a struggle.

Machine Learning 132

Machine Learning Data Science Database Building 132

Knowledge Hut

NOVEMBER 28, 2023

Hence, learning and developing the required data engineer skills set will ensure a better future and can even land you better salaries in good companies anywhere in the world. Data Engineer certification will aid in scaling up you knowledge and learning of data engineering. Who are Data Engineers? What do Data Engineers Do?

Data Engineering 52

Data Engineering Data Engineer Engineering Generalist 52

Cloudera

JUNE 18, 2022

These lakes power mission critical large scale data analytics, business intelligence (BI), and machine learning use cases, including enterprise data warehouses. If you want to learn more, join us on June 21 on our webinar with Ryan Blue, co-creator of Apache Iceberg and Anjali Norwood, Big Data Compute Lead at Netflix.

Data Lake 89

Data Lake Data Warehouse BI SQL 89

Data Engineering Podcast

JULY 22, 2019

And for your machine learning workloads, they just announced dedicated CPU instances. Go to dataengineeringpodcast.com/conferences to learn more and take advantage of our partner discounts when you register. And for your machine learning workloads, they just announced dedicated CPU instances.

Data Lake 100

Data Lake Hadoop Data Architecture 100

RandomTrees

SEPTEMBER 6, 2020

Once the data is loaded, analysis can be done quickly using Machine Learning and predictive analysis to produce useful insights. This process also requires a dedicated resource for ELT development and operations. The need to learn a new language is eliminated as it can be managed with SQL alone.

Cloud Storage 52

Cloud Storage Data Ingestion Data Cleanse Data Warehouse 52

dbt Developer Hub

MARCH 9, 2022

But there’s one baton pass that’s still relatively painful and undefined: the handoff between machine learning (ML) engineers and analytics engineers. Whatever the interface evolves into, it must center people, create a low bar and high ceiling, and focus on outcomes and not the mystique of high learning curves.

Machine Learning 52

Machine Learning Raw Data SQL Python 52