Data Engineering Podcast

NOVEMBER 4, 2021

Summary The precursor to widespread adoption of cloud data warehouses was the creation of customer data platforms. Acting as a centralized repository of information about how your customers interact with your organization they drove a wave of analytics about how to improve products based on actual usage data.

Data Warehouse 100

Data Warehouse Business Intelligence ETL System Data Lake 100

Data Engineering Podcast

SEPTEMBER 11, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode.

Systems 100

Systems Metadata Building MongoDB 100

Data Engineering Podcast

DECEMBER 19, 2021

In this episode Pardhu Gunnam and Mars Lan explain how they have designed the architecture and user experience to allow everyone to collaborate on the data lifecycle and provide opportunities for automation and extensible workflows. Start trusting your data with Monte Carlo today! No more scripts, just SQL.

Systems 100

Systems Building Metadata Data Warehouse 100

Data Engineering Podcast

MAY 8, 2022

In this episode Jon Herke shares how TigerGraph customers are taking advantage of those capabilities to achieve meaningful discoveries in their fields, the utilities that it provides for modeling and managing your connected data, and some of his own experiences working with the platform before joining the company.

Database 100

Database Data Lake BI Business Intelligence 100

Data Engineering Podcast

DECEMBER 11, 2021

In this episode he explains how it can be used to build rich and useful profiles of your users, the system architecture that powers it, and some of the ways that it is being integrated into an organization’s broader data ecosystem. Start trusting your data with Monte Carlo today! No more scripts, just SQL.

Data Warehouse 100

Data Warehouse Raw Data Data Lake BI 100

Data Engineering Podcast

DECEMBER 4, 2021

The whole process of hiring is an important organizational skill to cultivate and this is an interesting exploration of the specific challenges involved in finding data professionals. Start trusting your data with Monte Carlo today! Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Data Warehouse 100

Data Warehouse Data Lake BI Business Intelligence 100

Data Engineering Podcast

JANUARY 1, 2022

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. No more scripts, just SQL.

Data Warehouse 100

Data Warehouse Data Lake SQL Hadoop 100

Data Engineering Podcast

OCTOBER 28, 2021

He also explains why he started Decodable to address that limitation and the work that he and his team have done to let data engineers build streaming pipelines entirely in SQL. Start trusting your data with Monte Carlo today! Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads?

Data Pipeline 100

Data Pipeline SQL Data Warehouse Data Lake 100

Data Engineering Podcast

NOVEMBER 10, 2021

Summary A significant source of friction and wasted effort in building and integrating data management systems is the fragmentation of metadata across various tools. Start trusting your data with Monte Carlo today! Are you bored with writing scripts to move data into SaaS tools like Salesforce, Marketo, or Facebook Ads?

Metadata 100

Metadata Data Warehouse Data Lake BI 100

Data Engineering Podcast

NOVEMBER 20, 2021

In this episode he shares the journey that he and his team at Elementl have taken to understand the state of the ecosystem and how they can provide a foundational layer for a holistic data platform. Start trusting your data with Monte Carlo today! The data you’re looking for is already in your data warehouse and BI tools.

Data Warehouse 100

Data Warehouse Data Lake BI Business Intelligence 100

Data Engineering Podcast

OCTOBER 9, 2022

Summary The core of any data platform is the centralized storage and processing layer. For many that is a data warehouse, but in order to support a diverse and constantly changing set of uses and technologies the data lakehouse is a paradigm that offers a useful balance of scale and cost, with performance and ease of use.

Metadata 100

Metadata AWS MongoDB MySQL 100

Data Engineering Podcast

DECEMBER 21, 2020

If you are starting down the path of implementing a data governance strategy then this episode will provide a great overview of what is involved. If you hand a book to a new data engineer, what wisdom would you add to it? What is data governance? If you hand a book to a new data engineer, what wisdom would you add to it?

Data Governance 130

Data Governance Government Data Lake Banking 130

Data Engineering Podcast

MAY 19, 2019

Summary In recent years the traditional approach to building data warehouses has shifted from transforming records before loading, to transforming them afterwards. The data build tool (dbt) is designed to bring battle tested engineering practices to your analytics pipelines.

Data Analytics 100

Data Analytics Building Engineering Data Warehouse 100

Knowledge Hut

DECEMBER 29, 2023

A Data Engineer in the Data Science team is responsible for this sort of data manipulation. Big Data is a part of this umbrella term, which encompasses Data Warehousing and Business Intelligence as well. A Data Engineer's primary responsibility is the construction and upkeep of a data warehouse.

Data Science 98

Data Science Data Mining Deep Learning Programming Language 98

Data Science Blog: Data Engineering

SEPTEMBER 24, 2021

Data Warehousing is applied Big Data Management and a key success factor in almost every company. Without a data warehouse, no company today can control its processes and make the right decisions on a strategic level as there would be a lack of data transparency for all decision makers.

Data Lake 52

Data Lake Data Warehouse Data Transparency Database 52

Monte Carlo

JUNE 28, 2022

To help organizations realize the full potential of their data lake and lakehouse investments, Monte Carlo, the data observability leader, is proud to announce integrations with Delta Lake and Databricks’ Unity Catalog for full data observability coverage. billion in 2020 to 17.60 billion in 2020 to 17.60

Data Lake 52

Data Lake Metadata AWS Data Warehouse 52

Data Engineering Podcast

JANUARY 4, 2021

Summary As more organizations are gaining experience with data management and incorporating analytics into their decision making, their next move is to adopt machine learning. Monte Carlo’s end-to-end Data Observability Platform monitors and alerts for data issues across your data warehouses, data lakes, ETL, and business intelligence.

Python 100

Python Data Lake Machine Learning Computer Science 100

Data Engineering Podcast

JANUARY 11, 2021

This is a fascinating conversation on the technical challenges involved, the opportunities that such as system provides, and the complexities inherent to building a successful business on open source. There are a number of different reasons and methods for versioning data, such as the work being done with Datomic, LakeFS, DVC, etc.

PostgreSQL 100

PostgreSQL Python Data Lake Computer Science 100

Monte Carlo

AUGUST 31, 2023

Data is a priority for your CEO, as it often is for digital-first companies, and she is fluent in the latest and greatest business intelligence tools. What about a frantic email from your CTO about “duplicate data” in a business intelligence dashboard?

Data Lake 52

Data Lake Data Pipeline Unstructured Data Data Warehouse 52

Silectis

JANUARY 3, 2021

If you’re new to data engineering or are a practitioner of a related field, such as data science, or business intelligence, we thought it might be helpful to have a handy list of commonly used terms available for you to get up to speed. Big Data Large volumes of structured or unstructured data.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

Knowledge Hut

JUNE 30, 2023

Data engineers are experts who specialize in the design and execution of data systems and infrastructure. They have unique skills, including competence in software engineering, data management, and analytics. Investigate alternative data storage solutions, such as databases and data lakes.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Data Engineering Podcast

APRIL 16, 2023

In this episode Paul Blankley and Ryan Janssen explore the power of natural language driven data exploration combined with semantic modeling that enables an intuitive way for everyone in the business to access the data that they need to succeed in their work. Business intelligence is a crowded market.

Business Intelligence 147

Business Intelligence Building Data Lake BI 147

Knowledge Hut

DECEMBER 28, 2023

Today, organizations seek skilled professionals who can harness data’s power to drive informed decisions. As technology evolves, cloud platforms have emerged as the cornerstone of modern data management. Its comprehensive suite of services can handle data at scale. Among these, Microsoft Azure stands out as a leader.

Certification 52

Certification Data Engineering Data Engineer Engineering 52

Snowflake

NOVEMBER 2, 2023

Over the years, the technology landscape for data management has given rise to various architecture patterns, each thoughtfully designed to cater to specific use cases and requirements. Customers that require a hybrid of these to support many different tools and languages have built a data lakehouse.

Data Lake 85

Data Lake Data Warehouse Cloud Unstructured Data 85

Monte Carlo

AUGUST 25, 2023

Different vendors offering data warehouses, data lakes, and now data lakehouses all offer their own distinct advantages and disadvantages for data teams to consider. So let’s get to the bottom of the big question: what kind of data storage layer will provide the strongest foundation for your data platform?

Data Lake 52

Data Lake Data Warehouse Unstructured Data Raw Data 52

ProjectPro

DECEMBER 7, 2021

Generally, data pipelines are created to store data in a data warehouse or data lake or provide information directly to the machine learning model development. Keeping data in data warehouses or data lakes helps companies centralize the data for several data-driven initiatives.

Data Pipeline 52

Data Pipeline Architecture Kafka AWS 52

Rockset

JANUARY 3, 2023

In the future, we expect to see a shift from companies using data pipelines to manage their data streaming needs to allowing this data to serve as a central nervous system so more people can derive smarter insights from it.”

Data Warehouse 52

Data Warehouse Education Database Data 52

AltexSoft

FEBRUARY 11, 2023

It serves as a foundation for the entire data management strategy and consists of multiple components including data pipelines; , on-premises and cloud storage facilities – data lakes , data warehouses , data hubs ;, data streaming and Big Data analytics solutions ( Hadoop , Spark , Kafka , etc.);

Data Architect 87

Data Architect Certification Generalist Big Data 87

U-Next

SEPTEMBER 7, 2022

The terms “ Data Warehouse ” and “ Data Lake ” may have confused you, and you have some questions. Essentially, this is the difference between a lake and a warehouse. On the other hand, a data warehouse contains historical data that has been cleaned and arranged. .

Data Lake 52

Data Lake Data Warehouse Unstructured Data Amazon Web Services 52

ProjectPro

OCTOBER 6, 2021

The benefits it offers start from data management and manipulation to machine learning tools on the GCP platform. GCP offers 90 services that span computation, storage, databases, networking, operations, development, data analytics , machine learning , and artificial intelligence , to name a few.

Google Cloud 52

Google Cloud Project Data Lake Healthcare 52

Data Engineering Podcast

MAY 1, 2022

He describes how the platform is architected, the challenges related to selling cloud technologies into enterprise organizations, and how you can adopt Matillion for your own workflows to reduce the maintenance burden of data integration workflows. No more shipping and praying, you can now know exactly what will change in your database!

Data Warehouse 100

Data Warehouse Data Integration Cloud Google Cloud 100

ProjectPro

SEPTEMBER 26, 2021

Big Data Engineer Salary by Experience (Entry-Level, Mid-Level, and Senior) Entry-Level Big Data Engineer Salary An entry-level position does not demand years of experience in Big Data technology. However, one should have an educational background and theoretical knowledge in data management.

Big Data 52

Big Data Data Engineering Data Engineer Engineering 52

Data Engineering Podcast

DECEMBER 29, 2022

In this episode she shares the strategic and tactical elements of how to make more effective use of the technical and organizational resources that are available to you for getting work done with data. Trusted by the data teams at Fox, JetBlue, and PagerDuty, Monte Carlo solves the costly problem of broken data pipelines.

Management 130

Management Metadata Business Intelligence Data Lake 130

Data Engineering Podcast

JANUARY 8, 2023

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data and analytics leaders, 2023 is your year to sharpen your leadership skills, refine your strategies and lead with purpose. Missing data? Can you describe what the SQLake product is and the story behind it?

PostgreSQL 130

PostgreSQL Data Lake Data Warehouse BI 130

Data Engineering Podcast

FEBRUARY 5, 2023

Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Sign up now for early access to Materialize and get started with the power of streaming data with the same simplicity and low implementation cost as batch cloud data warehouses.

Data Engineering 147

Data Engineering Data Engineer Engineering PostgreSQL 147

Advancing Analytics: Data Engineering

JULY 2, 2019

It’s not always the most accurate indicator, but a quick glance at google trends sees Data Engineer rocketing in popularity, compared to more traditional functions such as BI and ETL Developer: google trends Now, that’s not saying that the other roles are going away, not by a long stretch. or visit www.advancinganalytics.co.uk

Business Intelligence 52

Business Intelligence Data Engineering Data Engineer Engineering 52

Data Engineering Podcast

DECEMBER 18, 2022

In this episode Abe Gong brings his experiences with the Great Expectations project and community to discuss the technical and organizational considerations involved in implementing these constraints to your data workflows. Can you describe what your conception of a data contract is? Closing Announcements Thank you for listening!

Metadata 130

Metadata Business Intelligence Data Lake BI 130