Data Engineering Podcast

NOVEMBER 27, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Struggling with broken pipelines? Missing data? Stale dashboards? If this resonates with you, you’re not alone.

Data Process 100

Data Process Process Metadata Business Intelligence 100

Data Engineering Podcast

NOVEMBER 20, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. The biggest challenge with modern data systems is understanding what data you have, where it is located, and who is using it.

Systems 130

Systems Metadata Data Pipeline MongoDB 130

Data Engineering Podcast

DECEMBER 25, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan's active metadata capabilities. Struggling with broken pipelines? Missing data? Again, be prepared to have metadata challenges especially.

Building 100

Building Metadata Business Intelligence Data Lake 100

Towards Data Science

NOVEMBER 4, 2023

I’d like to discuss some popular Data engineering questions: Modern data engineering (DE). Does your DE work well enough to fuel advanced data pipelines and Business intelligence (BI)? Are your data pipelines efficient? Data warehouse exmaple. What is it?

Data Engineering 82

Data Engineering Data Engineer Engineering BI 82

Data Engineering Podcast

JUNE 19, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 130

Metadata Unstructured Data MongoDB MySQL 130

Data Engineering Podcast

JULY 17, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Data Management 130

Data Management Management Metadata MongoDB 130

Data Engineering Podcast

JULY 3, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Architecture 100

Architecture Metadata MongoDB MySQL 100

Data Engineering Podcast

DECEMBER 19, 2021

Summary Building a well managed data ecosystem for your organization requires a holistic view of all of the producers, consumers, and processors of information. The team at Metaphor are building a fully connected metadata layer to provide both technical and social intelligence about your data. No more scripts, just SQL.

Systems 100

Systems Building Metadata Data Warehouse 100

Data Engineering Podcast

JUNE 26, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Datasets 130

Datasets Unstructured Data Metadata MongoDB 130

Data Engineering Podcast

JUNE 12, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking all of that information into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Go to dataengineeringpodcast.com/atlan today to learn more about how you can take advantage of active metadata and escape the chaos.

Metadata 100

Metadata Unstructured Data Business Intelligence MongoDB 100

Data Engineering Podcast

JULY 10, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code.

Metadata 100

Metadata MongoDB MySQL Data Pipeline 100

Cloudera

SEPTEMBER 9, 2018

We are proud to announce the general availability of Cloudera Altus Data Warehouse , the only cloud data warehousing service that brings the warehouse to the data. Modern data warehousing for the cloud. Modern data warehousing for the cloud. Using Cloudera Altus for your cloud data warehouse.

Data Warehouse 40

Data Warehouse Metadata Cloud Storage Cloud 40

Data Engineering Podcast

OCTOBER 9, 2022

Summary The core of any data platform is the centralized storage and processing layer. For many that is a data warehouse, but in order to support a diverse and constantly changing set of uses and technologies the data lakehouse is a paradigm that offers a useful balance of scale and cost, with performance and ease of use.

Metadata 100

Metadata AWS MongoDB MySQL 100

Data Engineering Podcast

JULY 31, 2022

Summary Data lineage is the roadmap for your data platform, providing visibility into all of the dependencies for any report, machine learning model, or data warehouse table that you are working with. Atlan is the metadata hub for your data ecosystem. Can you describe what Manta is and the story behind it?

IT 100

IT Metadata MongoDB MySQL 100

Data Engineering Podcast

JUNE 5, 2022

Atlan is the metadata hub for your data ecosystem. Instead of locking all of that information into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Go to dataengineeringpodcast.com/atlan today to learn more about how you can take advantage of active metadata and escape the chaos.

Data Security 100

Data Security Metadata MongoDB Scala 100

Ascend.io

AUGUST 31, 2023

Still, these methods have been overshadowed by EtLT — the predominant approach reshaping today’s data landscape. In this article, we assess: The role of the data warehouse on one hand, and the data lake on the other; The features of ETL and ELT in these two architectures; The evolution to EtLT; The emerging role of data pipelines.

Data Lake 52

Data Lake ETL Tools Data Warehouse Data Pipeline 52

Data Engineering Podcast

JANUARY 15, 2022

Summary Applications of data have grown well beyond the venerable business intelligence dashboards that organizations have relied on for decades. Today’s episode is Sponsored by Prophecy.io – the low-code data engineering platform for the cloud.

Engineering 100

Engineering Electronics ETL Tools Data Pipeline 100

Cloudera

AUGUST 30, 2022

Data Lakehouse: Data lakehouses integrate and unify the capabilities of data warehouses and data lakes, aiming to support artificial intelligence, business intelligence, machine learning, and data engineering use cases on a single platform. Towards Data Science ). Forrester ).

Data Architecture 78

Data Architecture Architecture Data Lake NoSQL 78

AltexSoft

NOVEMBER 10, 2021

The pun being obvious, there’s more to that than just a new term: Data lakehouses combine the best features of both data lakes and data warehouses and this post will explain this all. What is a data lakehouse? Data warehouse vs data lake vs data lakehouse: What’s the difference.

Architecture 52

Architecture Data Lake Data Warehouse Metadata 52

Data Engineering Podcast

NOVEMBER 20, 2021

In the same way that application performance monitoring ensures reliable software and keeps application downtime at bay, Monte Carlo solves the costly problem of broken data pipelines. Start trusting your data with Monte Carlo today! Hightouch is the easiest way to sync data into the platforms that your business teams rely on.

Data Warehouse 100

Data Warehouse Data Lake BI Business Intelligence 100

Monte Carlo

JANUARY 23, 2024

The speed of ETL and analytics has increased Maxime recalls a time, not too long ago, when data engineering would require running Hive jobs for hours at a time, frequent context switching between jobs and managing different elements of your data pipeline. The data warehouse is the mirror of the organization in many ways.

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

AltexSoft

AUGUST 29, 2023

In 2010, a transformative concept took root in the realm of data storage and analytics — a data lake. The term was coined by James Dixon , Back-End Java, Data, and Business Intelligence Engineer, and it started a new era in how organizations could store, manage, and analyze their data.

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Silectis

JANUARY 3, 2021

If you’re new to data engineering or are a practitioner of a related field, such as data science, or business intelligence, we thought it might be helpful to have a handy list of commonly used terms available for you to get up to speed. Big Data Large volumes of structured or unstructured data.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

Monte Carlo

AUGUST 19, 2023

We’ll cover: What is a data platform? With companies moving their data platforms to the cloud, the emergence of cloud-native solutions ( data warehouse vs data lake or even a data lakehouse ) have taken over the market, offering more accessible and affordable options for storing data relative to many on-premises solutions.

Building 52

Building BI Data Lake Data Governance 52

AltexSoft

OCTOBER 30, 2021

They’re integral specialists in data science projects and cooperate with data scientists by backing up their algorithms with solid data pipelines. Juxtaposing data scientist vs engineer tasks. One data scientist usually needs two or three data engineers. Managing data and metadata.

Data Engineering 98

Data Engineering Data Engineer Engineering Machine Learning 98

Monte Carlo

JUNE 28, 2022

Traditionally, data lakes held raw data in its native format and were known for their flexibility, speed, and open source ecosystem. By design, data was less structured with limited metadata and no ACID properties. Since then, Databricks has aggressively moved toward allowing users to add more structure to their data.

Data Lake 52

Data Lake Metadata AWS Data Warehouse 52

Monte Carlo

MARCH 3, 2021

After all, any solution dedicated to keeping data pipelines reliable should take its own medicine when it comes to building the actual product. Instead, we only extract query logs, metadata, and aggregated statistics about data usage to ensure that your most critical data assets are as trustworthy and reliable as possible.

Metadata 52

Metadata Data Lake Business Intelligence Certification 52

DataKitchen

AUGUST 3, 2021

The past decades of enterprise data platform architectures can be summarized in 69 words. First-generation – expensive, proprietary enterprise data warehouse and business intelligence platforms maintained by a specialized team drowning in technical debt. The product includes data and operations.

Pharmaceutical 98

Pharmaceutical Data Lake Data Architecture Scala 98

Cloudera

JUNE 16, 2022

CSP was recently recognized as a leader in the 2022 GigaOm Radar for Streaming Data Platforms report. The DevOps/app dev team wants to know how data flows between such entities and understand the key performance metrics (KPMs) of these entities. She is a smart data analyst and former DBA working at a planet-scale manufacturing company.

Kafka 86

Kafka Manufacturing Data Lake SQL 86

Monte Carlo

APRIL 27, 2023

In the battle against data downtime , data teams must be equipped with full visibility into their data pipelines from ingestion and orchestration to transformation and visualization. Each interconnected layer can introduce and be impacted by poor data quality.

BI 52

BI Business Intelligence Metadata Data Warehouse 52

Monte Carlo

JANUARY 12, 2024

On the data lake side, Databricks has launched Unity Catalog to help bring more metadata, structure, and governance to data assets. Real-time data/insights Being able to access real-time data for analysis might sound like overkill to some, but that’s just no longer the case.

Big Data 59

Big Data Data Analytics Data Science Data Lake 59

DataKitchen

SEPTEMBER 21, 2023

Data in Place refers to the organized structuring and storage of data within a specific storage medium, be it a database, bucket store, files, or other storage platforms. In the contemporary data landscape, data teams commonly utilize data warehouses or lakes to arrange their data into L1, L2, and L3 layers.

Raw Data 67

Raw Data Data Business Intelligence High Quality Data 67

Monte Carlo

JUNE 26, 2023

billion in cumulative rewards to its users— Head of Data Jeff Hepburn and his team rely on Monte Carlo to deliver end-to-end visibility into the health of their data pipelines from ingestion in Databricks right down to the business intelligence layer.

Data Lake 52

Data Lake Metadata Bytes Google Cloud 52

AltexSoft

MARCH 14, 2023

As the volume and complexity of data continue to grow, organizations seek faster, more efficient, and cost-effective ways to manage and analyze data. In recent years, cloud-based data warehouses have revolutionized data processing with their advanced massively parallel processing (MPP) capabilities and SQL support.

IT 59

IT Data Warehouse Data Governance Data Lake 59

ProjectPro

JANUARY 31, 2022

As the demand for big data grows, an increasing number of businesses are turning to cloud data warehouses. The cloud is the only platform to handle today's colossal data volumes because of its flexibility and scalability. Launched in 2014, Snowflake is one of the most popular cloud data solutions on the market.

Architecture 52

Architecture IT Data Warehouse Amazon Web Services 52

AltexSoft

MARCH 30, 2023

What is Databricks Databricks is an analytics platform with a unified set of tools for data engineering, data management , data science, and machine learning. It combines the best elements of a data warehouse, a centralized repository for structured data, and a data lake used to host large amounts of raw data.

Scala 64

Scala Data Lake BI Google Cloud 64

AltexSoft

FEBRUARY 11, 2023

It serves as a foundation for the entire data management strategy and consists of multiple components including data pipelines; , on-premises and cloud storage facilities – data lakes , data warehouses , data hubs ;, data streaming and Big Data analytics solutions ( Hadoop , Spark , Kafka , etc.);

Data Architect 87

Data Architect Certification Generalist Big Data 87