Blog Data Lake Metadata Structured Data 
Snowflake
NOVEMBER 2, 2023
Over the years, the technology landscape for data management has given rise to various architecture patterns, each thoughtfully designed to cater to specific use cases and requirements. These patterns include both centralized storage patterns like data warehouse , data lake and data lakehouse , and distributed patterns such as data mesh.
phData: Data Engineering
SEPTEMBER 19, 2023
With the amount of data companies are using growing to unprecedented levels, organizations are grappling with the challenge of efficiently managing and deriving insights from these vast volumes of structured and unstructured data. What is a Data Lake? Consistency of data throughout the data lake.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Monte Carlo
APRIL 1, 2021
Over the past few years, data lakes have emerged as a must-have for the modern data stack. But while the technologies powering our access and analysis of data have matured, the mechanics behind understanding this data in a distributed environment have lagged behind. Data discovery tools and platforms can help.
LinkedIn Engineering
JULY 19, 2023
Open source data lakehouse deployments are built on the foundations of compute engines (like Apache Spark, Trino, Apache Flink), distributed storage (HDFS, cloud blob stores), and metadata catalogs / table formats (like Apache Iceberg, Delta, Hudi, Apache Hive Metastore). Tables are governed as per agreed upon company standards.
Cloudera
JUNE 25, 2021
Using easy-to-define policies, Replication Manager solves one of the biggest barriers for the customers in their cloud adoption journey by allowing them to move both tables/structured data and files/unstructured data to the CDP cloud of their choice easily. CDP Data Lake cluster versions – CM 7.4.0,
ProjectPro
JANUARY 24, 2023
If you are preparing for your ETL developer or data engineer interview , you must possess a solid fundamental knowledge of AWS Glue, as you’re likely to get asked questions that test your ability to handle complex big data ETL tasks. You can leverage AWS Glue to discover, transform, and prepare your data for analytics.
Hepta Analytics
FEBRUARY 14, 2022
DE Zoomcamp 2.2.1 – Introduction to Workflow Orchestration Following last weeks blog , we move to data ingestion. We already had a script that downloaded a csv file, processed the data and pushed the data to postgres database. This week, we got to think about our data ingestion design.
Expert insights. Personalized for you.
89 
Let's personalize your content