Towards Data Science

JANUARY 17, 2024

Parquet vs ORC vs Avro vs Delta Lake Photo by Viktor Talashuk on Unsplash The big data world is full of various storage systems, heavily influenced by different file formats. It’s crucial to know which file format fits which scenario. Then you’ll learn to read and write data in each format.

Big Data 73

Big Data Data Data Storage SQL 73

Monte Carlo

JANUARY 5, 2024

Data lakehouse architecture combines the benefits of data warehouses and data lakes, bringing together the structure and performance of a data warehouse with the flexibility of a data lake. Data lakehouse architecture is an increasingly popular choice for many businesses because it supports interoperability between data lake formats.

Architecture 52

Architecture Data Lake Metadata Unstructured Data 52

Monte Carlo

JANUARY 5, 2024

Data lakehouse architecture combines the benefits of data warehouses and data lakes, bringing together the structure and performance of a data warehouse with the flexibility of a data lake. Data lakehouse architecture is an increasingly popular choice for many businesses because it supports interoperability between data lake formats.

Architecture 52

Architecture Data Lake Metadata Unstructured Data 52

Towards Data Science

FEBRUARY 15, 2023

Nowadays, almost by default, organizations will have to deal with data in different formats (CSV, pdf, video, parquet, etc), hence the success of blob storage like amazon’s S3. What is Delta Lake? Before going into further details on Delta Lake, we need to remember the concept of Data Lake, so let’s travel through some history.

Data Lake 80

Data Lake Data Warehouse Hadoop Data Architecture 80

Monte Carlo

AUGUST 25, 2023

Different vendors offering data warehouses, data lakes, and now data lakehouses all offer their own distinct advantages and disadvantages for data teams to consider. Commonly, you’ll find data formats such as JSON, Apache Parquet, and Apache Avro in these environments. But, the options for data storage are evolving quickly.

Data Lake 52

Data Lake Data Warehouse Unstructured Data Raw Data 52

Cloudera

MARCH 23, 2022

Apache Iceberg is a high-performance, open table format, born-in-the cloud that scales to petabytes independent of the underlying storage layer and the access engine layer. By being a truly open table format, Apache Iceberg fits well within the vision of the Cloudera Data Platform (CDP). 2: Open formats. What is Apache Iceberg?

Metadata 92

Metadata Data Architecture BI Machine Learning 92

Monte Carlo

NOVEMBER 14, 2023

Over the last few months, Apache Iceberg has come to the forefront as a promising new open-source table format that removes many of the largest barriers to lakehouse adoption – namely, the high-latency and lack of OLTP (Online Transaction Processing) support afforded by Apache Hive. Is your data lake a good fit for Iceberg?

Data Lake 64

Data Lake Metadata Data Warehouse SQL 64