Hevo

MAY 9, 2024

Apache Hive is a Data Warehouse system that facilitates writing, reading, and manipulating large datasets residing across distributed storage using SQL. SQL (Structured Query Language) is a querying language that is used to perform various operations on the records stored in a database.

SQL 52

SQL Data Warehouse Datasets Database 52

Data Engineering Podcast

FEBRUARY 8, 2021

Summary With all of the tools and services available for building a data platform it can be difficult to separate the signal from the noise. In this episode Zeeshan Qureshi and Michelle Ark share their experiences using DBT to manage the data warehouse for Shopify.

Data Warehouse 100

Data Warehouse Building BI SQL 100

Monte Carlo

JANUARY 25, 2023

Over the past year, data contracts have taken the data world by storm as a novel approach to ensuring data quality at scale in production services. In the last couple of posts , I’ve focused on implementing data contracts in production services. Where to start implementing data contracts?

Data Warehouse 52

Data Warehouse Data High Quality Data Metadata 52

Data Engineering Podcast

OCTOBER 14, 2019

Summary Managing a data warehouse can be challenging, especially when trying to maintain a common set of patterns. With 200Gbit private networking, scalable shared block storage, and a 40Gbit public network, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform.

Data Warehouse 100

Data Warehouse PostgreSQL AWS Programming Language 100

Cloudera

SEPTEMBER 24, 2020

Some of the most powerful results come from combining complementary superpowers, and the “dynamic duo” of Apache Hive LLAP and Apache Impala, both included in Cloudera Data Warehouse , is further evidence of this. Both Impala and Hive can operate at an unprecedented and massive scale, with many petabytes of data.

Data Warehouse 70

Data Warehouse SQL Engineering Metadata 70

Monte Carlo

FEBRUARY 6, 2023

So, you’re planning a cloud data warehouse migration. But be warned, a warehouse migration isn’t for the faint of heart. As you probably already know if you’re reading this, a data warehouse migration is the process of moving data from one warehouse to another. A worthy quest to be sure.

Data Warehouse 52

Data Warehouse AWS Data Validation Data 52

Data Engineering Podcast

DECEMBER 8, 2019

Summary Data warehouses have gone through many transformations, from standard relational databases on powerful hardware, to column oriented storage engines, to the current generation of cloud-native analytical engines. If you are evaluating your options for building or migrating a data platform, then this is definitely worth a listen.

Data Warehouse 100

Data Warehouse Cloud AWS Relational Database 100

Cloudera

JANUARY 15, 2021

Cloud data warehouses allow users to run analytic workloads with greater agility, better isolation and scale, and lower administrative overhead than ever before. The results demonstrate superior price performance of Cloudera Data Warehouse on the full set of 99 queries from the TPC-DS benchmark. Introduction. higher cost.

Data Warehouse 94

Data Warehouse Cloud Consulting SQL 94

Data Engineering Podcast

OCTOBER 28, 2021

Summary Streaming data systems have been growing more capable and flexible over the past few years. He also explains why he started Decodable to address that limitation and the work that he and his team have done to let data engineers build streaming pipelines entirely in SQL. Missing data? Stale dashboards?

Data Pipeline 100

Data Pipeline SQL Data Warehouse Data Lake 100

databricks

AUGUST 10, 2023

At this year's Data+AI Summit, Databricks SQL continued to push the boundaries of what a data warehouse can be, leveraging AI across the.

SQL 87

SQL Data Warehouse Data 87

Data Engineering Podcast

AUGUST 31, 2020

Summary Data warehouse technology has been around for decades and has gone through several generational shifts in that time. The current trends in data warehousing are oriented around cloud native architectures that take advantage of dynamic scaling and the separation of compute and storage.

Data Warehouse 100

Data Warehouse Cloud Building Data Lake 100

Seattle Data Guy

FEBRUARY 10, 2024

Photo by Tiger Lily Data warehouses and data lakes play a crucial role for many businesses. It gives businesses access to the data from all of their various systems. As well as often integrating data so that end-users can answer business critical questions.

Data Lake 162

Data Lake Data Warehouse Data Accessible 162

Start Data Engineering

JANUARY 16, 2021

Introduction Setup Code Conditional logic to read from mock input Custom macro to test for equality Setup environment specific test Run ELT using dbt Conclusion Further reading Introduction With the recent advancements in data warehouses and tools like dbt most transformations(T of ELT) are being done directly in the data warehouse.

SQL 130

SQL Data Warehouse Coding IT 130

databricks

MARCH 6, 2024

This blog continues our series looking at advancements from 2023 to the serverless data warehouse Databricks SQL. The best data warehouse is.

SQL 52

SQL Data Warehouse Data 52

Cloudera

JANUARY 19, 2024

As described in our recent blog post , an SQL AI Assistant has been integrated into Hue with the capability to leverage the power of large language models (LLMs) for a number of SQL tasks. This is a real game-changer for data analysts on all levels and will make SQL development faster, easier, and less error-prone.

SQL 72

SQL Data Warehouse AWS Architecture 72

Christophe Blefari

MARCH 1, 2023

dbt Core is an open-source framework that helps you organise data warehouse SQL transformation. dbt was born out of the analysis that more and more companies were switching from on-premise Hadoop data infrastructure to cloud data warehouses. This switch has been lead by modern data stack vision.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Monte Carlo

APRIL 16, 2022

The data warehouse is the foundation of the modern data stack, so it caught our attention when we saw Convoy head of data Chad Sanderson declare, “ the data warehouse is broken ” on LinkedIn. Treating data like an API. Immutable data warehouses have challenges too.

Data Warehouse 52

Data Warehouse Architecture Data Government 52

Christophe Blefari

APRIL 19, 2024

easy ( credits ) Hey, new Friday, new Data News. Structured generative AI — Oren explains how you can constraint generative algorithms to produce structured outputs (like JSON or SQL—seen as an AST). This week, I feel like the selection is smaller than usual, so enjoy the links. But don't worry it gonna be out soon.

MySQL 130

MySQL Data Datasets SQL 130

Hevo

AUGUST 20, 2023

If you’re looking to migrate your data from HubSpot to SQL Server in a secure and simple manner, you’ve come to the right place. There are many reasons why organizations might want to replicate data from their source systems to the destination of their choice.

SQL 52

SQL Data Warehouse Data Systems 52

Christophe Blefari

SEPTEMBER 15, 2023

Facing the News ( credits ) Hello Data News readers. If you're late to the party and you need fresh views on LLMs Daniel wrote an introduction demystifying the Large Language Models and Jesse wrote about LLMs impact from a Data Engineering perspective. How to reduce warehouse costs? A bittersweet feeling.

Data Warehouse 130

Data Warehouse Data SQL Python 130

Cloudera

SEPTEMBER 17, 2018

SQL development is not a new concept. However, as the data warehousing world shifts into a fast-paced, digital, and agile era, the demands to quickly generate reports and help guide data-driven decisions are constantly increasing. New data types need to be quickly joined with existing data sets.

Data Warehouse 53

Data Warehouse SQL Portfolio MySQL 53

ProjectPro

FEBRUARY 16, 2023

The demand for skilled data engineers who can build, maintain, and optimize large data infrastructures does not seem to slow down any sooner. At the heart of these data engineering skills lies SQL that helps data engineers manage and manipulate large amounts of data. of data engineer job postings on Indeed?

Data Engineering 52

Data Engineering Data Engineer SQL Engineering 52

Hevo

MAY 10, 2024

Companies use Data Warehouses to store and analyze their business data to make data-driven business decisions. Querying through huge volumes of data and reach to a specific piece of data can be challenging if the queries are not optimized or data is not well organized.

SQL 40

SQL Data Warehouse Cloud Management 40

Christophe Blefari

JANUARY 20, 2024

Learn data engineering, all the references ( credits ) This is a special edition of the Data News. But right now I'm in holidays finishing a hiking week in Corsica 🥾 So I wrote this special edition about: how to learn data engineering in 2024. The idea is to create a living reference about Data Engineering.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Christophe Blefari

FEBRUARY 3, 2024

Anyway this week will be a mixed Data News with links, stuff and ideas and a small wrap-up of the DuckCon + the stuff I presented on Wed. to a Modern Data Stack meetup in Paris about DuckDB WASM. The text-to-sql problem Every once in a while the people are trying to give a shot at the text-to-sql problem. conference.

MongoDB 130

MongoDB SQL Data Data Warehouse 130

Cloudera

DECEMBER 21, 2023

How long would it take you to find the data you need to even begin to come up with a data-driven response? Well imagine it no longer, as Cloudera’s SQL AI Assistant is exactly that! However, with the new SQL AI Assistant, this is no longer a chore. Have no fear, the SQL AI Assistant is here, and can help.

SQL 67

SQL Data Warehouse Business Analyst Data Schemas 67

Cloudera

FEBRUARY 9, 2021

Today’s customers have a growing need for a faster end to end data ingestion to meet the expected speed of insights and overall business demand. This ‘need for speed’ drives a rethink on building a more modern data warehouse solution, one that balances speed with platform cost management, performance, and reliability.

Data Warehouse 70

Data Warehouse Cloud Kafka Cloud Storage 70

Data Engineering Podcast

SEPTEMBER 3, 2023

Summary Cloud data warehouses and the introduction of the ELT paradigm has led to the creation of multiple options for flexible data integration, with a roughly equal distribution of commercial and open source options. It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products.

Data Integration 162

Data Integration BI SQL Python 162

Data Engineering Weekly

JULY 5, 2023

Welcome to another episode of Data Engineering Weekly. Aswin and I select 3 to 4 articles from each edition of Data Engineering Weekly and discuss them from the author’s and our perspectives. Can we test SQL business logic during the development phase itself? Perhaps unit test the pipeline? Perhaps unit test the pipeline?

Data Warehouse 52

Data Warehouse Database Data Pipeline SQL 52

databricks

JUNE 14, 2023

It's been only 18 months since we announced Databricks SQL general availability - the serverless data warehouse on the Lakehouse - and we.

SQL 52

SQL Data Warehouse Data IT 52

Christophe Blefari

JUNE 16, 2023

I'm now under the Berlin rain with 20° When I write in these conditions I feel like a tortured author writing a depressing novel while actually today I'll speak about the AI Act, Python, SQL and data platforms. Mainly he unit tests macros (the logic) with his framework and test data with soda and dbt contracts.

Programming Language 130

Programming Language SQL PostgreSQL Data 130

Hevo

APRIL 12, 2024

ETL processes often involve aggregating data from various sources into a data warehouse or data lake. Bucketing can be used during the transformation phase to aggregate data into predefined buckets or intervals. It plays a […]

Aggregated Data 40

Aggregated Data SQL Data Lake Data Warehouse 40

Christophe Blefari

SEPTEMBER 28, 2023

Make your data stack take-off ( credits ) Hello, another edition of Data News. This week, we're going to take a step back and look at the current state of data platforms. What are the current trends and why are people fighting around the concept of the modern data stack. Is the modern data stack dying?

Cloud Storage 147

Cloud Storage Big Data Hadoop SQL 147

Christophe Blefari

MAY 18, 2023

( credits ) Data Council Austin is a yearly conference that features a great panel of speakers giving talks about the future of the data field. Data Council 2023 YouTube playlist My personal selection If you had only 3 videos to watch it should be the 3 following: Malloy an experimental language — This is my favourite talk.

Data 130

Data BI Consulting Data Science 130

Knowledge Hut

APRIL 25, 2024

Big data in information technology is used to improve operations, provide better customer service, develop customized marketing campaigns, and take other actions to increase revenue and profits. It is especially true in the world of big data. It is especially true in the world of big data. What Are Big Data T echnologies?

Big Data 98

Big Data Technology NoSQL Hadoop 98

Knowledge Hut

MARCH 28, 2024

Data science has become one of the most trending fields today. Data engineering is one of them. According to AnalytixLabs , the data science market is expected to be worth USD 230.80 This demonstrates the increasing need for Microsoft Certified Data Engineers. That’s where data engineers are on the go.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Christophe Blefari

FEBRUARY 3, 2023

Delivering the data news ( credits ) Hey you, it's already February. The article shows our Netflix art creators are using past data to create new artworks. ebay, Variable Hub a data access layer for risk decisioning — Looks like a feature store but for risk topics. But BI Engineer for the modern data stack times.

BI 130

BI Google Cloud SQL Machine Learning 130