Christophe Blefari

MARCH 17, 2023

On my side I'm slowly starting to get on top of the things I had in queue. But, sadly, I work in LIFO so I feel that I'm never done. For people that are not use to it it means last in, first out. Which means that I get easily disturbed by a notification—or even a thought—and do something that I did not plan to do at first.

Data 130

Data SQL Deep Learning Kafka 130

Christophe Blefari

JULY 8, 2023

Read dbt metrics documentation As an extension I've seen 2 things this week that I feel makes sense here: VulcanSQL — A data API framework for DuckDB, Snowflake, BigQuery, PostgreSQL. CSV exports, Excel, Sheets, etc.) Having some kind of simulated real-time data would be the best. But it requires to write a simulation.

Kafka 130

Kafka PostgreSQL Data Transportation 130

Data Engineering Weekly

NOVEMBER 5, 2023

Visit rudderstack.com to learn more. Netflix: Streaming SQL in Data Mesh Learnings from our journey In hindsight, we wish we had invested in enabling Flink SQL on the DataMesh platform much earlier. BlaBlaCar writes about system design and patterns for controlling the data platform cost on top of BigQuery.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Knowledge Hut

JUNE 26, 2023

Welcome to the world of data engineering, where the power of big data unfolds. If you're aspiring to be a data engineer and seeking to showcase your skills or gain hands-on experience, you've landed in the right spot. Before working on these initiatives, you should be conversant with topics and technologies. What are Data Engineering Projects?

Data Engineering 98

Data Engineering Data Engineer Coding Project 98

Data Engineering Podcast

JANUARY 23, 2022

He also provides some excellent insights into the factors that play into the build vs. buy decision at different organizational sizes. In this episode he shares his career journey, the challenges related to management of data professionals, and the platform design that he and his team have built to power analytics at a large company.

Building 100

Building Management Data Pipeline Metadata 100

Rockset

JULY 29, 2022

Often though, they learn that data warehouses are too slow and too expensive to run low latency, high concurrency workloads on real-time data. In this article, we’ll explore the strengths and shortcomings of three prominent data warehouses today: Google BigQuery, Amazon Redshift, and Snowflake.

Data Analytics 52

Data Analytics Data Warehouse Datasets Google Cloud 52

Data Engineering Weekly

APRIL 30, 2023

Intuit provides insightful information about performing a set of tests called machine learning model sanity checks in a pre-production environment. Sign up free to test out the tool today. DoorDash identifies Five big areas for using Generative AI. DoorDash writes about its discovery of Generative AI and its application to boost its business.

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

Christophe Blefari

DECEMBER 31, 2023

However, some excellent articles have been written and I want to end 2023 with one last big wrap on these December articles. A lot of things have been happening at the same time in my professional and personal life. To be honest, everything's been going well, but I've found it hard to find time to write among other things.

Data 100

Data Cloud Storage Datasets Python 100

Data Engineering Podcast

MAY 20, 2021

This is an excellent conversation that spans the engineering and philosophical complexities of an important and ever-present aspect of working with data. It builds your customer data warehouse and your identity graph on your data warehouse, with support for Snowflake, Google BigQuery, Amazon Redshift, and more.

Data Governance 100

Data Governance Government Data Warehouse Data Pipeline 100

Striim

FEBRUARY 10, 2023

These five companies successfully harnessed data analytics through Striim and serve as excellent examples of the practical applications of this valuable tool across industries and use cases. According to a recent study by KX, US businesses could see a total revenue uplift of $2.6 trillion through investment in real-time data analytics.

Data Analytics 52

Data Analytics Telecommunication Insurance Google Cloud 52

Knowledge Hut

SEPTEMBER 21, 2023

Questions such as which is better and easier to learn—AWS, Azure, or GCP— are often asked by organization leaders before starting out on their cloud journey. Azure or Google Cloud—Which is better? This question is often asked as businesses continue to understand the cloud’s usefulness and services.

Cloud 52

Cloud Google Cloud Cloud Computing Cloud Storage 52

Knowledge Hut

NOVEMBER 27, 2023

GCP offers various services like computational power, storage, databases, machine learning, networking, and security. Top Google Cloud Platform Skills to Learn Mastering the top Google Cloud skills is an excellent way to unlock the full potential of cloud computing. Are you ready to take the google cloud skills challenge?

Google Cloud 52

Google Cloud Cloud Cloud Computing Pipeline-centric 52

ThoughtSpot

JULY 18, 2023

In order to achieve this vision, we knew we’d need to work with some of the best, most innovative technology companies across the modern data stack —companies that put their users first, deliver excellence at every turn, and think differently about how technology can drive business outcomes.

Google Cloud 84

Google Cloud Cloud BI Data Security 84

Monte Carlo

APRIL 24, 2023

By accommodating various data types, reducing preprocessing overhead, and offering scalability, data lakes have become an essential component of modern data platforms , particularly those serving streaming or machine learning use cases. Databricks Data Catalog and AWS Lake Formation are examples in this vein.

Data Lake 52

Data Lake Google Cloud Data Warehouse AWS 52

Monte Carlo

JULY 27, 2023

It’s not enough for data teams to be magicians transforming raw data into business value, they need to be responsible (and cost conscious) stewards as well. Unfortunately, data optimization is far from an exact science. Like much of the B2B SaaS universe, costs are based on platform usage. Each cost needs to be understood,” said Scott at BestEgg.

Data 52

Data Architecture Data Warehouse Data Engineering 52

Christophe Blefari

MARCH 15, 2024

I'd like to thank you for the excellent comments you sent me last week after the publication of the first version of the Recommendations. Mountains I hope this e-mail finds you well, wherever you are. This is just the beginning! I'd like to do a bit of user research about yato, if you consider using it drop me a message please.

Metadata 272

Metadata Datasets Data Data Warehouse 272

Workfall

JULY 4, 2023

DBT has become the stuff of legends, passed down through generations of data engineers, forever celebrated for its role in creating a world of data excellence. DBT has become the stuff of legends, passed down through generations of data engineers, forever celebrated for its role in creating a world of data excellence.

Data Warehouse 69

Data Warehouse SQL PostgreSQL Database 69

Monte Carlo

FEBRUARY 6, 2023

A worthy quest to be sure. But be warned, a warehouse migration isn’t for the faint of heart. As you probably already know if you’re reading this, a data warehouse migration is the process of moving data from one warehouse to another. Now, on its face, a data warehouse migration might seem like a relatively straightforward operation. Let’s jump in!

Data Warehouse 52

Data Warehouse AWS Data Validation Data 52

AltexSoft

AUGUST 29, 2023

If your organization fits into one of these categories and you’re considering implementing advanced data management and analytics solutions, keep reading to learn how data lakes work and how they can benefit your business. What is a data lake? A data lake allows to store data before a specific use case has been identified.

Data Lake 52

Data Lake Architecture IT Amazon Web Services 52

Ascend.io

AUGUST 31, 2023

Modern platforms like Redshift , Snowflake , and BigQuery have elevated the data warehouse model. In the dynamic world of data, many professionals are still fixated on traditional patterns of data warehousing and ETL, even while their organizations are migrating to the cloud and adopting cloud-native data services. Let’s take a closer look.

Data Lake 52

Data Lake ETL Tools Data Warehouse Data Pipeline 52

Monte Carlo

APRIL 12, 2022

You do a little SQL, a few joins, and after some time, you export into Excel and ship off the answer. Data teams love the idea of automating data engineering processes in principle. After all, who doesn’t want to move faster and eliminate the time consuming, boring aspects of their job? We’ve felt it ourselves.

Data Engineering 52

Data Engineering Data Engineer Process Engineering 52

Knowledge Hut

SEPTEMBER 25, 2023

Because a certification serves as tangible proof to potential employers that you possess the necessary training and expertise to excel in this field. In today's digital age, data is the lifeblood of any successful business. Is Data Analyst Certification worth it? In my opinion, Data analyst certification is definitely worth it.

Certification 52

Certification Business Analyst Big Data Data Analysis 52

Databand.ai

DECEMBER 13, 2022

Whether you’re a data engineering pro looking to stay up to date on the latest trends or new to the space and want to learn more, following the right leaders and joining the right conversations can make all the difference when it comes to plugging into the data engineering community. You’ve come to the right place.

Data Engineering 52

Data Engineering Data Engineer Engineering AWS 52

Towards Data Science

MARCH 5, 2024

Make the most out of your BigQuery usage, burn data rather than money to create real value with some practical techniques. · ? BigQuery basics and understanding costs ∘ Storage ∘ Compute · ? In this article, we will unravel the arts of BigQuery sorcery, to reduce costs while increasing efficiency, and beyond. Introduction · ?

Bytes 69

Bytes Google Cloud Cloud Storage Utilities 69

Christophe Blefari

MAY 12, 2023

In Minds of Data I'll met people from the data ecosystem in order to learn more about them. I hope you will enjoy it as much as me. Let's start with with my newly created podcast Minds of Data. I really liked Benoit and Samy presentation about Cloud Composer —Managed Airflow on GCP. for IT security or medicine).

Data 130

Data Data Storage SQL Coding 130

Knowledge Hut

JANUARY 24, 2024

Machine Learning Tableau supports Python machine learning features. Machine Learning Tableau supports Python machine learning features. It is an excellent data visualization and business intelligence tool for reporting and analyzing vast volumes of data. Compared to Tableau, Power BI is less expensive.

BI 98

BI Business Intelligence Non-relational Database Data Analysis 98

Knowledge Hut

APRIL 19, 2023

If you would like to learn more about different data aggregation techniques check out a Data Engineer certification program. If you would like to learn more about different data aggregation techniques check out a Data Engineer certification program. Aggregation of data is useful in this situation. What is Data Aggregation?

Process 59

Process Data Mining Aggregated Data Portfolio 59

ProjectPro

JANUARY 24, 2023

This blog is your comprehensive guide to Google BigQuery, its architecture, and a beginner-friendly tutorial on how to use Google BigQuery for your data warehousing activities. BigQuery can process upto 20 TB of data per day and has a storage limit of 1PB per table. Sounds exciting to learn more about Google BigQuery, read on!

Bytes 52

Bytes Google Cloud Data Warehouse Datasets 52

Data Engineering Weekly

MAY 21, 2023

link] Timo Dechau: The extensive guide for Server-Side Tracking The blog is an excellent overview of server-side event tracking. Sign up free to test out the tool today. Ramon Marrero: DBT Model Contracts - Importance and Pitfalls dbt introduces model contract with 1.5 Instacart adoption of Databricks ecosystem along with Snowflake.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Data Engineering Weekly

MAY 28, 2023

Over the weekend, I found this an excellent thread on how Slack uses Kafka; I want to highlight this one piece. HoneyComb writes an excellent article from a developer perspective showing the hard part of integrating LLM into a product experience. Sign up free to test out the tool today.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Data Engineering Podcast

MAY 18, 2021

This is an excellent conversation about how competing companies can still find mutual benefit in co-operating on open standards. It builds your customer data warehouse and your identity graph on your data warehouse, with support for Snowflake, Google BigQuery, Amazon Redshift, and more.

Metadata 100

Metadata Kafka Data Warehouse Hadoop 100

Monte Carlo

JUNE 14, 2023

Be sure to read to the end where we share 6 data pipeline architecture diagrams used by real data teams at companies like JetBlue, Fox Networks, Drata, and more. What is data pipeline architecture? Data pipeline architecture is the process of designing how data is surfaced from its source system to the consumption layer.

Data Pipeline 52

Data Pipeline Architecture Data Lake Data Warehouse 52

Monte Carlo

NOVEMBER 21, 2022

A few weeks ago, I had the opportunity to chat with famed venture capitalist, prolific blogger , and friend Tomasz Tunguz about his top 9 data engineering predictions for 2023. It looked like so much fun that I decided to grab my crystal ball and add a few suggestions to the mix. Snowflake went from a hundred million in revenue to about 1.2

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Monte Carlo

FEBRUARY 20, 2024

At the heart of every data-driven decision is a deceptively simple question: How do you get the right data to the right place at the right time? The growing field of data ingestion tools offers a range of answers, each with implications to ponder. Times have changed and there are better ways of doing things now.

Data Ingestion 52

Data Ingestion Google Cloud Kafka AWS 52

AltexSoft

JANUARY 26, 2022

They work with Excel, SQL code, and analytics tools to perform ad-hoc analyses and forecasting. They commonly prepare data and build machine learning (ML) models. Get acquainted with how data is prepared for machine learning projects in our dedicated video. Learn more about what a machine learning engineer is in our video.

Engineering 59

Engineering Software Engineer Software Engineering Data Warehouse 59

dbt Developer Hub

OCTOBER 17, 2022

For years working in data and analytics engineering roles, I treasured the daily camaraderie sharing a small office space with talented folks using a range of tools - from analysts using SQL and Excel to data scientists working in Python. But logistically, it just couldn’t be done in a scalable way.

SQL 52

SQL Python Programming Language Data Analysis 52

ProjectPro

JANUARY 31, 2022

RedShift Snowflake vs. BigQuery Snowflake vs. Databricks Snowflake Projects for Practice in 2022 Dive Deeper Into The Snowflake Architecture FAQs on Snowflake Architecture Snowflake Overview and Architecture With Data Explosion, acquiring, processing, and storing large or complicated datasets appears more challenging.

Architecture 52

Architecture IT Data Warehouse Amazon Web Services 52