The Pragmatic Engineer

APRIL 21, 2024

See a longer version of this article here: Scaling ChatGPT: Five Real-World Engineering Challenges. We  sample  the next most probable word, based on pre-trained data, the prompt, and the text generated so far. ” This was an expert from the article Scaling ChatGPT: Five Real-World Engineering Challenges.

Engineering 346

Engineering Software Engineer Software Engineering Programming 346

Christophe Blefari

AUGUST 4, 2023

Summer Edition ( credits ) This is the first article of the Data News Summer Edition: how to build a data platform. I tried to be as short as possible in this first article, details will come in the following ones. Let's design the most complete modern data stack, or rather the fancy data stack.

Google Cloud 225

Google Cloud MongoDB NoSQL Data 225

Analytics Vidhya

SEPTEMBER 20, 2023

Data engineering plays a pivotal role in the vast data ecosystem by collecting, transforming, and delivering data essential for analytics, reporting, and machine learning. Aspiring data engineers often seek real-world projects to gain hands-on experience and showcase their expertise.

Data Engineering 268

Data Engineering Data Engineer Coding Project 268

Christophe Blefari

MARCH 22, 2024

Friday routine ( credits ) It's Friday and it's Data News. I don't go into too much detail about the magic of Data News, but every Friday is the same. At first, I'm: oh s**t, here we go again and 10 minutes later I'm lost in reading the content and picking too many articles to fit into a thousand word edition.

Electronics 130

Electronics Media Data Python 130

Zalando Engineering

SEPTEMBER 1, 2015

This week the blog/e-newsletter Data Science Weekly released their list of the “Most Read Data Science Articles of 2015.” We were pretty excited to see that our very own Mikio Braun topped the list with his article, “Three Things About Data Science You Won’t Find In the Books.”

Data Science 40

Data Science Data Systems Big Data 40

Analytics Vidhya

OCTOBER 6, 2023

In today’s rapidly evolving digital landscape, seamless data, applications, and device integration are more pressing than ever. Enter Microsoft Fabric, a cutting-edge solution designed to revolutionize how we interact with technology.

Designing 262

Designing Technology Data Lake BI 262

Christophe Blefari

MARCH 15, 2024

Saying mainly that " Sora is a tool to extend creativity " Last point Mira has been mocked and criticised online because as a CTO she wasn't able to say on which public / licensed data Sora has been trained on. Pandera, a data validation library for dataframes, now supports Polars.

Metadata 272

Metadata Datasets Data Data Warehouse 272

The Pragmatic Engineer

MAY 1, 2024

This article was published on 14 March 2024 in The Pragmatic Engineer, for subscribers. Some articles about Cognition AI’s product launch. A 1959 survey had found that in any data processing installation, the programming cost US$800,000 on average and that translating programs to run on new hardware would cost $600,000.

Software Engineer 181

Software Engineer Software Engineering Programming Language Media 181

The Pragmatic Engineer

MARCH 12, 2024

In this article, we cover one out of five topics from today’s subscriber-only article What Changed in 50 Years of Computing. They come up with test cases and data. This was one out of the five topics covered in the article What Changed in 50 Years of Computing. ‘The Mythical Man Month’ by Frederick P.

Engineering 257

Engineering Programming Language Hospitality Programming 257

The Pragmatic Engineer

NOVEMBER 21, 2023

In this article, we cover thee out of nine topics from today’s subscriber-only issue: The Past and Future of Modern Backend Practices. “The more things change, the more they stay the same,” is one theme of this article. 👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter.

Engineering 285

Engineering Bytes Cloud Computing AWS 285

Christophe Blefari

JANUARY 20, 2024

Learn data engineering, all the references ( credits ) This is a special edition of the Data News. But right now I'm in holidays finishing a hiking week in Corsica 🥾 So I wrote this special edition about: how to learn data engineering in 2024. The idea is to create a living reference about Data Engineering.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Christophe Blefari

FEBRUARY 16, 2024

Italy Sora ( credits ) Hey you, time for the Data News. Because I did not send the news last week you will get articles from the 2 last weeks. Next Wednesday I will participate to a Data Night Talk a open discussion about AI & data engineering with other content creators. We will do it online / in-person. So tune-in.

Food 130

Food Data SQL Government 130

Christophe Blefari

JANUARY 14, 2024

Back to the usual Data News—with a little delay, I'm sorry. First of all, I'd like to thank you for your positive comments on last week 's article. It's a subject close to my heart and I was very happy to share it with you, because I never thought that Data News would become such a big part of my life.

Google Cloud 130

Google Cloud Data SQL Python 130

Data Engineering Podcast

MARCH 10, 2024

Summary Data lakehouse architectures are gaining popularity due to the flexibility and cost effectiveness that they offer. The link that bridges the gap between data lake and warehouse capabilities is the catalog. Data lakes are notoriously complex. Join us at the top event for the global data community, Data Council Austin.

Data Lake 147

Data Lake High Quality Data Data Pipeline Architecture 147

KDnuggets

APRIL 29, 2024

This article is an overview of a particular subset of data structures useful in machine learning and AI development, along with explanations and example implementations.

Machine Learning 111

Machine Learning Data Programming 111

Christophe Blefari

JANUARY 20, 2024

We are deep in the winter, it's time for comfy Data News to read near the fire 🔥 This week, on Monday, I started my annual university lecture. In exchange we can wear the new Ray Ban Meta glasses with AI inside to give Meta more training data. So it's normal that they don't have any questions at first.

Data 130

Data Google Cloud Python Algorithm 130

Analytics Vidhya

JUNE 12, 2023

Introduction Meet Tajinder, a seasoned Senior Data Scientist and ML Engineer who has excelled in the rapidly evolving field of data science. Tajinder’s passion for unraveling hidden patterns in complex datasets has driven impactful outcomes, transforming raw data into actionable intelligence.

Machine Learning 228

Machine Learning Engineering Raw Data Data Science 228

Analytics Vidhya

MARCH 21, 2023

Introduction In today’s world, data is growing exponentially with time with digitalization. to store and analyze this data to get valuable business insights from it. Organizations are using various cloud platforms like Azure, GCP, etc.,

Data 240

Data Cloud Data Lake Database 240

Christophe Blefari

APRIL 19, 2024

easy ( credits ) Hey, new Friday, new Data News. How we build Slack AI to be secure and private — How Slack uses VPC and Amazon SageMaker with your data secured and private. This article covers the whole ladder and the mix of skills needed to reach the top. But don't worry it gonna be out soon.

MySQL 130

MySQL Data Datasets SQL 130

The Pragmatic Engineer

OCTOBER 31, 2023

In this article, we cover three out of seven topics from today’s subscriber-only issue Three Cloud Providers, Three Outages: Three Different Responses. We’ll also learn how this article contributed to AWS publishing its first public postmortem in two years! Silence of AWS (until this article was about to publish) 1.

AWS 216

AWS Google Cloud Cloud Food 216

KDnuggets

APRIL 1, 2024

This article discusses the psychology of data visualization, including the principles and techniques that underpin the creation of persuasive and effective visuals.

Data 130

Data Data Science 130

Christophe Blefari

JULY 22, 2023

We can already feel summer, I found less articles to enter the selection this week. Also be ready for the Data News: Summer Edition. For the next 5 releases it will be a bit different than usual, less curation and more original articles written in advance to allow me to take a break. Classically boring stuff.

Data 130

Data Database Big Data Data Engineering 130

Data Engineering Podcast

APRIL 21, 2024

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. Data lakes are notoriously complex. From a product perspective, what are the data challenges that are posed by email?

Data Lake 182

Data Lake High Quality Data Data Pipeline Machine Learning 182

Christophe Blefari

JUNE 9, 2023

Rethinking the newsletter ( credits ) Here's a new edition of the Data News newsletter. I search and read articles first and then I write. This way, I'll dedicate my Fridays to write original articles, explore ideas and preparing articles stock for the summer holidays. So I need to schedule articles in advance.

Data 130

Data Government SQL Coding 130

Christophe Blefari

MARCH 1, 2023

This article is meant to be a resource hub in order to understand dbt basics and to help get started your dbt journey. dbt Core is an open-source framework that helps you organise data warehouse SQL transformation. This switch has been lead by modern data stack vision. When I write dbt, I often mean dbt Core. Enter the ELT.

Data Warehouse 278

Data Warehouse SQL Metadata Raw Data 278

Christophe Blefari

JULY 15, 2023

Have fun train models on this ( credits ) Hey, it's Saturday I hope you're enjoying July, taking deserve break, reading data engineering articles while at the beach or traveling to unknown places. And it's honestly a great dataset but it's a bit hard to download and format all the data for exploration.

Datasets 130

Datasets Python Machine Learning Data 130

Christophe Blefari

APRIL 5, 2024

Lost between ideas ( credits ) Hey, new Data News edition. MDS Fest is a free virtual 5 days conference about Modern Data Stack topics, a lot of awesome speakers, there are a few talks I can't wait to watch. It shows a few RAGs, Agents and parsers on documents to retrieve the data you need. But here we are.

SQL 130

SQL Data Python Datasets 130

Analytics Vidhya

APRIL 17, 2023

Introduction Kedro is an open-source Python framework for creating reproducible, maintainable, and modular data science code. It uses best practices of software engineering to build production-ready data science pipelines. This article will give you a glimpse of Kedro framework using news classification tasks.

Data Science 221

Data Science Software Engineer Software Engineering Machine Learning 221

Analytics Vidhya

FEBRUARY 28, 2023

Introduction One of the most important assets of any organization is the data it produces on a daily basis. This data is used by an organization to find valuable insights which help in improving an organization’s growth and strategies and give them an upper hand over its competitors.

Data Warehouse 229

Data Warehouse Data IT Database 229

Analytics Vidhya

AUGUST 22, 2023

Introduction In today’s data-driven world, seamless data integration plays a crucial role in driving business decisions and innovation. Two prominent methodologies have emerged to facilitate this process: Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT).

Data Integration 185

Data Integration Process Datasets Data 185

The Pragmatic Engineer

JANUARY 12, 2023

We cover one and a half out of eight topics in today’s subscriber-only issue, Inside Pollen's Transparent Compensation Data. I covered the collapse of the company at the time in the article Inside Pollen’s collapse: “$200M raised” but staff unpaid - exclusive. Tech compensation numbers.

Software Engineer 183

Software Engineer Software Engineering Engineering Raw Data 183

Analytics Vidhya

MARCH 7, 2023

Introduction Apache Flume is a tool/service/data ingestion mechanism for gathering, aggregating, and delivering huge amounts of streaming data from diverse sources, such as log files, events, and so on, to centralized data storage. Flume is a tool that is very dependable, distributed, and customizable. einsteinerupload of.

Data Ingestion 216

Data Ingestion Data Storage Hadoop Data 216

The Pragmatic Engineer

APRIL 11, 2024

In this article, we cover one out of four topics from today’s subscriber-only The Pulse issue. Banks regularly do maintenance during weekends; it’s when data migration projects are run, and things like schema updates, rolling out of new systems, and other riskier changes. To get full issues twice a week, subscribe here.

Banking 203

Banking Utilities Database Engineering 203

The Pragmatic Engineer

SEPTEMBER 26, 2023

In this article, we cover a fresh industry trends: Cloud Developent Environments — which is analysis full subscribers have received 3 weeks ago. And with cloud providers launching new data centers, zones and regions, it’s now easier than ever to have cloud environments near to software engineers’ locations. 

Cloud 241

Cloud Software Engineer Software Engineering Cloud Computing 241

Analytics Vidhya

JUNE 12, 2023

Introduction Meet Tajinder, a seasoned Senior Data Scientist and ML Engineer who has excelled in the rapidly evolving field of data science. Tajinder’s passion for unraveling hidden patterns in complex datasets has driven impactful outcomes, transforming raw data into actionable intelligence.

Machine Learning 183

Machine Learning Engineering Raw Data Data Science 183

Christophe Blefari

MARCH 4, 2023

Formula 1 is back (trying to jinx before it happens) (yes there is no link with the data news) ( credits ) Hello you, I hope this new Data News finds you well. This week I've published an compact article about how to get started with dbt. ” This is a quote from Cassie article which I find awesome.

Machine Learning 130

Machine Learning AWS Data Data Lake 130

Christophe Blefari

NOVEMBER 18, 2023

Still I've accumulated a lot of articles that I think should fit in the Data News so this week might be a huge recap of content that has been produce in the last month. Fast News ⚡️ Because the AI News is pretty packed and I still want you to enjoy this newsletter articles will be less commented than usual.

Food 147

Food Data Data Lake Retail 147