KDnuggets

JANUARY 15, 2022

In our first weekly roundup of data science nuggets from around the web, check out a list of curated articles on Kaggle datasets, Python debugging tools, what it is data scientists do, an overview of YOLO, 2-dimensional PyTorch tensors, and the secrets of machine learning deployment.

Datasets 158

Datasets Data Science Python Machine Learning 158

Meltano

JANUARY 25, 2023

SQL Jinja isn’t enough SQL + Jinja isn’t enough – why we need dataframes by Furcy Pin “In October 2021, Max Beauchemin, creator of Apache Airflow, wrote an excellent article about 12 trends that are currently affecting data engineers.

Pipeline-centric 52

Pipeline-centric Database-centric Data Warehouse SQL 52

Ascend.io

MAY 23, 2023

A prime example of such patterns is orphaned datasets. These are datasets that exist in a database or data storage system but no longer have a relevant link or relationship to other data, to any of the analytics, or to the main application — making them a deceptively challenging issue to tackle.

Datasets 52

Datasets Data Pipeline Metadata Database 52

AltexSoft

OCTOBER 18, 2022

In this post, we’ll briefly discuss challenges you face when working with medical data and make an overview of publucly available healthcare datasets, along with practical tasks they help solve. Read our article on HIPAA violations to avoid common mistakes and associated penalties. Medical datasets comparison chart .

Medical 52

Medical Datasets Machine Learning Hospitality 52

Christophe Blefari

MARCH 15, 2024

Forward thinking Dataviz is hierarchical — Malloy, once again, provides an excellent article about a new way to see data visualisations. Matthaus gives the dlt vision about creating the foundation for developers to be able to create sources in a wink creating a large ecosystem of APIs datasets easily maintainable.

Metadata 272

Metadata Datasets Data Data Warehouse 272

Analytics Vidhya

JUNE 12, 2023

Tajinder’s passion for unraveling hidden patterns in complex datasets has driven impactful outcomes, transforming raw data into actionable intelligence. In this article, we explore Tajinder’s inspiring success story.

Machine Learning 228

Machine Learning Engineering Raw Data Data Science 228

Towards Data Science

MARCH 29, 2023

TL;DR In this article, I explore using GPT-3.5-Turbo Turbo and GPT-4 to categorize datasets without the need for labeled data or model training, by prompting the model with data excerpts and category definitions. Is the Dataset in an Approved Category? Datasets that are not considered relevant are automatically excluded.

Datasets 62

Datasets Food Retail Hospitality 62

Christophe Blefari

JULY 15, 2023

Have fun train models on this ( credits ) Hey, it's Saturday I hope you're enjoying July, taking deserve break, reading data engineering articles while at the beach or traveling to unknown places. And it's honestly a great dataset but it's a bit hard to download and format all the data for exploration.

Datasets 130

Datasets Python Machine Learning Data 130

Analytics Vidhya

JUNE 12, 2023

Tajinder’s passion for unraveling hidden patterns in complex datasets has driven impactful outcomes, transforming raw data into actionable intelligence. In this article, we explore Tajinder’s inspiring success story.

Machine Learning 183

Machine Learning Engineering Raw Data Data Science 183

Christophe Blefari

APRIL 19, 2024

It was trained on a large dataset containing 15T tokens (compared to 2T for Llama 2). Fast News ⚡️ Theseus against really big data ( credits ) Principal Engineer — Although staffs and principals have been on the career ladder for a long time, there are very few articles on what it takes to become one of the greats.

MySQL 130

MySQL Data Datasets SQL 130

ProjectPro

JANUARY 15, 2021

And honestly, there are a lot of real-world machine learning datasets around you that you can opt to start practicing your fundamental data science and machine learning skills, even without having to complete a comprehensive data science or machine learning course. Table of Contents What is a dataset in machine learning?

Datasets 40

Datasets Machine Learning Retail Banking 40

Knowledge Hut

MAY 3, 2024

Before dwelling on this article, let's know more about r squared meaning here. Unsupervised Learning: If the available dataset has predefined features but lacks labels, then the Machine Learning algorithms perform operations on this data to assign labels to it or to reduce the dimensionality of the data.

Machine Learning 98

Machine Learning Algorithm Datasets Medical 98

Christophe Blefari

AUGUST 4, 2023

Summer Edition ( credits ) This is the first article of the Data News Summer Edition: how to build a data platform. I tried to be as short as possible in this first article, details will come in the following ones. In this article we will try to design the fancy data stack for a batch usage. This is not a paid article.

Google Cloud 225

Google Cloud MongoDB NoSQL Data 225

Knowledge Hut

FEBRUARY 29, 2024

Then, based on this information from the sample, defect or abnormality the rate for whole dataset is considered. Hypothesis testing is a part of inferential statistics which uses data from a sample to analyze results about whole dataset or population. While using Amazon SageMaker datasets are quick to access and load.

Data Science 98

Data Science Datasets Database Design Machine Learning 98

Christophe Blefari

MARCH 22, 2024

At first, I'm: oh s**t, here we go again and 10 minutes later I'm lost in reading the content and picking too many articles to fit into a thousand word edition. Then I use Feedly which is connected to ~500 websites, Reddit and Medium and opens interesting articles in tabs. Usually all the process takes me a whole Friday.

Electronics 130

Electronics Media Data Python 130

KDnuggets

OCTOBER 12, 2023

This article talks about several best practices for writing ETLs for building training datasets. It delves into several software engineering techniques and patterns applied to ML.

Building 108

Building Software Engineer Software Engineering Datasets 108

Christophe Blefari

APRIL 5, 2024

This article greatly explained how they did it. They deeply explain in this article why they choose ClickHouse to monitor their ClickHouse Cloud offering saving money on their Datadog bill. A training set for bike sharing forecasting — Max has created a large dataset of bike sharing providers in ~50 cities around the world.

SQL 130

SQL Data Python Datasets 130

Christophe Blefari

DECEMBER 31, 2023

However, some excellent articles have been written and I want to end 2023 with one last big wrap on these December articles. LLMs for builders: jargons, theory & history — Mehdi, compiled in a large article all the necessary vocab to understand the basic conversation when it comes to generative. Yeah, you know.

Data 100

Data Cloud Storage Datasets Python 100

Christophe Blefari

FEBRUARY 16, 2024

Because I did not send the news last week you will get articles from the 2 last weeks. Introducing the column explorer in MotherDuck — A cool feature in MotherDuck (DuckDB in the cloud) to add sparklines and columns distributions when looking at a dataset. I really like this article. This article discuss it.

Food 130

Food Data SQL Government 130

Christophe Blefari

APRIL 21, 2023

This article gives you an overview of the Kubernetes platform they implemented. Data Engineering at Adyen — "Data engineers at Adyen are responsible for creating high-quality, scalable, reusable and insightful datasets out of large volumes of raw data" This is a good definition of one of the possible responsibilities of DE.

Raw Data 130

Raw Data Data Datasets SQL 130

Christophe Blefari

NOVEMBER 11, 2022

Kovid wrote an article that tries to explain what are the ingredients of a data warehouse. Here are few articles that will give you few ideas about stuff to do—tbh, there isn't a one-stop solution to fix it: Programmatic schema management — Manage all your schema with some kind of code. The end-game dataset.

BI 130

BI Data Warehouse Data Database 130

ThoughtSpot

MARCH 5, 2024

These tables provide a centralized location to host both your raw data and transformed datasets optimized for AI-powered analytics with ThoughtSpot. This article provides a technical overview of this integration so you can make the most of your data investment.

BI 94

BI Datasets Raw Data SQL 94

Christophe Blefari

JUNE 9, 2023

I search and read articles first and then I write. This way, I'll dedicate my Fridays to write original articles, explore ideas and preparing articles stock for the summer holidays. So I need to schedule articles in advance. This week a natural cluster of articles have emerged.

Data 130

Data Government SQL Coding 130

Christophe Blefari

OCTOBER 14, 2023

A microservice approach for DAG authoring using datasets — The idea is to apply SE patterns to pipelines like migration , broadcast and aggregate. In addition you should create micropipelines which we can define as small, loosely coupled DAG which operates on one input Dataset and produces one output Dataset.

Python 130

Python Datasets Data Data Ingestion 130

Towards Data Science

MARCH 2, 2024

The primary objective here is to establish a metric that can effectively measure the cleanliness level of a dataset, translating this concept into a concrete optimisation problem. However, these metrics are task-specific and do not offer a unified measure for the overall cleanliness of a dataset that includes various types of errors.

Datasets 60

Datasets Data Data Science Python 60

Christophe Blefari

APRIL 14, 2023

Using Metrics Layer to standardize and scale experimentation at DoorDash — A very good exhaustive article about a metrics layer. Cybersyn is a data-as-a-service platform that provides public datasets for everyone. You can see it as a datasets marketplace of common public data. That’s why they built this system.

Datasets 130

Datasets Data Deep Learning SQL 130

KDnuggets

APRIL 24, 2023

The article shows effective coding procedures for fixing noisy labels in text data that improve the performance of any NLP model. The impact is proved by the comparison of the ML algorithm on starting and cleaning the dataset.

Datasets 106

Datasets Algorithm Data Coding 106

Picnic Engineering

JANUARY 9, 2024

One example is a model focussed on customers’ repeat purchases, called Customer Article Rebuy Prediction (or CARP in short ?). Most notably in the previous purchases page where all of your previously bought articles are ranked by CARP to create your personal shopping list. We currently have multiple recommendation models in operation.

Machine Learning 91

Machine Learning Datasets Algorithm Systems 91

Netflix Tech

NOVEMBER 13, 2023

In this blog post, we will introduce speech and music detection as an enabling technology for a variety of audio applications in Film & TV, as well as introduce our speech and music activity detection (SMAD) system which we recently published as a journal article in EURASIP Journal on Audio, Speech, and Music Processing.

Datasets 89

Datasets Metadata Algorithm Architecture 89

Knowledge Hut

MAY 3, 2024

There have been many articles written regarding overfitting and underfitting in machine learning, but virtually all of them are merely a list of tools. "Top In this article, I'd like to outline the fundamental principles for enhancing the quality of your model and, as a result, avoid underfitting and overfitting on a specific example.

Machine Learning 52

Machine Learning Datasets Algorithm Data Science 52

KDnuggets

NOVEMBER 4, 2021

This article is a brief summary of our observations on some common client misperceptions with respect to recent developments in NLP, especially the use of large-scale models and datasets.

Datasets 133

Datasets 133

Knowledge Hut

JANUARY 12, 2024

Here is an article on Measures of Dispersion. The main agenda is to remove the redundant and dependent features by changing the dataset onto a lower-dimensional space. The main agenda is to remove the redundant and dependent features by changing the dataset onto a lower-dimensional space. What is Dimensionality Reduction?

Machine Learning 97

Machine Learning Datasets Medical Algorithm 97

Christophe Blefari

SEPTEMBER 28, 2023

This article depicts well how composable our platforms will be in the future and why Apache Arrow have to be everywhere. Announcing BigQuery Omni cross-cloud joins — Join datasets located in BigQuery with datasets located in AWS or Azure. Respectively $0.04 per GB per month and $0.02 per GB per hour.

Cloud Storage 147

Cloud Storage Big Data Hadoop SQL 147

Towards Data Science

AUGUST 29, 2023

Today, we will delve into the intricacies the problem of missing data , discover the different types of missing data we may find in the wild, and explore how we can identify and mark missing values in real-world datasets. For that matter, we’ll take a look at the adolescent tobacco study example , used in the paper. Image by Author.

Datasets 103

Datasets Machine Learning Data Data Science 103

Data Engineering Weekly

MARCH 10, 2024

Google: Croissant- a metadata format for ML-ready datasets Google Research introduced Croissant, a new metadata format designed to make datasets ML-ready by standardizing the format, facilitating easier use in machine learning projects. Pradheep Arjunan - Shared insights on AZ's journey from on-prem to the cloud data warehouses.

Data Engineering 52

Data Engineering Data Engineer Engineering Datasets 52

Christophe Blefari

JULY 8, 2023

I'm working for the Series of articles in advance for August about "creating data platforms" and I'm looking for ideas about the data I could use for this. You can read my article on the semantic layer. This is an article giving tips to make your best Makefiles. Who's leading the data peloton?

Kafka 130

Kafka PostgreSQL Data Transportation 130

Monte Carlo

APRIL 1, 2024

And if you happen to be reading this article in a season other than spring, don’t worry: spring cleaning is acceptable (and encouraged, in fact) year-round. In this article, we’ll outline seven data cleaning best practices so your pipelines will be spick and span and ready for any season (or stakeholder). Spring cleaning is upon us.

High Quality Data 52

High Quality Data Datasets Data Data Pipeline 52