KDnuggets

NOVEMBER 30, 2023

The blog discusses five platforms designed for data scientists with specialized capabilities in managing large datasets, models, workflows, and collaboration beyond what GitHub offers.

Data Science 116

Data Science Project Datasets Data 116

KDnuggets

AUGUST 24, 2022

This blog post introduces seven techniques that are commonly applied in domains like intrusion detection or real-time bidding, because the datasets are often extremely imbalanced.

Datasets 159

Datasets Data Machine Learning 159

DataKitchen

MAY 10, 2024

This blog post introduces five critical use cases for data observability, each pivotal in maintaining the integrity and usability of data throughout its journey in any enterprise. Production During the production cycle, it’s crucial to oversee processes involving multiple tools and datasets, such as dashboard production or warehouse building.

Data Ingestion 58

Data Ingestion Datasets Data Coding 58

Uber Engineering

FEBRUARY 23, 2023

In this blog learn how we automated column-level drift detection in batch datasets at Uber scale, reducing the median time to detect issues in critical datasets by 5X. Data quality is of paramount importance at Uber, powering critical decisions and features.

Systems 93

Systems Datasets Data 93

databricks

MARCH 26, 2023

In today's business landscape, secure and cost-effective data sharing is more critical than ever for organizations looking to optimize their internal and external.

Datasets 75

Datasets Data 75

Team Data Science

MARCH 15, 2020

I am taking you through my recent experience to find a dataset for my project. I defined a few sources in my earlier blog post, which will give a sneak peek of techniques to extract industries. Criteria Define a simple layout to your dataset with elements like size, type of columns, format.

Insurance 130

Insurance Datasets Banking Finance 130

Workfall

JULY 18, 2023

As one of those wizards, we’ve seen the challenges we face: the struggle to transform massive datasets into meaningful insights, all while keeping queries fast and our system scalable. In this blog, we’ll whisk you away on an enchanting journey through DBT materializations. In this blog, we will cover: What is DBT?

Datasets 77

Datasets Entertainment Data Workflow Data 77

RandomTrees

APRIL 12, 2024

In this blog, we delve into the intricacies of model retraining, exploring its significance, various approaches, triggers, and best practices to empower organizations in mastering this essential component of MLOps.

Machine Learning 52

Machine Learning Datasets Systems Process 52

Databand.ai

AUGUST 30, 2023

Data profiling tools: Profiling plays a crucial role in understanding your dataset’s structure and content. Data testing tools provide insights into potential errors or discrepancies within datasets, allowing necessary corrections to be made promptly and enabling faster, more confident decision-making processes.

Data Cleanse 52

Data Cleanse Data Pipeline Datasets Data Validation 52

Christophe Blefari

APRIL 5, 2024

The data analyst every CEO wants — I really like this blog from Benoit, he gives practical advices about what you have to focus on if you're working as an data analyst for C-level of your company. In a nutshell, I'd say that both technologies are not yet mature, with a smaller advantage for Cube for being open.

SQL 130

SQL Data Python Datasets 130

DataKitchen

MAY 10, 2024

Overseeing multi-tool, multi-dataset, and multi-hop data processes ensures high-quality outputs. The Third of Five Use Cases in Data Observability Data Evaluation: This involves evaluating and cleansing new datasets before being added to production. Does the data maintain integrity without conflicting with other datasets?

Raw Data 51

Raw Data Data Ingestion Datasets Data 51

DataKitchen

MAY 10, 2024

This blog post delves into the third critical use case for Data Observation and Data Quality Validation: development and Deployment. The Fourth of Five Use Cases in Data Observability Data Evaluation: This involves evaluating and cleansing new datasets before being added to production.

Data Ingestion 51

Data Ingestion Datasets Coding Data 51

Team Data Science

NOVEMBER 28, 2020

During our first week, Andreas helped us navigate the industry that interests each of us, and this is how we picked a dataset to work on for the next following weeks. I ended up picking yelp dataset , which aligns with my interests in analyzing user behaviors. Stay tuned! Cheers, Liuna Say hello on LinkedIn: [link]

Engineering 130

Engineering Datasets Data Data Engineering 130

Uber Engineering

AUGUST 5, 2021

Uber delivers efficient and reliable transportation across the global marketplace, which is powered by hundreds of services, machine learning models, and tens of thousands of datasets.

Transportation 145

Transportation Datasets Machine Learning Data 145

Christophe Blefari

MARCH 22, 2024

❤️ I rarely say it, if Data News helps you save time you should consider taking a paid subscription (60€/year) to help me covers the blog fees and my writing Fridays. Commun Corpus — A HuggingFace dataset collection including public domain texts, newspapers and books in a lot of languages. on April 10.

Electronics 130

Electronics Media Data Python 130

Databand.ai

AUGUST 30, 2023

This includes defining roles and responsibilities related to managing datasets and setting guidelines for metadata management. Data profiling: Regularly analyze dataset content to identify inconsistencies or errors. Automated profiling tools can quickly detect anomalies or patterns indicating potential dataset integrity issues.

Building 52

Building Data Cleanse Data Governance Datasets 52

Netflix Tech

NOVEMBER 13, 2023

In this blog post, we will introduce speech and music detection as an enabling technology for a variety of audio applications in Film & TV, as well as introduce our speech and music activity detection (SMAD) system which we recently published as a journal article in EURASIP Journal on Audio, Speech, and Music Processing.

Datasets 89

Datasets Metadata Algorithm Architecture 89

Databand.ai

AUGUST 30, 2023

Data profiling tools: Profiling plays a crucial role in understanding your dataset’s structure and content. Data testing tools provide insights into potential errors or discrepancies within datasets, allowing necessary corrections to be made promptly—enabling faster, more confident decision-making processes.

Data Cleanse 52

Data Cleanse Data Validation Data Pipeline Datasets 52

Cloudera

AUGUST 26, 2021

In this blog post, we will ingest a real world dataset into Ozone, create a Hive table on top of it and analyze the data to study the correlation between new vaccinations and new cases per country using a Spark ML Jupyter notebook in CML. Learn more about the impacts of global data sharing in this blog, The Ethics of Data Exchange.

Data Science 109

Data Science Cloud Hadoop Metadata 109

FreshBI

SEPTEMBER 11, 2023

In this blog post, we will dig deeper into the transformative influence of automation in Power BI and explore a couple of Power Automate examples capable of reshaping your entire developmental journey. This proves particularly valuable when producing multiple reports with differing datasets but a standardized structure.

BI 59

BI Consulting Datasets Data Ingestion 59

KDnuggets

JUNE 14, 2023

This blog post explores the challenges of acquiring sufficient data and the limitations posed by biased datasets using RapidsAI cuDF.

Datasets 79

Datasets Machine Learning Data 79

Cloudera

DECEMBER 13, 2023

The seamless inflow of retained and external datasets via CDP allows teams to create datasets locally and make them available for others to consume without the need for assistance from data engineers. The post Cloudera Customer Story appeared first on Cloudera Blog. Please read the full story here.

Pipeline-centric 88

Pipeline-centric Professional Services Datasets BI 88

Knowledge Hut

NOVEMBER 28, 2023

Whether you are working on a personal project, learning the concepts, or working with datasets for your company, the primary focus is a data acquisition and data understanding. In this article, we will look at 31 different places to find free datasets for data science projects. What is a Data Science Dataset?

Datasets 96

Datasets Data Science Project Banking 96

Cloudera

NOVEMBER 30, 2022

We have divided the “ Transaction Support in Cloudera Operational Database (COD)” blog into two parts. var dataSet = List(Row(1, "1", 1), Row(2, "2", 2)). dataSet = dataSet :+ Row(w, "foo", w); }. var rowRDD = spark.sparkContext.parallelize(dataSet). dataSet = List(Row(501, "500", 500), Row(502, "502", 502)).

Database 79

Database Datasets NoSQL Big Data 79

Data Engineering Weekly

MARCH 10, 2024

Google: Croissant- a metadata format for ML-ready datasets Google Research introduced Croissant, a new metadata format designed to make datasets ML-ready by standardizing the format, facilitating easier use in machine learning projects. Pradheep Arjunan - Shared insights on AZ's journey from on-prem to the cloud data warehouses.

Data Engineering 52

Data Engineering Data Engineer Engineering Datasets 52

Data Engineering Weekly

APRIL 7, 2024

We index only top-tier tables, promoting the use of these higher-quality datasets. The model is trained using a dataset of code-diagnostic pairs and fine-tuned to predict line diffs that correct LSP-identified errors, showing promising results against larger models and existing benchmarks.

Data Engineering 70

Data Engineering Data Engineer Engineering Unstructured Data 70

RandomTrees

FEBRUARY 2, 2024

It creates summaries of large datasets and identifies anomalies in data. It supports keyword search in any type of document, such as a web page, PDF, email, or any other format. Generate: Cohere creates product descriptions, blog entries, and marketing materials. Its technology is based on transformer architecture.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Databand.ai

MAY 19, 2023

Snowflake dataset logging You can now see all the tables pulling in from Snowflake within the main Datasets screen. All Blogs May Product Update: Snowflake Data Quality Alerts and Groups All Blogs dbt Alerting and Monitoring with Databand All Blogs What is DataOps?

Datasets 52

Datasets Data Data Pipeline Accessible 52

Rockset

MARCH 9, 2023

In this blog post, we will explore three different ways to query the Ethereum blockchain. This method has been made particularly easy by companies like Google Cloud ( dataset released in 2018 ) and Amazon Web Services ( dataset released in 2022 ), who have each released public, actively maintained datasets for both Ethereum and Bitcoin.

Amazon Web Services 52

Amazon Web Services Datasets AWS Google Cloud 52

Pinterest Engineering

SEPTEMBER 20, 2023

In this case, thousands of fashion Pins¹ publicly available on Pinterest are gathered to serve as the raw dataset. The resulting structured dataset becomes the foundation to train and evaluate the machine learning model known as the body type signal. To explore and apply to open roles, visit our Careers page.

Building 85

Building Pipeline-centric Machine Learning Datasets 85

DataKitchen

MAY 10, 2024

This blog post explores the challenges and solutions associated with data ingestion monitoring, focusing on the unique capabilities of DataKitchen’s Open Source Data Observability software.

Data Ingestion 54

Data Ingestion Transportation High Quality Data Data Schemas 54

dbt Developer Hub

OCTOBER 30, 2023

In this blog post, I’ll attempt to provide this guidance by answering these FAQs: What is dbt clone ? One of the coolest moments of my career here thus far has been shipping the new dbt clone command as part of the dbt-core v1.6 How is it different from deferral? Should I defer or should I clone?

BI 52

BI Datasets Metadata Data Warehouse 52

Data Engineering Weekly

MARCH 5, 2023

The basics of the best practices are to establish Meta’s Ground Truth Maturity Framework [GTMF] [link] Google: Datasets at your fingertips in Google Search Easy access to the datasets is 80% of the problem solved in data engineering. link] The blog highlights six key principles of the value creation of data.

Data Engineering 52

Data Engineering Data Engineer Engineering Datasets 52

Cloudera

SEPTEMBER 15, 2021

The edge is a critical component of many digital transformation implementations, and particularly IoT deployments, for three main reasons — immediacy, fast-changing datasets and scalability. As Bernard Marr , a futurist and technology consultant, explained in a Cloudera digital event , that today’s datasets have a short shelf life.

Medical 117

Medical Retail Datasets Algorithm 117

Workfall

FEBRUARY 28, 2023

In this blog, we will cover: What Is ChatGPT? The model is trained on a massive text dataset before being fine-tuned for specific tasks such as: Translation of a language Summarization of text Debugging code Answering questions, and so on. On November 30, 2022, OpenAI launched ChatGPT. How Does ChatGPT Function? Who Can Use ChatGPT?

Datasets 62

Datasets Coding Algorithm Technology 62

Databand.ai

AUGUST 30, 2023

Data cleansing: This involves identifying and correcting (or removing) errors and inconsistencies in datasets. Data profiling: It involves examining datasets to identify patterns, trends, and anomalies. It can be done at the time of data entry or afterward.

Data Integration 52

Data Integration Data Cleanse Data Validation Data Governance 52

Knowledge Hut

OCTOBER 26, 2023

In this blog, we'll explore a curated selection of beginner-friendly machine learning projects that will not only help you grasp the fundamentals but also inspire your passion for this ever-evolving technology. It includes the UCI machine learning repository and dataset.

Machine Learning 52

Machine Learning Project Datasets Algorithm 52