Analytics Vidhya

MARCH 5, 2023

A distributed file system runs on commodity hardware and manages massive data collections. It is a fully managed cloud-based environment for analyzing and processing enormous volumes of data. Introduction Microsoft Azure HDInsight(or Microsoft HDFS) is a cloud-based Hadoop Distributed File System version.

Hadoop 254

Hadoop Data Collection Cloud Technology 254

Knowledge Hut

AUGUST 19, 2024

A Deloitte survey reveals the following: 49% of the respondents said data analytics helps them make better business decisions. What i s a Data Collection Plan ? A Data collection plan is a detailed document that describes the exact steps and sequence that must be followed in gathering data for a project.

Data Collection 52

Data Collection Electronics Media Bytes 52

Analytics Vidhya

FEBRUARY 21, 2023

Organizations are converting them to cloud-based technologies for the convenience of data collecting, reporting, and analysis. This is where data warehousing is a critical component of any business, allowing companies to store and manage vast amounts of data.

Data Warehouse 211

Data Warehouse IT Data Collection Data 211

The Pragmatic Engineer

OCTOBER 17, 2024

Storing data: data collected is stored to allow for historical comparisons. Benchmarking: for new server types identified – or ones that need an updated benchmark executed to avoid data becoming stale – those instances have a benchmark started on them.

Cloud 332

Cloud AWS Metadata Cloud Computing 332

Cloudera

DECEMBER 6, 2024

Data collectives are going to merge over time, and industry value chains will consolidate and share information. A big retailer might partner with the manufacturer and a distributor to share information on demand or intervention on pricing elasticity or about available supply. It’s not direct competitors.

Manufacturing 82

Manufacturing Retail Finance Metadata 82

KDnuggets

JANUARY 30, 2023

The ChatGPT Cheat Sheet • ChatGPT as a Python Programming Assistant • How to Select Rows and Columns in Pandas Using [ ],loc, iloc,at and.iat • 5 Free Data Science Books You Must Read in 2023 • From Data Collection to Model Deployment: 6 Stages of a Data Science Project

Data Science 139

Data Science Data Collection Python Programming 139

Snowflake

SEPTEMBER 18, 2023

Bring data and ML models to life Interactive visualizations Data teams now have the ability to build a whole range of new and exciting applications that were not possible before. With Streamlit, app builders can create interactive data apps that allow for much more than just static visualizations.

Python 143

Python Building Cloud Amazon Web Services 143

KDnuggets

NOVEMBER 4, 2021

Toloka is a crowdsourced data labeling platform that handles data collection and annotation projects for machine learning at any scale. In this Nov 11 Live Demo, Learn how to get reliable training data for machine learning.

Machine Learning 133

Machine Learning Data Collection Data Project 133

Snowflake

MARCH 12, 2025

On the business side, a fan 360 strategy can improve the performance of ticketing and merchandise marketing campaigns by enabling targeted campaigns that deliver a tailored message to a specific audience at the right time and on the right channel.

Media 80

Media Cloud Data Collection Programming 80

Cloudera

JANUARY 20, 2021

The data journey is not linear, but it is an infinite loop data lifecycle – initiating at the edge, weaving through a data platform, and resulting in business imperative insights applied to real business-critical problems that result in new data-led initiatives. Data Collection Challenge. Factory ID.

Manufacturing 110

Manufacturing Data Warehouse Kafka Retail 110

Cloudera

APRIL 13, 2022

It means your company has automated the processes of collecting, understanding and acting on data across the board, from production to purchasing to product development to understanding customer priorities and preferences. Data collection and interpretation when purchasing products and services can make a big difference.

Data Collection 112

Data Collection Algorithm Machine Learning Education 112

KDnuggets

JANUARY 25, 2023

ChatGPT as a Python Programming Assistant • How to Use Python and Machine Learning to Predict Football Match Winners • 20 Questions (with Answers) to Detect Fake Data Scientists: ChatGPT Edition, Part 1 • From Data Collection to Model Deployment: 6 Stages of a Data Science Project • 5 Free Data Science Books You Must Read in 2023

Machine Learning 112

Machine Learning Python Programming Data Science 112

Cloudera

FEBRUARY 8, 2021

To accomplish this, ECC is leveraging the Cloudera Data Platform (CDP) to predict events and to have a top-down view of the car’s manufacturing process within its factories located across the globe. . Having completed the Data Collection step in the previous blog, ECC’s next step in the data lifecycle is Data Enrichment.

Data Pipeline 118

Data Pipeline Building Manufacturing Data Warehouse 118

Knowledge Hut

JANUARY 18, 2024

For more information, check out the best Data Science certification. A data scientist’s job description focuses on the following – Automating the collection process and identifying the valuable data. To pursue a career in BI development, one must have a strong understanding of data mining, data warehouse design, and SQL.

Data Science 98

Data Science Business Analyst Data Architect ETL Method 98

Data Engineering Podcast

APRIL 28, 2024

In order to simplify the integration of AI capabilities into developer workflows Tsavo Knott helped create Pieces, a powerful collection of tools that complements the tools that developers already use.

Building 147

Building Data Lake High Quality Data Machine Learning 147

Engineering at Meta

APRIL 17, 2023

How it works: Millisampler comprises userspace code to schedule runs, store data, and serve data, and an eBPF-based tc filter that runs in the kernel to collect fine-timescale data. The user code attaches the tc filter and enables data collection.

Bytes 115

Bytes Transportation Data Collection Coding 115

Cloudera

APRIL 9, 2021

This blog series follows the manufacturing and operations data lifecycle stages of an electric car manufacturer – typically experienced in large, data-driven manufacturing companies. The first blog introduced a mock vehicle manufacturing company, The Electric Car Company (ECC) and focused on Data Collection.

Machine Learning 98

Machine Learning Manufacturing Data Collection Data Science 98

Data Engineering Weekly

MARCH 2, 2025

[link] Netflix: Cloud Efficiency at Netflix Data is the Key Optimization starts with collecting data and asking the right questions. Netflix writes an excellent article describing its approach to cloud efficiency, starting with data collection to questioning the business process.

Data Engineering 63

Data Engineering Data Engineer Engineering Datasets 63

Precisely

DECEMBER 12, 2024

Understanding Bias in AI Bias in AI arises when the data used to train machine learning models reflects historical inequalities, stereotypes, or inaccuracies. This bias can be introduced at various stages of the AI development process, from data collection to algorithm design, and it can have far-reaching consequences.

Healthcare 64

Healthcare Algorithm Finance Data Integration 64

Netflix Tech

FEBRUARY 14, 2025

The data collected feeds into a comprehensive quality dashboard and supports a tiered threshold-based alerting system. We accomplish this by gathering detailed column-level metrics that offer insights into the state and quality of each impression.

Kafka 66

Kafka Datasets Metadata Utilities 66

Cloudera

JUNE 2, 2022

Companies have not treated the collection, distribution, and tracking of data throughout their data estate as a first-class problem requiring a first-class solution. Instead they built or purchased tools for data collection that are confined with a class of sources and destinations.

Systems 104

Systems Data Lake Google Cloud Data Collection 104

Confluent

JULY 29, 2021

Data is at the center of our world today, especially with the ever-increasing amount of machine-generated log data collected from applications, devices, and sensors from almost every modern technology. The […].

Data Collection 90

Data Collection Data Technology Kafka 90

Data Engineering Podcast

OCTOBER 8, 2023

In this episode he shares his journey of data collection and analysis and the challenges of automating an intentionally manual industry. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles.

Insurance 162

Insurance BI SQL Machine Learning 162

Cloudera

FEBRUARY 8, 2021

The goal is to define, implement and offer a data lifecycle platform enabling and optimizing future connected and autonomous vehicle systems that would train connected vehicle AI/ML models faster with higher accuracy and delivering a lower cost.

Manufacturing 108

Manufacturing Machine Learning Data Ingestion Electronics 108

Snowflake

JULY 8, 2024

These select EU deployments will be connected to and will send all usage data to the EU repository and only select usage data will be sent to the global repository. European Union (EU) data sovereignty Snowflake’s first zonal repository outside of the US will be located in the EU to house usage data collected from the region.

Cloud 111

Cloud Google Cloud Education AWS 111

Cloudera

MAY 4, 2022

The availability and maturity of automated data collection and analysis systems is making it possible for businesses to implement AI across their entire operations to boost efficiency and agility. Artificial intelligence (AI) has been a focus for research for decades, but has only recently become truly viable.

Insurance 130

Insurance Retail Finance Medical 130

RandomTrees

NOVEMBER 25, 2024

Solution: Generative AI-Driven Customer Insights In the project, Random Trees, a Generative AI algorithm was created as part of a suite of models for data mining the patterns from patterns in data collections that were too large for traditional models to easily extract insights from.

Telecommunication 52

Telecommunication IT Unstructured Data Data Mining 52

AltexSoft

JUNE 14, 2021

Insurers use data collected from smart devices to notify customers about harmful activities and lifestyles. Then, make sure you have data collection channels that provide you with relevant data needed for your tasks. You’ll need a data engineering team for that. Personalized communications.

Insurance 88

Insurance Medical Machine Learning Data Collection 88

Snowflake

MARCH 7, 2023

For example, utilizing data infrastructures that can scale compute resources up and down to handle fluctuating demand will inherently be more energy efficient than a data warehouse with regimented sizing. You should use the data you already have. Data collection and disclosure requirements keep shifting.

Transportation 100

Transportation Data Data Lake Retail 100

Databand.ai

MAY 30, 2023

Data quality refers to the degree of accuracy, consistency, completeness, reliability, and relevance of the data collected, stored, and used within an organization or a specific context. High-quality data is essential for making well-informed decisions, performing accurate analyses, and developing effective strategies.

Data Cleanse 85

Data Cleanse Datasets Data Governance Data Validation 85

Engineering at Meta

JANUARY 27, 2023

Millisampler data allows us to characterize microbursts at millisecond or even microsecond granularity. And simultaneous data collection enables analysis of how synchronized bursts interact in rack buffers.

Engineering 130

Engineering Software Engineer Software Engineering Transportation 130

Confluent

DECEMBER 5, 2023

Confluent Cloud enables organizations to unlock real-time visibility into manufacturing processes, using real-time data collection and analytics to prevent re-work and tooling failures, delivering an outsized impact on production volume and quality.

Manufacturing 75

Manufacturing Data Collection Cloud Process 75

U-Next

MARCH 7, 2023

Data Integration and Identification Clarification: You can gain helpful insights into previous consumer activities through data unification, also known as identity resolution, which combines data from many sources and links it to specific customer profiles. Salesforce’s CDP is one example.

Bytes 98

Bytes Media Data Data Collection 98

Data Engineering Podcast

NOVEMBER 6, 2022

How has the emergence of the "modern data stack" influenced the product direction? What are the most interesting, innovative, or unexpected ways that you have seen Snowplow used for data generation/behavioral data collection? When is Snowplow the wrong choice? What do you have planned for the future of Snowplow?

Building 130

Building IT Metadata MongoDB 130

Pinterest Engineering

MARCH 26, 2025

Bootstrap Phase To ensure users could discover Holiday Finds, we implemented a fixed-position strategy: Three-day bootstrap period with Holiday Finds locked to position 1 (immediately afterAll) Existing Board More Ideas tabs maintain their engagement-based ranking User behavior tracking begins immediately to inform future positioning This approach (..)

Building 59

Building Engineering Algorithm Systems 59

Christophe Blefari

SEPTEMBER 15, 2023

— Hugo propose 7 hacks to optimise data warehouse cost. How to reduce warehouse costs?

Data Warehouse 130

Data Warehouse Data SQL Python 130

Snowflake

OCTOBER 9, 2023

Users can proxy these libraries through sub-domains to mitigate the impact of ad blockers on data collection. A Data Contracts feature allows Snowflake users to predefine expected event data formats and quarantine non-compliant data.

Cloud 125

Cloud Building SQL Data 125