Pinterest Engineering

JANUARY 18, 2024

Understanding this evolving user journey while balancing across multiple objectives is crucial to bring the best experience to Pinterest users and is supported by multiple recommendation models, with each providing real-time inference with an overall latency of 200–300 milliseconds.

Systems 78

Systems Machine Learning Data Mining Algorithm 78

DoorDash Engineering

FEBRUARY 13, 2024

In certain domains, such as campaign management in CRM, it can be time-consuming to designate and apply business policies to different user segments. In a perfect world, we would test all possible policies and run a heterogeneous treatment effect, or HTE, model to identify the best policy for each consumer’s historical data.

Machine Learning 79

Machine Learning Designing Data Science Building 79

Pinterest Engineering

MAY 17, 2023

In this blog post, we will walk through: The Warden Anomaly Detection Platform. Detection and removal of spam and users who create spam is a priority in keeping our systems safe and providing a great experience for our users. Let’s assume Pinterest users (Pinners) are looking for clothing ideas. What is Warden?

Algorithm 68

Algorithm Engineering Machine Learning Project 68

DoorDash Engineering

JANUARY 31, 2023

App startup time is a critical metric for users, as it’s their first interaction with the app, and even minor improvements can have significant benefits for the user experience. Historically, we circumvented the need of having to think about hashing by using AnyHashable.

Architecture 113

Architecture Engineering Process Coding 113

Rockset

FEBRUARY 24, 2022

Events are commonly used by systems in the following ways: Reacting to changes in other systems; e.g. when a payment is completed, send the user a receipt. All older segments are read-only and immutable. If events arrive late and have to update historical segments, those segments need to be rewritten.

AWS 52

AWS Amazon Web Services Kafka SQL 52

Rockset

MARCH 27, 2023

This blog will clarify some conceptual differences, provide an overview of popular tools, and offer a framework for deciding which tools are best suited for specific technical requirements. If you’re looking to execute incredibly fast aggregations on historical data, you’ll likely choose a column-oriented database with a primary index.

Database 52

Database Process Scala SQL 52

ProjectPro

OCTOBER 27, 2021

This blog presents the topmost useful machine learning applications in finance to help you understand how financial markets thrive by adopting AI and ML solutions. With the quantitative nature of the finance sector and large volumes of historical data available, machine learning in finance is poised to enhance several aspects of the industry.

Finance 52

Finance Machine Learning Project Banking 52

ProjectPro

JANUARY 31, 2023

Read this blog to learn more about how the insurance industry benefits from implementing AI and ML solutions, which highlights the most practical applications and real-world use cases of machine learning in insurance and how it brings a new level of precision and effciency to the industry. The company has saved $5.7

Insurance 75

Insurance Machine Learning Project Algorithm 75

ProjectPro

NOVEMBER 18, 2021

Walmart Sales Forecasting Project uses historical sales data for 45 Walmart stores located in different regions. You can also try your hands-on Inventory Demand Forecasting Data Science Project to develop a machine learning model to forecast inventory demand accurately based on historical sales data.

Data Science 52

Data Science Project Food Pharmaceutical 52

Grouparoo

SEPTEMBER 10, 2020

Historically, marketers were responsible primarily for defining the company’s brand and driving awareness. As an organization grows, one of the most important aspects of marketing is segmenting your users correctly before you start any campaigns. All of this often requires someone with SQL skills to help you out.

Data Warehouse 52

Data Warehouse SQL Engineering Technology 52

ProjectPro

JANUARY 15, 2021

Here in this blog, we’ll provide you with over 100 worthwhile datasets for machine learning, particularly for beginners, that will surely help validate your basic data science and machine learning skills. transactions, and clicks along with information on different item properties for 417053 unique items.

Datasets 40

Datasets Machine Learning Retail Banking 40

Knowledge Hut

MAY 31, 2023

This blog will explore the top 10 business analytics project ideas you can do online as a beginner or an experienced professional. Customer Segmentation Source Code Dataset Fraud Detection Credit card fraud, identity theft, and cyber-attack are common fraudster challenges faced across various industries.

Project 52

Project Datasets Media Python 52

ProjectPro

AUGUST 16, 2021

Machine Learning Projects on Prediction Predictive modelling often uses historical data to learn and predict the likelihood of an event in the future. Historical data provides insights and patterns for making valuable business predictions—for example, predicting customer churn for an organisation in the next 30 days.

Machine Learning 40

Machine Learning Coding Project Deep Learning 40

phData: Data Engineering

MARCH 11, 2024

This blog was co-written by Alex Roed, Luke Kline, Sam Hall, Alec Haase, and Christian Franklin. Audience Management: Building and segmenting cohorts based on events, attributes, and user traits (e.g., an audience of high LTV users in the United States who abandoned their shopping cart in the last seven days.)

Data Warehouse 52

Data Warehouse Data Data Storage Cloud 52

Maxime Beauchemin

JANUARY 7, 2018

Batch data processing  — historically known as ETL —  is extremely challenging. Knowing this, the retention policy on derived tables can be shorter, knowing that it’s possible to backfill historical data at will. Another modern approach to store historical values in dimensions is to use complex or nested data types.

Data Engineering 183

Data Engineering Data Engineer Data Process Process 183

Netflix Tech

SEPTEMBER 2, 2020

In an earlier blog post, we discussed Telltale , our health monitoring system. We built Edgar to ease this burden, by empowering our users to troubleshoot distributed systems efficiently with the help of a summarized presentation of request tracing, logs, analysis, and metadata. Has a start time and an end time.

Metadata 118

Metadata Transportation Engineering Systems 118

LinkedIn Engineering

AUGUST 23, 2023

In this blog post, we are delighted to introduce a significant upgrade to our model's capabilities. By mapping new items or users to the existing embedding space, the system can generate meaningful recommendations based on similarities with known items or members.

Architecture 82

Architecture Deep Learning Datasets Machine Learning 82

DoorDash Engineering

JUNE 20, 2023

More modularization and extensibility As we built a standardized module wrapper in the ensemble modeling framework, our implementation improved flexibility by supporting any user-defined model. The post How DoorDash Built an Ensemble Learning Model for Time Series Forecasting appeared first on DoorDash Engineering Blog.

Machine Learning 72

Machine Learning Deep Learning Architecture Designing 72

ProjectPro

MAY 23, 2015

2014 Kaggle Competition Walmart Recruiting – Predicting Store Sales using Historical Data Description of Walmart Dataset for Predicting Store Sales What kind of big data and hadoop projects you can work with using Walmart Dataset? petabytes of unstructured data from 1 million customers every hour. How Walmart uses Big Data?

Big Data 52

Big Data Data Analysis Hadoop Retail 52

ProjectPro

AUGUST 24, 2021

And, out of these professions, this blog will discuss the data engineering job role. Source Code: Smart IoT Infrastructure Data Engineering Project with Source Code 2) Aviation Data Analysis Aviation Data can segment passengers, observe their behavioral patterns, and reach out to them with relevant, targeted promotions.

Data Engineering 52

Data Engineering Data Engineer Coding Project 52

ProjectPro

MARCH 12, 2016

Reader's Choice: The topic for this article has been recommended by one of our Blog subscribers. With more than- -165 million active users as of August 2015; -10+ million logins every day; -13 million transactions; - Processing more than 1.1 PB of data; - $250 billion worth of payments processed every year; -12.5

Big Data 52

Big Data Data Analytics Hadoop Algorithm 52

Monte Carlo

MAY 6, 2021

Incidents can be detected through data monitoring and alerting , which can be both implemented manually on your data pipelines and triggered based on specific thresholds, or layered as part of a data observability solution and triggered automatically at regular intervals based on historical data patterns and custom rules.

Management 40

Management Data Pipeline Software Engineer Software Engineering 40

Knowledge Hut

JULY 20, 2023

In this blog, we look at how Python Comments work, pinpoint the good uses for them and talk about how they might make coding easier. Even if users are not aware of the original developer's intentions, they make it simpler for them to understand and modify code. Commenting promotes developer collaboration and knowledge transfer.

Python 40

Python Algorithm Coding Entertainment 40

ProjectPro

JULY 27, 2021

This blog is your one-stop solution for the top 100+ Data Engineer Interview Questions and Answers. In this blog, we have collated the frequently asked data engineer interview questions based on tools and technologies that are highly useful for a data engineer in the Big Data industry. Hadoop is a user-friendly open source framework.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

ProjectPro

JULY 30, 2021

Data Description The dataset contains the following information about the products purchased by different users. Description: This describes the product that a user purchased. In this project, you will work on Google’s Cloud Platform (GCP) to build an Image segmentation system using Mask RCNN deep learning algorithm.

Machine Learning 40

Machine Learning Project Deep Learning Banking 40

Monte Carlo

SEPTEMBER 12, 2023

For every decision or process, Whatnot leaders consider three “legs of the stool”: user research (every engineer is encouraged to talk directly to users and dogfood the app), product strategy, and data. “If Learn more about data engineering at Whatnot on their blog. Data is very much part of the culture.”

Machine Learning 52

Machine Learning Data Entertainment SQL 52

Monte Carlo

APRIL 1, 2021

it also might be looking at what’s the right tool for the right user, and use case managing their BI tool portfolio. Historically, because of limited compute and desktop analytics products, so much of the analysis had to be aggregated and simplified for performance reasons.

Building 52

Building BI Business Intelligence Data 52

Monte Carlo

MAY 17, 2023

In less than three years it has gone from an idea sketched out in a Barr Moses blog post to climbing the Gartner Hype Cycle for Emerging Technology. Fintech Checkout.com monitors 4,600+ data sets, supports 300+ active data platform users every day, and supports 1,200+ dbt models that run various transformations on a daily basis.

Data 52

Data Data Pipeline Data Engineering Data Engineer 52

Monte Carlo

MAY 17, 2023

In less than three years it has gone from an idea sketched out in a Barr Moses blog post to climbing the Gartner Hype Cycle for Emerging Technology. Fintech Checkout.com monitors 4,600+ data sets, supports 300+ active data platform users every day, and supports 1,200+ dbt models that run various transformations on a daily basis.

Data Pipeline 52

Data Pipeline Data Data Engineering Data Engineer 52