ProjectPro

OCTOBER 18, 2021

Recommendation engines are popular in media, entertainment, and shopping. All modern apps come with a recommendation engine that suggests users for more engagement. Sales forecasting helps business owners get a clear idea of what products are in demand. Modern e-commerce sites like Flipkart, Amazon, Alibaba, etc.,

Machine Learning 52

Machine Learning Project Datasets Algorithm 52

Zalando Engineering

JULY 27, 2016

The efficiency of your software engineering work depends not only on your deep knowledge and expertise, but also on the toolset, proper environment configuration, and team collaboration activities. What does your AndroidManifest really look like? Merged Manifest Viewer. Yes, everything is fine.

Metadata 40

Metadata Coding Java Project 40

Zalando Engineering

JULY 10, 2023

Welcome back to our web platform blog series! We are excited now to reconnect and share with you some substantial enhancements we've made to the streaming and rendering architecture of our Rendering Engine framework. The building blocks of RE's language for defining what to render, are Entities.

Engineering 40

Engineering Architecture Coding Utilities 40

DoorDash Engineering

MARCH 7, 2023

While most engineering tooling at DoorDash is focused on making safe incremental improvements to existing systems, in part by testing in production (learn more about our end-to-end testing strategy ), this is not always the best approach when launching an entirely new business line. Additionally, some require a paid subscription.

AWS 119

AWS PostgreSQL Database SQL 119

Knowledge Hut

DECEMBER 26, 2023

Data Engineering is typically a software engineering role that focuses deeply on data – namely, data workflows, data pipelines, and the ETL (Extract, Transform, Load) process. According to reports by DICE Insights, the job of a Data Engineer is considered the top job in the technology industry in the third quarter of 2020.

Data Engineering 76

Data Engineering Data Engineer Engineering Pipeline-centric 76

AltexSoft

SEPTEMBER 13, 2023

If you’re an engineer or a decision-maker at a company planning to add generative AI features to its applications, the prompts you use are crucial. This makes understanding how to create good prompts, also known as prompt engineering , very important for your projects. What is prompt engineering and why does it matter?

Engineering 59

Engineering Hospitality Transportation Algorithm 59

Confluent

MAY 21, 2019

At first, you may think to use REST APIs—most programming languages have frameworks that make it very easy to implement REST APIs, so this is a common first choice. REST APIs define the HTTP methods that are used and the request and response payloads that are expected. What will the profile service publish?

Kafka 110

Kafka Insurance Architecture Database 110

Snowflake

APRIL 19, 2023

Welcome to the third blog post in our series highlighting Snowflake’s data ingestion capabilities, covering the latest on Snowpipe Streaming (currently in public preview) and how streaming ingestion can accelerate data engineering on Snowflake. What is Snowpipe Streaming?

Data Ingestion 54

Data Ingestion Kafka Java Data Pipeline 54

Lyft Engineering

JANUARY 30, 2023

These model-based decisions include price optimization for rides, incentives allocation for drivers, fraud detection, ETA prediction, and innumerable others that impact how riders move and drivers earn. Making real-time inferences with machine learning (ML) models at scale is complex. Constraints imposed by our legacy system.

Machine Learning 92

Machine Learning Systems Python Coding 92

ProjectPro

FEBRUARY 8, 2023

Read this blog to understand everything about AWS Glue that makes it one of the most popular data integration solutions in the industry. Let us dive deeper into this data integration solution by AWS and understand how and why big data professionals leverage it in their data engineering projects. Table of Contents What is AWS Glue?

AWS 98

AWS Scala Metadata Data Lake 98

Knowledge Hut

SEPTEMBER 22, 2023

What are cloud security certifications? There’s various types of cloud infrastructures available on the market, and each group offers its own certification. We’ll cover these types of certifications - and many more - in the following sections, in more detail.

Certification 52

Certification Cloud AWS Google Cloud 52

AltexSoft

OCTOBER 21, 2022

Kafka can continue the list of brand names that became generic terms for the entire type of technology. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. What is Kafka? What does the high-performance data project have to do with the real Franz Kafka’s heritage?

Kafka 93

Kafka Hadoop ETL Tools Big Data 93

Scott Logic

MAY 31, 2023

For a while now I’ve been keeping a notebook where I record brief definitions of these new terms as I encounter them. I find it such a useful reference, I thought I’d share it in this blog post. There isn’t really a formal definition of what makes a model ‘large’, but typically they will have billions of parameters.

Datasets 59

Datasets Media Architecture Finance 59

Cloudera

AUGUST 10, 2022

In CSP, Kafka serves as the storage streaming substrate, and Flink as the core in-stream processing engine that supports SQL and REST interfaces. In this blog post we’ll introduce CSP-CE, show how easy and quick it is to get started with it, and list a few interesting examples of what you can do with it. Apache Kafka and SMM.

Process 86

Process Kafka PostgreSQL MySQL 86

DareData

AUGUST 12, 2021

The goal of this blog post is to hopefully shed some light on what a healthy data ecosystem looks like, so that you might explore your company's data to its full potential. What is a data ecosystem? Each of these requires a different type of processed oil. Introduction "Data is the 21st century oil".

Transportation 52

Transportation Data Lake Data Warehouse Data 52

Netflix Tech

DECEMBER 14, 2018

In the previous blog posts in this series, we introduced the N etflix M edia D ata B ase ( NMDB ) and its salient “Media Document” data model. All NMDB APIs are authenticated (AuthN) so that the identity of an accessing application is known up front. A specific media analysis that has been performed on various media assets (e.g.,

Media 94

Media Database Metadata Data Schemas 94

Confluent

JULY 10, 2019

The repository’s README contains a bit more detail, but in a nutshell, we check out the repo and then use Gradle to initiate docker-compose : git clone [link] cd kafka-examples git checkout confluent-blog./gradlew We are using the Spock framework for writing our test specifications. gradlew composeUp. text1 : defaultvalue } [.] }.

Kafka 87

Kafka Java Bytes SQL 87

Data Engineering Podcast

JANUARY 28, 2018

Preamble Hello and welcome to the Data Engineering Podcast, the show about modern data infrastructure When you’re ready to launch your next project you’ll need somewhere to deploy it. Summary Sharing data across multiple computers, particularly when it is large and changing, is a difficult problem to solve.

Data 100

Data Project Electronics Data Management 100

Rockset

NOVEMBER 1, 2022

Something to avoid, something to fear and definitely not something to do on a whim. This blog outlines best practices from customers I have helped migrate from Elasticsearch to Rockset , reducing risk and avoiding common pitfalls. So popular still today that Rockset engineers use it for our own internal log search functions.

Database-centric 52

Database-centric Pipeline-centric SQL Aggregated Data 52

Booking.com Engineering

DECEMBER 10, 2020

Based on benchmarks and blog posts out in the wild, brotli is able to get text-like payloads (HTML, Javascript, CSS) about 5–15% smaller than the gzipped size, and it’s not especially slower or more resource-intensive during decompression. But you’re gonna have to work for it. Photo CC BY-SA 2.0 by Antony Stanley , from Flickr.

Bytes 52

Bytes Recruitment Coding Engineering 52

Zalando Engineering

APRIL 27, 2022

Anyone who has been following the topic of Site Reliability Engineering (SRE) has likely heard of Service Level Objectives (SLOs) , and Service Level Indicators (SLIs). View Catalog" , "Add Item to Cart" ), rather than a specific application (Catalog Service, Cart Service). But what if that first fallback also fails?

Architecture 52

Architecture Engineering Systems Algorithm 52

ProjectPro

JULY 27, 2021

This blog is your one-stop solution for the top 100+ Data Engineer Interview Questions and Answers. In this blog, we have collated the frequently asked data engineer interview questions based on tools and technologies that are highly useful for a data engineer in the Big Data industry.

Data Engineering 40

Data Engineering Data Engineer Engineering Hadoop 40

ProjectPro

DECEMBER 10, 2021

This blog covers the top 50 most frequently asked Azure interview questions and answers. It will provide you with a good sense of what areas you should focus on as you prepare for your next Azure interview. What makes Azure better than AWS? What do you understand about cloud computing? Why is Azure Diagnostics API needed?

BI 52

BI Cloud Computing SQL Database 52

phData: Data Engineering

JANUARY 3, 2022

DataOps: What Is It, Core Principles, and Tools For Implementation Nick Goble January 3, 2022 When building a successful company, it’s critical to have a strategy around how you build and scale your business from a technology and data perspective. What’s Data Strategy? What Are The Components of a DataOps Strategy? Why is that?

IT 52

IT AWS Software Engineer Software Engineering 52

Airbnb Tech

JUNE 1, 2021

Specifically, we will showcase how we standardize dataset definitions through declarative configurations, explain how data versioning enables us to ensure cross-dataset consistency, and illustrate how we backfill data efficiently with zero downtime. Anyone can look up definitions without confusion.

Datasets 52

Datasets Pipeline-centric Metadata Data Science 52

ProjectPro

OCTOBER 20, 2021

It is important to understand how data flows in the real world and what kind of AI interview questions are being discussed across companies. It will explain what an instance of the best-in-class answers would sound like. The value of a company lies solely in the interview process. Gartner is a market leader in market research.

Machine Learning 52

Machine Learning Algorithm Government Data Science 52

Cloudera

DECEMBER 12, 2018

I would like to start off by asking you to tell us about your background and what kicked off your 20-year career in relational database technology? I was part of this migration project, and then after undergrad, I went on to be a software engineer for a utility company, who was using DB2 on the mainframe and migrating to Oracle on Unix.

Data Warehouse 42

Data Warehouse Relational Database Hadoop BI 42

ProjectPro

APRIL 12, 2021

What kind of deep learning interview questions they are going to ask me? What questions should I ask them? We’ve collated 100 deep learning technical interview questions from the insights of our industry experts on what kind of questions they ask most often. How do I prepare for my upcoming deep learning job interview?

Deep Learning 40

Deep Learning Datasets Machine Learning Algorithm 40

Monte Carlo

JANUARY 23, 2024

In the world of data engineering, Maxime Beauchemin is someone who needs no introduction. One of the first data engineers at Facebook and Airbnb, he wrote and open sourced the wildly popular orchestrator, Apache Airflow , followed shortly thereafter by Apache Superset , a data exploration tool that’s taking the data viz landscape by storm.

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Monte Carlo

MAY 17, 2023

Data observability, an organization’s ability to fully understand the health and quality of the data in their systems, has become one of the hottest technologies in modern data engineering. In less than three years it has gone from an idea sketched out in a Barr Moses blog post to climbing the Gartner Hype Cycle for Emerging Technology.

Data 52

Data Data Pipeline Data Engineering Data Engineer 52

Monte Carlo

MAY 17, 2023

Data observability, an organization’s ability to fully understand the health and quality of the data in their systems, has become one of the hottest technologies in modern data engineering. In less than three years it has gone from an idea sketched out in a Barr Moses blog post to climbing the Gartner Hype Cycle for Emerging Technology.

Data Pipeline 52

Data Pipeline Data Data Engineering Data Engineer 52

Snowflake

SEPTEMBER 5, 2023

Snowpark ML unifies data pre-processing, feature engineering, model training, and model management and integrated deployment into a single, easy-to-use Python toolkit. Snowpark ML unifies data pre-processing, feature engineering, model training, and model management and integrated deployment into a single, easy-to-use Python toolkit.

Medical 114

Medical Python Government Datasets 114

Airbnb Tech

APRIL 30, 2021

Specifically, we will set the context on why we built it, describe its core features and the ecosystem of tools it has enabled, and highlight the impact it has had on Airbnb. As a result, data scientists and engineers spent countless hours debugging data discrepancies, fighting fires, and often feeling unproductive and defeated.

Data Warehouse 52

Data Warehouse Finance Metadata Aggregated Data 52