Knowledge Hut

DECEMBER 26, 2023

So don't memorize answers - instead, use these questions as a starting point to help demonstrate your expertise as a Full Stack Developer. So don't memorize answers - instead, use these questions as a starting point to help demonstrate your expertise as a Full Stack Developer. Who is a Full stack Developer?

Programming Language 52

Programming Language Python Programming Coding 52

Rockset

APRIL 29, 2019

In this blog post I compare options for real-time analytics on DynamoDB - Elasticsearch , Athena, and Spark - in terms of ease of setup, maintenance, query capability, latency. I also evaluate which use cases each of them are best suited for. There is limited support for SQL analytics with some of these options.

NoSQL 52

NoSQL PostgreSQL AWS SQL 52

Maxime Beauchemin

JANUARY 20, 2017

In relation to previously existing roles , the data engineering field could be thought of as a superset of business intelligence and data warehousing that brings more elements from software engineering. I joined Facebook in 2011 as a business intelligence engineer. By the time I left in 2013, I was a data engineer. We were pioneers.

Data Engineering 130

Data Engineering Data Engineer Engineering ETL Tools 130

Confluent

SEPTEMBER 26, 2019

In the early days, many companies simply used Apache Kafka ® for data ingestion into Hadoop or another data lake. The significant difference today is that companies use Apache Kafka as an event streaming platform for building mission-critical infrastructures and core operations platforms.

Kafka 21

Kafka BI SQL Datasets 21

Data Engineering Weekly

DECEMBER 11, 2022

Etsy: Understanding the collective impact of experiments A collective understanding of the impact of the experiments is essential to understand the overall business impact of the changes made across all teams. Etsy writes about using holdout groups to estimate the collective impact of individual experiments.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Pinterest Engineering

JANUARY 18, 2024

Understanding this evolving user journey while balancing across multiple objectives is crucial to bring the best experience to Pinterest users and is supported by multiple recommendation models, with each providing real-time inference with an overall latency of 200–300 milliseconds.

Systems 78

Systems Machine Learning Data Mining Algorithm 78

Knowledge Hut

APRIL 23, 2024

' export default function Hello() { return ( <div> Hello World! </div> ) } With an introduction to the React 16, functional components will do all the class-based components’ jobs using React hooks. Most developers create functional components using the Arrow function.

Coding 52

Coding Python Programming Accessible 52

Knowledge Hut

DECEMBER 28, 2023

Web development language is the backbone of the internet, powering everything from simple blogs to complex online applications. HTML, Python, JavaScript, PHP, and Java are some of the simplest languages to understand and are the best programming languages for web development. What is a Web Development Language?

Programming Language 52

Programming Language Java Python SQL 52

Airbnb Tech

APRIL 13, 2021

Minerva , Apache Druid , DataPortal , Apache Superset , SLA monitoring ) to make data-informed decisions. Minerva , Apache Druid , DataPortal , Apache Superset , SLA monitoring ) to make data-informed decisions. A foundation of robust and actionable data is essential for a successful efficiency program.

AWS 52

AWS Raw Data Amazon Web Services Cloud 52

Picnic Engineering

JULY 26, 2022

Since the publication of the first blog post in this series, we have received numerous questions via social media, direct messages, public posts, and meet-up discussions. Though many of the ongoing questions were already covered in the previous blog posts, a few warranted their own article. Any advice? Let’s dive in!

Data Engineering 52

Data Engineering Data Engineer Engineering Data Warehouse 52

Rockset

NOVEMBER 6, 2019

With event-driven architectures powered by systems like Apache Kafka becoming more prominent, there are now many applications in the modern software stack that make use of events and messages to operate effectively. Recording changes that can then be used to recompute state as needed: e.g. a transaction log.

Kafka 40

Kafka Data Lake SQL Hadoop 40

Rockset

FEBRUARY 25, 2021

In this article we’ll compare the ease and flexibility of querying data using Rocket and Elasticsearch. For example, a company might use real-time analytics on data such as daily active users and page load times to help detect outages of their apps on a regional level. This can be a challenge, though.

SQL 40

SQL Data Pipeline Kafka Database 40

ProjectPro

NOVEMBER 15, 2021

Anyone can freely use, study, modify and improve the project, enhancing it for good. The adaptability and technical superiority of such open-source big data projects make them stand out for community use. This blog will walk through the most popular and fascinating open source big data projects.

Big Data 52

Big Data Project Metadata Programming Language 52

Monte Carlo

JANUARY 23, 2024

One of the first data engineers at Facebook and Airbnb, he wrote and open sourced the wildly popular orchestrator, Apache Airflow , followed shortly thereafter by Apache Superset , a data exploration tool that’s taking the data viz landscape by storm. Enter, the data engineer. What is a data engineer today? What do data engineers do?

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Databand.ai

DECEMBER 20, 2022

He is also an open-source developer at The Apache Software Foundation and the author of Hysterical , a popular blog on tech careers and topics like data, coding, and engineering. But for all the nuggets of wisdom, there’s a lot of noise to sift through. But for all the nuggets of wisdom, there’s a lot of noise to sift through.

Data Analytics 52

Data Analytics Google Cloud Data Science Data Mining 52

Monte Carlo

MAY 19, 2022

In his seminal 2017 blog post, The Downfall of the Data Engineer , Maxime Beauchemin wrote that the data engineer had the worst seat at the table. This was also, uncoincidentally, directly following the introduction of the World Wide Web, email, and widespread use of personal computers. Quite the opposite in fact.

Generalist 52

Generalist Data Governance Government Metadata 52

Maxime Beauchemin

MARCH 8, 2021

we were stirred towards using Tableau Extracts. Extracts crumbled under the data volumes we had at Airbnb, creating a whole lot of challenges around non-additive metrics (think distinct user counts) and forcing us to intricately pre-compute multiple “grouping sets” which broke down some of the Tableau paradigms and confused users.

Business Intelligence 130

Business Intelligence BI Database-centric Google Cloud 130

Airbnb Tech

APRIL 30, 2021

In this post, we will share our journey in building Minerva, Airbnb’s metric platform that is used across the company as the single source of truth for analytics, reporting, and experimentation. In fact, the proliferation of data and use cases caused serious growing pains, both for data producers and for data consumers.

Data Warehouse 52

Data Warehouse Finance Metadata Aggregated Data 52