Remove sharing-is-caring-multi-tenancy-in-distributed-data-systems
article thumbnail

PinCompute: A Kubernetes Backed General Purpose Compute Platform for Pinterest

Pinterest Engineering

This will accelerate innovation through platform agility, scalability, and a reduced cost of keeping systems up to date, and will improve efficiency by running our users’ applications on Kubernetes-based compute. PinCompute is multi-tenant, where a variety of types of workloads from different teams and organizations share the same platform.

article thumbnail

Optimizing data warehouse storage

Netflix Tech

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Top 5 Questions about Apache NiFi

Cloudera

Over the last few weeks, I delivered four live NiFi demo sessions, showing how to use NiFi connectors and processors to connect to various systems, with 1000 attendees in different geographic regions. MiNiFi are agents used to collect subsets of data from sensors and devices situated in remote locations.

Kafka 60
article thumbnail

Implementing the Netflix Media Database

Netflix Tech

In the previous blog posts in this series, we introduced the N etflix M edia D ata B ase ( NMDB ) and its salient “Media Document” data model. In this post we will provide details of the NMDB system architecture beginning with the system requirements?—?these key value stores generally allow storing any data under a key).

Media 94
article thumbnail

100+ Kafka Interview Questions and Answers for 2023

ProjectPro

This blog brings you the most popular Kafka interview questions and answers divided into various categories such as Apache Kafka interview questions for beginners, Advanced Kafka interview questions/Apache Kafka interview questions for experienced, Apache Kafka Zookeeper interview questions, etc. How to study for Kafka interview?

Kafka 40