Blog - Data Engineering Digest

PinCompute: A Kubernetes Backed General Purpose Compute Platform for Pinterest

Pinterest Engineering

OCTOBER 31, 2023

PinCompute API is also fully integrated with Pinterest’s proprietary security primitives to ensure authentication, authorization, and auditing to follow paved paths. Such integration enables us to provide Pinterest developers with unified access control experience with granularity at API call and API resource level.

Architecture

Architecture Pipeline-centric Accessible Accessibility

Reducing Apache Spark Application Dependencies Upload by 99%

LinkedIn Engineering

MARCH 9, 2023

Co-authors: Shu Wang , Biao He , and Minchu Yang At LinkedIn, Apache Spark is our primary compute engine for offline data analytics such as data warehousing, data science, machine learning, A/B testing, and metrics reporting. These applications rely heavily on dependencies ( JAR files ) for their computation needs.

Hadoop

Hadoop Machine Learning Designing Data Pipeline

Data Engineering Digest

PinCompute: A Kubernetes Backed General Purpose Compute Platform for Pinterest

Reducing Apache Spark Application Dependencies Upload by 99%

Webinars

Stay Connected