Remove author lu-wang
article thumbnail

PinCompute: A Kubernetes Backed General Purpose Compute Platform for Pinterest

Pinterest Engineering

Harry Zhang, Jiajun Wang, Yi Li, Shunyao Li, Ming Zong, Haniel Martino, Cathy Lu, Quentin Miao, Hao Jiang, James Wen, David Westbrook | Cloud Runtime Team Image Source: [link] Overview Modern compute platforms are foundational to accelerating innovation and running applications more efficiently.

article thumbnail

Unified Streaming And Batch Pipelines At LinkedIn: Reducing Processing time by 94% with Apache Beam

LinkedIn Engineering

Co-Authors: Yuhong Cheng , Shangjin Zhang , Xinyu Liu, and Yi Pan Efficient data processing is crucial in reducing learning curves, simplifying maintenance efforts, and decreasing operational complexity. In this blog post, we will share our progress, challenges, and lessons learned from implementing Apache Beam.

Process 97
article thumbnail

Upscaling LinkedIn's Profile Datastore While Reducing Costs

LinkedIn Engineering

Co-Authors: Estella Pham and Guanlin Lu At peak, LinkedIn serves over 1.4 In this blog post we’ll discuss our decision to leverage Couchbase, the challenges that arose, and how we addressed each challenge in our final solution. million member profiles per second.

Database 133