LinkedIn Engineering

OCTOBER 19, 2023

Authors: Bingfeng Xia and Xinyu Liu Background At LinkedIn, Apache Beam plays a pivotal role in stream processing infrastructures that process over 4 trillion events daily through more than 3,000 pipelines across multiple production data centers. More details about this use case can be found on LinkedIn’s engineering blog.

Process 119

Process Lambda Architecture Kafka Machine Learning 119

Pinterest Engineering

OCTOBER 31, 2023

Harry Zhang, Jiajun Wang, Yi Li, Shunyao Li, Ming Zong, Haniel Martino, Cathy Lu, Quentin Miao, Hao Jiang, James Wen, David Westbrook | Cloud Runtime Team Image Source: [link] Overview Modern compute platforms are foundational to accelerating innovation and running applications more efficiently.

Architecture 96

Architecture Pipeline-centric Accessible Accessibility 96

LinkedIn Engineering

MARCH 23, 2023

Co-Authors: Yuhong Cheng , Shangjin Zhang , Xinyu Liu, and Yi Pan Efficient data processing is crucial in reducing learning curves, simplifying maintenance efforts, and decreasing operational complexity. In this blog post, we will share our progress, challenges, and lessons learned from implementing Apache Beam.

Process 97

Process Lambda Architecture Kafka Datasets 97

LinkedIn Engineering

AUGUST 23, 2023

Co-authors: Jason (Siyu) Zhu , Amol Ghoting , Birjodh Tiwanna , Maneesh Varshney Introduction At LinkedIn, we strive to provide our members with valuable content that can help them build professional networks, learn new skills, and discover exciting job opportunities.

Architecture 82

Architecture Deep Learning Datasets Machine Learning 82