Lyft Engineering

APRIL 21, 2023

Building a large scale unsupervised model anomaly detection system — Part 1 Distributed Profiling of Model Inference Logs By Anindya Saha , Han Wang , Rajeev Prabhakar Introduction LyftLearn is Lyft’s ML Platform. In a previous blog post , we explored the architecture and challenges of the platform. As always, Lyft is hiring!

Systems 105

Systems Building Machine Learning Datasets 105

Uber Engineering

SEPTEMBER 9, 2021

One such technology is called Schemaless , which enables the modeling of related entries in one single row of multiple columns, as well as … The post Jellyfish: Cost-Effective Data Tiering for Uber’s Largest Storage System appeared first on Uber Engineering Blog.

Systems 134

Systems Data Technology Engineering 134

Cloudera

JUNE 2, 2022

This blog aims to answer two questions: What is a universal data distribution service? In a recent customer workshop with a large retail data science media company, one of the attendees, an engineering leader, made the following observation: “Everytime I go to your competitor website, they only care about their system.

Systems 99

Systems Data Lake Google Cloud Data Collection 99

Knowledge Hut

JANUARY 3, 2024

A major computer system component is its operating system (OS). A computer would be a little more than a useless computer without an operating system. And at least one operating system must be installed on your computer to run simple programs like browsers. What is Operating System (OS)?

Systems 52

Systems Programming Python Accessible 52

Knowledge Hut

JANUARY 18, 2024

CISSP stands for Certified Information Systems Security Professional, and it is a certification in cyber security. This professional certification is developed and offered by (ISC)2, also known as International Information Systems Security Certification Consortium. This blog will help you understand the question “what is CISSP?

Systems 52

Systems Certification Consulting Education 52

Eventbrite Engineering

JUNE 13, 2022

Obviously, the answer is that we need to … Continue reading "Monitoring Your System" The post Monitoring Your System appeared first on Engineering Blog.

Systems 52

Systems Engineering Technology IT 52

DataKitchen

DECEMBER 9, 2022

This can include the use of tools for data preparation, model training, and deployment, as well as technologies for monitoring and managing data-related systems and processes. One of the key benefits of DataOps observability is the ability to improve collaboration and communication across teams and systems.

Data Preparation 52

Data Preparation Machine Learning Government Data Analytics 52

Uber Engineering

FEBRUARY 23, 2023

In this blog learn how we automated column-level drift detection in batch datasets at Uber scale, reducing the median time to detect issues in critical datasets by 5X. Data quality is of paramount importance at Uber, powering critical decisions and features.

Systems 98

Systems Datasets Data 98

Lyft Engineering

APRIL 25, 2023

Building a large scale unsupervised model anomaly detection system — Part 2 Building ML Models with Observability at Scale By Rajeev Prabhakar , Han Wang , Anindya Saha Photo by Octavian Rosca on Unsplash In our previous blog we discussed the different challenges we faced for model monitoring and our strategy for addressing some of these problems.

Systems 75

Systems Building Machine Learning Datasets 75

Knowledge Hut

JULY 26, 2023

But what precisely is operating system virtualization? This blog will provide you all the information about the Operating System virtualization along with AWS Solution Architect syllabus. What is Operating System Virtualization? Uses of Operating System Virtualization?

Systems 52

Systems Cloud Computing Utilities Cloud 52

Rockset

DECEMBER 19, 2023

From his early days at Quora to leading projects at Facebook and his current venture at Fennel (a real-time feature store for ML), Nikhil has traversed the evolving landscape of machine learning engineering and machine learning infrastructure specifically in the context of recommendation systems.

Systems 52

Systems Machine Learning Deep Learning Media 52

LinkedIn Engineering

DECEMBER 6, 2022

With a reasonably sizable footprint of servers in data centers, LinkedIn is responsible for ensuring that these hosts are always on an operating system (OS) version deemed the ���latest and greatest��� for all intents and purposes.

Systems 55

Systems Metadata Datasets Designing 55

Cloudera

MAY 25, 2021

Recommendation systems have become a cornerstone of modern life, spanning sectors that include online retail, music and video streaming, and even content publishing. These systems help us navigate the sheer volume of content on the internet, allowing us to discover what’s interesting or important to us.

Systems 59

Systems Algorithm Retail Utilities 59

Data Science Blog: Data Engineering

SEPTEMBER 19, 2023

In the contemporary age of Big Data, Data Warehouse Systems and Data Science Analytics Infrastructures have become an essential component for organizations to store, analyze, and make data-driven decisions. Scalability Tailored for Data Needs Data Warehouse Systems often require to scale quickly to handle larger datasets or more queries.

Data Warehouse 52

Data Warehouse Coding Systems Cloud 52

Zalando Engineering

JUNE 30, 2020

Our Engineering Blog was launched in June 2020 after a long break of the previous tech blog. What customizations we applied to design the blog and the publishing process. Static Site Generator Our previous tech blog used a CMS which only a limited number of people had access to. So which static site generator to choose?

Engineering 40

Engineering Bytes AWS Python 40

Zalando Engineering

JULY 20, 2022

Yet Another "What is a Design System?" There is a lot of literature and countless blog posts around the very definition of the concept of design systems. You can also find more information about the creation process from a design point of view in this blog post.

Designing 52

Designing Systems Electronics Architecture 52

Pinterest Engineering

JANUARY 10, 2023

Liang Ma | Software Engineer, Core Eng; Wei Zhu | Software Engineer, Observability In early 2020, during a critical iOS out of memory incident (we have a blogpost for that), we realized that we didn’t have much visibility of how the app is running or a good system to look up for monitoring and troubleshooting.

Systems 54

Systems Building Software Engineer Software Engineering 54

Uber Engineering

APRIL 5, 2018

This analysis powers our services and enables the delivery of more seamless and reliable user … The post Scaling Uber’s Apache Hadoop Distributed File System for Growth appeared first on Uber Engineering Blog.

Hadoop 109

Hadoop Systems Big Data Data Analysis 109

Data Engineering Podcast

FEBRUARY 18, 2019

Summary Distributed storage systems are the foundational layer of any big data stack. Alluxio is a distributed virtual filesystem which integrates with multiple persistent storage systems to provide a scalable, in-memory storage layer for scaling computational workloads independent of the size of your data.

Systems 100

Systems Java Media Algorithm 100

Palantir

APRIL 10, 2023

These run the risk of taking critical systems offline without ever accomplishing the necessary operational outcomes. Now that FCE allows containerized code to run in Foundry, unlocking the full value of legacy systems has never been easier.

Systems 52

Systems Engineering Data Lake Coding 52

Scott Logic

NOVEMBER 15, 2023

Until now, you’ve probably expected all your tests to pass, but what happens when you test systems with inherent inbuilt randomness? You may think this will require you to run many tests, but this blog will give you a method for ensuring that we extract statistically valid results and how to do it quickly. What is the t-test?

Systems 52

Systems Architecture IT Data 52

The Pragmatic Engineer

JUNE 1, 2023

Juraj included system monitoring parts which monitor the server’s capacity he runs the app on: The monitoring page on the Rides app And it doesn’t end here. Juraj created a systems design explainer on how he built this project, and the technologies used: The systems design diagram for the Rides application The app uses: Node.js

Education 363

Education Project PostgreSQL Software Engineer 363

Rockset

SEPTEMBER 2, 2022

I recently had the good fortune to host a small-group discussion on personalization and recommendation systems with two technical experts with years of experience at FAANG and other web-scale companies. Garg also blogs regularly on real-time data and recommendation systems – read and subscribe here. That’s not machine learning.

Systems 52

Systems Building Machine Learning NoSQL 52

Uber Engineering

AUGUST 7, 2019

As we scaled up to our present level of support for 14 million trips per day, the car in that … The post Migrating Functionality Between Large-scale Production Systems Seamlessly appeared first on Uber Engineering Blog.

Systems 84

Systems Engineering Building IT 84

U-Next

OCTOBER 27, 2022

An MIS ( Management Information Systems ) executive is responsible for the management of an organization’s computer systems, applications, and networks. In this blog, we will break down the MIS Executive salary in India based on these factors, making it easier for you to negotiate for a fair amount. .

Systems 52

Systems Management Telecommunication Technology 52

The Pragmatic Engineer

APRIL 9, 2024

Context My initial motivation with the job board was to add a kind of "helpful monetization" to both my blog, and The Pragmatic Engineer Newsletter. In the blog – and at the end of all free newsletter issues – jobs posted by customers of the job board were shown.

Software Engineer 253

Software Engineer Software Engineering Engineering Recruitment 253

databricks

DECEMBER 21, 2023

We are excited to announce that Gartner has recognized Databricks as a Leader for a third consecutive year in the 2023 Gartner® Magic.

Systems 132

Systems Database Cloud Management 132

databricks

SEPTEMBER 12, 2023

As the lakehouse becomes increasingly mission-critical to data-forward organizations, so too grows the risk that unexpected events, outages, and security incidents may derail.

Systems 84

Systems Data 84

Data Engineering Podcast

APRIL 21, 2024

How do you manage the personalization of the AI functionality in your system for each user/team? Contact Info LinkedIn Blog Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? When is Shortwave the wrong choice? What do you have planned for the future of Shortwave?

Data Lake 173

Data Lake High Quality Data Data Pipeline Machine Learning 173

databricks

FEBRUARY 29, 2024

Special thanks to Phillip Jones, Senior Product Manager, and Harshal Brahmbhatt, Systems Engineer from Cloudflare for their contributions to this blog. Organizations across.

Engineering 119

Engineering Systems Management 119

databricks

JUNE 13, 2023

We are incredibly excited to announce that the team behind Rubicon is joining Databricks. Founded by large scale infrastructure builders, Akhil Gupta and.

Systems 81

Systems 81

Data Engineering Weekly

NOVEMBER 11, 2022

Not too long ago, I worked for a famous messaging platform; You may be typing on it as you read this blog. I took a break from data engineering and focused on system monitoring and observability for some time. Little did I know I would be building data pipelines to measure system efficiency instead of measuring business efficiency.

Systems 40

Systems High Quality Data Consulting Metadata 40

Netflix Tech

MAY 4, 2023

Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations. This blog series will examine the tools, techniques, and strategies we have utilized to achieve this goal.

Utilities 135

Utilities Systems Architecture Coding 135

Yelp Engineering

MARCH 25, 2024

In this blog post, we highlight how phone number masking helps build consumer trust in the services marketplace at Yelp, decreases the friction in communication with service professionals, and allows for seamless switching between the Yelp app and a user’s phone.

Project 103

Project Consulting Engineering Systems 103

Rockset

JULY 6, 2022

This is the fifth post in a series by Rockset's CTO and Co-founder Dhruba Borthakur on Designing the Next Generation of Data Systems for Real-Time Analytics. We'll be publishing more posts in the series in the near future, so subscribe to our blog so you don't miss them!

NoSQL 52

NoSQL SQL Systems PostgreSQL 52

Data Engineering Weekly

MARCH 31, 2024

The blog further emphasizes its increased investment in Data Mesh and clean data. The blog is an excellent overview of all the improvements made to PySpark in 2023. The blog narrates how Uber utilizes the existing infrastructure to support various AI/ML use cases.

Data Engineering 62

Data Engineering Data Engineer Engineering Scala 62

Engineering at Meta

OCTOBER 23, 2023

To optimize build and performance, we developed our own build system called Buck , which was first open-sourced in 2013. We debated between using Java (like Buck1), Haskell (like the Shake build system ) or Go for the core programming language. Meta has a very large monorepo, with many different programming languages.

Programming Language 109

Programming Language Java Python Programming 109