The Pragmatic Engineer

FEBRUARY 29, 2024

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. You can imagine the engineer who was asked by a business stakeholder: “is it fixed yet?

Software Engineer 203

Software Engineer Software Engineering Banking Engineering 203

Snowflake

APRIL 17, 2024

In today’s data-driven world, developer productivity is essential for organizations to build effective and reliable products, accelerate time to value, and fuel ongoing innovation. This allows your applications to handle large data sets and complex workflows efficiently.

Data Pipeline 95

Data Pipeline Python Data Engineering Data Engineer 95

Data Engineering Podcast

NOVEMBER 6, 2022

Summary Despite the best efforts of data engineers, data is as messy as the real world. Entity resolution and fuzzy matching are powerful utilities for cleaning up data from disconnected sources, but it has typically required custom development and training machine learning models.

MongoDB 130

MongoDB Scala MySQL Data Lake 130

KDnuggets

OCTOBER 24, 2022

Preprocessing data for machine learning models is a core general skill for any Data Scientist or Machine Learning Engineer. Follow this guide using Pandas and Scikit-learn to improve your techniques and make sure your data leads to the best possible outcome.

Python 160

Python Machine Learning Data Engineering 160

Pinterest Engineering

FEBRUARY 22, 2024

Staff Software Engineer At Pinterest we are continuously looking for ways to improve our developer experience, and we have recently shipped AI-assisted development for everyone while balancing safety, security, and cost. We looked at both qualitative and quantitative data — and spent time sampling real-time user feedback.

Scala 78

Scala Engineering Software Engineer Software Engineering 78

Tweag

APRIL 17, 2024

And Nix was in general a great entry point for conversations at this year’s Kubecon. IDPs don’t replace other tools but offer a unified interface, usually with customized presets, to interact with repositories, internal data and infrastructure. No better way to start the day than to meet with fellow Nix users!

Cloud 76

Cloud Consulting Portfolio Programming 76

Netflix Tech

DECEMBER 14, 2023

Engineers from across the company came together to share best practices on everything from Data Processing Patterns to Building Reliable Data Pipelines. The result was a series of talks which we are now sharing with the rest of the Data Engineering community!

Data Engineering 103

Data Engineering Data Engineer Engineering Metadata 103

Knowledge Hut

APRIL 26, 2024

Machine Learning is a sub-branch of Artificial Intelligence, used for the analysis of data. It learns from the data that is input and predicts the output from the data rather than being explicitly programmed. If ML algorithms consume more amounts of data while solving a problem, it results in better performance.

Machine Learning 98

Machine Learning Algorithm Deep Learning Programming Language 98

Snowflake

MARCH 5, 2024

Because human-machine interaction using natural language is now possible with large language models (LLMs), more data teams and developers can bring AI to their daily workflows. So how does Snowflake Cortex make AI easy, whether you are doing prompt engineering or RAG? Let’s dive into the details and check out some code along the way.

Government 84

Government AWS Data Preparation Data Governance 84

Knowledge Hut

JANUARY 19, 2024

"Data is the pollution problem of the information age, and protecting privacy is the environmental challenge" — Bruce Schneier. It is a general-purpose and high-level program language for hacking. It is a straightforward widely-used open source general-purpose scripting language. Its speed is the biggest strength.

Programming Language 98

Programming Language Programming Java SQL 98

LinkedIn Engineering

DECEMBER 19, 2023

We found that the best way to answer this question is through data, usually meaning metrics and feedback systems. When your data systems are in place, you’ll need to translate that data into concrete metrics, reports, and visualizations.

Data Schemas 121

Data Schemas Software Engineer Software Engineering Data Pipeline 121

Knowledge Hut

NOVEMBER 28, 2023

As a software engineer , you will be responsible for developing and maintaining software applications. They are also constantly innovating and expanding, which creates opportunities for software engineers to grow their skills and careers. The top companies for software engineers include Apple, Microsoft, and Uber.

Software Engineer 97

Software Engineer Software Engineering Engineering Electronics 97

LinkedIn Engineering

JUNE 20, 2023

Co-authors: Shivansh Mundra , Gonzalo Aniano Porcile , Smit Marvaniya , Hany Farid A core part of what we do on the Trust Data Team at LinkedIn is create, deploy, and maintain models that detect and prevent many types of abuse. Our approaches learn simple/compact embeddings to capture the regularities in GAN-generated photos.

Media 132

Media Datasets Programming Engineering 132

Knowledge Hut

MARCH 28, 2024

Netflix , the streaming giant, is not just a platform for binge-watching your favorite shows; it's also a tech powerhouse with a rigorous interview process for software engineer roles. Preparing for a Netflix software engineer interview is crucial to landing your dream job at one of the most innovative companies in the world.

Software Engineer 52

Software Engineer Software Engineering Engineering Algorithm 52

Knowledge Hut

DECEMBER 29, 2023

The market for analytics is flourishing, as is the usage of the phrase Data Science. Professionals from a variety of disciplines use data in their day-to-day operations and feel the need to understand cutting-edge technology to get maximum insights from the data, therefore contributing to the growth of the organization.

Data Science 98

Data Science Data Mining Deep Learning Programming Language 98

databricks

AUGUST 13, 2023

Today we are announcing the general availability of Delta Live Tables (DLT) on Google Cloud. DLT pipelines empower data engineers to build reliable.

Google Cloud 81

Google Cloud Cloud Data Engineering Data Engineer 81

Cloudera

DECEMBER 13, 2023

Legal & General Investment Management (LGIM) is one of the largest global asset managers, managing £1.2 Implementing a cohesive data vision LGIM’s Global Chief Operating Office is responsible for the company’s technology, data, client servicing and the management of its strategic operating agenda.

Pipeline-centric 88

Pipeline-centric Professional Services Datasets BI 88

Rockset

JANUARY 30, 2024

With this architecture, users can separate ingestion compute from query compute, all while accessing the same real-time data. General purpose instance class Rockset introduces the concept of an instance class , or different ratios of compute and memory resources for virtual instances. minutes to batch load the data.

Data Ingestion 64

Data Ingestion Utilities Architecture SQL 64

Knowledge Hut

MARCH 21, 2024

For aspiring software engineers, landing a job at Microsoft can be a dream come true. The journey to becoming a Microsoft software engineer involves a comprehensive interview process designed to assess candidates thoroughly. What to Expect in a Microsoft Software Engineer Interview Process?

Software Engineer 52

Software Engineer Software Engineering Process Engineering 52

Knowledge Hut

MARCH 29, 2024

One of the most important steps in joining one of the world’s most inventive tech businesses is the Apple software engineer interview process. Who is a Software Engineer? Software engineers are in charge of designing, developing, testing, deploying, maintaining, and enhancing the company’s software.

Software Engineer 52

Software Engineer Software Engineering Process Engineering 52

Knowledge Hut

JANUARY 18, 2024

Most organizations are still in the initial stages of learning how to apply data science to gain business benefits and healthy returns. To maximize the benefits of data science, people need to have good technical knowledge of managing the data in a production environment, without which the awareness of full stack data science is insufficient.

Data Science 96

Data Science Computer Science Programming Language Machine Learning 96

Knowledge Hut

MARCH 29, 2024

It can be difficult to distinguish between a software engineer and a software architect in the digital era. This post will clarify the argument between software engineers and architects and assist you in determining which would be most appropriate for you. What is a Software Engineer? What is a Software Architect?

Software Engineer 52

Software Engineer Software Engineering Engineering Programming Language 52

Snowflake

APRIL 8, 2024

The list of Top 10 semi-finalists is a perfect example: we have use cases for cybersecurity, gen AI, food safety, restaurant chain pricing, quantitative trading analytics, geospatial data, sales pipeline measurement, marketing tech and healthcare. Our sincere thanks go out to everyone who participated in this year’s competition.

Pipeline-centric 117

Pipeline-centric Food Healthcare Unstructured Data 117

Knowledge Hut

JANUARY 12, 2024

Well, in a technologically advanced world, Cloud refers to a place where you can store and manage data on a device. Mostly, all companies have started using the Cloud for storing and managing data. Data centers and warehouses typically operate cloud computing systems of databases and software. What is Cloud Computing?

Cloud Computing 98

Cloud Computing Cloud Computer Science Programming Language 98

Monte Carlo

OCTOBER 31, 2023

I’m not sure what’s harder to believe – that we’re just a week away from IMPACT OR that it’s our third iteration of the world’s only Data Observability Summit ! 2023 has been a crazy year in data, with the rise of GenAI and LLMs eclipsing nearly everything else in tech. Another AI-focused session not to miss?

Data Engineering 72

Data Engineering Data Engineer Data Engineering 72

Knowledge Hut

JANUARY 19, 2024

Adding data online has led to many businesses being wary of certain security flaws that could lead to hackers infiltrating this information. Ethical hacking refers to much more than stealing data and cracking passwords. This mainly refers to operating systems that are generally based on Linux Kernel.

Programming Language 52

Programming Language Media Database Certification 52

Towards Data Science

AUGUST 22, 2023

Advice from a startup founder in the data space on how to find a startup that works for you Photo by Leeloo Thefirst from Pexels.com So you want to join a startup huh? As a founder of a VC funded startup in the data / technology space, I’ve seen a lot of things and have spoken to a lot of other founders. Sounds like fun.

Data Engineering 92

Data Engineering Data Engineer Engineering Data Pipeline 92

Ascend.io

NOVEMBER 28, 2023

The terms ‘data orchestration’ and ‘data pipeline orchestration’ are often used interchangeably, yet they diverge significantly in function and scope. Data orchestration refers to a wide collection of methods and tools that coordinate any and all types of data-related computing tasks.

Data Pipeline 52

Data Pipeline IT Data Metadata 52

Knowledge Hut

NOVEMBER 30, 2023

Are you interested in knowing how to become a data scientist with no experience  but not sure how to go about it? Here you will learn how to get your first data science job. To get a data science job without prior experience, you can first pursue o n line courses in Data Science. How to get a job as a data scientist ?

Data Science 98

Data Science Portfolio Programming Language Programming 98

Netflix Tech

MARCH 7, 2024

Berg , Romain Cledat , Kayla Seeley , Shashank Srikanth , Chaoying Wang , Darin Yu Netflix uses data science and machine learning across all facets of the company, powering a wide range of business applications from our internal infrastructure and content demand modeling to media understanding.

Systems 90

Systems Media Machine Learning Data Warehouse 90

Knowledge Hut

JANUARY 18, 2024

Speaking of job vacancies, the two careers have high demands till date and in upcoming years are Data Scientist and a Software Engineer. Per the BLS, the expected growth rate of job vacancies for data scientists and software engineers is around 22% by 2030. What is Data Science? Get data and analyze the problem.

Software Engineer 97

Software Engineer Software Engineering Data Science Engineering 97

Knowledge Hut

MARCH 13, 2024

The search engineer giant provides a platform for many exceptional software engineers beyond problem-solving. As an experienced software engineer with the skills to solve problems I have not encountered before, I have some important tips to share. Who is a Google Software Engineer? What Do Google Software Engineers Do?

Software Engineer 52

Software Engineer Software Engineering Process Engineering 52

Data Engineering Weekly

MARCH 3, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Editor’s Note: Chennai, India Meetup - March-08 Update We are thankful to Ideas2IT to host our first Data Hero’s meetup.

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Knowledge Hut

MARCH 14, 2024

What is Computer Science A more general term that includes nearly all software-related and related fields is computer science. Parameters Computer Science Artificial Intelligence Definition Broad field encompassing various aspects of computing such as algorithms, data structures, software development, and more.

Computer Science 52

Computer Science Programming Language Electronics MongoDB 52

Knowledge Hut

DECEMBER 27, 2023

What will be the key skills for a software engineer in 2024? The top software developers stand out from the crowd thanks to their extensive technical skills as software engineers. Thus, to become a highly sought-after software developer, you must increase and update the technical skills required for software engineering.

Software Engineer 98

Software Engineer Software Engineering Engineering Programming Language 98

Knowledge Hut

MARCH 20, 2024

In fact, with Google receiving over 3 million job applications in a year, your application for the position of Software engineer must be attractive enough to make the cut. Without further delay, let us now answer your burning question, “How to get a job in Google as a software engineer?” ” How do I apply at Google?

Software Engineer 52

Software Engineer Software Engineering Engineering Recruitment 52

Snowflake

AUGUST 16, 2023

Applications Snowflake Native App Framework now available in AWS – public preview Snowflake Native Apps are an entirely new way to put data to work. Streaming and Continuous Pipelines & Data Onboarding Snowpipe Streaming – general availability Streaming pipelines are often challenging to build. Learn more here.

Scala 52

Scala Transportation Kafka Data Lake 52