DoorDash Engineering

FEBRUARY 27, 2024

Two primary aspects of that search engine were causing the trouble: its document-replication mechanism and its lack of support for complex document relationships. We decided the best way to address these challenges was to move away from Elasticsearch to a homegrown search engine. latency reduction and a 75% hardware cost decrease.

Engineering 129

Engineering Systems Designing Architecture 129

The Pragmatic Engineer

AUGUST 10, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In what is neat: The Pragmatic Engineer is now the first-ever newsletter listed on Learnerbly.

Engineering 191

Engineering Recruitment Retail Software Engineer 191

Jesse Anderson

SEPTEMBER 14, 2023

I haven’t seen much from the data engineering side. that summarizes blog posts using LLMs. How can we use LLMs in data engineering? This will make senior engineers more productive. I explain to non-technical people that code generation is the most difficult part of software engineering. Be careful.

Data Engineering 147

Data Engineering Data Engineer Engineering NoSQL 147

Data Engineering Weekly

APRIL 21, 2024

[link] Stripe: Shepherd - How Stripe adapted Chronon to scale ML feature development Airbnb recently open-sourced Chronon , a declarative feature engineering framework. The blog narrates how Chronon fits into Stripe’s online and offline requirements. Counting is the hardest problem in data engineering.

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

The Pragmatic Engineer

JANUARY 12, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. Here are a few details from the data points, focusing on software engineering compensation. Source: The Pragmatic Engineer blog. Source: The Pragmatic Engineer blog. To get this newsletter every week, subscribe here.

Software Engineer 183

Software Engineer Software Engineering Engineering Raw Data 183

Data Engineering Weekly

APRIL 14, 2024

link] Github: 4 ways GitHub engineers use GitHub Copilot The impact of LLM on software development is undeniable. Github shares some insights on how Github engineers use Github Copilot. The blog post brings in-depth insights into how Apache Hudi works on the right side, read side & underlying storage formats.

Data Engineering 62

Data Engineering Data Engineer Engineering Business Intelligence 62

Zalando Engineering

JUNE 30, 2020

Our Engineering Blog was launched in June 2020 after a long break of the previous tech blog. What customizations we applied to design the blog and the publishing process. Static Site Generator Our previous tech blog used a CMS which only a limited number of people had access to. It's actively developed.

Engineering 40

Engineering Bytes AWS Python 40

Data Engineering Weekly

APRIL 28, 2024

Intuit: The Data Mesh Strategy Behind Intuit’s Global Financial Technology Platform The Data Product Builder platform is becoming increasingly important in enterprise data engineering. LinkedIn writes an exciting blog about what is and is not working. link] Grab: Ensuring data reliability and observability in risk systems.

Data Engineering 52

Data Engineering Data Engineer Engineering Hospitality 52

Data Engineering Weekly

APRIL 7, 2024

dbt: 2024 State of Analytics Engineering The 2024 dbt’s state of analytical engineering report is out. What will the future of software engineers be? I strongly believe the concept of Data Product will play a bigger role in data engineering. Data Mesh continuously gaining popularity among the enterprises.

Data Engineering 70

Data Engineering Data Engineer Engineering Unstructured Data 70

Waitingforcode

FEBRUARY 3, 2023

This time I won't blog about my discoveries. Instead, I'm going to see other blog posts from the data engineering space and share some key takeaways with you. Hi and welcome to the new series. I don't know how regular it will be yet but hopefully will be able to share some of the notes every month.

Data Engineering 130

Data Engineering Data Engineer Engineering Data 130

KDnuggets

NOVEMBER 15, 2021

The October blogs that won KDnuggets Rewards include: How I Tripled My Income With Data Science in 18 Months; What Google Recommends You do Before Taking Their Machine Learning or Data Science Course; How to Build Strong Data Science Portfolio as a Beginner; Data Scientist vs Data Engineer Salary.

Portfolio 94

Portfolio Data Science Machine Learning Data Engineering 94

Data Engineering Weekly

MARCH 31, 2024

The blog further emphasizes its increased investment in Data Mesh and clean data. The blog is an excellent overview of all the improvements made to PySpark in 2023. The blog narrates how Uber utilizes the existing infrastructure to support various AI/ML use cases.

Data Engineering 62

Data Engineering Data Engineer Engineering Scala 62

Data Engineering Weekly

MARCH 3, 2024

There will be food, networking, and real-world talks around data engineering. This approach led to a successful expansion of Copilot access across the engineering team, resulting in a significant increase in productivity and adoption, demonstrating a commitment to enhancing developer experience while maintaining safety and security standards.

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Monte Carlo

JANUARY 23, 2024

In the world of data engineering, Maxime Beauchemin is someone who needs no introduction. One of the first data engineers at Facebook and Airbnb, he wrote and open sourced the wildly popular orchestrator, Apache Airflow , followed shortly thereafter by Apache Superset , a data exploration tool that’s taking the data viz landscape by storm.

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Waitingforcode

FEBRUARY 3, 2023

In this blog post I'll share with you a list of Java and Scala classes I use almost every time in data engineering projects. We all have our habits and as programmers, libraries and frameworks are definitely a part of the group. The part for Python will follow next week!

Scala 130

Scala Java Data Engineering Data Engineer 130

Data Engineering Weekly

FEBRUARY 4, 2024

Joe Reis: Definition of Data Modeling & What Data Modeling Is not Joe raised a very fundamental question in data engineering. He rightly points out that many data engineers will name any of the modeling techniques as data modeling. Visit rudderstack.com to learn more. What is Data Modeling, and what is not?

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

U-Next

DECEMBER 21, 2022

As another year comes to an end, we decided the best way to look back was to revisit the most popular and sought-after blogs of Cybersecurity and list the same for all our Cybersecurity enthusiasts. Interesting Cyber Security Projects For Final Year Engineering Students 2022 . Then you are at the perfect place.

Education 105

Education Government Project Engineering 105

DoorDash Engineering

OCTOBER 31, 2023

At DoorDash, the growth and development of our engineering talent is critical to our success and ability to continue innovating. Launched in 2019, the program is a mandatory part of preparing senior engineers for promotion. The Apprentice Engineering Manager Program offers them an additional platform to hone leadership capabilities.

Programming 79

Programming Engineering Management Education 79

Jesse Anderson

JANUARY 29, 2024

The premier of my latest talk covering The State of Data Engineering. I finish off by showing how data engineering can avoid the same fate as data warehousing and data science. I go through the history of the industry to see where we’re heading. This starts with data warehousing and goes into data science.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Science 130

Knowledge Hut

DECEMBER 6, 2023

Voice technologies have played a very important role in making websites and blogs more accessible to people with disabilities and will continue to do so in the coming years. Let’s explore how we can add voice to a blog or a webpage. How can you make a web page talk or add voice to your blog? utterance.volume = 0.5

Telecommunication 52

Telecommunication Education Coding Technology 52

Data Engineering Weekly

DECEMBER 3, 2023

Github writes an excellent blog to capture the current state of the LLM integration architecture. link] Microsoft: Generative AI for Beginners Understanding Gen-AI becomes a mandatory skill for application developers and data engineers. I experienced similar drawbacks to what Lyft is talking about in Druid.

Data Engineering 70

Data Engineering Data Engineer Engineering Bytes 70

The Modern Data Company

JANUARY 31, 2023

These were our most popular blog posts in 2022 according to reader statistics. “ Thus begins another popular Modern blog post from 2022. And our blog post shows the first steps towards unifying data once and for all. If you haven’t been with us long, we had some incredible pieces in the past few years. Why is DataOS?

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

Data Engineering Weekly

FEBRUARY 25, 2024

Are you ready to take Europe's data engineering scene to the next level? link] Pinterest: Unlocking AI-Assisted Development Safely: From Idea to GA Pinterest Engineering shared its journey towards implementing AI-assisted development, focusing on balancing innovation, safety, security, and cost. 88% of people voted, Yes for DEWCon!!!

Data Engineering 62

Data Engineering Data Engineer Engineering Kafka 62

Edureka

APRIL 19, 2024

In other words, you don’t need to be a programmer or have any coding background to dive into this creative process of building a chatbot using prompt engineering. In this tutorial blog, we’ll be building a chatbot using prompt engineering inspired by Steve Harvey, offering tailored advice on life motivation and personal growth.

Building 52

Building Engineering Coding Systems 52

Jesse Anderson

DECEMBER 12, 2022

Apache Spark came in 2009 and gave a unified batch and streaming engine. Apache Flink came in 2011 and gave us our first real streaming engine. As clearly evident as that sounds now, my writing about needing data engineering went heavily against the grain of everything that was written at the time.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

DoorDash Engineering

SEPTEMBER 19, 2023

As a tech company, it’s our products and platform – and the engineers that build them – that power what DoorDash is able to offer our Consumers, Dashers, and Merchants every day. It’s for this reason that in Q2 2023, we revisited our performance expectations for all engineers at DoorDash.

Engineering 88

Engineering Recruitment Certification Data Analysis 88

Data Engineering Weekly

SEPTEMBER 24, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. The blog specifically focuses on the following areas of LLM implementations.

Data Engineering 67

Data Engineering Data Engineer Engineering Kafka 67

Data Engineering Weekly

FEBRUARY 18, 2024

One thing I learned while writing Data Engineering Weekly is that persistence and consistency are the keys to success. See how recent product updates make it easier for developers and data engineers to debug data models, work in a BI-like UI, and fine-tune deployments for better query performance. Was this simply too ambitious?

Data Engineering 62

Data Engineering Data Engineer Engineering Data 62

Uber Engineering

MARCH 2, 2023

Join us in this blog post exploring some of the innovative tools, services, and practices we have developed to promote sustainable usage of engineering resources. As a leader in tech, Uber is a major proponent of sustainability and reducing carbon emissions.

Engineering 82

Engineering 82

Data Engineering Weekly

FEBRUARY 11, 2024

The blog is an excellent guide on when and how to build a data team, the important qualities of a data team, and the organizational structure. link] Alireza Sadeghi: Open Source Data Engineering Landscape 2024 Popular open-source tools like Airbyte and Snowplow have recently changed their licenses. Let me know in the comments.

Data Engineering 62

Data Engineering Data Engineer Engineering SQL 62

Pinterest Engineering

JANUARY 26, 2024

Jeremy Krach | Staff Security Engineer, Platform Security Background Several years ago, Pinterest had a short incident due to oversights in the policy delivery engine. To learn more about engineering at Pinterest, check out the rest of our Engineering Blog and visit our Pinterest Labs site.

Engineering 75

Engineering Architecture Systems Metadata 75

dbt Developer Hub

NOVEMBER 28, 2021

Welcome to the dbt Developer Blog ​ The purpose of this blog is to double down on our long running commitment to contributing to the knowledge loop. This blog represents the exact same philosophy, just applied via the lens of written guides rather than code.

SQL 52

SQL Database Project Engineering 52

Data Engineering Weekly

OCTOBER 1, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. Learn more and reach out for early access on the blog. See how it works today.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Knowledge Hut

NOVEMBER 20, 2023

DevOps engineers are a group of influential individuals who encapsulates depth of knowledge and years of hands-on experience around a wide variety of open-source technologies and tools. DevOps engineer always shows interest and comfort working with frequent, incremental code testing and deployment.

Engineering 98

Engineering Google Cloud Portfolio Certification 98

Data Engineering Weekly

SEPTEMBER 11, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. The blog narrates the key concepts of the Kimball model and a modern outlook on the concepts.

Data Engineering 62

Data Engineering Data Engineer Engineering Kafka 62

Data Engineering Weekly

JANUARY 21, 2024

Interesting article on the impact of search engine optimization (SEO) on the quality of search engine results. The article claims that modern search engines are significantly affected by SEO strategies, with search results being biased towards those who can profit the most from specific terms.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

RandomTrees

FEBRUARY 2, 2024

GPT-based data engineering accelerators make the working of data more accessible. Overall, GPT-based data engineering accelerators are very efficient and beneficial for organizations. GPT-Based Data Engineering Accelerators: Given below is the list of some of the GPT-based data engineering accelerators. 1.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52