Jesse Anderson

SEPTEMBER 14, 2023

I haven’t seen much from the data engineering side. that summarizes blog posts using LLMs. How can we use LLMs in data engineering? This will make senior engineers more productive. I explain to non-technical people that code generation is the most difficult part of software engineering. Be careful.

Data Engineering 147

Data Engineering Data Engineer Engineering NoSQL 147

Zalando Engineering

JUNE 30, 2020

Our Engineering Blog was launched in June 2020 after a long break of the previous tech blog. What customizations we applied to design the blog and the publishing process. Static Site Generator Our previous tech blog used a CMS which only a limited number of people had access to. It's actively developed.

Engineering 40

Engineering Bytes AWS Python 40

The Pragmatic Engineer

JANUARY 12, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. Here are a few details from the data points, focusing on software engineering compensation. Source: The Pragmatic Engineer blog. Source: The Pragmatic Engineer blog. To get this newsletter every week, subscribe here.

Software Engineer 181

Software Engineer Software Engineering Engineering Raw Data 181

KDnuggets

NOVEMBER 15, 2021

The October blogs that won KDnuggets Rewards include: How I Tripled My Income With Data Science in 18 Months; What Google Recommends You do Before Taking Their Machine Learning or Data Science Course; How to Build Strong Data Science Portfolio as a Beginner; Data Scientist vs Data Engineer Salary.

Portfolio 100

Portfolio Data Science Machine Learning Data Engineering 100

Knowledge Hut

APRIL 23, 2024

Software engineering is a rapidly growing field with vast career opportunities. With emerging technologies like AI, machine learning, and blockchain, the demand for software engineers has skyrocketed. With emerging technologies like AI, machine learning, and blockchain, the demand for software engineers has skyrocketed.

Software Engineer 98

Software Engineer Software Engineering Engineering Programming Language 98

Waitingforcode

FEBRUARY 3, 2023

This time I won't blog about my discoveries. Instead, I'm going to see other blog posts from the data engineering space and share some key takeaways with you. Hi and welcome to the new series. I don't know how regular it will be yet but hopefully will be able to share some of the notes every month.

Data Engineering 130

Data Engineering Data Engineer Engineering Data 130

Data Engineering Weekly

MAY 12, 2024

link] Gradient Flow: Learning from the Past - Comparing the Hype Cycles of Big Data and GenAI The blogs compare the hype cycle of Big Data with Gen AI. The blog narrates the initial hype of Big Data, followed by talent shortages and lead time to build production applications. Ready to give DoubleCloud a try?

Data Engineering 52

Data Engineering Data Engineer Engineering Data Lake 52

Data Engineering Weekly

APRIL 14, 2024

link] Github: 4 ways GitHub engineers use GitHub Copilot The impact of LLM on software development is undeniable. Github shares some insights on how Github engineers use Github Copilot. The blog post brings in-depth insights into how Apache Hudi works on the right side, read side & underlying storage formats.

Data Engineering 62

Data Engineering Data Engineer Engineering Business Intelligence 62

Waitingforcode

FEBRUARY 3, 2023

In this blog post I'll share with you a list of Java and Scala classes I use almost every time in data engineering projects. We all have our habits and as programmers, libraries and frameworks are definitely a part of the group. The part for Python will follow next week!

Scala 130

Scala Java Data Engineering Data Engineer 130

Data Engineering Weekly

APRIL 21, 2024

[link] Stripe: Shepherd - How Stripe adapted Chronon to scale ML feature development Airbnb recently open-sourced Chronon , a declarative feature engineering framework. The blog narrates how Chronon fits into Stripe’s online and offline requirements. Counting is the hardest problem in data engineering.

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

U-Next

DECEMBER 21, 2022

As another year comes to an end, we decided the best way to look back was to revisit the most popular and sought-after blogs of Cybersecurity and list the same for all our Cybersecurity enthusiasts. Interesting Cyber Security Projects For Final Year Engineering Students 2022 . Then you are at the perfect place.

Education 105

Education Government Project Engineering 105

Data Engineering Weekly

APRIL 7, 2024

dbt: 2024 State of Analytics Engineering The 2024 dbt’s state of analytical engineering report is out. What will the future of software engineers be? I strongly believe the concept of Data Product will play a bigger role in data engineering. Data Mesh continuously gaining popularity among the enterprises.

Data Engineering 70

Data Engineering Data Engineer Engineering Unstructured Data 70

Data Engineering Weekly

MARCH 3, 2024

There will be food, networking, and real-world talks around data engineering. This approach led to a successful expansion of Copilot access across the engineering team, resulting in a significant increase in productivity and adoption, demonstrating a commitment to enhancing developer experience while maintaining safety and security standards.

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Data Engineering Weekly

FEBRUARY 4, 2024

Joe Reis: Definition of Data Modeling & What Data Modeling Is not Joe raised a very fundamental question in data engineering. He rightly points out that many data engineers will name any of the modeling techniques as data modeling. Visit rudderstack.com to learn more. What is Data Modeling, and what is not?

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

DoorDash Engineering

OCTOBER 31, 2023

At DoorDash, the growth and development of our engineering talent is critical to our success and ability to continue innovating. Launched in 2019, the program is a mandatory part of preparing senior engineers for promotion. The Apprentice Engineering Manager Program offers them an additional platform to hone leadership capabilities.

Programming 80

Programming Engineering Management Education 80

The Modern Data Company

JANUARY 31, 2023

These were our most popular blog posts in 2022 according to reader statistics. “ Thus begins another popular Modern blog post from 2022. And our blog post shows the first steps towards unifying data once and for all. If you haven’t been with us long, we had some incredible pieces in the past few years. Why is DataOS?

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

Jesse Anderson

JANUARY 29, 2024

The premier of my latest talk covering The State of Data Engineering. I finish off by showing how data engineering can avoid the same fate as data warehousing and data science. I go through the history of the industry to see where we’re heading. This starts with data warehousing and goes into data science.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Science 130

Data Engineering Weekly

DECEMBER 3, 2023

Github writes an excellent blog to capture the current state of the LLM integration architecture. link] Microsoft: Generative AI for Beginners Understanding Gen-AI becomes a mandatory skill for application developers and data engineers. I experienced similar drawbacks to what Lyft is talking about in Druid.

Data Engineering 70

Data Engineering Data Engineer Engineering Bytes 70

Jesse Anderson

DECEMBER 12, 2022

Apache Spark came in 2009 and gave a unified batch and streaming engine. Apache Flink came in 2011 and gave us our first real streaming engine. As clearly evident as that sounds now, my writing about needing data engineering went heavily against the grain of everything that was written at the time.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

dbt Developer Hub

NOVEMBER 28, 2021

Welcome to the dbt Developer Blog ​ The purpose of this blog is to double down on our long running commitment to contributing to the knowledge loop. This blog represents the exact same philosophy, just applied via the lens of written guides rather than code.

SQL 52

SQL Database Project Engineering 52

Data Engineering Weekly

SEPTEMBER 24, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. The blog specifically focuses on the following areas of LLM implementations.

Data Engineering 67

Data Engineering Data Engineer Engineering Kafka 67

Data Engineering Weekly

OCTOBER 1, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. Learn more and reach out for early access on the blog. See how it works today.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

KDnuggets

NOVEMBER 15, 2021

The October blogs that won KDnuggets Rewards include: How I Tripled My Income With Data Science in 18 Months; What Google Recommends You do Before Taking Their Machine Learning or Data Science Course; How to Build Strong Data Science Portfolio as a Beginner; Data Scientist vs Data Engineer Salary.

Portfolio 58

Portfolio Data Science Machine Learning Data Engineering 58

Data Engineering Weekly

FEBRUARY 18, 2024

One thing I learned while writing Data Engineering Weekly is that persistence and consistency are the keys to success. See how recent product updates make it easier for developers and data engineers to debug data models, work in a BI-like UI, and fine-tune deployments for better query performance. Was this simply too ambitious?

Data Engineering 62

Data Engineering Data Engineer Engineering Data 62

Data Engineering Weekly

JANUARY 21, 2024

Interesting article on the impact of search engine optimization (SEO) on the quality of search engine results. The article claims that modern search engines are significantly affected by SEO strategies, with search results being biased towards those who can profit the most from specific terms.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

RandomTrees

FEBRUARY 2, 2024

GPT-based data engineering accelerators make the working of data more accessible. Overall, GPT-based data engineering accelerators are very efficient and beneficial for organizations. GPT-Based Data Engineering Accelerators: Given below is the list of some of the GPT-based data engineering accelerators. 1.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Edureka

APRIL 19, 2024

In other words, you don’t need to be a programmer or have any coding background to dive into this creative process of building a chatbot using prompt engineering. In this tutorial blog, we’ll be building a chatbot using prompt engineering inspired by Steve Harvey, offering tailored advice on life motivation and personal growth.

Building 52

Building Engineering Coding Systems 52

Pinterest Engineering

AUGUST 3, 2023

An interview with Behnam Rezaei | Pinterest VP, Engineering At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love. Joining Pinterest in March 2023, Behnam Rezaei is Pinterest’s VP for Monetization — Machine Learning Engineering and Data Science based in San Francisco. What’s the goal of the team?

Engineering 79

Engineering Machine Learning Data Science Technology 79

Uber Engineering

MARCH 2, 2023

Join us in this blog post exploring some of the innovative tools, services, and practices we have developed to promote sustainable usage of engineering resources. As a leader in tech, Uber is a major proponent of sustainability and reducing carbon emissions.

Engineering 75

Engineering 75

Data Engineering Weekly

MARCH 10, 2024

NVIDIA's blog post on Trustworthy AI defines it as an AI development approach prioritizing safety and transparency, emphasizing the importance of developing AI systems that people can trust. Data engineers build the systems that store and process sensitive information. link] NVIDIA: What Is Trustworthy AI?

Data Engineering 52

Data Engineering Data Engineer Engineering Datasets 52

Data Engineering Weekly

NOVEMBER 5, 2023

If we had the Data Mesh SQL Processor earlier, we would’ve been able to avoid spending engineering resources to build smaller building blocks such as the Union Processor, Column Rename Processor, Projection, and Filtering Processor. That is a very wise quote from the Netflix engineering team.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Data Engineering Weekly

AUGUST 6, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. The first one that caught my eye is Astronomer published a blog post Introducing Cosmos 1.0:

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

databricks

OCTOBER 3, 2023

This blog was written in collaboration with Sukh Sekhon, Software Engineer, Cloud Infrastructure and Helen Li, Sr. Director of Engineering at Exai Bio.

Software Engineer 96

Software Engineer Software Engineering Engineering Cloud 96

Cloudera

JULY 13, 2021

After the launch of CDP Data Engineering (CDE) on AWS a few months ago, we are thrilled to announce that CDE, the only cloud-native service purpose built for enterprise data engineers, is now available on Microsoft Azure. . Prerequisites for deploying CDP Data Engineering on Azure can be found here. Easy job deployment.

Data Engineering 128

Data Engineering Data Engineer Engineering Pipeline-centric 128

Data Engineering Weekly

MARCH 19, 2023

link] Data Engineering Weekly talks in detail about adopting functional data engineering principles, and Apache Hudi certainly supports it out of the box. Data Engineering Weekly Functional Data Engineering - A Blueprint The Rise of Data Modeling Data modeling has been one of the hot topics in Data LinkedIn.

Data Engineering 57

Data Engineering Data Engineer Engineering Media 57

Cloudera

FEBRUARY 15, 2023

This month’s #ClouderaLife Spotlight features software engineer Amogh Desai. Meet Amogh Desai Amogh lives in Bangalore and joined Cloudera, first as an intern and then full-time in July of 2021 as a software engineer. Amogh has the unique experience of working on CDP Data Engineering during his internship.

Software Engineer 85

Software Engineer Software Engineering Engineering Recruitment 85

Data Engineering Weekly

AUGUST 20, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork, and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. See how it works today. 📣 Exciting news!

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52