Data Engineering Weekly

APRIL 21, 2024

[link] Stripe: Shepherd - How Stripe adapted Chronon to scale ML feature development Airbnb recently open-sourced Chronon , a declarative feature engineering framework. The blog narrates how Chronon fits into Stripe’s online and offline requirements. Counting is the hardest problem in data engineering.

Data Engineering 52

Data Engineering Data Engineer Engineering Medical 52

Data Engineering Weekly

APRIL 14, 2024

link] Github: 4 ways GitHub engineers use GitHub Copilot The impact of LLM on software development is undeniable. Github shares some insights on how Github engineers use Github Copilot. The blog post brings in-depth insights into how Apache Hudi works on the right side, read side & underlying storage formats.

Data Engineering 62

Data Engineering Data Engineer Engineering Business Intelligence 62

The Pragmatic Engineer

JANUARY 12, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. Here are a few details from the data points, focusing on software engineering compensation. Source: The Pragmatic Engineer blog. Source: The Pragmatic Engineer blog. To get this newsletter every week, subscribe here.

Software Engineer 183

Software Engineer Software Engineering Engineering Raw Data 183

Zalando Engineering

JUNE 30, 2020

Our Engineering Blog was launched in June 2020 after a long break of the previous tech blog. What customizations we applied to design the blog and the publishing process. Static Site Generator Our previous tech blog used a CMS which only a limited number of people had access to. It's actively developed.

Engineering 40

Engineering Bytes AWS Python 40

Data Engineering Weekly

APRIL 7, 2024

dbt: 2024 State of Analytics Engineering The 2024 dbt’s state of analytical engineering report is out. What will the future of software engineers be? I strongly believe the concept of Data Product will play a bigger role in data engineering. Data Mesh continuously gaining popularity among the enterprises.

Data Engineering 70

Data Engineering Data Engineer Engineering Unstructured Data 70

KDnuggets

NOVEMBER 15, 2021

The October blogs that won KDnuggets Rewards include: How I Tripled My Income With Data Science in 18 Months; What Google Recommends You do Before Taking Their Machine Learning or Data Science Course; How to Build Strong Data Science Portfolio as a Beginner; Data Scientist vs Data Engineer Salary.

Portfolio 97

Portfolio Data Science Machine Learning Data Engineering 97

Waitingforcode

FEBRUARY 3, 2023

This time I won't blog about my discoveries. Instead, I'm going to see other blog posts from the data engineering space and share some key takeaways with you. Hi and welcome to the new series. I don't know how regular it will be yet but hopefully will be able to share some of the notes every month.

Data Engineering 130

Data Engineering Data Engineer Engineering Data 130

Data Engineering Weekly

MARCH 3, 2024

There will be food, networking, and real-world talks around data engineering. This approach led to a successful expansion of Copilot access across the engineering team, resulting in a significant increase in productivity and adoption, demonstrating a commitment to enhancing developer experience while maintaining safety and security standards.

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Waitingforcode

FEBRUARY 3, 2023

In this blog post I'll share with you a list of Java and Scala classes I use almost every time in data engineering projects. We all have our habits and as programmers, libraries and frameworks are definitely a part of the group. The part for Python will follow next week!

Scala 130

Scala Java Data Engineering Data Engineer 130

Data Engineering Weekly

FEBRUARY 4, 2024

Joe Reis: Definition of Data Modeling & What Data Modeling Is not Joe raised a very fundamental question in data engineering. He rightly points out that many data engineers will name any of the modeling techniques as data modeling. Visit rudderstack.com to learn more. What is Data Modeling, and what is not?

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

U-Next

DECEMBER 21, 2022

As another year comes to an end, we decided the best way to look back was to revisit the most popular and sought-after blogs of Cybersecurity and list the same for all our Cybersecurity enthusiasts. Interesting Cyber Security Projects For Final Year Engineering Students 2022 . Then you are at the perfect place.

Education 105

Education Government Project Engineering 105

DoorDash Engineering

OCTOBER 31, 2023

At DoorDash, the growth and development of our engineering talent is critical to our success and ability to continue innovating. Launched in 2019, the program is a mandatory part of preparing senior engineers for promotion. The Apprentice Engineering Manager Program offers them an additional platform to hone leadership capabilities.

Programming 79

Programming Engineering Management Education 79

Jesse Anderson

JANUARY 29, 2024

The premier of my latest talk covering The State of Data Engineering. I finish off by showing how data engineering can avoid the same fate as data warehousing and data science. I go through the history of the industry to see where we’re heading. This starts with data warehousing and goes into data science.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Science 130

Data Engineering Weekly

DECEMBER 3, 2023

Github writes an excellent blog to capture the current state of the LLM integration architecture. link] Microsoft: Generative AI for Beginners Understanding Gen-AI becomes a mandatory skill for application developers and data engineers. I experienced similar drawbacks to what Lyft is talking about in Druid.

Data Engineering 70

Data Engineering Data Engineer Engineering Bytes 70

The Modern Data Company

JANUARY 31, 2023

These were our most popular blog posts in 2022 according to reader statistics. “ Thus begins another popular Modern blog post from 2022. And our blog post shows the first steps towards unifying data once and for all. If you haven’t been with us long, we had some incredible pieces in the past few years. Why is DataOS?

Retail 98

Retail Healthcare Data Architecture Data Pipeline 98

Edureka

APRIL 19, 2024

In other words, you don’t need to be a programmer or have any coding background to dive into this creative process of building a chatbot using prompt engineering. In this tutorial blog, we’ll be building a chatbot using prompt engineering inspired by Steve Harvey, offering tailored advice on life motivation and personal growth.

Building 52

Building Engineering Coding Systems 52

Jesse Anderson

DECEMBER 12, 2022

Apache Spark came in 2009 and gave a unified batch and streaming engine. Apache Flink came in 2011 and gave us our first real streaming engine. As clearly evident as that sounds now, my writing about needing data engineering went heavily against the grain of everything that was written at the time.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Data Engineering Weekly

FEBRUARY 18, 2024

One thing I learned while writing Data Engineering Weekly is that persistence and consistency are the keys to success. See how recent product updates make it easier for developers and data engineers to debug data models, work in a BI-like UI, and fine-tune deployments for better query performance. Was this simply too ambitious?

Data Engineering 62

Data Engineering Data Engineer Engineering Data 62

Data Engineering Weekly

SEPTEMBER 24, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. The blog specifically focuses on the following areas of LLM implementations.

Data Engineering 67

Data Engineering Data Engineer Engineering Kafka 67

Uber Engineering

MARCH 2, 2023

Join us in this blog post exploring some of the innovative tools, services, and practices we have developed to promote sustainable usage of engineering resources. As a leader in tech, Uber is a major proponent of sustainability and reducing carbon emissions.

Engineering 90

Engineering 90

dbt Developer Hub

NOVEMBER 28, 2021

Welcome to the dbt Developer Blog ​ The purpose of this blog is to double down on our long running commitment to contributing to the knowledge loop. This blog represents the exact same philosophy, just applied via the lens of written guides rather than code.

SQL 52

SQL Database Project Coding 52

Data Engineering Weekly

OCTOBER 1, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. Learn more and reach out for early access on the blog. See how it works today.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Data Engineering Weekly

JANUARY 21, 2024

Interesting article on the impact of search engine optimization (SEO) on the quality of search engine results. The article claims that modern search engines are significantly affected by SEO strategies, with search results being biased towards those who can profit the most from specific terms.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

KDnuggets

NOVEMBER 15, 2021

The October blogs that won KDnuggets Rewards include: How I Tripled My Income With Data Science in 18 Months; What Google Recommends You do Before Taking Their Machine Learning or Data Science Course; How to Build Strong Data Science Portfolio as a Beginner; Data Scientist vs Data Engineer Salary.

Portfolio 57

Portfolio Data Science Machine Learning Data Engineering 57

RandomTrees

FEBRUARY 2, 2024

GPT-based data engineering accelerators make the working of data more accessible. Overall, GPT-based data engineering accelerators are very efficient and beneficial for organizations. GPT-Based Data Engineering Accelerators: Given below is the list of some of the GPT-based data engineering accelerators. 1.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Data Engineering Weekly

MARCH 10, 2024

NVIDIA's blog post on Trustworthy AI defines it as an AI development approach prioritizing safety and transparency, emphasizing the importance of developing AI systems that people can trust. Data engineers build the systems that store and process sensitive information. link] NVIDIA: What Is Trustworthy AI?

Data Engineering 52

Data Engineering Data Engineer Engineering Datasets 52

Edureka

APRIL 19, 2024

Fundamentally, social engineering is not the same as a cyberattack. Rather, social engineering relies heavily on the psychology of convincing, attacking the mind like a classic with artists. Even though recovering from a social engineering attack is much more difficult, preventing one can have far less disastrous effects.

Engineering 52

Engineering Media Banking Education 52

Data Engineering Weekly

NOVEMBER 5, 2023

If we had the Data Mesh SQL Processor earlier, we would’ve been able to avoid spending engineering resources to build smaller building blocks such as the Union Processor, Column Rename Processor, Projection, and Filtering Processor. That is a very wise quote from the Netflix engineering team.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Pinterest Engineering

AUGUST 3, 2023

An interview with Behnam Rezaei | Pinterest VP, Engineering At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love. Joining Pinterest in March 2023, Behnam Rezaei is Pinterest’s VP for Monetization — Machine Learning Engineering and Data Science based in San Francisco. What’s the goal of the team?

Engineering 77

Engineering Machine Learning Data Science Technology 77

Data Engineering Weekly

AUGUST 6, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. The first one that caught my eye is Astronomer published a blog post Introducing Cosmos 1.0:

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Data Engineering Weekly

AUGUST 20, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork, and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. See how it works today. 📣 Exciting news!

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Data Engineering Weekly

MARCH 19, 2023

link] Data Engineering Weekly talks in detail about adopting functional data engineering principles, and Apache Hudi certainly supports it out of the box. Data Engineering Weekly Functional Data Engineering - A Blueprint The Rise of Data Modeling Data modeling has been one of the hot topics in Data LinkedIn.

Data Engineering 57

Data Engineering Data Engineer Engineering Media 57

Knowledge Hut

MARCH 29, 2024

One of the most important steps in joining one of the world’s most inventive tech businesses is the Apple software engineer interview process. Who is a Software Engineer? Software engineers are in charge of designing, developing, testing, deploying, maintaining, and enhancing the company’s software.

Software Engineer 52

Software Engineer Software Engineering Process Engineering 52

Knowledge Hut

MARCH 20, 2024

Because of the surge in cyber-attacks, there is a massive demand for certified network security engineers. Thus, if you're interested in starting a career in cyber security or if you've already started one and want to know more about network security engineer salary and other pertinent information, read the blog through to the end.

Engineering 52

Engineering Banking Amazon Web Services Certification 52

Knowledge Hut

MARCH 19, 2024

Machine Learning Software Engineers are at the forefront of this revolution, applying their expertise to develop intelligent systems and algorithms. In this blog, I will describe the role of a Machine Learning Software Engineer, their responsibilities, required skills, and the path to becoming one.

Software Engineer 52

Software Engineer Software Engineering Machine Learning Engineering 52

Knowledge Hut

MARCH 29, 2024

A Manufacturing Engineer’s job is no easy task. And when it comes to Manufacturing Engineer wages, I have quite the secrets for you! However, to be honest, as complex as this job role is, acquiring an attractive Manufacturing Engineer salary can be equally challenging for fresh graduates. Who Is a Manufacturing Engineer?

Manufacturing 52

Manufacturing Engineering Certification Programming Language 52

Knowledge Hut

MARCH 20, 2024

The rise in demand for automation software engineers has become indispensable across industries. Automation Software Engineers are at the forefront of this revolution, playing a crucial role in developing and implementing automated solutions to enhance efficiency and productivity. Who is an Automation Software Engineer ?

Software Engineer 52

Software Engineer Software Engineering Engineering Programming Language 52