Jesse Anderson

SEPTEMBER 14, 2023

I haven’t seen much from the data engineering side. How can we use LLMs in data engineering? This will make senior engineers more productive. I explain to non-technical people that code generation is the most difficult part of software engineering. That isn’t the most difficult part of software engineering.

Data Engineering 147

Data Engineering Data Engineer Engineering Data 147

The Pragmatic Engineer

APRIL 19, 2023

When I think back on the software engineers I looked up to, they all shared this trait where they never took anything at face value. After the meeting, I asked other engineers in private who were nodding along if they knew what idempotency was. Some senior+ engineers loved using jargon, and used it all the time.

Software Engineer 317

Software Engineer Software Engineering Engineering Media 317

Seattle Data Guy

JANUARY 17, 2023

With all the recent data events I have put together I inevitably run into new data engineers who are either finishing up college or looking to transition into a data engineer or data scientist position. In fact I have talked to several newly graduated engineers who are struggling to find work.

Data Engineering 223

Data Engineering Data Engineer Engineering Data 223

Analytics Vidhya

JUNE 20, 2023

Traditional data engineering solutions, such as Apache Airflow, have played an important role in orchestrating and controlling data operations in order to tackle these difficulties.

Data Engineering 217

Data Engineering Data Engineer Data Process Engineering 217

Analytics Vidhya

APRIL 3, 2023

Nevertheless, setting up a streaming data pipeline to power such dashboards may […] The post Data Engineering for Streaming Data on GCP appeared first on Analytics Vidhya. Real-time dashboards such as GCP provide strong data visualization and actionable information for decision-makers.

Data Engineering 250

Data Engineering Data Engineer Engineering Data 250

Confessions of a Data Guy

AUGUST 11, 2023

After working in and around Data Teams for well over a decade, with both the smartest people to touch the keyboard, and the others, it’s become quite clear to me what the number one skill that identifies a Senior level Engineering […] The post Senior Engineer – The Number One Skill appeared first on Confessions of a Data Guy.

Engineering 130

Engineering Data Data Engineering Data Engineer 130

Analytics Vidhya

FEBRUARY 7, 2023

Introduction Data engineering is the field of study that deals with the design, construction, deployment, and maintenance of data processing systems. This includes designing and implementing […] The post Most Essential 2023 Interview Questions on Data Engineering appeared first on Analytics Vidhya.

Data Engineering 240

Data Engineering Data Engineer Engineering Data 240

LinkedIn Engineering

SEPTEMBER 20, 2023

Since learning frontend and backend skills, Rishika’s passion for engineering has expanded beyond her team at LinkedIn to grow into her own digital community. As she develops as an engineer, giving back has become the most rewarding part of her role. And the enthusiasm from my team inspires me to keep going!

Engineering 95

Engineering Java Programming Language Software Engineer 95

Medium Data Engineering

OCTOBER 3, 2023

The practice of data engineering in digital product engineering, involving data collection, transformation, and organization for analysis… Continue reading on Medium »

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Medium Data Engineering

SEPTEMBER 9, 2023

Data engineering is the backbone of modern data-driven businesses. Data engineers design, build, and maintain the infrastructure that… Continue reading on The Resume Whisperer »

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Medium Data Engineering

SEPTEMBER 11, 2023

During an SQL coding screen in a data engineer interview, you can typically expect to be presented with real-world scenarios or problems… Continue reading on Medium »

Data Engineering 98

Data Engineering Data Engineer SQL Engineering 98

DoorDash Engineering

SEPTEMBER 19, 2023

As an Engineering org, we are tremendously proud of our accomplishments throughout the history of DoorDash, particularly in recent years as we’ve grown and scaled in service of our customers. Business Outcome comprises two subcategories: Direction and Delivery. Team Building measures how effectively we are building and shaping our teams.

Engineering 80

Engineering Management Building Process 80

Data Engineering Weekly

OCTOBER 1, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. I wish more data engineers would start sharing their on-call experience soon.

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Medium Data Engineering

SEPTEMBER 22, 2023

Data engineering is the practice of developing, implementing, and maintaining systems and processes to collect, clean, store, and prepare… Continue reading on Medium »

Data Engineering 52

Data Engineering Data Engineer Engineering AWS 52

Medium Data Engineering

MAY 12, 2023

Prompt engineering is the art and science of designing prompts that elicit the desired responses from a natural language generation (NLG)… Continue reading on Medium »

Engineering 98

Engineering Designing Computer Science Data Engineering 98

Medium Data Engineering

OCTOBER 2, 2023

If you work as a data engineer, data analytics, or data scientist in an organization that needs you on a project and are using a pretty… Continue reading on Medium »

Data Engineering 52

Data Engineering Data Engineer Data Pipeline Engineering 52

DoorDash Engineering

SEPTEMBER 19, 2023

As a tech company, it’s our products and platform – and the engineers that build them – that power what DoorDash is able to offer our Consumers, Dashers, and Merchants every day. It’s for this reason that in Q2 2023, we revisited our performance expectations for all engineers at DoorDash.

Engineering 83

Engineering Recruitment Coding Data Analysis 83

Medium Data Engineering

SEPTEMBER 6, 2023

There is a lot of variance in the types of coding challenges you will face in a data engineering interview. I’ve compiled a number of… Continue reading on Medium »

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Medium Data Engineering

AUGUST 19, 2023

In my experience the coding screen for data engineers can range wildly, from non-existent, to algorithmic challenges in line with software… Continue reading on Medium »

Data Engineering 94

Data Engineering Data Engineer Coding Engineering 94

Towards Data Science

MAY 22, 2023

Solving data preparation tasks with ChatGPT Photo by Ricardo Gomez Angel on Unsplash Data engineering makes up a large part of the data science process. In our article we solve typical data engineering tasks using ChatGPT and Python. Specifying an action to be performed by an LLM is referred to as “prompt engineering”.

Data Engineering 89

Data Engineering Data Engineer Engineering Data 89

Medium Data Engineering

SEPTEMBER 4, 2023

Optimize your LinkedIn profile to include data-engineering specific keywords and skills to increase your visibility. In addition the a… Continue reading on Medium »

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Medium Data Engineering

SEPTEMBER 2, 2023

Are you aspiring to embark on a rewarding career as a data engineer? Well, the journey begins with setting clear and powerful career… Continue reading on The Resume Whisperer »

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Medium Data Engineering

OCTOBER 2, 2023

As part of this course, you will learn all the Data Engineering Essentials related to building Data Pipelines using SQL, Python as Hadoop… Continue reading on Medium »

Data Engineering 52

Data Engineering Data Engineer SQL Python 52

Data Engineering Weekly

SEPTEMBER 24, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. See how it works today.

Data Engineering 69

Data Engineering Data Engineer Engineering Kafka 69

Knowledge Hut

OCTOBER 3, 2023

Within the Microsoft Azure ecosystem, the role of an Azure data engineer stands out as one of the most sought-after positions. What Does an Azure Data Engineer Do? Azure Data engineers collaborate with Azure AI services built on top of Azure Cognitive Services APIs to offer end customers a variety of pre-built models.

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Medium Data Engineering

AUGUST 30, 2023

Large Language Models (LLMs) like GPT-4 can indeed be applied creatively to address various challenges in the field of data engineering. Continue reading on Medium »

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Medium Data Engineering

SEPTEMBER 26, 2023

How Engineering Blogs Can Elevate Your Data Engineering Career Continue reading on Art of Data Engineering »

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Medium Data Engineering

SEPTEMBER 17, 2023

I have been working for some time as a data engineer and I am actually amazed by how vast this domain really is. Few skills people expect… Continue reading on Medium »

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Medium Data Engineering

SEPTEMBER 23, 2023

During the Data Engineering Botcamp, i would like to share my experiences with you. Actually Data Engineering… I’m starting a new series. Today i want to talk about Data Enginnering. Let’s start it.

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Medium Data Engineering

SEPTEMBER 23, 2023

In this blog, we’ll delve into some common data engineering questions and solutions, showcasing the techniques and best practices that… Continue reading on Medium »

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Medium Data Engineering

OCTOBER 2, 2023

In the dynamic landscape of data engineering, two tools have recently caught attention: DuckDB and Polars. DuckDB impresses with its unique…

Engineering 52

Engineering Python Data Engineering Data Engineer 52

Medium Data Engineering

SEPTEMBER 16, 2023

First time I tried to shift from Data science and Machine learning to data engineering all my focus was on three main skills : Continue reading on Medium »

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Towards Data Science

AUGUST 19, 2023

How I made the transition to an analytics engineer Photo by Campaign Creators on Unsplash A few years ago, I was at a point where I was feeling unfulfilled in my career. I had been working in data engineering for three years and the initial excitement of starting in the world of tech had faded. But with analytics engineering, you do.

Data Engineering 77

Data Engineering Data Engineer Engineering Data 77

Medium Data Engineering

OCTOBER 1, 2023

Data engineering is the discipline of designing, building, and maintaining data pipelines that enable data-driven decision making and… Continue reading on Medium »

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

Medium Data Engineering

JULY 21, 2023

Data engineering is a crucial field that plays a fundamental role in the data-driven world we live in today. The demand for data engineers… Continue reading on Medium »

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Medium Data Engineering

AUGUST 18, 2023

Data engineering is a crucial field that plays a pivotal role in modern data-driven businesses. It involves the collection, transformation… Continue reading on Medium »

Data Engineering 94

Data Engineering Data Engineer Engineering Data 94

Medium Data Engineering

SEPTEMBER 21, 2023

Big data engineering offers numerous career opportunities and it is a rapidly growing field. As a big data engineer, your role involves… Continue reading on Medium »

Data Engineering 52

Data Engineering Data Engineer Big Data Engineering 52