Data Engineering Weekly

APRIL 14, 2024

link] Github: 4 ways GitHub engineers use GitHub Copilot The impact of LLM on software development is undeniable. Github shares some insights on how Github engineers use Github Copilot. The blog post brings in-depth insights into how Apache Hudi works on the right side, read side & underlying storage formats.

Data Engineering 62

Data Engineering Data Engineer Engineering Business Intelligence 62

Data Engineering Weekly

APRIL 7, 2024

dbt: 2024 State of Analytics Engineering The 2024 dbt’s state of analytical engineering report is out. Poor data quality and unlcear data ownership remains the top challenges for the data teams. Data Mesh continuously gaining popularity among the enterprises.

Data Engineering 70

Data Engineering Data Engineer Engineering Unstructured Data 70

Jesse Anderson

JANUARY 29, 2024

The premier of my latest talk covering The State of Data Engineering. This starts with data warehousing and goes into data science. I finish off by showing how data engineering can avoid the same fate as data warehousing and data science.

Data Engineering 130

Data Engineering Data Engineer Engineering Data Science 130

Data Engineering Weekly

MARCH 31, 2024

Intuit: How Intuit data analysts write SQL 2x faster with the internal GenAI tool The productivity increase with GenAI is undeniable, and several startups are trying to solve the Text2SQL generation problem. My key highlight is that Excellent data documentation and “clean data” improve results.

Data Engineering 62

Data Engineering Data Engineer Engineering Scala 62

Monte Carlo

JANUARY 23, 2024

In the world of data engineering, Maxime Beauchemin is someone who needs no introduction. Currently, Maxime is CEO and co-founder of Preset , a fast-growing startup that’s paving the way forward for AI-enabled data visualization for modern companies. Enter, the data engineer. What is a data engineer today?

Data Engineering 98

Data Engineering Data Engineer Engineering Data 98

Data Engineering Weekly

MARCH 3, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Editor’s Note: Chennai, India Meetup - March-08 Update We are thankful to Ideas2IT to host our first Data Hero’s meetup.

Data Engineering 67

Data Engineering Data Engineer Pipeline-centric Engineering 67

Waitingforcode

FEBRUARY 3, 2023

This time I won't blog about my discoveries. Instead, I'm going to see other blog posts from the data engineering space and share some key takeaways with you. Hi and welcome to the new series. I don't know how regular it will be yet but hopefully will be able to share some of the notes every month.

Data Engineering 130

Data Engineering Data Engineer Engineering Data 130

Data Engineering Weekly

FEBRUARY 4, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Joe Reis: Definition of Data Modeling & What Data Modeling Is not Joe raised a very fundamental question in data engineering.

Data Engineering 62

Data Engineering Data Engineer Engineering PostgreSQL 62

Waitingforcode

FEBRUARY 3, 2023

In this blog post I'll share with you a list of Java and Scala classes I use almost every time in data engineering projects. We all have our habits and as programmers, libraries and frameworks are definitely a part of the group. The part for Python will follow next week!

Scala 130

Scala Java Data Engineering Data Engineer 130

Knowledge Hut

APRIL 23, 2024

Software engineering is a rapidly growing field with vast career opportunities. With emerging technologies like AI, machine learning, and blockchain, the demand for software engineers has skyrocketed. With emerging technologies like AI, machine learning, and blockchain, the demand for software engineers has skyrocketed.

Software Engineer 98

Software Engineer Software Engineering Engineering Programming Language 98

Data Engineering Weekly

FEBRUARY 25, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Editor’s Note: DEWCon Europe Update & Data Hero’s Chennai Chapter Meetup Last week, we asked our readers if we should bring DEWCon to Europe.

Data Engineering 62

Data Engineering Data Engineer Engineering Kafka 62

Data Engineering Weekly

DECEMBER 3, 2023

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Github writes an excellent blog to capture the current state of the LLM integration architecture. Lackluster AI/ML results often stem from poor data quality.

Data Engineering 70

Data Engineering Data Engineer Engineering Bytes 70

Jesse Anderson

DECEMBER 12, 2022

They were the first companies to commercialize open source big data technologies and pushed the marketing and commercialization of Hadoop. With an immutable file system like HDFS, we needed scalable databases to read and write data randomly. Apache Spark came in 2009 and gave a unified batch and streaming engine.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Data Engineering Weekly

FEBRUARY 18, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Our hope is only with the amazing community of data practitioners who constantly support us. We are so over the Big Data Era to Modern Data Stack.

Data Engineering 62

Data Engineering Data Engineer Engineering Data 62

Data Engineering Weekly

FEBRUARY 11, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Editors Note: A Survey on the current state of Data Products. What do you think of Data Products? Is Modern Data Stack still a useful idea?

Data Engineering 62

Data Engineering Data Engineer Engineering SQL 62

The Pragmatic Engineer

JANUARY 12, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. We cover one and a half out of eight topics in today’s subscriber-only issue, Inside Pollen's Transparent Compensation Data. Here are a few details from the data points, focusing on software engineering compensation.

Software Engineer 183

Software Engineer Software Engineering Engineering Raw Data 183

DoorDash Engineering

FEBRUARY 27, 2024

Two primary aspects of that search engine were causing the trouble: its document-replication mechanism and its lack of support for complex document relationships. We decided the best way to address these challenges was to move away from Elasticsearch to a homegrown search engine. latency reduction and a 75% hardware cost decrease.

Engineering 129

Engineering Systems Designing Architecture 129

The Pragmatic Engineer

AUGUST 10, 2023

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In what is neat: The Pragmatic Engineer is now the first-ever newsletter listed on Learnerbly.

Engineering 191

Engineering Recruitment Retail Software Engineer 191

Data Engineering Weekly

SEPTEMBER 24, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. Airflow isn’t meant to process the data. See how it works today.

Data Engineering 67

Data Engineering Data Engineer Engineering Kafka 67

Data Engineering Weekly

MARCH 10, 2024

Editor’s Note: Chennai Meetup Wrap-Up & Preparation work started for DEWCon I am so grateful for the enthusiastic participants who made our Chennai Data Heroes- Community for Data Folks meetup vibrant! Big thanks to our insightful speakers, Hareshkumar Selvakumar - Talks about his work on Data Products for PayPal.

Data Engineering 52

Data Engineering Data Engineer Engineering Datasets 52

Data Engineering Weekly

JANUARY 21, 2024

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Interesting article on the impact of search engine optimization (SEO) on the quality of search engine results. Visit rudderstack.com to learn more.

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Data Engineering Weekly

SEPTEMBER 11, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. All these concepts fundamentally try to achieve data consistency across the board.

Data Engineering 62

Data Engineering Data Engineer Engineering Kafka 62

Data Engineering Weekly

OCTOBER 1, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. What is the data behavior? See how it works today. Dropbox: Is this a date?

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Data Engineering Weekly

DECEMBER 24, 2023

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. I love the rising, stable, and declining format for categorizing data engineering trends. Visit rudderstack.com to learn more.

Data Engineering 62

Data Engineering Data Engineer Engineering Deep Learning 62

RandomTrees

FEBRUARY 2, 2024

GPT-based data engineering accelerators make the working of data more accessible. These accelerators use GPT models to do data tasks faster, fix any issues, and save a lot of time. GPT models change data in simple language and also provide summaries and explanations. One can rely on this information.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Pipeline 52

Data Engineering Weekly

DECEMBER 10, 2023

RudderStack, one of the leading alternatives to Segment , is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Lackluster AI/ML results often stem from poor data quality. Visit rudderstack.com to learn more.

Data Engineering 76

Data Engineering Data Engineer Engineering Metadata 76

Christophe Blefari

APRIL 12, 2024

The fest we deserve ( credits ) I hope this Data News finds you well. This is an episode in French and we talked mainly about the eventual end of the modern data stack. Build analytics at Hive.co — The journey Oleg and his team went through to implement a modern data stack.

BI 130

BI SQL Data Coding 130

KDnuggets

NOVEMBER 15, 2021

The October blogs that won KDnuggets Rewards include: How I Tripled My Income With Data Science in 18 Months; What Google Recommends You do Before Taking Their Machine Learning or Data Science Course; How to Build Strong Data Science Portfolio as a Beginner; Data Scientist vs Data Engineer Salary.

Portfolio 99

Portfolio Data Science Machine Learning Data Engineering 99

Christophe Blefari

APRIL 19, 2024

easy ( credits ) Hey, new Friday, new Data News. This blog shows how you can use Gen AI to evaluate inputs like translations with added reasons. How we build Slack AI to be secure and private — How Slack uses VPC and Amazon SageMaker with your data secured and private. — A great blog to answer a great question.

MySQL 130

MySQL Data Datasets SQL 130

Data Engineering Weekly

NOVEMBER 5, 2023

RudderStack is the Warehouse Native CDP, built to help data teams deliver value across the entire data activation lifecycle, from collection to unification and activation. Netflix: Streaming SQL in Data Mesh Learnings from our journey In hindsight, we wish we had invested in enabling Flink SQL on the DataMesh platform much earlier.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

Knowledge Hut

DECEMBER 26, 2023

Data Engineering is typically a software engineering role that focuses deeply on data – namely, data workflows, data pipelines, and the ETL (Extract, Transform, Load) process. What is Data Science? What are the roles and responsibilities of a Data Engineer? What is Data Science?

Data Engineering 98

Data Engineering Data Engineer Engineering Pipeline-centric 98

Team Data Science

NOVEMBER 28, 2020

As a data scientist, I always felt a missing link between my developed models and putting them in the production process. This thought led me to my data engineering adventure. I am confident that learning data engineering will make me a better data scientist. Where did I start? Stay tuned!

Engineering 130

Engineering Datasets Data Data Engineering 130

Data Engineering Weekly

SEPTEMBER 3, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. See how it works today. 📣 Exciting news! . or "Don't reinvent the wheel.".

Data Engineering 73

Data Engineering Data Engineer Engineering Data 73

Data Engineering Weekly

JULY 23, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. Overall an exciting time for a data contract in 2024. See how it works today.

Data Engineering 52

Data Engineering Data Engineer Engineering Deep Learning 52

Data Engineering Weekly

AUGUST 6, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. See how it works today. Editor’s Note: DewCon.ai

Data Engineering 52

Data Engineering Data Engineer Engineering Kafka 52

Data Engineering Weekly

AUGUST 20, 2023

Data Engineering Weekly Is Brought to You by RudderStack RudderStack Profiles takes the SaaS guesswork, and SQL grunt work out of building complete customer profiles, so you can quickly ship actionable, enriched data to every downstream team. But 4 years later, in 2023 — where has the data mesh gotten us?

Data Engineering 52

Data Engineering Data Engineer Engineering Data 52

U-Next

DECEMBER 21, 2022

As another year comes to an end, we decided the best way to look back was to revisit the most popular and sought-after blogs of Cybersecurity and list the same for all our Cybersecurity enthusiasts. This just means two-thirds of the world has gone digital, making their data more vulnerable to breach if proper precautions are not taken.

Education 105

Education Government Project Engineering 105