Jesse Anderson

FEBRUARY 20, 2024

This week on Unapologetically Technical, I had the wonderful pleasure of interviewing Gunnar Morling, the creator of the Billion Row Challenge and Senior Staff Software Engineer at Decodable. In this episode, we talk about why it is so important to stay in a position long enough to gain experience and see the success or failure of decisions. He also shares his experiences at RedHat and working on Debezium.

Software Engineer 100

Software Engineer Software Engineering Engineering Process 100

Jesse Anderson

NOVEMBER 30, 2023

Lately, I’ve been learning how to trade options. Although there’s data and programming involved in options trading, it isn’t as technical as data engineering or software engineering. However, it reflects the current state of learning, whether that’s data engineering or options trading. It gave me a look into learning a skill using videos. Each lesson I learned will directly apply to your learning or skill improvement.

Data Engineering 147

Data Engineering Data Engineer Algorithm Software Engineer 147

Jesse Anderson

NOVEMBER 14, 2023

A Guest Post by Ole Olesen-Bagneux In this blog post I would like to describe a new data team, that I call ‘the data discovery team’. It’s a team that connects naturally into the constellation of the three data teams Operations team Data engineering team Data Science team as described in Jesse Anderson’s book Data Teams (2020) Before I explain what the data discovery team should do, it is necessary to add a bit of context on the concept of data discovery itself.

Metadata 147

Metadata Data Science Big Data Data 147

Jesse Anderson

OCTOBER 5, 2023

Confluent had their Current Conference (Videos: day one and day two ). There were many announcements that both technologists and investors need to know about. Confluent had two moats (replication and Confluent Cloud), and now they anticipate three moats (replication, Confluent Cloud, serverless Flink). As expected with a vendor conference, there is a lot of marketing from the stage.

Kafka 195

Kafka Finance Cloud Designing 195

Jesse Anderson

SEPTEMBER 14, 2023

There has been quite a bit of writing covering GPT and LLMs from data science and business perspectives. I haven’t seen much from the data engineering side. Let me share my perspective, having been in data and AI for a while and using LLMs before they became popular. It is interesting to see the general public having the same amount of excitement as there was a year ago in the LLM space.

Data Engineering 147

Data Engineering Data Engineer Engineering NoSQL 147

Jesse Anderson

MAY 9, 2023

The results and analysis from my 2023 Data Teams Survey left a few open questions. Let’s revisit these questions with some answers. Methodologies and Size of Company Figure 1 – Methodologies Broken Down By Size of Company Using Them We see a few commonalities across different company sizes, as shown in Figure 1. One striking commonality is that so many companies are using data mesh.

Software Engineer 147

Software Engineer Software Engineering Consulting Data 147

Jesse Anderson

MARCH 6, 2023

Between January 24, 2023, and February 28, 2023, I ran a survey to get more data for my latest book Data Teams , and to update my previous survey from late 2020. Overall, we had 81 respondents. This survey was designed to get information about how management uses data teams, the value they’re creating, and how they’re creating it. The survey asked about the best and worst practices that teams are using or experiencing.

Consulting 130

Consulting Data Data Science Big Data 130

Jesse Anderson

MARCH 30, 2022

Data Teams are like Role Playing Games (RPG). If you’re not familiar with RPGs, there is a person or group of characters all working together for a common goal. A crucial part of the individual characters are their levels, skills, and stats. In many games, higher levels are required to unlock specific skills. Likewise, stats show how well a character can utilize their skills.

Coding Skills 100

Coding Skills Big Data Machine Learning Data 100

Jesse Anderson

APRIL 13, 2022

Data mesh is quickly becoming a way for companies to roll out their data strategy. If you haven’t already learned about data mesh , I suggest doing so. It comes with organizational and technical changes. I think a crucial part of your data mesh revolves around the choice of publish/subscribe technologies. At the crux of data mesh is a desire for flexibility.

Kafka 124

Kafka Data Cloud Technology 124

Jesse Anderson

JUNE 7, 2022

In 2021 I had the pleasure to first get to know and speak with Zhamak Dheghani, Director of Emerging Technologies at ThoughtWorks, in season one of the Data Dream Team series. Zhamak is a software engineer and architect who is (in)famously known as the founder of the data mesh concept, a paradigm shift in how we manage data-driven value at scale. I interviewed Zhamak last season as more of an introduction to Data Mesh.

Generalist 130

Generalist Software Engineer Software Engineering Data 130

Jesse Anderson

JULY 12, 2022

My guest this week is Drew Smith , Vice President of Global Data and Analytics at Little Caesars Enterprises and Ilitch Companies. Little Caesars is a pizza franchise that is mainly in the United States. Illitch Companies owns the Detroit Tigers (baseball), Detroit Red Wings (hockey), and several stadiums. Before that, Drew worked at International Institute for Analytics (IIA), an analytics consulting company, and IKEA, the furniture retailer and manufacturer.

Recruitment 130

Recruitment Manufacturing Retail Consulting 130

Jesse Anderson

OCTOBER 5, 2022

My guest this week is Brent Dykes , Founder and Chief Data Storyteller at Analytics Hero. Before he founded his own company, he was at Omniture, Adobe, and Domo. Analytics Hero is a consulting business based around data storytelling Data storytelling was a new concept to me. Brent defines it as “as a structured approach for communicating insights to a targeted audience using narrative elements and explanatory visuals.

Consulting 130

Consulting Data IT 130

Jesse Anderson

OCTOBER 19, 2022

I have a calendar reminder that tells me when I founded Big Data Institute. It just told me I founded the company eight years ago. The reminder is called “Independent Anniversary.” It’s the day I split off and executed my vision for an independent, big data consulting company. Independence has all sorts of manifestations. For you, it’s an independent look at technology and vendors from someone who’s worked at a vendor (Cloudera) and worked in distributed systems for even longer.

Consulting 130

Consulting Big Data Technology Designing 130

Jesse Anderson

NOVEMBER 2, 2022

My guest this week is Erick Webbe , Head of Data Science at bol.com. Bol.com is the biggest online retailer in northwestern Europe, serving about 12 million customers, as a general retailer similar to Amazon.com. Erick has a Master’s degree in Applied Physics. His background in physics forms a basis for his philosophy on life and work. That’s a “philosophy that I still apply to my work every single day […] we think about how we can best help them overcome that problem or solve it, and then

Generalist 130

Generalist Data Science Building Retail 130

Jesse Anderson

NOVEMBER 15, 2022

My guest this week is Kulani Likotsi , the Head of Data Management and Data Governance at one of the four biggest banks in Africa. She’s had a rising career journey going from an analyst, to a Business Intelligence developer, to the data warehouse team, to the data governance team. I was impressed with Kulani’s volunteer spirit. Whenever there was a need, she volunteered.

Data Governance 130

Data Governance Government Banking Business Intelligence 130

Jesse Anderson

DECEMBER 12, 2022

In the beginning, there was Google. Google looked over the expanse of the growing internet and realized they’d need scalable systems. They created MapReduce and GFS in 2004. They published the papers for them in the same year. Doug Cutting took those papers and created Apache Hadoop in 2005. Cloudera was started in 2008, and HortonWorks started in 2011.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Jesse Anderson

JANUARY 9, 2023

If you haven’t heard, Confluent announced they’re buying Immerok. This purchase represents a significant shift in strategy for Confluent. I started a Twitter thread with some of my initial thoughts, but I want to write a post giving more analysis and opinions. In short, I still echo the sentiment from my original tweet “This was always the way it should have been.

Kafka 147

Kafka Coding Technology SQL 147