article thumbnail

Building Data Platforms (from scratch)

Confessions of a Data Guy

Build new pipeline, update pipeline, new data model, fix bug, etc, etc. It’s a constant stream of data, new and old, spilling into our Data Warehouses and […] The post Building Data Platforms (from scratch) appeared first on Confessions of a Data Guy. It’s never-ending.

Building 184
article thumbnail

Zenlytic Is Building You A Better Coworker With AI Agents

Data Engineering Podcast

The team at Zenlytic have leaned on the promise of large language models to build an AI agent that lets you converse with your data. Are data agents harder to build? What are the most interesting, unexpected, or challenging lessons that you have learned while working on building an AI agent for business intelligence?

Building 278
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Building cost effective data pipelines with Python & DuckDB

Start Data Engineering

Building efficient data pipelines with DuckDB 4.1. Introduction 2. Project demo 3. Use DuckDB to process data, not for multiple users to access data 4.2. Cost calculation: DuckDB + Ephemeral VMs = dirt cheap data processing 4.3. Processing data less than 100GB? Use DuckDB 4.4.

article thumbnail

Kafka to MongoDB: Building a Streamlined Data Pipeline

Analytics Vidhya

We know that streaming data is data that is emitted at high volume […] The post Kafka to MongoDB: Building a Streamlined Data Pipeline appeared first on Analytics Vidhya. Handling and processing the streaming data is the hardest work for Data Analysis.

MongoDB 217
article thumbnail

Entity Resolution: Your Guide to Deciding Whether to Build It or Buy It

This will help you decide whether to build an in-house entity resolution system or utilize an existing solution like the Senzing® API for entity resolution. This guide will walk you through the requirements and challenges of implementing entity resolution.

article thumbnail

Building Linked Data Products With JSON-LD

Data Engineering Podcast

Summary A significant amount of time in data engineering is dedicated to building connections and semantic meaning around pieces of information. In this episode Brian Platz explains how JSON-LD can be used as a shared representation of linked data for building semantic data products. Hex brings everything together.

Building 189
article thumbnail

Building ETL Pipelines With Generative AI

Data Engineering Podcast

Now that AI has reached the level of sophistication seen in the various generative models it is being used to build new ETL workflows. In this episode Jay Mishra shares his experiences and insights building ETL pipelines with the help of generative AI. How can you get the best results for your use case?

Building 162
article thumbnail

Why “Build or Buy?” Is the Wrong Question for Analytics

Every time an application team gets caught up in the “build vs buy” debate, it stalls projects and delays time to revenue. Partnering with an analytics development platform gives you the freedom to customize a solution without the risks and long-term costs of building your own. There is a third option.

article thumbnail

How to Build Data Experiences for End Users

Organizational data literacy is regularly addressed, but it’s uncommon for product managers to consider users’ data literacy levels when building products. Product managers need to research and recognize their end users' data literacy when building an application with analytic features.

article thumbnail

The Essential Guide to Building Analytic Applications

Download this eBook to discover insights from 16 top product experts, and learn what it takes to build a successful application with analytics at its core. What should product managers keep in mind when adding an analytics project to their roadmap?

article thumbnail

How to Optimize the Developer Experience for Monumental Impact

Speaker: Anne Steiner and David Laribee

You’ll walk away with a comprehensive understanding of: The transformative power of strategic DX improvements in fostering a product-oriented strategy, driving up customer satisfaction and revenue 📈 DX’s impact on the entire Product Development Lifecycle (PDLC), with a review of tangible, real-world illustrations 🌐 Actionable strategies (..)

article thumbnail

Driving Business Impact for PMs

Speaker: Jon Harmer, Product Manager for Google Cloud

Create actual artifacts for your products: With the practical experience provided in this session, apply these tools to real-world product management scenarios to build journey and impact maps for actual users & products.

article thumbnail

3 Challenges of Building Complex Dashboards with Open Source Components

Speaker: Ryan MacCarrigan, Founding Principal, LeanStudio

Many product teams use charting components and open source code libraries to get dashboards and reporting functionality quickly. But what happens when you have a growing user base and additional feature requests?

article thumbnail

The Definitive Guide to Predictive Analytics

The Definitive Guide to Predictive Analytics has everything you need to get started, including real-world examples, steps to build your models, and solutions to common data challenges. What You'll Learn: 7 steps to embed predictive analytics in your application—from identifying a problem to solve to building your prototype.

article thumbnail

Embedded Analytics Insights for 2024

To better understand the factors behind the decision to build or buy analytics, insightsoftware partnered with Hanover Research to survey IT, software development, and analytics professionals on why they make the embedded analytics choices they do.