Sat.Mar 09, 2024 - Fri.Mar 15, 2024

article thumbnail

Data News — Week 24.11

Christophe Blefari

Mountains I hope this e-mail finds you well, wherever you are. I'd like to thank you for the excellent comments you sent me last week after the publication of the first version of the Recommendations. This is just the beginning! This week I've added a subscribe button in the Recommendations page in order for you to opt-in for the weekly recommendation email—every Tuesday.

Metadata 272
article thumbnail

The “10x engineer:" 50 years ago and now

The Pragmatic Engineer

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of five topics from today’s subscriber-only article What Changed in 50 Years of Computing.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Version Your Data Lakehouse Like Your Software With Nessie

Data Engineering Podcast

Summary Data lakehouse architectures are gaining popularity due to the flexibility and cost effectiveness that they offer. The link that bridges the gap between data lake and warehouse capabilities is the catalog. The primary purpose of the catalog is to inform the query engine of what data exists and where, but the Nessie project aims to go beyond that simple utility.

Data Lake 147
article thumbnail

Processing time trigger, to be or not to be?

Waitingforcode

That's the question. The lack of the processing time trigger means more a reactive micro-batch triggering but it cannot be considered as the single true best practice. Let's see why.

Process 130
article thumbnail

Get Better Network Graphs & Save Analysts Time

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

article thumbnail

Apache Druid’s Architecture – How Druid Processes Data In Real Time At Scale

Seattle Data Guy

Recently, I wrote an article diving into what Druid is and which companies are using it. Now I wanted to do a deeper dive into Apache Druid’s architecture. Apache Druid has several unique features that allow it to be used as a real-time OLAP. Everything from its various nodes and processes that each have unique… Read more The post Apache Druid’s Architecture – How Druid Processes Data In Real Time At Scale appeared first on Seattle Data Guy.

article thumbnail

Databricks invests in Mistral AI and integrates Mistral AI’s models into the Databricks Data Intelligence Platform

databricks

Sharing a belief that open source solutions will foster innovation and transparency in generative AI development, Databricks has announced a partnership and participation.

Data 133

More Trending

article thumbnail

Don’t Be So Smart

Confessions of a Data Guy

Most Software Engineers think of themselves as too smart. They think they are the best and brightest coder alive or that has ever lived. Doing so, they stunt themselves from becoming Senior Engineers and become hard to work with, the nightmare of the PR process. You don’t need to be the smartest person in the […] The post Don’t Be So Smart appeared first on Confessions of a Data Guy.

article thumbnail

Announcing {arcgis}, an R package for ArcGIS Location Services

ArcGIS

A new R package created by the R-ArcGIS Bridge team enables integration with ArcGIS location services, enhancing their combined powers.

143
143
article thumbnail

Keeping track of engineering-wide goals and migrations

Yelp Engineering

What is Engineering Effectiveness Metrics (EE Metrics)? EE Metrics was envisioned as a hub that helps teams manage their technical debt. EE Metrics provides every team with a detailed web page that contains information about technical debt that needs to be addressed. It also serves as a platform to highlight top engineering initiatives at the organization level.

article thumbnail

5 Ways To Use LLMs On Your Laptop

KDnuggets

Run large language models on your local PC for customized AI capabilities with more control, privacy, and personalization.

129
129
article thumbnail

Understanding User Needs and Satisfying Them

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

article thumbnail

Data Modeling Is Easy

Confessions of a Data Guy

When you’ve been data modeling as long as I have, it gets to be the same old … same old. People make data modeling harder than it has to be. There is a lot of jargon that gets thrown around … third-normal-form, OLAP, OLTP … I give you the 3-4 basics that are at the […] The post Data Modeling Is Easy appeared first on Confessions of a Data Guy.

Data 100
article thumbnail

AI Regulation is Rolling Out…And the Data Intelligence Platform is Here to Help

databricks

Policymakers around the world are paying increased attention to artificial intelligence. The world’s most comprehensive AI regulation to date was just passed by.

Data 114
article thumbnail

Benefits of PMP Certification for Professionals and Employers

Knowledge Hut

Project managers are in high demand, and companies are willing to pay top dollar for qualified individuals. In today's fast-paced business world, having a Project Management Professional (PMP) certification can give you a significant advantage over other job candidates. As per the PMI statistics, a certified PMP is entitled to a pay increase of 20%.

article thumbnail

Boost Your Data Science Skills: The Essential SQL Certifications You Need

KDnuggets

If you are a data scientist who works with large amounts of data and hasn’t learned SQL yet - now might be the time.

SQL 124
article thumbnail

Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You Need to Know

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

article thumbnail

Is Devin Going To Take My Software Engineering Job?

Confessions of a Data Guy

Unless you’ve been hiding a rock you’ve probably heard the hubbub over Devin the new AI Software Engineer that is going to take your job. While this is a genius piece of marketing … it’s a bunch of crud. Never fear, you are in no more danger of losing your job in Software than when […] The post Is Devin Going To Take My Software Engineering Job?

article thumbnail

Implementing LLM Guardrails for Safe and Responsible Generative AI Deployment on Databricks

databricks

Introduction Let’s explore a common scenario – your team is eager to leverage open source LLMs to build chatbots for customer support interactions.

Building 119
article thumbnail

Mapbox Snowflake Native App Opens Geospatial Analytics to New Audiences 

Snowflake

Geospatial data can give a business a competitive edge — especially when it’s combined with the company’s own data resources. Considering a new store location? You’ll want to analyze not just where your nearest competitors and potential customers are, but also retail footfall numbers, historical traffic patterns, distance from distribution centers, environmental factors, potential delivery times to customers and more.

article thumbnail

Build An AI Application with Python in 10 Easy Steps

KDnuggets

Explore the fundamental steps for creating a successful AI Application with Python and other tools.

Python 128
article thumbnail

Peak Performance: Continuous Testing & Evaluation of LLM-Based Applications

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

article thumbnail

Improving ETAs with Multi-Task Models, Deep Learning, and Probabilistic Forecasts

DoorDash Engineering

The DoorDash ETA team is committed to providing an accurate and reliable estimated time of arrival (ETA) as a cornerstone DoorDash consumer experience. We want to ensure that every customer can trust our ETAs, ensuring a high-quality experience in which their food arrives on time every time. With more than 2 billion orders annually, our dynamic engineering challenge is to improve and maintain accuracy at scale while managing a variety of conditions within diverse delivery and merchant scenarios.

article thumbnail

Announcing the General Availability of Databricks Feature Serving

databricks

Today, we are excited to announce the general availability of Feature Serving. Features play a pivotal role in AI Applications, typically requiring considerable.

article thumbnail

SNP Unlocks SAP Data for Advanced Analytics with Its Snowflake Native App

Snowflake

As a cohesive ERP solution, SAP is often one of the largest data resources in an organization, containing everything from financial and transactional data to master information about customers, vendors, materials, facilities, planning and even HR. But SAP has limited analytics capabilities, and directly ingesting SAP data into Snowflake can present a challenge.

IT 85
article thumbnail

5 Essential Skills Every Data Scientist Needs in 2024

KDnuggets

Want to move into the data science field? Or advance your career in the data? Don’t miss these must-have skills.

article thumbnail

Entity Resolution Checklist: What to Consider When Evaluating Options

Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.

article thumbnail

Developer Summit 2024: A tour of the ArcGIS Well-Architected Framework

ArcGIS

The ArcGIS Well-Architected Framework and ArcGIS Architecture Center provides guidance for implementing systems with ArcGIS.

article thumbnail

Building an AI-Ready Retail Organization with Improved Data Governance

databricks

Artificial Intelligence is top-of-mind with every C-suite in Retail & Consumer Goods. Companies see the potential to deliver better customer service, derive faster.

Retail 91
article thumbnail

How Financial Services and Retail Companies Are Accelerating their Data, Apps and AI Strategy in the Data Cloud

Snowflake

Last year, we held our first Accelerate event , to explore industry trends, data and technology innovations, and data strategy case studies in financial services. This year, we are expanding to five industry events, featuring leaders in financial services; retail and consumer goods; manufacturing; media, advertising and entertainment; and healthcare and life sciences.

Retail 82
article thumbnail

5 Free University Courses to Learn Computer Science

KDnuggets

Want to switch to a tech career? Make it happen with these free computer science courses.

article thumbnail

From Developer Experience to Product Experience: How a Shared Focus Fuels Product Success

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

article thumbnail

In the spotlight with Rahul Mani, ThoughtSpot’s Selfless Excellence champion

ThoughtSpot

This is part of our ongoing spotlight series which highlights ThougthSpot’s quarterly Selfless Excellence champion. ThoughtSpot's culture is rooted in our core value of Selfless Excellence. This means we consider our teammates, customers, and society at large ahead of our own personal wins without the distraction of office politics. Our common ground ensures that we are moving together with intention and integrity in everything we do—when we run the business, plan our go-to-market strategy,

article thumbnail

Postgres Aurora DB major version upgrade with minimal downtime

Lyft Engineering

Photo by Frank Olsen UNDER CC BY-SA 3.0 DEED Introduction Our payment platform team had the unique challenge to upgrade our Aurora Postgres DB from v10 to v13. This DB was responsible for storing transactions within Lyft and contains ~400 tables (with partitions) and ~30TB of data. Upgrading the database in-place would have resulted in ~30 mins of downtime.

Bytes 78
article thumbnail

Yelp’s AI pipeline for inappropriate language detection in reviews

Yelp Engineering

Yelp’s mission is to connect consumers with great local businesses by giving them access to reliable and useful information. Consumer trust is one of our top priorities, which is why we make significant investments in technology and human moderation to protect the integrity and quality of content on Yelp. As a platform for user-generated content, we rely on our community of users and business owners to help report reviews that they believe may violate our Terms of Service and Content Guidelines.

article thumbnail

Getting Started With Go Programing For Data Science

KDnuggets

Learn how to perform data analysis, data visualization, and model training in GoLang, just like Python.

article thumbnail

How to Build an Experimentation Culture for Data-Driven Product Development

Speaker: Margaret-Ann Seger, Head of Product, Statsig

Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating