March, 2024

article thumbnail

The “10x engineer:" 50 years ago and now

The Pragmatic Engineer

👋 Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover topics related to Big Tech and startups through the lens of engineering managers and senior engineers. In this article, we cover one out of five topics from today’s subscriber-only article What Changed in 50 Years of Computing.

article thumbnail

The Best Piece of Software Engineering Advice

Confessions of a Data Guy

You probably think this is another internet clickbait title uh? Just trying to get you to clickty clickty and sell you some Google Ads. Two problems. I don’t have Google Ads, and I know a small percentage of people will actually listen to this advice. Whatever. There is a reason some developers struggle to move […] The post The Best Piece of Software Engineering Advice appeared first on Confessions of a Data Guy.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Apache Flink and the input data reading

Waitingforcode

I'm writing this unexpected blog post because I got stuck with watermarks and checkpoints and felt that I was missing some basics. Even though this introduction is a bit negative, the exploration for the data reading enabled my other discoveries.

Data 130
article thumbnail

Databricks invests in Mistral AI and integrates Mistral AI’s models into the Databricks Data Intelligence Platform

databricks

Sharing a belief that open source solutions will foster innovation and transparency in generative AI development, Databricks has announced a partnership and participation.

Data 130
article thumbnail

Driving Business Impact for PMs

Speaker: Jon Harmer, Product Manager for Google Cloud

Move from feature factory to customer outcomes and drive impact in your business! This session will provide you with a comprehensive set of tools to help you develop impactful products by shifting from output-based thinking to outcome-based thinking. You will deepen your understanding of your customers and their needs as well as identifying and de-risking the different kinds of hypotheses built into your roadmap.

article thumbnail

Statistics for Machine Learning: What you need to know to become a certified expert

KDnuggets

Ready to become a SAS Certified Specialist in Statistics for Machine Learning? Here’s everything you need to know about the recently released certification from SAS.

article thumbnail

Snowflake Ventures Invests in Landing AI, Boosting Visual AI in the Data Cloud

Snowflake

As Large Language Models are revolutionizing natural language prompts, Large Vision Models (LVMs) represent another new, exciting frontier for AI. An estimated 90% of the world’s data is unstructured, much of it in the form of visual content such as images and videos. Insights from analyzing this visual data can open up powerful new use cases that significantly boost productivity and efficiency, but enterprises need sophisticated computer vision technologies to achieve this.

Cloud 125

More Trending

article thumbnail

Never Put Databricks Notebooks in Production

Confessions of a Data Guy

Recently an Architecture at Databricks recommended people use Notebooks for Production workloads. Very bad and horrible idea. Very expensive compute for most people (All Purpose Clusters) and it leads to horrible development practices. It set off a firestorm on Linkedin when I commented people SHOULD NOT follow this advice. Read here and here The post Never Put Databricks Notebooks in Production appeared first on Confessions of a Data Guy.

article thumbnail

Announcing {arcgis}, an R package for ArcGIS Location Services

ArcGIS

A new R package created by the R-ArcGIS Bridge team enables integration with ArcGIS location services, enhancing their combined powers.

130
130
article thumbnail

Keeping track of engineering-wide goals and migrations

Yelp Engineering

What is Engineering Effectiveness Metrics (EE Metrics)? EE Metrics was envisioned as a hub that helps teams manage their technical debt. EE Metrics provides every team with a detailed web page that contains information about technical debt that needs to be addressed. It also serves as a platform to highlight top engineering initiatives at the organization level.

article thumbnail

Top 6 YouTube Series for Data Science Beginners

KDnuggets

Want to start your data science journey from home, for free, and work at your own pace? Have a dive into this data science roadmap using the YouTube series.

article thumbnail

How to Build an Experimentation Culture for Data-Driven Product Development

Speaker: Margaret-Ann Seger, Head of Product, Statsig

Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating

article thumbnail

A Look Ahead at the Gartner Data & Analytics Summit

Cloudera

As we enter into a new month, the Cloudera team is getting ready to head off to the Gartner Data & Analytics Summit in Orlando, Florida for one of the most important events of the year for Chief Data Analytics Officers (CDAOs) and the field of data and analytics. We’re at a crucial point in time where the excitement and potential surrounding AI has elevated the importance of improving access to the mission-critical data that helps organizations implement it at scale.

article thumbnail

AI Regulation is Rolling Out…And the Data Intelligence Platform is Here to Help

databricks

Policymakers around the world are paying increased attention to artificial intelligence. The world’s most comprehensive AI regulation to date was just passed by.

Data 106
article thumbnail

DuckDB has MAJOR Problems! OOM Errors.

Confessions of a Data Guy

I recently did a challenge. The results were clear. DuckDB CANNOT handle larger-than-memory datasets. OOM Errors. See link below for more details. … DuckDB vs Polars – Thunderdome. 16GB on 4GB machine Challenge. The post DuckDB has MAJOR Problems! OOM Errors. appeared first on Confessions of a Data Guy.

Datasets 130
article thumbnail

Benefits of PMP Certification for Professionals and Employers

Knowledge Hut

Project managers are in high demand, and companies are willing to pay top dollar for qualified individuals. In today's fast-paced business world, having a Project Management Professional (PMP) certification can give you a significant advantage over other job candidates. As per the PMI statistics, a certified PMP is entitled to a pay increase of 20%.

article thumbnail

Reimagined: Building Products with Generative AI

“Reimagined: Building Products with Generative AI” is an extensive guide for integrating generative AI into product strategy and careers featuring over 150 real-world examples, 30 case studies, and 20+ frameworks, and endorsed by over 20 leading AI and product executives, inventors, entrepreneurs, and researchers.

article thumbnail

Easy and Secure LLM Inference and Retrieval Augmented Generation (RAG) Using Snowflake Cortex

Snowflake

Because human-machine interaction using natural language is now possible with large language models (LLMs), more data teams and developers can bring AI to their daily workflows. To do this efficiently and securely, teams must decide how they want to combine the knowledge of pre-trained LLMs with their organization’s private enterprise data in order to deal with the hallucinations (that is, incorrect responses) that LLMs can generate due to the fact that they’ve only been trained on data availabl

article thumbnail

5 Free University Courses to Learn Databases and SQL

KDnuggets

Looking to learn SQL and databases to level up your data science skills? Learn SQL, database internals, and much more with these free university courses.

SQL 120
article thumbnail

Simplifying BI pipelines with Snowflake dynamic tables

ThoughtSpot

Managing complex data pipelines is a major challenge for data-driven organizations looking to accelerate analytics initiatives. While AI-powered, self-service BI platforms like ThoughtSpot can fully operationalize insights at scale by delivering visual data exploration and discovery, it still requires robust underlying data management. Now, that’s changing.

BI 92
article thumbnail

StreamNative and Databricks Unite to Power Real-Time Data Processing with Pulsar-Spark Connector

databricks

StreamNative, a leading Apache Pulsar-based real-time data platform solutions provider, and Databricks, the Data Intelligence Platform, are thrilled to announce the enhanced Pulsar-Spark.

article thumbnail

Entity Resolution Checklist: What to Consider When Evaluating Options

Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.

article thumbnail

Supporting Diverse ML Systems at Netflix

Netflix Tech

David J. Berg , Romain Cledat , Kayla Seeley , Shashank Srikanth , Chaoying Wang , Darin Yu Netflix uses data science and machine learning across all facets of the company, powering a wide range of business applications from our internal infrastructure and content demand modeling to media understanding. The Machine Learning Platform (MLP) team at Netflix provides an entire ecosystem of tools around Metaflow , an open source machine learning infrastructure framework we started, to empower data sc

Systems 87
article thumbnail

5 Big Data Challenges in 2024

Knowledge Hut

The year 2024 saw some enthralling changes in volume and variety of data across businesses worldwide. The surge in data generation is only going to continue. Foresighted enterprises are the ones who will be able to leverage this data for maximum profitability through data processing and handling techniques. With the rise in opportunities related to Big Data, challenges are also bound to increase.

article thumbnail

SNP Unlocks SAP Data for Advanced Analytics with Its Snowflake Native App

Snowflake

As a cohesive ERP solution, SAP is often one of the largest data resources in an organization, containing everything from financial and transactional data to master information about customers, vendors, materials, facilities, planning and even HR. But SAP has limited analytics capabilities, and directly ingesting SAP data into Snowflake can present a challenge.

IT 78
article thumbnail

Master Data Science in a Year: The Ultimate Guide to Affordable, Self-Paced Learning

KDnuggets

Ready to start a career in data science? Put your commitment hat on because I found 4 courses you need to become a master in a year!

article thumbnail

The Big Payoff of Application Analytics

Outdated or absent analytics won’t cut it in today’s data-driven applications – not for your end users, your development team, or your business. That’s what drove the five companies in this e-book to change their approach to analytics. Download this e-book to learn about the unique problems each company faced and how they achieved huge returns beyond expectation by embedding analytics into applications.

article thumbnail

Yelp’s AI pipeline for inappropriate language detection in reviews

Yelp Engineering

Yelp’s mission is to connect consumers with great local businesses by giving them access to reliable and useful information. Consumer trust is one of our top priorities, which is why we make significant investments in technology and human moderation to protect the integrity and quality of content on Yelp. As a platform for user-generated content, we rely on our community of users and business owners to help report reviews that they believe may violate our Terms of Service and Content Guidelines.

article thumbnail

Announcing the General Availability of Databricks Feature Serving

databricks

Today, we are excited to announce the general availability of Feature Serving. Features play a pivotal role in AI Applications, typically requiring considerable.

article thumbnail

Evolving from Rule-based Classifier: Machine Learning Powered Auto Remediation in Netflix Data…

Netflix Tech

Evolving from Rule-based Classifier: Machine Learning Powered Auto Remediation in Netflix Data Platform by Binbing Hou , Stephanie Vezich Tamayo , Xiao Chen , Liang Tian , Troy Ristow , Haoyuan Wang , Snehal Chennuru , Pawan Dixit This is the first of the series of our work at Netflix on leveraging data insights and Machine Learning (ML) to improve the operational automation around the performance and cost efficiency of big data jobs.

article thumbnail

Top Underlying Competencies for Business Analysts in 2024

Knowledge Hut

Business Analysts play a pivotal role in digital transformation projects carried out by organizations. BAs are thus expected to have knowledge about key concepts of business analysis and be skilled in using different tools and techniques for eliciting, analyzing, and managing requirements. In order to facilitate the five core responsibilities of a business analyst and communicate requirements, and in evaluating solutions, the BA is expected to have a set of competencies.

article thumbnail

Monetizing Analytics Features

Think your customers will pay more for data visualizations in your application? Five years ago, they may have. But today, dashboards and visualizations have become table stakes. Turning analytics into a source of revenue means integrating advanced features in unique, hard-to-steal ways. Download this white paper to discover which features will differentiate your application and maximize the ROI of your analytics.

article thumbnail

Data Engineering Weekly #163

Data Engineering Weekly

Stephanie Kirmer: Uncovering the EU AI Act Large language models have taken the world by storm, and every country is trying to evaluate its potential impact. India recently announced that all AI apps require government approval and dropped the plan later. On similar trends, the article navigates to the complex EU AI Act, recently passed by the European Parliament, which introduces comprehensive regulations for machine learning models impacting EU citizens, focusing on mitigating risks to health,

article thumbnail

Best Free Resources to Learn Data Analysis and Data Science

KDnuggets

This article introduces six top-notch, free data science resources ideal for aspiring data analysts, data scientists, or anyone aiming to enhance their analytical skills.

article thumbnail

Snowflake Startup Spotlight: ZeroError

Snowflake

Welcome to Snowflake’s Startup Spotlight, where we learn about companies building their businesses on Snowflake. In this edition, we’ll hear how Maria Marti, founder and CEO of ZeroError , used her experiences as an engineer and an executive to build a team and create the AI analytics assistant she always wanted — but never had. What inspires you as a founder?

BI 76
article thumbnail

KX and Databricks Integration: Advancing Time-series Data Analytics in Capital Markets and Beyond

databricks

KX and Databricks have partnered to develop time series analytics solutions for the capital markets sector to support many use cases including quant.

article thumbnail

How To Package & Price Embedded Analytics

Just by embedding analytics, app owners can charge 24% more for their product. How much value could you add? This framework from Software Pricing Partners explains how application enhancements can extend your product offerings. You’ll learn: How to take a disciplined approach to pricing The three elements of the Packaging Decision Framework Ways to structure your new embedded analytics offering Download the White Paper to learn about How To Package & Price Embedded Analytics.