KDnuggets

AUGUST 31, 2023

The Burtch Works 2023 Data Science & AI Professionals salary report is here, and includes insightful data such as hiring and marketplace trends, compensation changes over time, and salary data. Get your copy here.

Data Science 138

Data Science Data 138

Confluent

AUGUST 31, 2023

Learn how Confluent’s cloud-native Apache Kafka engine stood out from other data management systems with its uniquely elastic, reliable, and cost-efficient design

Kafka 89

Kafka Cloud Designing Data Management 89

KDnuggets

AUGUST 31, 2023

Buckle up for the Generative AI gold rush! Will BigTech rule with its picks and shovels? Which startups will strike it rich? Will “copilot for X” be the business strategy to hit pay dirt? How can startups dig moats to keep out other prospectors? And will the US once again have the richest gold seams?

IT 112

IT 112

Towards Data Science

AUGUST 31, 2023

Optimizing queries, improving runtimes, and geospatial data science applications Photo by Tamas Tuzes-Katai on Unsplash Intro: why is a spatial index useful? In doing geospatial data science work, it is very important to think about optimizing the code you are writing. How can you make datasets with hundreds of millions of rows aggregate or join faster?

Data Engineering 78

Data Engineering Data Engineer Engineering Data Science 78

Advertisement

Generative AI is upending the way product developers & end-users alike are interacting with data. Despite the potential of AI, many are left with questions about the future of product development: How will AI impact my business and contribute to its success? What can product managers and developers expect in the future with the widespread adoption of AI?

Data

KDnuggets

AUGUST 31, 2023

These are my predictions based on my personal experiences, recent research and reports from leading platforms.

Data Management 110

Data Management Management Data Data Engineering 110

databricks

AUGUST 31, 2023

For a limited time, we are offering 20% off our public instructor-led training with the code: dU0ChfGA1 Value of Databricks Training The explosion.

Coding 87

Coding 87

Monte Carlo

AUGUST 31, 2023

Every data science team has that moment where it realizes it has outgrown its initial architecture. That moment came quickly for our team and our SQL-based feature store. Within a few years our organization had scaled from supporting dozens to hundreds of clients, each of whom seemed to require different feature requests for more advanced use cases.

SQL 64

SQL Data Science Machine Learning Algorithm 64

Confluent

AUGUST 31, 2023

Get a high-level overview of source connector tuning: What can and cannot be tuned, and tuning methodology for any and all source connectors.

Kafka 70

Kafka 70

Monte Carlo

AUGUST 31, 2023

Raise your hand (or spit out your coffee, sigh deeply, and shake your head) if this scenario rings a bell. Data is a priority for your CEO, as it often is for digital-first companies, and she is fluent in the latest and greatest business intelligence tools. Your CTO is excited about migrating to the cloud, and constantly sends your team articles highlighting performance measurements against some of the latest technologies.

Data Lake 52

Data Lake Data Pipeline Unstructured Data Data Warehouse 52

Confluent

AUGUST 31, 2023

How a modern business is bringing personalized notifications and recommendations via mobile apps on users’ phones with help from data streaming.

Data 67

Data 67

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

Ascend.io

AUGUST 31, 2023

In the dynamic world of data, many professionals are still fixated on traditional patterns of data warehousing and ETL, even while their organizations are migrating to the cloud and adopting cloud-native data services. But the capabilities of the clouds have eclipsed traditional data architectures, and have upset the roles of data acquisition (“Extract”), logical processing (“Transform”), and populating a schema (“Load”).

Data Lake 52

Data Lake ETL Tools Data Warehouse Data Pipeline 52

DoorDash Engineering

AUGUST 31, 2023

At DoorDash, we generate supply and demand forecasts to proactively plan operations such as acquiring the right number of Dashers (delivery drivers) and adding extra pay when we anticipate low supply. It is challenging to generate accurate forecasts during holidays because certain machine learning techniques (e.g., XGBoost , Gradient Boosting , Random Forest ) have difficulty handling high variation with limited data.

Machine Learning 75

Machine Learning Finance Designing Architecture 75

Striim

AUGUST 31, 2023

Kappa Architectures are becoming a popular way of unifying real-time (streaming) and historical (batch) analytics giving you a faster path to realizing business value with your pipelines. Treating batch and streaming as separate pipelines for separate use cases drives up complexity, cost, and ultimately deters data teams from solving business problems that truly require data streaming architectures.

Data Integration 69

Data Integration Architecture Amazon Web Services ETL System 69

LinkedIn Engineering

AUGUST 31, 2023

LinkedIn Information Security is committed to help foster a community that is safe and secure for our members. The Application Security team is responsible for safeguarding LinkedIn member data through the implementation and management of various security features, focusing primarily on framework-level security. One of our core responsibilities at the web framework layer is to configure and manage security headers to enhance web application security, some of which include Content Security Policy

Transportation 83

Transportation Architecture Coding Government 83

Advertisement

Start-ups & SMBs launching products quickly must bundle dashboards, reports, & self-service analytics into apps. Customers expect rapid value from your product (time-to-value), data security, and access to advanced capabilities. Traditional Business Intelligence (BI) tools can provide valuable data analysis capabilities, but they have a barrier to entry that can stop small and midsize businesses from capitalizing on them.

Data Analysis

Snowflake

AUGUST 31, 2023

Welcome to Snowflake’s Startup Spotlight, where we learn about startups building amazing things on Snowflake. In this edition, we’ll hear from Bobby Pinero, Co-Founder of Equals , about how his preference for doing analysis in spreadsheets fueled his drive to create a modern spreadsheet that can handle today’s data analysis needs. Tell us a little about yourself and what inspired you to build Equals.

BI 81

BI Finance Data Analysis SQL 81

ThoughtSpot

AUGUST 31, 2023

I’m calling it now. The next battleground for analytics adoption among business users will be the productivity suite. Let’s unpack that statement by considering these two examples: You finally get your data visualization just how you want it for your presentation. Now, you take a screenshot and copy-paste it into your slide deck. You pull your dashboard data into Google Sheets so you can perform ad-hoc analysis and collaborate with various stakeholders who don’t have dashboard access.

Google Cloud 86

Google Cloud BI Government Cloud 86

Snowflake

AUGUST 31, 2023

Apache Iceberg continues to grow in popularity as the industry standard for open table formats. Because of its leading ecosystem of diverse adopters, contributors and commercial offerings, Iceberg helps prevent storage lock-in and eliminates the need to move or copy tables between different systems, which often translates to lower compute and storage costs for your overall data stack.

Metadata 85

Metadata AWS Data Lake Datasets 85

Monte Carlo

AUGUST 31, 2023

Below, we share an excerpt from Data Quality Fundamentals: A Practical Guide to Building Trustworthy Data Pipelines , written by Barr Moses, Lior Gavish, and Molly Vorwerck. Access the first 10 pages for free, here. If you’ve experienced any of the following scenarios, raise your hand. (Or, you can just nod in solidarity – there’s no way we’ll know otherwise). 5,000 rows in a critical (and relatively predictable) table suddenly turns into 500, with no rhyme or reason.

Data Pipeline 40

Data Pipeline Data Lake Data Data Warehouse 40

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

Certification