Confessions of a Data Guy

DECEMBER 1, 2022

Nothing captures the imagination and heart like a tale of betrayal and heartbreak, and that is a tale I want to bring to you today. It’s a tale of Databricks Workflows and Jobs, version changes, new features, API’s, and insidious little hidden gems that will make you pull your hair out when you find them. […] The post A Tale of Betrayal and Heartbreak – Databricks Workflows and Jobs. appeared first on Confessions of a Data Guy.

Data 130

Data IT Big Data Data Engineering 130

KDnuggets

DECEMBER 1, 2022

In this blog, I shared my story on getting 4 data science job offers including Airbnb, Lyft and Twitter after being laid off. Any data scientist who was laid off due to the pandemic or who is actively looking for a data science position can find something here to which they can relate.

Data Science 160

Data Science Data 160

Confluent

DECEMBER 2, 2022

ksqlDB use case: see how apps can use ksqlDB to ingest, filter, enrich, aggregate, and query data directly with Kafka—no complex architectures or data stores needed.

Kafka 144

Kafka Building Architecture Data 144

Data Engineering Podcast

NOVEMBER 27, 2022

Summary The most expensive part of working with massive data sets is the work of retrieving and processing the files that contain the raw information. FeatureBase (formerly Pilosa) avoids that overhead by converting the data into bitmaps. In this episode Matt Jaffee explains how to model your data as bitmaps and the benefits that this representation provides for fast aggregate computation.

Data Lake 100

Data Lake Data Warehouse MongoDB Data Pipeline 100

Speaker: Maher Hanafi, VP of Engineering at Betterworks & Tony Karrer, CTO at Aggregage

Executive leaders and board members are pushing their teams to adopt Generative AI to gain a competitive edge, save money, and otherwise take advantage of the promise of this new era of artificial intelligence. There's no question that it is challenging to figure out where to focus and how to advance when it’s a new field that is evolving everyday. 💡 This new webinar featuring Maher Hanafi, CTO of Betterworks, will explore a practical framework to transform Generative AI prototypes into

Data Collection

Teradata

NOVEMBER 30, 2022

Teradata has joined the Amazon SageMaker Ready Program which differentiates Teradata as an AWS Partner Network member with a product that works with Amazon SageMaker & fully supports AWS customers.

Programming 97

Programming Designing AWS 97

KDnuggets

DECEMBER 2, 2022

The data science field is full of job opportunities, yet there is still a lot of confusion about what data scientists actually do. This confusion is largely due to the many myths that exist about the role of a data scientist. In this article, we will bust the top 10 myths about data science. By the end of this article, you will have a better understanding of the role of a data scientist and what it takes to be one.

Data Science 157

Data Science Data IT 157

Data Engineering Podcast

NOVEMBER 27, 2022

Summary The data ecosystem has been growing rapidly, with new communities joining and bringing their preferred programming languages to the mix. This has led to inefficiencies in how data is stored, accessed, and shared across process and system boundaries. The Arrow project is designed to eliminate wasted effort in translating between languages, and Voltron Data was created to help grow and support its technology and community.

Data Process 100

Data Process Process Metadata Business Intelligence 100

DoorDash Engineering

NOVEMBER 29, 2022

As DoorDash’s business grows, engineers strive for a better network infrastructure to ensure more third-party services could be integrated into our system while keeping data securely transmitted. Due to security and compliance concerns, some vendors handling such sensitive data cannot expose services to the public Internet and therefore host their own on-premise data centers.

Cloud 97

Cloud AWS Amazon Web Services Data 97

KDnuggets

DECEMBER 1, 2022

The latest KDnuggets exclusive cheatsheet covers the essentials of machine learning with Scikit-learn.

Machine Learning 154

Machine Learning 154

Confluent

DECEMBER 2, 2022

Major improvements to the Kafka consumer, Streams, and ksqlDB for incremental cooperative rebalancing while maintaining at-least-once and exactly-once guarantees.

Kafka 138

Kafka Process 138

Speaker: Jason Tanner

While growth of software-enabled solutions generates momentum, growth alone is not enough to ensure sustainability. The probability of success dramatically improves with early planning for profitability. A sustainable business model contains a system of interrelated choices made not once but over time. Join this webinar for an iterative approach to ensuring solution, economic and relationship sustainability.

Certification

Cloudera

DECEMBER 1, 2022

Full-stack observability is a critical requirement for effective modern data platforms to deliver the agile, flexible, and cost-effective environment organizations are looking for. For analytic applications to properly leverage a hybrid, multi-cloud ecosystem to support modern data architectures, data observability has become even more important. I spoke to Mark Ramsey of Ramsey International (RI) to dive deeper into that last subject.

Data Lake 81

Data Lake Data Pipeline Data Governance Analytics Application 81

Engineering at Meta

NOVEMBER 30, 2022

UPM is our internal standalone library to perform static analysis of SQL code and enhance SQL authoring. UPM takes SQL code as input and represents it as a data structure called a semantic tree. Infrastructure teams at Meta leverage UPM to build SQL linters, catch user mistakes in SQL code, and perform data lineage analysis at scale. Executing SQL queries against our data warehouse is important to the workflows of many engineers and data scientists at Meta for analytics and monitoring use cases

SQL 76

SQL Data Warehouse Metadata Coding 76

KDnuggets

NOVEMBER 30, 2022

The best way to learn Data Science is by solving real-world problems with the data and building your own portfolio. In this article, we will discuss three projects that you can work on to build your portfolio and impress interviewers.

Data Science 114

Data Science Project Portfolio Data 114

Confluent

NOVEMBER 29, 2022

Apache Kafka’s Streams API embeds Machine Learning into any app or microservice (Java, Docker, Kubernetes, etc.) to add business value.

Machine Learning 145

Machine Learning Kafka Java Building 145

Advertisement

Generative AI is upending the way product developers & end-users alike are interacting with data. Despite the potential of AI, many are left with questions about the future of product development: How will AI impact my business and contribute to its success? What can product managers and developers expect in the future with the widespread adoption of AI?

Data

Cloudera

NOVEMBER 30, 2022

What is CDP Operational Database (COD). CDP Operational Database enables developers to quickly build future-proof applications that are architected to handle data evolution. It helps developers automate and simplify database management with capabilities like auto-scale, and is fully integrated with Cloudera Data Platform (CDP). For more information and to get started with COD, refer to Getting Started with Cloudera Data Platform Operational Database (COD).

Database 80

Database Datasets NoSQL Big Data 80

Booking.com Engineering

DECEMBER 2, 2022

Booking.com’s mission is to make it easier for everyone to experience the world. To help people discover destinations, we are a leading travel advertiser on Google Pay Per Click (PPC). Booking Holdings, as a whole, spent $4.7 billion in marketing across all brands in the first nine months of 2022[1]. How do we run PPC at our scale, and efficiently? In this article, we want to illustrate our extensive use of the public cloud, specifically Google Cloud Platform (GCP).

Systems 52

Systems Cloud MySQL Relational Database 52

KDnuggets

NOVEMBER 29, 2022

First steps to learning data science & machine learning are the foundations.

Machine Learning 160

Machine Learning Data Science Data 160

Confluent

DECEMBER 2, 2022

Build fast, break nothing. Learn about the unique challenges Confluent's engineering team has faced building ksqlDB and continuously shipping the latest, greatest features.

Building 123

Building Engineering Process 123

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

Pinterest Engineering

DECEMBER 2, 2022

Grey Skold | (former Android Video Engineer) ; Lin Wang | Android Performance Engineer; Sheng Liu | Android Performance Engineer Pinterest Android App offers a rare experience with a mix of images and videos on a two-column grid. In order to maintain a performant video experience on Android devices, we focused on: Warming up Configurations Pooling players Warming Up In order to reduce the startup latency, we establish a video network connection by sending a dummy HTTP HEAD request during the ear

Media 52

Media Engineering Utilities Architecture 52

Scott Logic

NOVEMBER 30, 2022

An introduction to Markdown Markdown is a brilliant tool for quickly writing up universally accessible documents. Created by John Gruber and Aaron Schwartz in 2004, it stands as one of the most popular and widely used markup languages around. It uses simple and intuitive formatting that can be easily read and understood. “A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions” John Gruber, creator of

Programming Language 52

Programming Language Utilities Designing Coding 52

KDnuggets

DECEMBER 1, 2022

Introduction to PyTorch Lightning and how it can be used for the model building process. It also provides a brief overview of the PyTorch characteristics and how they are different from TensorFlow.

Building 108

Building Process IT Machine Learning 108

Confluent

DECEMBER 2, 2022

Datadog and Confluent integration brings new monitoring, metrics, and enterprise capabilities for Kafka. Monitor Kafka Connect, ksqlDB, Schema Registry, REST Proxy, and more.

Kafka 117

Kafka 117

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science

Tweag

NOVEMBER 30, 2022

There is an inherent beauty to be found in simple, pervasive ideas that shift our perspective on familiar objects. Such ideas can help tame the complexity of abstruse abstractions by offering a more intuitive angle from which to understand them. The aim of this post is to present an alternative angle — that of interactive semantics — from which to view one of the fundamental notion of functional programming: higher-order functions.

Programming Language 52

Programming Language Programming Computer Science Coding 52

Ascend.io

DECEMBER 1, 2022

Data migration is one of the most common undertakings for data teams. Yet, many businesses underestimate the process—resulting in extra time and money spent. A data migration process usually takes longer than it should and requires several teams. In addition, it is highly visible to both users and executives. How can you keep from making the same mistake?

Process 52

Process Data ETL Tools Database 52

KDnuggets

NOVEMBER 29, 2022

What is Chebychev's Theorem and How Does it Apply to Data Science? • How to Select Rows and Columns in Pandas Using [ ],loc, iloc,at and.iat • Linux for Data Science Cheatsheet • How Much Math Do You Need in Data Science? • Git for Data Science Cheatsheet.

Data Science 108

Data Science IT Data 108

Confluent

NOVEMBER 30, 2022

What is stream processing, or complex event processing (CEP), and how does it work? Learn about real-time data and event stream analytics in this tutorial.

Process 125

Process Data IT 125

Speaker: John Mansour

If you're looking to advance your career in product management, there are more options than just climbing the management ladder. Join our upcoming webinar to learn about highly rewarding career paths that don't involve management responsibilities. We'll cover both career tracks and provide tips on how to position yourself for success in the one that's right for you.

Management

DataKitchen

DECEMBER 1, 2022

Data Team members, have you ever felt overwhelmed? The never-ending flow of new information can be stressful, and it’s hard to know where to start. Well, don’t worry because DataOps is here to help! In this post, we’ll discuss how DataOps Observability and Automation can relieve team stress and show you how to get started. So don’t wait any longer.

Data Engineering 52

Data Engineering Data Engineer Data Pipeline Data Analytics 52

Databand.ai

NOVEMBER 30, 2022

The Ravit Show Q&A: How More Data Observability Leads to Better Governance Ryan Yackel 2022-11-30 10:18:32 We recently had the opportunity to join an episode of The Ravit Show , a community for data science and AI professionals to upskill, grow, share, and learn from each other. Ryan Yackel, Product Evangelist at Databand, and Kip Yego, Program Director at IBM, joined Ravit Jain to talk about all things data observability and data governance.

Government 52

Government Data Governance Architecture Google Cloud 52

KDnuggets

DECEMBER 2, 2022

Personalized learning, smart grading, skill gap assessment, and better ROI: The importance of incorporating Machine Learning in Online Learning cannot be overstated.

Machine Learning 115

Machine Learning 115

Confluent

DECEMBER 2, 2022

With over 4,700 stores, learn how Walmart used Kafka to build an event-driven architecture for real-time inventory management, providing a seamless omnichannel experience.

Kafka 117

Kafka Systems Architecture Building 117

Advertisement

Start-ups & SMBs launching products quickly must bundle dashboards, reports, & self-service analytics into apps. Customers expect rapid value from your product (time-to-value), data security, and access to advanced capabilities. Traditional Business Intelligence (BI) tools can provide valuable data analysis capabilities, but they have a barrier to entry that can stop small and midsize businesses from capitalizing on them.

Data Analysis