Confessions of a Data Guy

DECEMBER 1, 2022

Nothing captures the imagination and heart like a tale of betrayal and heartbreak, and that is a tale I want to bring to you today. It’s a tale of Databricks Workflows and Jobs, version changes, new features, API’s, and insidious little hidden gems that will make you pull your hair out when you find them. […] The post A Tale of Betrayal and Heartbreak – Databricks Workflows and Jobs. appeared first on Confessions of a Data Guy.

Data 130

Data IT Big Data Data Engineering 130

KDnuggets

DECEMBER 1, 2022

In this blog, I shared my story on getting 4 data science job offers including Airbnb, Lyft and Twitter after being laid off. Any data scientist who was laid off due to the pandemic or who is actively looking for a data science position can find something here to which they can relate.

Data Science 160

Data Science Data 160

Confluent

DECEMBER 2, 2022

ksqlDB use case: see how apps can use ksqlDB to ingest, filter, enrich, aggregate, and query data directly with Kafka—no complex architectures or data stores needed.

Kafka 144

Kafka Building Architecture Data 144

Teradata

NOVEMBER 30, 2022

Teradata has joined the Amazon SageMaker Ready Program which differentiates Teradata as an AWS Partner Network member with a product that works with Amazon SageMaker & fully supports AWS customers.

Programming 97

Programming Designing AWS 97

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science

Cloudera

DECEMBER 1, 2022

Full-stack observability is a critical requirement for effective modern data platforms to deliver the agile, flexible, and cost-effective environment organizations are looking for. For analytic applications to properly leverage a hybrid, multi-cloud ecosystem to support modern data architectures, data observability has become even more important. I spoke to Mark Ramsey of Ramsey International (RI) to dive deeper into that last subject.

Data Lake 82

Data Lake Data Pipeline Data Governance Analytics Application 82

KDnuggets

DECEMBER 2, 2022

The data science field is full of job opportunities, yet there is still a lot of confusion about what data scientists actually do. This confusion is largely due to the many myths that exist about the role of a data scientist. In this article, we will bust the top 10 myths about data science. By the end of this article, you will have a better understanding of the role of a data scientist and what it takes to be one.

Data Science 158

Data Science Data IT 158

Booking.com Engineering

DECEMBER 2, 2022

Booking.com’s mission is to make it easier for everyone to experience the world. To help people discover destinations, we are a leading travel advertiser on Google Pay Per Click (PPC). Booking Holdings, as a whole, spent $4.7 billion in marketing across all brands in the first nine months of 2022[1]. How do we run PPC at our scale, and efficiently? In this article, we want to illustrate our extensive use of the public cloud, specifically Google Cloud Platform (GCP).

Systems 52

Systems Cloud MySQL Relational Database 52

Cloudera

NOVEMBER 30, 2022

What is CDP Operational Database (COD). CDP Operational Database enables developers to quickly build future-proof applications that are architected to handle data evolution. It helps developers automate and simplify database management with capabilities like auto-scale, and is fully integrated with Cloudera Data Platform (CDP). For more information and to get started with COD, refer to Getting Started with Cloudera Data Platform Operational Database (COD).

Database 81

Database Datasets NoSQL Big Data 81

KDnuggets

DECEMBER 1, 2022

The latest KDnuggets exclusive cheatsheet covers the essentials of machine learning with Scikit-learn.

Machine Learning 159

Machine Learning 159

Confluent

DECEMBER 2, 2022

Major improvements to the Kafka consumer, Streams, and ksqlDB for incremental cooperative rebalancing while maintaining at-least-once and exactly-once guarantees.

Kafka 138

Kafka Process 138

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

Engineering

Pinterest Engineering

DECEMBER 2, 2022

Grey Skold | (former Android Video Engineer) ; Lin Wang | Android Performance Engineer; Sheng Liu | Android Performance Engineer Pinterest Android App offers a rare experience with a mix of images and videos on a two-column grid. In order to maintain a performant video experience on Android devices, we focused on: Warming up Configurations Pooling players Warming Up In order to reduce the startup latency, we establish a video network connection by sending a dummy HTTP HEAD request during the ear

Media 52

Media Engineering Utilities Architecture 52

Scott Logic

NOVEMBER 30, 2022

An introduction to Markdown Markdown is a brilliant tool for quickly writing up universally accessible documents. Created by John Gruber and Aaron Schwartz in 2004, it stands as one of the most popular and widely used markup languages around. It uses simple and intuitive formatting that can be easily read and understood. “A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions” John Gruber, creator of

Programming Language 52

Programming Language Utilities Designing Coding 52

KDnuggets

NOVEMBER 30, 2022

The best way to learn Data Science is by solving real-world problems with the data and building your own portfolio. In this article, we will discuss three projects that you can work on to build your portfolio and impress interviewers.

Data Science 115

Data Science Project Portfolio Data 115

Confluent

NOVEMBER 29, 2022

Apache Kafka’s Streams API embeds Machine Learning into any app or microservice (Java, Docker, Kubernetes, etc.) to add business value.

Machine Learning 145

Machine Learning Kafka Java Building 145

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

Building

Tweag

NOVEMBER 30, 2022

There is an inherent beauty to be found in simple, pervasive ideas that shift our perspective on familiar objects. Such ideas can help tame the complexity of abstruse abstractions by offering a more intuitive angle from which to understand them. The aim of this post is to present an alternative angle — that of interactive semantics — from which to view one of the fundamental notion of functional programming: higher-order functions.

Programming Language 52

Programming Language Programming Computer Science Coding 52

DataKitchen

DECEMBER 1, 2022

Data Team members, have you ever felt overwhelmed? The never-ending flow of new information can be stressful, and it’s hard to know where to start. Well, don’t worry because DataOps is here to help! In this post, we’ll discuss how DataOps Observability and Automation can relieve team stress and show you how to get started. So don’t wait any longer.

Data Engineering 52

Data Engineering Data Engineer Data Pipeline Data Analytics 52

KDnuggets

DECEMBER 2, 2022

Personalized learning, smart grading, skill gap assessment, and better ROI: The importance of incorporating Machine Learning in Online Learning cannot be overstated.

Machine Learning 122

Machine Learning 122

Confluent

DECEMBER 2, 2022

Build fast, break nothing. Learn about the unique challenges Confluent's engineering team has faced building ksqlDB and continuously shipping the latest, greatest features.

Building 123

Building Engineering Process 123

Advertisement

Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.

Project

Striim

NOVEMBER 30, 2022

Businesses of all scales and industries have access to increasingly large amounts of data, which need to be harnessed effectively. According to an IDG Market Pulse survey , companies collect data from 400 sources on average. Companies that can’t process and analyze it to glean useful insights for their operations are falling behind. Thousands of companies are centralizing their analytics and applications on the AWS ecosystem.

AWS 52

AWS Cloud Management MySQL 52

Workfall

NOVEMBER 29, 2022

Reading Time: 10 minutes MongoDB is one of the most popular No-SQL databases in the developer community today. Instead of SQL objects, No-SQL databases allow developers to send and retrieve data as JSON documents. In this blog, we will demonstrate how to connect to MongoDB using Mongoose and MongoDB Atlas in Node.js. Let’s get started! In this blog, we will cover: What is MongoDB?

MongoDB 52

MongoDB Database NoSQL SQL 52

KDnuggets

NOVEMBER 29, 2022

First steps to learning data science & machine learning are the foundations.

Machine Learning 160

Machine Learning Data Science Data 160

Confluent

DECEMBER 2, 2022

Datadog and Confluent integration brings new monitoring, metrics, and enterprise capabilities for Kafka. Monitor Kafka Connect, ksqlDB, Schema Registry, REST Proxy, and more.

Kafka 117

Kafka 117

Speaker: Margaret-Ann Seger, Head of Product, Statsig

Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating

Building

Scott Logic

NOVEMBER 29, 2022

For over a decade, the representation of women at senior levels has been on the agenda in the corporate world, and yet women are still underrepresented in UK businesses – with women occupying only 30% of management roles. That number is worse still in the tech industry, with women occupying just 5% of the top jobs. Last week I attended ‘Women of Silicon Roundabout’ – the UK’s largest tech event for women.

Building 52

Building Designing Process Management 52

Cloudyard

NOVEMBER 28, 2022

Read Time: 1 Minute, 36 Second During this post we will discuss how Schema object PRIVILEGES behaves during Cloning the Database and Schema. Consider the Scenario when entire source Database needs to be CLONE at target end. In addition to it we want all the GRANTS applied to Source DB should be readily available in CLONED Database. At first glance it seems pretty straightforward.

Database 52

Database IT 52

KDnuggets

DECEMBER 1, 2022

Introduction to PyTorch Lightning and how it can be used for the model building process. It also provides a brief overview of the PyTorch characteristics and how they are different from TensorFlow.

Building 108

Building Process IT Machine Learning 108

Confluent

NOVEMBER 30, 2022

What is stream processing, or complex event processing (CEP), and how does it work? Learn about real-time data and event stream analytics in this tutorial.

Process 125

Process Data IT 125

Speaker: David Bard, Principal at VP Product Coaching

In the fast-paced world of digital innovation, success is often accompanied by a multitude of challenges - like the pitfalls lurking at every turn, threatening to derail the most promising projects. But fret not, this webinar is your key to effective product development! Join us for an enlightening session to empower you to lead your team to greater heights.

Certification

Data Engineering Weekly

NOVEMBER 27, 2022

Data Engineering Weekly Is Brought to You by RudderStack RudderStack provides data pipelines that make it easy to collect data from every application, website, and SaaS platform, then activate it in your warehouse and business tools. Sign up free to test out the tool today. Data Contracts for SaaS Developers with Benn Stancil The Thanksgiving break gives me enough time to catch up on a few podcasts.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

DataKitchen

NOVEMBER 28, 2022

Part 4: Reviewing the Benefits. This is the final post in DataKitchen’s four-part series on DataOps Observability. Observability is a methodology for providing visibility of every journey that data takes from source to customer value across every tool, environment, data store, team, and customer so that problems are detected and addressed immediately.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Process 52

KDnuggets

NOVEMBER 29, 2022

What is Chebychev's Theorem and How Does it Apply to Data Science? • How to Select Rows and Columns in Pandas Using [ ],loc, iloc,at and.iat • Linux for Data Science Cheatsheet • How Much Math Do You Need in Data Science? • Git for Data Science Cheatsheet.

Data Science 108

Data Science IT Data 108

Confluent

DECEMBER 2, 2022

With over 4,700 stores, learn how Walmart used Kafka to build an event-driven architecture for real-time inventory management, providing a seamless omnichannel experience.

Kafka 117

Kafka Systems Architecture Building 117

Advertisement

“Reimagined: Building Products with Generative AI” is an extensive guide for integrating generative AI into product strategy and careers featuring over 150 real-world examples, 30 case studies, and 20+ frameworks, and endorsed by over 20 leading AI and product executives, inventors, entrepreneurs, and researchers.

Building