Data Engineering Podcast

OCTOBER 14, 2019

Summary Managing a data warehouse can be challenging, especially when trying to maintain a common set of patterns. Dataform is a platform that helps you apply engineering principles to your data transformations and table definitions, including unit testing SQL scripts, defining repeatable pipelines, and adding metadata to your warehouse to improve your team’s communication.

Data Warehouse 100

Data Warehouse PostgreSQL AWS Programming Language 100

KDnuggets

OCTOBER 16, 2019

A guide covering the things you should learn to become a data scientist, including the basics of business intelligence, statistics, programming, and machine learning.

Business Intelligence 120

Business Intelligence Machine Learning Programming Data 120

Uber Engineering

OCTOBER 16, 2019

Michelangelo , Uber’s machine learning (ML) platform, supports the training and serving of thousands of models in production across the company. Designed to cover the end-to-end ML workflow, the system currently supports classical machine learning, time series forecasting, and deep … The post Evolving Michelangelo Model Representation for Flexibility at Scale appeared first on Uber Engineering Blog.

Machine Learning 111

Machine Learning Engineering Designing Systems 111

Netflix Tech

OCTOBER 15, 2019

Part I: Overview Andreas Andreakis , Falguni Jhaveri , Ioannis Papapanagiotou , Mark Cho , Poorna Reddy , Tongliang Liu Overview It is a commonly observed pattern for applications to utilize multiple datastores where each is used to serve a specific need such as storing the canonical form of data (MySQL etc.), providing advanced search capabilities (ElasticSearch etc.), caching (Memcached etc.), and more.

Transportation 77

Transportation MySQL Kafka Data 77

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

Dataquest

OCTOBER 16, 2019

Exciting news: we just launched a totally revamped Data Engineering path that offers from-scratch training for anyone who wants to become a data engineer or learn some data engineering skills. Looks cool, right? But it begs the question: why learn data engineering in the first place? Typically, data science teams are comprised of data analysts, data scientists, and data engineers.

Data Engineering 52

Data Engineering Data Engineer Engineering Data Science 52

KDnuggets

OCTOBER 14, 2019

As an engineer, scientist, or researcher, you may want to take advantage of this new and growing technology, but where do you start? The best place to begin is to understand what the concept is, how to implement it, and whether it’s the right approach for a given problem.

Technology 112

Technology Engineering IT 112

Teradata

OCTOBER 15, 2019

Find out how our UX team is going to radically simplify the Teradata user experience. To be unveiled at Teradata Universe!

65

65

Dataquest

OCTOBER 16, 2019

We’ve got some really exciting news: we’ve just launched a total revamp of our Data Engineering learning path ! This revamped path is designed to be more like our other course paths. You can start it even if you have no prior experience with coding , and it’ll take you from total beginner to experienced practitioner with all of the core skills needed to become a data engineer.

Data Engineering 52

Data Engineering Data Engineer Engineering SQL 52

KDnuggets

OCTOBER 17, 2019

While the average salary for a Software Engineer is around $100,000 to $150,000, to make the big bucks you want to be an AI or Machine Learning (Specialist/Scientist/Engineer.).

Machine Learning 108

Machine Learning Software Engineer Software Engineering Engineering 108

Netflix Tech

OCTOBER 18, 2019

Faisal Siddiqi Infrastructure for Contextual Bandits and Reinforcement Learning?—? theme of the ML Platform meetup hosted at Netflix, Los Gatos on Sep 12, 2019. Contextual and Multi-armed Bandits enable faster and adaptive alternatives to traditional A/B Testing. They enable rapid learning and better decision-making for product rollouts. Broadly speaking, these approaches can be seen as a stepping stone to full-on Reinforcement Learning (RL) with closed-loop, on-policy evaluation and model objec

Algorithm 50

Algorithm Architecture Machine Learning Deep Learning 50

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

Certification

Confluent

OCTOBER 16, 2019

Trains are an excellent source of streaming data—their movements around the network are an unbounded series of events. Using this data, Apache Kafka ® and Confluent Platform can provide the foundations for both event-driven applications as well as an analytical platform. With tools like KSQL and Kafka Connect, the concept of streaming ETL is made accessible to a much wider audience of developers and data engineers.

Kafka 19

Kafka Building Data Coding 19

KDnuggets

OCTOBER 17, 2019

This post aims to make you get started with putting your trained machine learning models into production using Flask API.

Machine Learning 117

Machine Learning Python 117

KDnuggets

OCTOBER 18, 2019

If you want to launch your data science skills into freelance work, then check out these important tips to help you kick start your next adventure in data.

Data Science 90

Data Science Data Consulting 90

KDnuggets

OCTOBER 14, 2019

Selecting the perfect machine learning model is part art and part science. Learn how to review multiple models and pick the best in both competitive and real-world applications.

Machine Learning 81

Machine Learning 81

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science

KDnuggets

OCTOBER 16, 2019

This post is about various evaluation metrics and how and when to use them.

Data 120

Data Machine Learning Python 120

KDnuggets

OCTOBER 18, 2019

Read this quick overview of neural networks and learn how to implement your first in very few lines using Keras.

Coding 100

Coding Python 100

KDnuggets

OCTOBER 14, 2019

Density estimation is estimating the probability density function of the population from the sample. This post examines and compares a number of approaches to density estimation.

75

75

KDnuggets

OCTOBER 15, 2019

The way we control our data isn’t working. Data is as vulnerable as ever. Download this white paper, which outlines lessons about how data science and governance programs can, if implemented properly, reinforce each other’s objective.

Government 60

Government Data Governance Data Science Data 60

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

Building

KDnuggets

OCTOBER 15, 2019

In this research guide, we’ll look at deep learning papers aimed at synthesizing video frames within an existing video.

Deep Learning 75

Deep Learning 75

KDnuggets

OCTOBER 18, 2019

In this second part we want to outline our own experience building an AI application and reflect on why we chose not to utilise deep learning as the core technology used.

Deep Learning 59

Deep Learning Building Technology Machine Learning 59

KDnuggets

OCTOBER 17, 2019

While effective anonymization technology remains elusive, understanding the history of this challenge can guide data science practitioners to address these important concerns through ethical and responsible use of sensitive information.

Data Science 57

Data Science Data Technology 57

KDnuggets

OCTOBER 16, 2019

This week on KDnuggets: Beyond Word Embedding: Key Ideas in Document Embedding; The problem with metrics is a big problem for AI; Activation maps for deep learning models in a few lines of code; There is No Such Thing as a Free Lunch; 8 Paths to Getting a Machine Learning Job Interview; and much, much more.

Deep Learning 53

Deep Learning Machine Learning Coding Data Science 53

Advertisement

Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.

Project

KDnuggets

OCTOBER 16, 2019

Also: Kannada-MNIST: A new handwritten digits dataset in ML town; Math for Programmers; The 4 Quadrants of Data Science Skills and 7 Principles for Creating a Viral Data Visualization; The Last SQL Guide for Data Analysis You’ll Ever Need.

Process 52

Process Datasets Data Science Data Analysis 52

KDnuggets

OCTOBER 15, 2019

Follow this step-by-step tutorial using Tensorflow to setup a DC/OS Data Science Engine as a PaaS for enabling distributed multi-node, multi-GPU model training.

Data Science 55

Data Science Data Engineering Cloud Computing 55

KDnuggets

OCTOBER 16, 2019

Learn about one of the fundamental theorems of probability with an easy everyday example.

71

71

KDnuggets

OCTOBER 18, 2019

ODSC West comes to San Francisco on Oct 29 - Nov 1. With over 300 hours of content, 200+ speakers, and thousands of attendees, there is certainly a lot to see, learn, and do at the conference. Register by Friday for 10% off your pass.

Data Science 47

Data Science Data 47

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

Engineering

KDnuggets

OCTOBER 14, 2019

Also: Activation maps for deep learning models in a few lines of code; The 4 Quadrants of Data Science Skills and 7 Principles for Creating a Viral Data Visualization; OpenAI Tried to Train AI Agents to Play Hide-And-Seek but Instead They Were Shocked by What They Learned; 10 Great Python Resources for Aspiring Data Scientists.

Data Analysis 47

Data Analysis Deep Learning SQL Process 47

KDnuggets

OCTOBER 17, 2019

Watch UChicago Master of Science in Analytics capstone projects unfold in Real Data, Big Impact and see how students collaborate with their clients to deliver successful analytics projects.

Project 46

Project Data Education 46

KDnuggets

OCTOBER 14, 2019

A recent survey outlined the main neural architecture search methods used to automate the design of deep learning systems.

Architecture 57

Architecture Designing Deep Learning Systems 57

KDnuggets

OCTOBER 15, 2019

Even though I’m still in my studies, here’s a list of the most important things I’ve learned (as of yet).

Data Science 60

Data Science Data Education 60

Speaker: Margaret-Ann Seger, Head of Product, Statsig

Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating

Building