Data Engineering Podcast

SEPTEMBER 22, 2019

Summary Object storage is quickly becoming the unifying layer for data intensive applications and analytics. Modern, cloud oriented data warehouses and data lakes both rely on the durability and ease of use that it provides. S3 from Amazon has quickly become the de-facto API for interacting with this service, so the team at MinIO have built a production grade, easy to manage storage engine that replicates that interface.

AWS 100

AWS Google Cloud Cloud Storage Data Lake 100

KDnuggets

SEPTEMBER 23, 2019

Our list of deep learning researchers and industry leaders are the people you should follow to stay current with this wildly expanding field in AI. From early practitioners and established academics to entrepreneurs and today’s top corporate influencers, this diverse group of individuals is leading the way into tomorrow’s deep learning landscape.

Deep Learning 122

Deep Learning 122

Confluent

SEPTEMBER 26, 2019

In the early days, many companies simply used Apache Kafka ® for data ingestion into Hadoop or another data lake. However, Apache Kafka is more than just messaging. The significant difference today is that companies use Apache Kafka as an event streaming platform for building mission-critical infrastructures and core operations platforms. Examples include microservice architectures, mainframe integration, instant payment, fraud detection, sensor analytics, real-time monitoring, and many more—dri

Kafka 21

Kafka BI SQL Datasets 21

Teradata

SEPTEMBER 24, 2019

Time series data is found everywhere from stock prices to public health. Vantage's Machine Learning Engine helps turn that data into answers. Find out how.

Machine Learning 12

Machine Learning Engineering Data 12

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

Netflix Tech

SEPTEMBER 23, 2019

Niosha Behnam | Demand Engineering @ Netflix At Netflix we prioritize innovation and velocity in pursuit of the best experience for our 150+ million global customers. This means that our microservices constantly evolve and change, but what doesn’t change is our responsibility to provide a highly available service that delivers 100+ million hours of daily streaming to our subscribers.

Amazon Web Services 76

Amazon Web Services Electronics Java AWS 76

KDnuggets

SEPTEMBER 24, 2019

Deep Learning is/has become the hottest skill in Data Science at the moment. There is a plethora of articles, courses, technologies, influencers and resources that we can leverage to gain the Deep Learning skills.

Deep Learning 122

Deep Learning Data Science Technology Data 122

Teradata

SEPTEMBER 22, 2019

Clean data is critical to your business. Find out what three things you need to know about clean data for the health of your organization. Read more.

Data 10

Data 10

Airbnb Tech

SEPTEMBER 24, 2019

Scaling a Mature Data Pipeline — Managing Overhead There is often a hidden performance cost tied to the complexity of data pipelines — the overhead. In this post, we will introduce its concept, and examine the techniques we use to avoid it in our data pipelines. Author : Zachary Ennenga The view from the third floor at Airbnb HQ! Background There is often a natural evolution in the tooling, organization, and technical underpinning of data pipelines.

Data Pipeline 52

Data Pipeline Management Data Scala 52

KDnuggets

SEPTEMBER 23, 2019

We show, step-by-step, how to construct a single, generalized, utility function to pull images automatically from a directory and train a convolutional neural net model.

Utilities 118

Utilities Python 118

Confluent

SEPTEMBER 25, 2019

In 2011, Marc Andressen wrote an article called Why Software is Eating the World. The central idea is that any process that can be moved into software, will be. This has become a kind of shorthand for the investment thesis behind Silicon Valley’s current wave of unicorn startups. It’s also a unifying idea behind the larger set of technology trends we see today, such as machine learning, IoT, ubiquitous mobile connectivity, SaaS, and cloud computing.

Database-centric 56

Database-centric Pipeline-centric Kafka Retail 56

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

Certification

KDnuggets

SEPTEMBER 24, 2019

How can you keep your machine learning models and data organized so you can collaborate effectively? Discover this new tool set available for better version control designed for the data scientist workflow.

Machine Learning 110

Machine Learning Data Designing 110

KDnuggets

SEPTEMBER 26, 2019

Learn about the the current and future issues of data science and possible solutions from this interview with IADSS Co-founder, Dr. Usama Fayyad following his keynote speech at ODSC Boston 2019.

Data Science 107

Data Science Data 107

KDnuggets

SEPTEMBER 25, 2019

As a data scientist, you can get lost in your daily dives into the data. Consider this advice to be certain to follow in your work for being diligent and more impactful for your organization.

Data 105

Data 105

KDnuggets

SEPTEMBER 26, 2019

This article provides a brief introduction to working with natural language (sometimes called “text analytics”) in Python using spaCy and related libraries.

Python 109

Python 109

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science

KDnuggets

SEPTEMBER 26, 2019

This article shows you how to separate your customers into distinct groups based on their purchase behavior. For the R enthusiasts out there, I demonstrated what you can do with r/stats, ggradar, ggplot2, animation, and factoextra.

97

97

KDnuggets

SEPTEMBER 24, 2019

In this article, we’ll look at a couple of papers aimed at solving the problem of automated speech recognition with machine and deep learning.

Deep Learning 109

Deep Learning 109

KDnuggets

SEPTEMBER 26, 2019

Join the Crunch Data Conference in Budapest, Oct 16-18, with stellar speakers from companies like Facebook, Netflix and LinkedIn. Use the discount code ‘KDNuggets’ to save $100 off your conference ticket.

Data 96

Data Coding Data Analytics Data Science 96

KDnuggets

SEPTEMBER 27, 2019

Take me out to the ballgame! Take me out to the crowd! For the 2,829 seasons that have been played for 101 baseball teams since 1880, which seasons were unlike any others? Using SAX Encoding to recognize patterns in time series data, the most special years in baseball can be found.

Data 94

Data 94

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

Building

KDnuggets

SEPTEMBER 27, 2019

The article contains a brief introduction to various concepts related to Hierarchical clustering algorithm.

Algorithm 115

Algorithm Machine Learning Python 115

KDnuggets

SEPTEMBER 25, 2019

Today, as companies have finally come to understand the value that data science can bring, more and more emphasis is being placed on the implementation of data science in production systems. And as these implementations have required models that can perform on larger and larger datasets in real-time, an awful lot of data science problems have become engineering problems.

Data Science 87

Data Science Data Engineering Data Engineer Engineering 87

KDnuggets

SEPTEMBER 27, 2019

This live webinar, Oct 2 2019, will instruct data scientists and machine learning engineers how to build manage and deploy auto-adaptive machine learning models in production. Save your spot now.

Machine Learning 84

Machine Learning Building Engineering Management 84

KDnuggets

SEPTEMBER 23, 2019

Register now for this webinar, Sep 25 @ 12 PM ET, for a clear approach on how to apply machine learning language technology to massive, unstructured data sets in order to create predictive models of what may be the next “it” ingredient, color, flavor or pack size.

Unstructured Data 82

Unstructured Data Machine Learning Data Technology 82

Advertisement

Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.

Project

KDnuggets

SEPTEMBER 23, 2019

Also: Explore the world of Bioinformatics with Machine Learning; My journey path from a Software Engineer to BI Specialist to a Data Scientist; 5 Beginner Friendly Steps to Learn Machine Learning and Data Science with Python; 10 Great Python Resources for Aspiring Data Scientists.

Data Science 81

Data Science BI Software Engineer Software Engineering 81

KDnuggets

SEPTEMBER 25, 2019

This article covers the beta distribution, and explains it using baseball batting averages.

IT 112

IT 112

KDnuggets

SEPTEMBER 25, 2019

Penn State’s fully online data analytics program uniquely prepares students to advance their career in data science. Penn State offers 3 intakes every year and reviews applications on a rolling basis. GMAT or GRE waivers are available to highly qualified candidates. Learn more now.

Data Science 79

Data Science Data Analytics Programming Education 79

KDnuggets

SEPTEMBER 24, 2019

Of all data quality characteristics, we consider consistency and accuracy to be the most difficult ones to measure. Here, we describe the challenges that you may encounter and the ways to overcome them.

Data 70

Data 70

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

Engineering

KDnuggets

SEPTEMBER 27, 2019

Data mapping is a way to organize various bits of data into a manageable and easy-to-understand system.

Machine Learning 94

Machine Learning Data Systems Management 94

KDnuggets

SEPTEMBER 23, 2019

The new open source framework that brings multi-task learning to conversational agents.

93

93

KDnuggets

SEPTEMBER 24, 2019

AI World Conference & Expo has become the industry’s largest independent business event focused on the state of the practice of AI in the enterprise. Join us in Boston, Oct 23-25. Use the discount code 1968-KDN and SAVE $200.

Coding 52

Coding 52

KDnuggets

SEPTEMBER 25, 2019

Python Libraries for Interpretable Machine Learning; Scikit-Learn: A silver bullet for basic machine learning; I wasn't getting hired as a Data Scientist. So I sought data on who is; Which Data Science Skills are core and which are hot/emerging ones?

Machine Learning 48

Machine Learning Python Data Science Data 48

Speaker: Margaret-Ann Seger, Head of Product, Statsig

Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating

Building