Data Engineering Podcast

SEPTEMBER 18, 2019

Summary The conventional approach to analytics involves collecting large amounts of data that can be cleaned, followed by a separate step for analysis and interpretation. Unfortunately this strategy is not viable for handling real-time, real-world use cases such as traffic management or supply chain logistics. In this episode Simon Crosby, CTO of Swim Inc., explains how the SwimOS kernel and the enterprise data fabric built on top of it enable brand new use cases for instant insights.

Data Lake 100

Data Lake Hadoop BI Kafka 100

KDnuggets

SEPTEMBER 17, 2019

We identify two main groups of Data Science skills: A: 13 core, stable skills that most respondents have and B: a group of hot, emerging skills that most do not have (yet) but want to add. See our detailed analysis.

Data Science 123

Data Science Data Deep Learning Scala 123

Teradata

SEPTEMBER 17, 2019

Learn how to better classify data & analytics within the analytic ecosystem by analyzing the various states of data & analytics within organizations. Read more.

Data Analytics 45

Data Analytics Data 45

Confluent

SEPTEMBER 19, 2019

When people ask me the very top-level question “why do people use Kafka,” I usually lead with the story in my last post , where I talked about how Apache Kafka ® is helping us deliver on the promises the cloud made to us a decade ago. But I follow it up quickly with a second and potentially unrelated pattern: real-time data pipelines. These provide a different set of motivations for using an event streaming platform than scaling and microservices: specifically, the need to produce analytics resu

Kafka 14

Kafka Data Pipeline Bytes Data Architect 14

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data

Rockset

SEPTEMBER 20, 2019

Airbnb Prices Around Major Events Operational analytics on real-time data streams requires being able to slice and dice it along all the axes that matter to people, including time and space. We can see how important it is to analyze data spatially by looking at an app that’s all about location: Airbnb. Major events in San Francisco cause huge influxes of people, and Airbnb prices increase accordingly.

IT 40

IT Data 40

KDnuggets

SEPTEMBER 17, 2019

Lately, varying improvements over BERT have been shown — and here I will contrast the main similarities and differences so you can choose which one to use in your research or application.

122

122

Confluent

SEPTEMBER 20, 2019

As a distributed system for collecting, storing, and processing data at scale, Apache Kafka ® comes with its own deployment complexities. Luckily for on-premises scenarios, a myriad of deployment options are available, such as the Confluent Platform which can be deployed on bare metal, virtual machines, containers, etc. But deployment is just the tip of the iceberg.

Kafka 21

Kafka Management Cloud AWS 21

KDnuggets

SEPTEMBER 17, 2019

The article contains a brief introduction of Bioinformatics and how a machine learning classification algorithm can be used to classify the type of cancer in each patient by their gene expressions.

Machine Learning 121

Machine Learning Algorithm Python 121

KDnuggets

SEPTEMBER 16, 2019

The career path of the Data Scientist remains a hot target for many with its continuing high demand. Becoming one requires developing a broad set of skills including statistics, programming, and even business acumen. Learn more about one person's experience making this journey, and discover the many resources available to help you find your way into a world of data science.

Software Engineer 121

Software Engineer Software Engineering BI Engineering 121

KDnuggets

SEPTEMBER 20, 2019

With recent advances in AI being enabled through access to so much “Big Data” and cheap computing power, there is incredible momentum in the field. Can big data really deliver on all this hype, and what can go wrong?

Big Data 112

Big Data Data Accessible Accessibility 112

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

Engineering

KDnuggets

SEPTEMBER 17, 2019

For some people anything below 60% is acceptable and for certain others, even a correlation of 30% to 40% is considered too high because it one variable may just end up exaggerating the performance of the model or completely messing up parameter estimates.

IT 110

IT 110

KDnuggets

SEPTEMBER 18, 2019

Algorithms are at the core of data science and sampling is a critical technical that can make or break a project. Learn more about the most common sampling techniques used, so you can select the best approach while working with your data.

Algorithm 102

Algorithm Data Science Data Project 102

KDnuggets

SEPTEMBER 20, 2019

This comprehensive tutorial aims to introduce the fundamentals of PyTorch building blocks for training neural networks.

Building 117

Building Python 117

KDnuggets

SEPTEMBER 18, 2019

Also: Cartoon: Unsupervised #MachineLearning?; Cartoon: Unsupervised Machine Learning ? How to Become More Marketable as a Data Scientist; Ensemble Methods for Machine Learning: AdaBoost.

Machine Learning 91

Machine Learning Python Data 91

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

Building

KDnuggets

SEPTEMBER 19, 2019

While mature algorithms and extensive open-source libraries are widely available for machine learning practitioners, sufficient data to apply these techniques remains a core challenge. Discover how to leverage scikit-learn and other tools to generate synthetic data appropriate for optimizing and fine-tuning your models.

Machine Learning 91

Machine Learning Datasets Algorithm Data 91

KDnuggets

SEPTEMBER 20, 2019

When we create our machine learning models, a common task that falls on us is how to tune them. So that brings us to the quintessential question: Can we automate this process?

Machine Learning 90

Machine Learning Process 90

KDnuggets

SEPTEMBER 17, 2019

What other creative tools for data science beyond Python and R can you use to make an impression? It's not about the tool -- it's about its impact.

Data Science 91

Data Science Python Data IT 91

KDnuggets

SEPTEMBER 19, 2019

“I want to learn machine learning and artificial intelligence, where do I start?” Here.

Machine Learning 113

Machine Learning Data Science Python Data 113

Advertisement

Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.

Project

KDnuggets

SEPTEMBER 14, 2019

New KDnuggets Cartoon looks at one of the hottest directions in Machine Learning and asks can Machine Learning be too unsupervised?

Machine Learning 93

Machine Learning 93

KDnuggets

SEPTEMBER 18, 2019

This article covers the implementation of a data scraping and natural language processing project which had two parts: scrape as many posts from Reddit’s API as allowed &then use classification models to predict the origin of the posts.

Project 72

Project Process Data 72

KDnuggets

SEPTEMBER 19, 2019

Check out this detailed tutorial on applying data science to the cybersecurity domain, written by an individual with backgrounds in both fields.

Data Science 78

Data Science Data Machine Learning Python 78

KDnuggets

SEPTEMBER 18, 2019

Support for Python 2 will expire on Jan. 1, 2020, after which the Python core language and many third-party packages will no longer be supported or maintained. Take this survey to help determine and share your level of preparation.

Python 69

Python 69

Speaker: Margaret-Ann Seger, Head of Product, Statsig

Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating

Building

KDnuggets

SEPTEMBER 16, 2019

Also: The 5 Graph Algorithms That Data Scientists Should Know; Many Heads Are Better Than One: The Case For Ensemble Learning; BERT is changing the NLP landscape; I wasn't getting hired as a Data Scientist; There is No Free Lunch in Data Science.

Python 67

Python Algorithm Data Science Data 67

KDnuggets

SEPTEMBER 17, 2019

Join this technical webinar on Oct 3, where Domino Chief Data Scientist Josh Poduska will dive into popular open source and proprietary AutoML tools, and walk through hands-on examples of how to install and use these tools, so you can start using these technologies in your work right away.

Data Science 67

Data Science Data Technology 67

KDnuggets

SEPTEMBER 16, 2019

How to turn a typical pytorch script into a scalable d6tflow DAG for faster research & development.

Deep Learning 89

Deep Learning Python 89

KDnuggets

SEPTEMBER 16, 2019

The new emerging field that wants to study AI agents the way social scientists study humans.

90

90

Speaker: David Bard, Principal at VP Product Coaching

In the fast-paced world of digital innovation, success is often accompanied by a multitude of challenges - like the pitfalls lurking at every turn, threatening to derail the most promising projects. But fret not, this webinar is your key to effective product development! Join us for an enlightening session to empower you to lead your team to greater heights.

Certification

KDnuggets

SEPTEMBER 18, 2019

Read about how one data scientist copes with his boring days of deploying machine learning.

Data Science 87

Data Science Machine Learning Data 87

KDnuggets

SEPTEMBER 16, 2019

The UC Center for Business Analytics will present the Data Science Symposium 2019 on Oct 10 & 11, featuring 3 keynote speakers and 16 tech talks/tutorials on a wide range of data science topics and tools.

Data Science 60

Data Science Data 60

KDnuggets

SEPTEMBER 19, 2019

Whether it’s demand forecasting, supply chain management, or any other application, getting it right requires balancing the need for performance with the constraints of implementation and complexity. Learn more in this free webinar, Data-Driven Approaches to Forecasting, Sep 26.

Data 59

Data Management IT 59

KDnuggets

SEPTEMBER 20, 2019

This white paper provides the first-ever standard for managing risk in AI and ML, focusing on both practical processes and technical best practices “beyond explainability” alone. Download now.

Machine Learning 49

Machine Learning Management Process 49

Advertisement

“Reimagined: Building Products with Generative AI” is an extensive guide for integrating generative AI into product strategy and careers featuring over 150 real-world examples, 30 case studies, and 20+ frameworks, and endorsed by over 20 leading AI and product executives, inventors, entrepreneurs, and researchers.

Building