Data Engineering Podcast

SEPTEMBER 30, 2019

Summary Building an end-to-end data pipeline for your machine learning projects is a complex task, made more difficult by the variety of ways that you can structure it. Kedro is a framework that provides an opinionated workflow that lets you focus on the parts that matter, so that you don’t waste time on gluing the steps together. In this episode Tom Goldenberg explains how it works, how it is being used at Quantum Black for customer projects, and how it can help you structure your own.

Data Pipeline 100

Data Pipeline Machine Learning Media Data 100

KDnuggets

SEPTEMBER 17, 2019

We identify two main groups of Data Science skills: A: 13 core, stable skills that most respondents have and B: a group of hot, emerging skills that most do not have (yet) but want to add. See our detailed analysis.

Data Science 123

Data Science Data Deep Learning Scala 123

Teradata

SEPTEMBER 11, 2019

AI & Deep Learning allow organizations to maximize player performance while minimizing player risk through better insights from performance and wellness data.

Deep Learning 98

Deep Learning Data 98

Rockset

SEPTEMBER 13, 2019

What Is Grafana? Grafana is an open-source software platform for time series analytics and monitoring. You can connect Grafana to a large number of data sources, from PostgreSQL to Prometheus. Once your data source is connected, you can use a built-in query control or editor to fetch data, and build dashboards from your data source. Grafana is frequently deployed for a wide variety of use cases, including DevOps and AdTech.

PostgreSQL 52

PostgreSQL SQL Building Engineering 52

Advertisement

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

Database

nodeSWAT

SEPTEMBER 9, 2019

Story about unexpected slowdown during AWS RDS upgrade to AWS Aurora and InnoDB adaptive hash index parameter TL;DR at the end. The parameter. MySQL 5.7 documentation about InnoDB adaptive hash index. Turning this parameter ON enables the database engine to analyze index searches and to automatically adapt to the queries/searches you are running. It does so by making custom indexes for these specific cases, in return making your queries run faster because they can now use the automatically gener

AWS 52

AWS MySQL Database SQL 52

Pandora Engineering

SEPTEMBER 6, 2019

When embarking on the journey of developing a new application, a team must establish the foundational technologies upon which their… Continue reading on Algorithm and Blues »

Programming 52

Programming Algorithm Technology 52

KDnuggets

SEPTEMBER 9, 2019

This is a collection of 10 interesting resources in the form of articles and tutorials for the aspiring data scientist new to Python, meant to provide both insight and practical instruction when starting on your journey.

Python 122

Python Data Programming Data Science 122

Teradata

SEPTEMBER 29, 2019

The Teradata Certification program is celebrating its 20th anniversary! Find out how it can advance your career by making you a certified expert on Vantage.

Programming 66

Programming Certification IT 66

Confluent

SEPTEMBER 26, 2019

In the early days, many companies simply used Apache Kafka ® for data ingestion into Hadoop or another data lake. However, Apache Kafka is more than just messaging. The significant difference today is that companies use Apache Kafka as an event streaming platform for building mission-critical infrastructures and core operations platforms. Examples include microservice architectures, mainframe integration, instant payment, fraud detection, sensor analytics, real-time monitoring, and many more—dri

Kafka 21

Kafka BI SQL Datasets 21

Netflix Tech

SEPTEMBER 23, 2019

Niosha Behnam | Demand Engineering @ Netflix At Netflix we prioritize innovation and velocity in pursuit of the best experience for our 150+ million global customers. This means that our microservices constantly evolve and change, but what doesn’t change is our responsibility to provide a highly available service that delivers 100+ million hours of daily streaming to our subscribers.

Amazon Web Services 76

Amazon Web Services Electronics Java AWS 76

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

Certification

Pandora Engineering

SEPTEMBER 5, 2019

Credit: Sally Anscombe An Android Apprentice’s journey to understand Pandora’s migration from AsyncTask to newer APIs During my second month as an Android Engineer Apprentice, I was tasked with migrating AsyncTask to newer APIs. Early on, I was asked, “Do you know why we are migrating from AsyncTask?” I wracked my brain and answered shyly, “It has something to do with memory leaks?

Computer Science 52

Computer Science Java Software Engineer Software Engineering 52

Data Engineering Podcast

SEPTEMBER 18, 2019

Summary The conventional approach to analytics involves collecting large amounts of data that can be cleaned, followed by a separate step for analysis and interpretation. Unfortunately this strategy is not viable for handling real-time, real-world use cases such as traffic management or supply chain logistics. In this episode Simon Crosby, CTO of Swim Inc., explains how the SwimOS kernel and the enterprise data fabric built on top of it enable brand new use cases for instant insights.

Data Lake 100

Data Lake Hadoop BI Kafka 100

KDnuggets

SEPTEMBER 23, 2019

Our list of deep learning researchers and industry leaders are the people you should follow to stay current with this wildly expanding field in AI. From early practitioners and established academics to entrepreneurs and today’s top corporate influencers, this diverse group of individuals is leading the way into tomorrow’s deep learning landscape.

Deep Learning 122

Deep Learning 122

Teradata

SEPTEMBER 9, 2019

There are a lot of misperceptions about Teradata. Learn more about what Teradata Vantage really is: a cloud-first integrated data and analytics platform.

Cloud 63

Cloud Data Analytics Data 63

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

Data Science

Confluent

SEPTEMBER 5, 2019

TL;DR. Following on from How to Work with Apache Kafka in Your Spring Boot Application , which shows how to get started with Spring Boot and Apache Kafka ® , here I will demonstrate how to enable usage of Confluent Schema Registry and Avro serialization format in your Spring Boot applications. Using Avro schemas, you can establish a data contract between your microservices applications.

Java 20

Java Kafka Food Cloud 20

Airbnb Tech

SEPTEMBER 24, 2019

Scaling a Mature Data Pipeline — Managing Overhead There is often a hidden performance cost tied to the complexity of data pipelines — the overhead. In this post, we will introduce its concept, and examine the techniques we use to avoid it in our data pipelines. Author : Zachary Ennenga The view from the third floor at Airbnb HQ! Background There is often a natural evolution in the tooling, organization, and technical underpinning of data pipelines.

Data Pipeline 52

Data Pipeline Management Data Scala 52

Rockset

SEPTEMBER 20, 2019

Airbnb Prices Around Major Events Operational analytics on real-time data streams requires being able to slice and dice it along all the axes that matter to people, including time and space. We can see how important it is to analyze data spatially by looking at an app that’s all about location: Airbnb. Major events in San Francisco cause huge influxes of people, and Airbnb prices increase accordingly.

IT 40

IT Data 40

Data Engineering Podcast

SEPTEMBER 2, 2019

Summary Data professionals are working in a domain that is rapidly evolving. In order to stay current we need access to deeply technical presentations that aren’t burdened by extraneous marketing. To fulfill that need Pete Soderling and his team have been running the Data Council series of conferences and meetups around the world. In this episode Pete discusses his motivation for starting these events, how they serve to bring the data community together, and the observations that he has ma

Building 100

Building Data Database Design Media 100

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

Building

KDnuggets

SEPTEMBER 3, 2019

These three deep learning frameworks are your go-to tools for NLP, so which is the best? Check out this comparative analysis based on the needs of NLP, and find out where things are headed in the future.

Deep Learning 122

Deep Learning 122

Teradata

SEPTEMBER 2, 2019

Companies are increasingly exploring opportunities to apply reinforcement learning to their most challenging problems. Learn what applications work the best.

56

56

Confluent

SEPTEMBER 23, 2019

Kafka Summit San Francisco is just one week away. Conferences can be busy affairs, so here are some tips on getting the most out of your time there. Plan. Go and check out the schedule. Spend a bit of time familiarising yourself with what sessions you want to get to, and mark them on your calendar. How do you pick which sessions to attend? My advice: diversify!

Kafka 18

Kafka Hadoop Architecture Database 18

Netflix Tech

SEPTEMBER 10, 2019

Toby Mao , Sri Sri Perangur , Colin McFarland Another day, another custom script to analyze an A/B test. Maybe you’ve done this before and have an old script lying around. If it’s new, it’s probably going to take some time to set up, right? Not at Netflix. ABlaze: The standard view of analyses in the XP UI Suppose you’re running a new video encoding test and theorize that the two new encodes should reduce play delay, a metric describing how long it takes for a video to play after you press the s

Raw Data 86

Raw Data Python Datasets SQL 86

Advertisement

Outdated or absent analytics won’t cut it in today’s data-driven applications – not for your end users, your development team, or your business. That’s what drove the five companies in this e-book to change their approach to analytics. Download this e-book to learn about the unique problems each company faced and how they achieved huge returns beyond expectation by embedding analytics into applications.

Building

Rockset

SEPTEMBER 6, 2019

"A fast-moving technology field where new tools, technologies and platforms are introduced very frequently and where it is very hard to keep up with new trends." I could be describing either the VR space or Data Engineering, but in fact this post is about the intersection of both. Virtual Reality – The Next Frontier in Media I work as a Data Engineer at a leading company in the VR space, with a mission to capture and transmit reality in perfect fidelity.

Metadata 40

Metadata Kafka Data Cleanse SQL 40

Data Engineering Podcast

SEPTEMBER 9, 2019

Summary The first stage in every data project is collecting information and routing it to a storage system for later analysis. For operational data this typically means collecting log messages and system metrics. Often a different tool is used for each class of data, increasing the overall complexity and number of moving parts. The engineers at Timber.io decided to build a new tool in the form of Vector that allows for processing both of these data types in a single framework that is reliable an

Building 100

Building Kafka Media Data 100

KDnuggets

SEPTEMBER 5, 2019

This blog summarizes the career advice/reading research papers lecture in the CS230 Deep learning course by Stanford University on YouTube, and includes advice from Andrew Ng on how to read research papers.

Machine Learning 122

Machine Learning Deep Learning Building 122

Teradata

SEPTEMBER 5, 2019

4D analytics combines geospatial, temporal and time series data to do advanced analysis of time and space. Learn how to uncover new insights today.

Data 56

Data 56

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

Engineering

Confluent

SEPTEMBER 30, 2019

Day one of the event, summarized for your convenience. They say you never forget your first Kafka Summit. Mine was in New York City in 2017, and it had, what, 300 people? Today we welcomed nearly 2,000 to a giant ballroom in San Francisco. There were laser beams. There were two Tony Stark allusions in the first 60 seconds. And most importantly, there was a vibrant and ever-growing community present.

Kafka 17

Kafka Architecture Cloud Designing 17

KDnuggets

SEPTEMBER 24, 2019

Deep Learning is/has become the hottest skill in Data Science at the moment. There is a plethora of articles, courses, technologies, influencers and resources that we can leverage to gain the Deep Learning skills.

Deep Learning 122

Deep Learning Data Science Technology Data 122

KDnuggets

SEPTEMBER 11, 2019

As compute gets cheaper and time to market for machine learning solutions becomes more critical, we’ve explored options for speeding up model training. One of those solutions is to combine elements from Spark and scikit-learn into our own hybrid solution.

Machine Learning 122

Machine Learning Systems Python 122

KDnuggets

SEPTEMBER 17, 2019

Lately, varying improvements over BERT have been shown — and here I will contrast the main similarities and differences so you can choose which one to use in your research or application.

121

121

Speaker: David Bard, Principal at VP Product Coaching

In the fast-paced world of digital innovation, success is often accompanied by a multitude of challenges - like the pitfalls lurking at every turn, threatening to derail the most promising projects. But fret not, this webinar is your key to effective product development! Join us for an enlightening session to empower you to lead your team to greater heights.

Certification