February, 2020

article thumbnail

99th Percentile Latency at Scale with Apache Kafka

Confluent

Fraud detection, payment systems, and stock trading platforms are only a few of many Apache Kafka® use cases that require both fast and predictable delivery of data. For example, detecting […].

Kafka 145
article thumbnail

Shining A Light on Shadow IT In Data And Analytics

Data Engineering Podcast

Summary Misaligned priorities across business units can lead to tensions that drive members of the organization to build data and analytics projects without the guidance or support of engineering or IT staff. The availability of cloud platforms and managed services makes this a viable option, but can lead to downstream challenges. In this episode Sean Knapp and Charlie Crocker share their experiences of working in and with companies that have dealt with shadow IT projects and the importance of e

IT 100
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Why 2020 is the Year for 5G and IoT

Teradata

In 2020, various use cases enabled by marrying 5G & IoT will become possible, ushering in a generation where there is more data & transactions than ever before.

Data 114
article thumbnail

Essential Suite?—?Artwork Producer Assistant

Netflix Tech

Essential Suite?—?Artwork Producer Assistant By: Hamid Shahid & Syed Haq Introduction Netflix continues to invest in content for a global audience with a diverse range of unique tastes and interests. Correspondingly, the member experience must also evolve to connect this global audience to the content that most appeals to each of them. Images that represent titles on Netflix (what we at Netflix call “ artwork” ) have proven to be one of the most effective ways to help our members discover th

article thumbnail

Navigating the Future: Generative AI, Application Analytics, and Data

Generative AI is upending the way product developers & end-users alike are interacting with data. Despite the potential of AI, many are left with questions about the future of product development: How will AI impact my business and contribute to its success? What can product managers and developers expect in the future with the widespread adoption of AI?

article thumbnail

Steps for Marketing Tests

Grouparoo

In a previous post , I talked about how powerful it is to make as many trips around the build/measure/learn loop as possible. This is an abstract concept that applies just as well to product development as marketing tests. As such, it is a little abstract. I thought it would be useful to go through the steps specific for marketers, show where the current pain is felt, and how Grouparoo makes things better.

article thumbnail

Real-Time Analytics on Connected Car IoT Data Streams from Apache Kafka

Rockset

In this IoT example, we examine how to enable complex analytic queries on real-time Kafka streams from connected car sensors. Understanding IoT and Connected Cars With an increasing number of data-generating sensors being embedded in all manner of smart devices and objects, there is a clear, growing need to harness and analyze IoT data. Embodying this trend is the burgeoning field of connected cars, where suitably equipped vehicles are able to communicate traffic and operating information, such

Kafka 40

More Trending

article thumbnail

Data Infrastructure Automation For Private SaaS At Snowplow

Data Engineering Podcast

Summary One of the biggest challenges in building reliable platforms for processing event pipelines is managing the underlying infrastructure. At Snowplow Analytics the complexity is compounded by the need to manage multiple instances of their platform across customer environments. In this episode Josh Beemster, the technical operations lead at Snowplow, explains how they manage automation, deployment, monitoring, scaling, and maintenance of their streaming analytics pipeline for event data.

AWS 100
article thumbnail

What do you mean UX design is horizontal?

Teradata

There's a trend in enterprise organizations where folks are starting to refer to user experience design as a "horizontal" function, but what does this mean?

article thumbnail

Netflix Now Streaming AV1 on Android

Netflix Tech

By Liwei Guo , Vivian Li , Julie Beckley , Venkatesh Selvaraj , and Jeff Watts Today we are excited to announce that Netflix has started streaming AV1 to our Android mobile app. AV1 is a high performance, royalty-free video codec that provides 20% improved compression efficiency over our VP9† encodes. AV1 is made possible by the wide-ranging industry commitment of expertise and intellectual property within the Alliance for Open Media (AOMedia), of which Netflix is a founding member.

Media 64
article thumbnail

Introducing Confluent Developer

Confluent

Today, I am pleased to announce the launch of Confluent Developer, the one and only portal for everything you need to get started with Apache Kafka®, Confluent Platform, and Confluent […].

Kafka 140
article thumbnail

Get Better Network Graphs & Save Analysts Time

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

article thumbnail

Turning Data at REST into Data in Motion with Kafka Streams

Confluent

The world is changing fast, and keeping up can be hard. Companies must evolve their IT to stay modern, providing services that are more and more sophisticated to their customers. […].

Kafka 132
article thumbnail

Celebrating Over 100 Supported Apache Kafka Connectors

Confluent

We just released Confluent Platform 5.4, which is one of our most important releases to date in terms of the features we’ve delivered to help enterprises take Apache Kafka® and […].

Kafka 129
article thumbnail

Building a Materialized Cache with ksqlDB

Confluent

When a company becomes overreliant on a centralized database, a world of bad things start to happen. Queries become slow, taxing an overburdened execution engine. Engineering decisions come to a […].

Building 124
article thumbnail

Data Modeling That Evolves With Your Business Using Data Vault

Data Engineering Podcast

Summary Designing the structure for your data warehouse is a complex and challenging process. As businesses deal with a growing number of sources and types of information that they need to integrate, they need a data modeling strategy that provides them with flexibility and speed. Data Vault is an approach that allows for evolving a data model in place without requiring destructive transformations and massive up front design to answer valuable questions.

Data Lake 100
article thumbnail

Understanding User Needs and Satisfying Them

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

article thumbnail

The Benefits And Challenges Of Building A Data Trust

Data Engineering Podcast

Summary Every business collects data in some fashion, but sometimes the true value of the collected information only comes when it is combined with other data sources. Data trusts are a legal framework for allowing businesses to collaboratively pool their data. This allows the members of the trust to increase the value of their individual repositories and gain new insights which would otherwise require substantial effort in duplicating the data owned by their peers.

Building 100
article thumbnail

Integrating Elasticsearch and ksqlDB for Powerful Data Enrichment and Analytics

Confluent

Apache Kafka® is often deployed alongside Elasticsearch to perform log exploration, metrics monitoring and alerting, data visualisation, and analytics. It is complementary to Elasticsearch but also overlaps in some ways, […].

Kafka 119
article thumbnail

Announcing ksqlDB 0.7.0

Confluent

We are pleased to announce the release of ksqlDB 0.7.0. This release features highly available state, security enhancements for queries, a broadened range of language/data expressions, performance improvements, bug fixes, […].

Data 95
article thumbnail

Seamless SIEM – Part 1: Osquery Event Log Aggregation and Confluent Platform

Confluent

Osquery (developed by Facebook) is an open source tool used to gather audit log events from an operating system (OS). What’s unique about osquery is that it uses basic SQL […].

SQL 90
article thumbnail

Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You Need to Know

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

article thumbnail

Teradata Has Been Named One of the World's Most Ethical Companies 2020

Teradata

For the 11th consecutive year, Teradata - a leader in data analytics - has been named one of the World's Most Ethical Companies. Read more!

article thumbnail

Seamless SIEM – Part 2: Anomaly Detection with Machine Learning and ksqlDB

Confluent

We talked about how easy it is to send osquery logs to the Confluent Platform in part 1. Now, we’ll consume streams of osquery logs, detect anomalous behavior using machine […].

article thumbnail

Teradata is Launch Partner for New AWS Features

Teradata

Teradata is a launch partner for Amazon Web Services's brand-new capability: the Elastic Block Store multi-attach feature.

article thumbnail

Netflix Now Streaming AV1 on Android

Netflix Tech

By Liwei Guo , Vivian Li , Julie Beckley , Venkatesh Selvaraj , and Jeff Watts Today we are excited to announce that Netflix has started streaming AV1 to our Android mobile app. AV1 is a high performance, royalty-free video codec that provides 20% improved compression efficiency over our VP9† encodes. AV1 is made possible by the wide-ranging industry commitment of expertise and intellectual property within the Alliance for Open Media (AOMedia), of which Netflix is a founding member.

Media 64
article thumbnail

Peak Performance: Continuous Testing & Evaluation of LLM-Based Applications

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

article thumbnail

Teradata Does Open Source! Introduction to the R and Python Packages for Vantage

Teradata

In part two of this three-part series, you’ll learn how to use Teradata's R and Python packages, tdplyr and teradataml, to run machine learning and predictive analytics in Vantage at scale.

Python 52
article thumbnail

Teradata Taking Home All the Gold

Teradata

Teradata is ranked as the top solution across all leading analyst studies for Data Analytics. Read more!

article thumbnail

Kafka Summit London 2020 Agenda, Keynotes, and Other News

Confluent

Do you make New Year’s resolutions? The most I personally hear about them is people making a big show about how they don’t do them. And sure enough, I don’t […].

Kafka 18
article thumbnail

Real-Time External Indexing For Aggregations and Joins on MongoDB Collections

Rockset

Tech Preview TL;DR Join the Tech Deep Dive to learn how Rockset works with MongoDB! This is a tech preview of the MongoDB integration with Rockset to support millisecond-latency SQL queries such as joins and aggregations in real-time. Rockset builds fully mutable external indexes on any fields, including deeply nested fields in JSON documents, from your MongoDB collections.

MongoDB 52
article thumbnail

How Embedded Analytics Gets You to Market Faster with a SAAS Offering

Start-ups & SMBs launching products quickly must bundle dashboards, reports, & self-service analytics into apps. Customers expect rapid value from your product (time-to-value), data security, and access to advanced capabilities. Traditional Business Intelligence (BI) tools can provide valuable data analysis capabilities, but they have a barrier to entry that can stop small and midsize businesses from capitalizing on them.

article thumbnail

AVIF for Next-Generation Image Coding

Netflix Tech

By Aditya Mavlankar, Jan De C**k¹, Cyril Concolato, Kyle Swanson, Anush Moorthy and Anne Aaron TL; DR We need an alternative to JPEG that a) is widely supported, b) has better compression efficiency and c) has a wider feature set. We believe AV1 Image File Format (AVIF) has the potential. Using the framework we have open sourced, AVIF compression efficiency can be seen at work and compared against a whole range of image codecs that came before it.

Coding 84
article thumbnail

Where's My Tesla? Creating a Data API Using Kafka, Rockset and Postman to Find Out

Rockset

In this post I’m going to show you how I tracked the location of my Tesla Model 3 in real time and plotted it on a map. I walk through an end to end integration of requesting data from the car, streaming it into a Kafka Topic and using Rockset to expose the data via its API to create real time visualisations in D3. Getting started with Kafka When starting with any new tool I find it best to look around and see the art of the possible.

Kafka 40