April, 2020

article thumbnail

Review: Building a Real Time Data Warehouse

Start Data Engineering

Many data engineers coming from traditional batch processing frameworks have questions about real time data processing systems, like “What kind of data model did you implement, for real-time processing?

article thumbnail

Preventing Fraud and Fighting Account Takeovers with Kafka Streams

Confluent

Many companies have recently started to take cybersecurity and data protection even more seriously, particularly driven by the recent General Data Protection Regulation (GDPR) legislation. They are increasing their investment […].

Kafka 145
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Building Real Time Applications On Streaming Data With Eventador

Data Engineering Podcast

Summary Modern applications frequently require access to real-time data, but building and maintaining the systems that make that possible is a complex and time consuming endeavor. Eventador is a managed platform designed to let you focus on using the data that you collect, without worrying about how to make it reliable. In this episode Eventador Founder and CEO Kenny Gorman describes how the platform is architected, the challenges inherent to managing reliable streams of data, the simplicity off

Building 100
article thumbnail

Teradata Supports China’s Fight Against COVID-19

Teradata

By fully utilizing the data for telco operators in China, Teradata helped communities battle the COVID-19 epidemic through ongoing public health communication, travel updates and inquiries.

Utilities 111
article thumbnail

Get Better Network Graphs & Save Analysts Time

Many organizations today are unlocking the power of their data by using graph databases to feed downstream analytics, enahance visualizations, and more. Yet, when different graph nodes represent the same entity, graphs get messy. Watch this essential video with Senzing CEO Jeff Jonas on how adding entity resolution to a graph database condenses network graphs to improve analytics and save your analysts time.

article thumbnail

How Netflix brings safer and faster streaming experience to the living room on crowded networks…

Netflix Tech

How Netflix brings safer and faster streaming experience to the living room on crowded networks using TLS 1.3 By Sekwon Choi At Netflix, we are obsessed with the best streaming experiences. We want playback to start instantly and to never stop unexpectedly in any network environment. We are also committed to protecting users’ privacy and service security without sacrificing any part of the playback experience.

article thumbnail

5 Must Know Workforce Analytics as an HR Manager

U-Next

Workforce analytics – what sounds like a complex technical term is relatively easy and inevitable when you understand its importance and how to leverage its full potential to your benefit. Some of the elite HR analysts out there have implemented workforce analytics in their businesses and have seen results that their competitors couldn’t dream of. As an HR manager, you need to be aware of these analytics concepts and theories to pave way for organizational and departmental success and curb any b

More Trending

article thumbnail

What’s New in Apache Kafka 2.5

Confluent

On behalf of the Apache Kafka® community, it is my pleasure to announce the release of Apache Kafka 2.5.0. The community has created another exciting release. We are making progress […].

Kafka 144
article thumbnail

Building A Knowledge Graph Of Commercial Real Estate At Cherre

Data Engineering Podcast

Summary Knowledge graphs are a data resource that can answer questions beyond the scope of traditional data analytics. By organizing and storing data to emphasize the relationship between entities, we can discover the complex connections between multiple sources of information. In this episode John Maiden talks about how Cherre builds knowledge graphs that provide powerful insights for their customers and the engineering challenges of building a scalable graph.

Building 100
article thumbnail

All Models Are Wrong (But Some Are Useful)

Teradata

Lots of smart people have created many predictive analytics models to help us manage the COVID-19 pandemic. But many of these models use different inputs, different heuristics, and come to different conclusions.

article thumbnail

Bringing 4K and HDR to Anime at Netflix with Sol Levante

Netflix Tech

By Haruka Miyagawa & Kylee Peña Continue reading on Netflix TechBlog ».

96
article thumbnail

Understanding User Needs and Satisfying Them

Speaker: Scott Sehlhorst

We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.

article thumbnail

Scala For Big Data Engineering – Why should you care?

Advancing Analytics: Data Engineering

The thought of learning Scala fills many with fear, its very name often causes feelings of terror. This suggests it’s either doing something very good, or very bad! The truth is Scala can be used for many things; from a simple web application to complex ML (Machine Learning). Moreover, it unusually fully incorporates two programming paradigms: OOP (Object Orientated Programming) and FP (Functional programming).

Scala 52
article thumbnail

Advantages of Using dbt(Data Build Tool)

Start Data Engineering

In this article we aim to go over the reasoning behind why someone might want to use dbt. If you are interested in learning dbt checkout this article.

Building 130
article thumbnail

Confluent Raises $250M and Kicks Off Project Metamorphosis

Confluent

Confluent Raises $250M and Kicks Off Project Metamorphosis It’s an exciting day for Confluent, in the middle of a very unusual and difficult time in the larger world. Nonetheless, I […].

Project 142
article thumbnail

AI and Automation Quick Wins that HR Teams Should Focus On

U-Next

Is your HR department failing to embrace the digital transformation (especially AI) revolution? If so, you are not alone. Human resources departments are notorious for lagging behind in adopting new technologies. It comes as no surprise that the use of automation and artificial intelligence in HR is still relatively rare compared to other departments in organizations across different industries.

article thumbnail

Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You Need to Know

Speaker: Timothy Chan, PhD., Head of Data Science

Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.

article thumbnail

Breaking the COVID-19 Chain with Data Analytics

Teradata

How can Teradata's data analytics platform help communities stop the spread of COVID-19? Find out more.

article thumbnail

Lessons Gleaned from Attending and Speaking at the World Economic Forum for Africa 2019 Gathering

Hepta Analytics

Last year in September, Hepta Analytics was amongst the few startup companies invited to participate in the World Economic Forum for Africa in Cape Town, South Africa. Such a rare opportunity for a young company like ours given the high profile individuals, such as heads of states invited to attend these types of events. It also included a great mix of local, regional and international companies execs, academic and civil society leaders, all coming together to discuss one thing: Shaping inclusi

Food 52
article thumbnail

Google Sheets Source

Grouparoo

Grouparoo is the Reverse ETL platform to connect Google Sheets data to your SaaS tools. This enables all of those crazy sheets out there to be the source of truth for your profiles and be fed into your marketing tools. Don't forget: with great power comes great responsibility! Google setup In Grouparoo, apps make the connection to facilitate data movement in the form of sources and destinations.

MySQL 52
article thumbnail

Apache Airflow Review: the good, the bad

Start Data Engineering

When getting started with Apache Airflow , data engineers have questions similar to the two below “What are people’s opinions of Airflow?

article thumbnail

Peak Performance: Continuous Testing & Evaluation of LLM-Based Applications

Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage

Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.

article thumbnail

Introducing Confluent Platform 5.5

Confluent

We are pleased to announce the release of Confluent Platform 5.5. With this release, Confluent makes event streaming more broadly accessible to developers of all backgrounds, enhancing three categories of […].

article thumbnail

Predict Attrition in a Company by Help of Analytics

U-Next

There’s always a sense of apprehension when someone walks down to the HR desk to put down their papers. More so if it is a key employee whose loss is going to be a definite setback. Then people wonder – the upper management, the line manager, the HR department – how it is that they never saw this coming. There used to be a time when employee retention processes would kick in only after an employee resigned.

Systems 52
article thumbnail

I’m Sorry CXOs, but You’re Mostly Doing Analytics All Wrong

Teradata

There is no ROI in technology - specifically in data analytics, AI & Machine Learning - until we deploy in production and change the way we do business.

article thumbnail

Index Scan: Using Rockset's Search Index to Speed up Range Scans Over a Specific Field

Rockset

Recently, InfoWorld’s Martin Heller described Rockset as a "one-of-a-kind database for operational analytics." After testing Rockset with a variety of queries on a large collection, Heller rated Rockset 4.5 out of 5 stars. Heller’s review of Rockset can be found here. Only one of the test queries timed out: SELECT * FROM commons."twitter-firehose" ORDER BY "twitter-firehose".favorite_count DESC LIMIT 10 For context, twitter-firehose is one of Rockset’s demo collections.

article thumbnail

The Big Payoff of Application Analytics

Outdated or absent analytics won’t cut it in today’s data-driven applications – not for your end users, your development team, or your business. That’s what drove the five companies in this e-book to change their approach to analytics. Download this e-book to learn about the unique problems each company faced and how they achieved huge returns beyond expectation by embedding analytics into applications.

article thumbnail

Open Sourcing a GitHub Engagement Dashboard

Preset

This post details the process of building a GitHub community dashboard by extracting data out of the GitHub API, loading it into a database, and building a Superset dashboard on top of it.

article thumbnail

Taming Complexity In Your Data Driven Organization With DataOps

Data Engineering Podcast

Summary Data is a critical element to every role in an organization, which is also what makes managing it so challenging. With so many different opinions about which pieces of information are most important, how it needs to be accessed, and what to do with it, many data projects are doomed to failure. In this episode Chris Bergh explains how taking an agile approach to delivering value can drive down the complexity that grows out of the varied needs of the business.

Hadoop 100
article thumbnail

Confluent Platform Now Supports Protobuf, JSON Schema, and Custom Formats

Confluent

When Confluent Schema Registry was first introduced, Apache Avro™ was initially chosen as the default format. While Avro has worked well for many users, over the years, we’ve received many […].

Data 102
article thumbnail

How to Make the Most of HR Analytics?

U-Next

It’s time that organizations realize great skillsets like HR Analytics, are the key to bigger businesses. Companies now find it fancy to promote contests around “Best place to work” and the popularity of LinkedIn and Employee Relations departments have gained a significant amount of importance in the last few years. “Human Resources isn’t a thing we do.

Process 52
article thumbnail

From Developer Experience to Product Experience: How a Shared Focus Fuels Product Success

Speaker: Anne Steiner and David Laribee

As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.

article thumbnail

Teradata and the MIT COVID Challenge Hackathon

Teradata

Teradata participated in the MIT COVID Challenge Hackathon to design approaches and mentor teams focused on stamping out the pandemic. Learn more.

article thumbnail

Case Study: Fleet Management System – An End-to-End Streaming Data Pipeline

Rockset

PROBLEM STATEMENT: Fleet operators often suffer business and monetary losses due to a lack of information on the health of their fleet and inventory it carries. This problem arises due to a lack of real-time data on vehicle health or inventory health, to take preemptive action or real-time action. EXAMPLES: A vehicle’s coolant is leaking and engine temperature is going up.

article thumbnail

Unlock Value From Your Data Lake with Dremio and Superset

Preset

Unlock Value From Your Data Lake with Dremio and Superset

article thumbnail

Making Data Collection In Your Code Easy With Rookout

Data Engineering Podcast

Summary The software applications that we build for our businesses are a rich source of data, but accessing and extracting that data is often a slow and error-prone process. Rookout has built a platform to separate the data collection process from the lifecycle of your code. In this episode, CTO Liran Haimovitch discusses the benefits of shortening the iteration cycle and bringing non-engineers into the process of identifying useful data.

article thumbnail

The Path to Product Excellence: Avoiding Common Pitfalls and Enhancing Communication

Speaker: David Bard, Principal at VP Product Coaching

In the fast-paced world of digital innovation, success is often accompanied by a multitude of challenges - like the pitfalls lurking at every turn, threatening to derail the most promising projects. But fret not, this webinar is your key to effective product development! Join us for an enlightening session to empower you to lead your team to greater heights.