Start Data Engineering
OCTOBER 5, 2021
1. Introduction 2. What is a staging area 3. The advantages of having a staging area 5. Conclusion 6. Further reading 1. Introduction Working with data pipelines, you might have noticed a staging area in most data pipelines. If you work in the data space and have questions like Why is there a staging area? Can’t we just load data into the destination tables?
Confluent
OCTOBER 6, 2021
Today, an organization’s strategic objective is to deliver innovations for a connected life and to improve the quality of life worldwide. With connected devices comes data, and with data comes […].
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Cloudera
OCTOBER 7, 2021
October sees the launch of Partner Appreciation Month and during the next few weeks we will be sharing success stories, updates and interviews with our valued partners across the world. . We’re on a mission to make data and analytics easy and accessible, for everyone, and the hybrid data cloud is how we’ll get there. Today’s world is a hybrid world—there’s hybrid data, hybrid infrastructure, hybrid work—and leading businesses are embracing these changes, unafraid to transform their processes and
Teradata
OCTOBER 4, 2021
An interdisciplinary team from Volkswagen, AWS and Teradata have created an intelligent solution that enables greater transparency and efficiency in car body construction. Find out more.
Speaker: Timothy Chan, PhD., Head of Data Science
Are you ready to move beyond the basics and take a deep dive into the cutting-edge techniques that are reshaping the landscape of experimentation? 🌐 From Sequential Testing to Multi-Armed Bandits, Switchback Experiments to Stratified Sampling, Timothy Chan, Data Science Lead, is here to unravel the mysteries of these powerful methodologies that are revolutionizing how we approach testing.
Start Data Engineering
OCTOBER 3, 2021
1. Introduction 2. Business requirements: dashboards and analytics 3. What is a data warehouse 4. OLTP vs OLAP based data warehouses 5. Conclusion 6. Further reading 7. References 1. Introduction If you are a student, analyst, engineer, or anyone in the data space, it’s important to understand what a data warehouse is. If you are wondering What is a data warehouse?
Netflix Tech
OCTOBER 7, 2021
Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , and Colin McFarland This is the third post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. Need to catch up? Have a look at Part 1 (Decision Making at Netflix) and Part 2 (What is an A/B Test?). Subsequent posts will go into more details on experimentation across Netflix, how Netflix has invested in infrastructure to support and scale experimentation, and the i
Data Engineering Digest brings together the best content for data engineering professionals from the widest variety of industry thought leaders.
DataKitchen
OCTOBER 7, 2021
The post How Predictive and Prescriptive Analytics Improve the Call Center Experience first appeared on DataKitchen.
Pipeline Data Engineering
OCTOBER 8, 2021
In this interview series we’ll share some of the stories that Daniel and I get to watch unfold at Pipeline Academy. Check out what our graduates have to say about the course, how they’ve tackled its challenges and what they are doing now with their new data engineering superpowers. Peter: Michele, it's great to see you again. Thanks for taking the time to have a chat with me.
Netflix Tech
OCTOBER 7, 2021
By Minal Mishra Quality of a client application is of paramount importance to global digital products, as it is the primary way customers interact with a brand. At Netflix, we have significant investments in ensuring new versions of our applications are well tested. However, Netflix is available for streaming on thousands of types of devices and it is powered by hundreds of micro-services which are deployed independently, making it extremely challenging to comprehensively test internally.
Cloudera
OCTOBER 5, 2021
If your organization is using multi-tenant big data clusters (and everyone should be), do you know the usage and cost efficiency of resources in the cluster by tenants? A chargeback or showback model allows IT to determine costs and resource usage by the actual analytic users in the multi-tenant cluster, instead of attributing those to the platform (“overhead’) or IT department.
Speaker: Anne Steiner and David Laribee
As a concept, Developer Experience (DX) has gained significant attention in the tech industry. It emphasizes engineers’ efficiency and satisfaction during the product development process. As product managers, we need to understand how a good DX can contribute not only to the well-being of our development teams but also to the broader objectives of product success and customer satisfaction.
DataKitchen
OCTOBER 8, 2021
The post DataOps Lowers The Cost Of Asking Analytic Questions first appeared on DataKitchen.
ProjectPro
OCTOBER 8, 2021
Retail is one of the first industries that started leveraging the power of machine learning and artificial intelligence. There are machine learning projects for almost every retail use case - right from inventory management to customer satisfaction. Machine learning projects in retail directly convert into profits and increase an organization’s market share with better customer acquisition and satisfaction.
Monte Carlo
OCTOBER 7, 2021
I first “joined” Monte Carlo exactly a year ago, as a data science intern. I met Lior , our co-founder, on Zoom in August of 2020. I had cast a volley of solicitous emails into my network — “(sort of) Stanford C.S. student looking to be (sort of) hired and avoid school for a while” — and one opportunity had come back from Oren and Glenn , former colleagues and now mentors of mine at GGV.
Grouparoo
OCTOBER 6, 2021
Last month, we decided that we should all read a book and talk about it as a company. It was a fun experience and I think we made a good choice by picking 97 Things Every Data Engineer Should Know. This was the first book I have read in this series and I liked the format. It is made up of 97 small vignettes that are 2-3 pages each. This provided a nice overview of the breadth of topics that are relevant to data engineering including data warehouses/lakes, pipelines, metadata, security, complianc
Speaker: Aarushi Kansal, AI Leader & Author and Tony Karrer, Founder & CTO at Aggregage
Software leaders who are building applications based on Large Language Models (LLMs) often find it a challenge to achieve reliability. It’s no surprise given the non-deterministic nature of LLMs. To effectively create reliable LLM-based (often with RAG) applications, extensive testing and evaluation processes are crucial. This often ends up involving meticulous adjustments to prompts.
DataKitchen
OCTOBER 6, 2021
The post Why Enterprise AI Needs Human Intervention first appeared on DataKitchen.
A Cloud Guru: Data Engineering
OCTOBER 6, 2021
In this post, we’ll discuss the difference between ETL vs ELT and when you might choose ETL or ELT. We’ll also include a flowchart to help walk you through the ETL vs ELT decision-making process. The difference between ETL vs ELT What’s the difference between ETL and ELT? The short answer is it’s all about […] The post ETL vs ELT flowchart: When to use each appeared first on A Cloud Guru.
ProjectPro
OCTOBER 6, 2021
With 67 zones, 140 edge locations, over 90 services, and 940163 organizations using GCP across 200 countries - GCP is slowly garnering the attention of cloud users in the market. Flexera’s State of Cloud report highlighted that 41% of the survey respondents showed the most interest in using Google Cloud Platform for their future cloud computing projects.
Big Data Tools
OCTOBER 5, 2021
In most countries, students start learning in September. As data engineers, let’s follow their lead and learn something new, too! I’m Pasha Finkelshteyn , and I’ll be your guide through this month’s news. I’ll offer my impressions of developments and highlight ideas from the wider community. If you think I missed something worthwhile, ping me on Twitter and suggest a topic, link, or anything else.
Advertisement
Are you trying to decide which entity resolution capabilities you need? It can be confusing to determine which features are most important for your project. And sometimes key features are overlooked. Get the Entity Resolution Evaluation Checklist to make sure you’ve thought of everything to make your project a success! The list was created by Senzing’s team of leading entity resolution experts, based on their real-world experience.
DataKitchen
OCTOBER 5, 2021
A DataOps Engineer owns the assembly line that’s used to build a data and analytic product. Data operations (or data production) is a series of pipeline procedures that take raw data, progress through a series of processing and transformation steps, and output finished products in the form of dashboards, predictions, data warehouses or whatever the business requires.
Teradata
OCTOBER 6, 2021
This how-to guide will help you connect Teradata Vantage with the AWS Data Exchange service. Read more for step-by-step instructions.
ProjectPro
OCTOBER 6, 2021
This blog is a step-by-step guide for a beginner in NLP. If you are someone who wants to know what is the best way to learn NLP from scratch, then please go through our blog till the end. We assure you will build the confidence and gear up yourself to make a career transition into data science as an NLP Engineer. We will first begin with what are the essential subjects one must be aware of, to prepare them for diving into the world of Natural Language Processing.
Big Data Tools
OCTOBER 5, 2021
In most countries, students start learning in September. As data engineers, let’s follow their lead and learn something new, too! I’m Pasha Finkelshteyn , and I’ll be your guide through this month’s news. I’ll offer my impressions of developments and highlight ideas from the wider community. If you think I missed something worthwhile, ping me on Twitter and suggest a topic, link, or anything else.
Speaker: Margaret-Ann Seger, Head of Product, Statsig
Experimentation is often seen as an aspirational practice, especially at smaller, fast-moving companies who are strapped for time and resources. So, how can you get your team making decisions in a more data-driven way while continuing to remain lean and maintaining ship velocity? In this webinar, Margaret-Ann Seger, Head of Product at Statsig, will teach you how to build an experimentation culture from the ground-up, graduating from just getting started with data-driven development to operating
Preset
OCTOBER 4, 2021
We discuss how to use dbt transformation (data build tool) to convert JSON data from GitHub into clean, tidy data for visualization.
Eventbrite Engineering
OCTOBER 5, 2021
JC Fant IV Oct-5th-2021 History Over the course of the last 21 years I’ve spent time in nearly every aspect of the technical stack, however, I’ve always been drawn to the frontend as the best place to be able to impact customers. I’ve enjoyed the rapid iterations, and the ability to visualize those changes in … Continue reading "Creating a 3 Year Frontend Vision" The post Creating a 3 Year Frontend Vision appeared first on Engineering Blog.
RudderStack
OCTOBER 5, 2021
In this update, we share incremental uploads for Reverse ETL and new integrations.
Cloudera
OCTOBER 5, 2021
Cloudera Data Platform (CDP) supports access controls on tables and columns, as well as on files and directories via Apache Ranger since its first release. It is common to have different workloads using the same data – some require authorizations at the table level (Apache Hive queries) and others at the underlying files (Apache Spark jobs). Unfortunately, in such instances you would have to create and maintain separate Ranger policies for both Hive and HDFS, that correspond to each othe
Speaker: David Bard, Principal at VP Product Coaching
In the fast-paced world of digital innovation, success is often accompanied by a multitude of challenges - like the pitfalls lurking at every turn, threatening to derail the most promising projects. But fret not, this webinar is your key to effective product development! Join us for an enlightening session to empower you to lead your team to greater heights.
Data Engineering Podcast
OCTOBER 8, 2021
Summary The key to making data valuable to business users is the ability to calculate meaningful metrics and explore them along useful dimensions. Business intelligence tools have provided this capability for years, but they don’t offer a means of exposing those metrics to other systems. Metriql is an open source project that provides a headless BI system where you can define your metrics and share them with all of your other processes.
Zalando Engineering
OCTOBER 4, 2021
The problem When building Machine Learning (ML) applications - such as recommender systems - there is often a need to provide a "feature store" which can enrich the request to the system with additional ML features. For example: whether a user had looked at an article before is often very informative about whether the user will click or buy that article this time.
AltexSoft
OCTOBER 8, 2021
To drive deeper business insights and greater revenues, organizations — whether they are big or small — need quality data. But more often than not data is scattered across a myriad of disparate platforms, databases, and file systems. What’s more, that data comes in different forms and its volumes keep growing rapidly every day — hence the name of Big Data.
Datakin
OCTOBER 5, 2021
Last week, Matt Turck and John Wu published the latest annual report on the state of data, the 2021 Machine Learning, AI and Data (MAD) Landscape. If you haven’t read it yet, we recommend it as a comprehensive snapshot of the intricate world of AI, machine learning, and data science & engineering. Our team enjoyed reading it. We represent several of the pixels on this chart (hey, cool!
Advertisement
“Reimagined: Building Products with Generative AI” is an extensive guide for integrating generative AI into product strategy and careers featuring over 150 real-world examples, 30 case studies, and 20+ frameworks, and endorsed by over 20 leading AI and product executives, inventors, entrepreneurs, and researchers.
Expert insights. Personalized for you.
130 
Let's personalize your content