DoorDash Engineering

OCTOBER 17, 2023

SRM represents one of the most egregious data quality issues in A/B tests because it fundamentally compromises the basic assumption of random assignment. High-fives are exchanged and your team is already setting its sights on the next big project. The term itself conjures a sense of rigor, validity, and trust.

Education 87

Education Kafka Algorithm Data Warehouse 87

Data Engineering Podcast

NOVEMBER 19, 2023

Summary The dbt project has become overwhelmingly popular across analytics and data engineering teams. While it is easy to adopt, there are many potential pitfalls. Dustin Dorsey and Cameron Cyr co-authored a practical guide to building your dbt project. Find simplicity in your most complex projects with Miro.

Project 147

Project Data Lake High Quality Data SQL 147

ProjectPro

AUGUST 6, 2015

Let’s take a look at how Amazon uses Big Data- Amazon has approximately 1 million hadoop clusters to support their risk management, affiliate network, website updates, machine learning systems and more. Amazon is collecting intelligence and valuable pricing information (big data) from its competitors. ” Interesting?

Big Data 52

Big Data Hadoop Insurance Media 52

Data Engineering Podcast

APRIL 3, 2022

Summary The flexibility of software oriented data workflows is useful for fulfilling complex requirements, but for simple and repetitious use cases it adds significant complexity. Coalesce is a platform designed to reduce repetitive work for common workflows by adopting a visual pipeline builder to support your data warehouse transformations.

Data Warehouse 100

Data Warehouse Data Workflow Data Architecture SQL 100

Knowledge Hut

DECEMBER 27, 2023

An example of sequencing a business analysis service framework could be like the one mentioned below: Business Model Analysis Framework Analytical frameworks in business are useful tools often used by many business analysts and strategy consultants. What Is a Business Analysis Framework ?

Business Analyst 52

Business Analyst Portfolio Consulting Certification 52

DoorDash Engineering

JANUARY 30, 2024

However, Cassandra also is notorious for being hard to tune for performance and for the pitfalls that can arise during that process. Apache Cassandra is an open-source, distributed NoSQL database management system designed to handle large amounts of data across a wide range of commodity servers. What is Apache Cassandra?

NoSQL 84

NoSQL Database Systems Relational Database 84

AltexSoft

OCTOBER 21, 2022

Similar to Google in web browsing and Photoshop in image processing, it became a gold standard in data streaming, preferred by 70 percent of Fortune 500 companies. Apache Kafka is an open-source, distributed streaming platform for messaging, storing, processing, and integrating large data volumes in real time. What is Kafka?

Kafka 93

Kafka Hadoop ETL Tools Big Data 93

Data Engineering Podcast

JUNE 1, 2020

Summary Data lakes offer a great deal of flexibility and the potential for reduced cost for your analytics, but they also introduce a great deal of complexity. In order to bring the DBA into the new era of data management the team at Upsolver added a SQL interface to their data lake platform.

Data Lake 100

Data Lake Database Building Lambda Architecture 100

Rockset

NOVEMBER 1, 2022

It’s an understandable position given the risk and horror stories associated with “Migration Projects”. This blog outlines best practices from customers I have helped migrate from Elasticsearch to Rockset , reducing risk and avoiding common pitfalls. With our confidence boosted, let’s take a look at Elasticsearch.

Database-centric 52

Database-centric Pipeline-centric SQL Aggregated Data 52

ProjectPro

NOVEMBER 15, 2021

.” From month-long open-source contribution programs for students to recruiters preferring candidates based on their contribution to open-source projects or tech-giants deploying open-source software in their organization, open-source projects have successfully set their mark in the industry.

Big Data 52

Big Data Project Metadata Programming Language 52

dbt Developer Hub

JUNE 19, 2020

If you’ve been using dbt for over a year, your project is out-of-date. Over the last year, I and others on the Fishtown Analytics (now dbt Labs!) ✅ dbt_project.yml ​ Project naming conventions What is the name of your project? . ✅ dbt_project.yml ​ Project naming conventions What is the name of your project?

Project 40

Project BI SQL Utilities 40

Airbnb Tech

NOVEMBER 3, 2020

Part 1 —  Rebuilding at Scale Authors: Jonathan Parks, Vaughn Quoss, Paul Ellwood Introduction At Airbnb, we’ve always had a data-driven culture. During this transformation, Airbnb experienced the typical growth challenges that most companies do, including those that affect the data warehouse.

Data Warehouse 52

Data Warehouse Scala Datasets Data Engineering 52

ProjectPro

APRIL 28, 2016

Hadoop job interview is a tough road to cross with many pitfalls, that can make good opportunities fall off the edge. RDBMS vs Hadoop MapReduce Feature RDBMS MapReduce Size of Data Traditional RDBMS can handle upto gigabytes of data. Hadoop MapReduce can hadnle upto petabytes of data or more.

Hadoop 40

Hadoop Java Big Data Programming Language 40

ProjectPro

JUNE 23, 2016

Hadoop job interview is a tough road to cross with many pitfalls, that can make good opportunities fall off the edge. Thus, this solution is not practically recommended and this is when Apache Sqoop comes to the rescues of users that allows users to import data on HDFS. the bandwidth of the resources would be flooded).

Hadoop 40

Hadoop MySQL Relational Database Java 40

Knowledge Hut

JANUARY 2, 2024

Benefits of Learning Programming Languages Logical Thinking: Coding cultivates a structured and logical thought process, enhancing analytical skills. Diverse Career Opportunities: Beyond just software development, skills in coding open doors to roles in data analysis, system administration, and digital marketing.

Programming Language 52

Programming Language Programming Java NoSQL 52

Towards Data Science

OCTOBER 7, 2023

Here are 5 popular applications data teams at OpenAI, Vimeo, and other companies are putting into practice today. The hype around generative AI is real, and data and ML teams are feeling the heat. Getting started with LLMs? Image courtesy of author. Let’s explore how a few key sectors are putting gen AI to use.

Unstructured Data 76

Unstructured Data Finance SQL Media 76

Monte Carlo

OCTOBER 4, 2023

The hype around generative AI is real, and data and ML teams are feeling the heat. Across industries, executives are pushing their data leaders to build AI-powered products that will save time, drive revenue, or give them a competitive advantage. But where do most companies actually start when it comes to incorporating generative AI?

Unstructured Data 72

Unstructured Data Finance SQL Media 72

AltexSoft

MARCH 30, 2023

But what do the gas and oil corporation, the computer software giant, the luxury fashion house, the top outdoor brand, and the multinational pharmaceutical enterprise have in common? The answer is simple: They use the same technology to make the most of data. Let’s see what exactly Databricks has to offer.

Scala 64

Scala Data Lake BI Google Cloud 64