article thumbnail

Brief History of Data Engineering

Jesse Anderson

Doug Cutting took those papers and created Apache Hadoop in 2005. Cloudera was started in 2008, and HortonWorks started in 2011. They were the first companies to commercialize open source big data technologies and pushed the marketing and commercialization of Hadoop. It gained in usage and eventually displaced Hadoop.

article thumbnail

A Detailed Guide of Interview Questions on Apache Kafka

Analytics Vidhya

Introduction Apache Kafka is an open-source publish-subscribe messaging application initially developed by LinkedIn in early 2011. It is a famous Scala-coded data processing tool that offers low latency, extensive throughput, and a unified platform to handle the data in real-time.

Kafka 201
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Recap of Hadoop News for March 2018

ProjectPro

News on Hadoop - March 2018 Kyvos Insights to Host Session "BI on Big Data - With Instant Response Times" at the Gartner Data and Analytics Summit 2018.PRNewswire.com, RTInsights.com, March 15, 2018 Information Builders is letting the users of its WebFOCUS product to tap into the power of Hadoop. Datanami.com, March 26, 2018.

Hadoop 40
article thumbnail

Data Engineers of Netflix?—?Interview with Kevin Wylie

Netflix Tech

His favorite TV shows: Ozark, Breaking Bad, Black Mirror, Barry, and Chernobyl Since I joined Netflix back in 2011, my favorite project has been designing and building the first version of our entertainment knowledge graph. When I joined Netflix back in 2011, our content analytics team was just 3 people.

article thumbnail

Avec Snowflake, Peaksys concilie pour Cdiscount une data platform unique et le cloisonnement des données entre toutes les filiales 

Snowflake

Cdiscount : du commerce en ligne aux services orientés B2B Figure historique du commerce en ligne français, créée en 1998 et marketplace depuis 2011, Cdiscount s’appuie aujourd’hui sur ses savoir-faire pour compléter sa stratégie avec des offres B2B : services de logistique, déploiement de marketplace et même cybersécurité.

Hadoop 52
article thumbnail

The Rise of the Data Engineer

Maxime Beauchemin

I joined Facebook in 2011 as a business intelligence engineer. This discipline also integrates specialization around the operation of so called “big data” distributed systems, along with concepts around the extended Hadoop ecosystem, stream processing, and in computation at scale. By the time I left in 2013, I was a data engineer.

article thumbnail

Five Tech Jobs That Didn’t Exist Five Years Ago

Zalando Engineering

A 2011 McKinsey Global Institute report revealed that nearly all sectors in the US economy had at least 200 terabytes of stored data per company, thus the need for specialised engineers to solve Big Data problems was conceded.