Jesse Anderson

DECEMBER 12, 2022

They created MapReduce and GFS in 2004. Doug Cutting took those papers and created Apache Hadoop in 2005. They were the first companies to commercialize open source big data technologies and pushed the marketing and commercialization of Hadoop. Hadoop was hard to program, and Apache Hive came along in 2010 to add SQL.

Data Engineering 130

Data Engineering Data Engineer Engineering Hadoop 130

Towards Data Science

JANUARY 16, 2024

Image by the author 2004 to 2010 — The elephant enters the room New wave of applications emerged — Social Media, Software observability, etc. Result: Hadoop & NoSQL frameworks emerged. Result: Companies started to sell pre-configured data warehouses as products. The concept of `Data Marts` was introduced.

Data Warehouse 69

Data Warehouse Data Architecture Relational Database NoSQL 69

Ascend.io

FEBRUARY 15, 2023

Back in 2004, I got to work with MapReduce at Google years before Apache Hadoop was even released, using it on a nearly daily basis to analyze user activity on web search and analyze the efficacy of user experiments. I’ve had the good fortune to work at or start companies that were breaking new ground. Big data would be a big deal.

Cloud 52

Cloud Amazon Web Services Hadoop Telecommunication 52

Ascend.io

FEBRUARY 15, 2023

Back in 2004, I got to work with MapReduce at Google years before Apache Hadoop was even released, using it on a nearly daily basis to analyze user activity on web search and analyze the efficacy of user experiments. I’ve had the good fortune to work at or start companies that were breaking new ground. Big data would be a big deal.

Cloud 52

Cloud Amazon Web Services Hadoop Telecommunication 52

Zalando Engineering

FEBRUARY 28, 2018

For the sake of comparison, let’s recap the Hadoop way of working: Hadoop saves intermediate states to disk and communicates over a network. In fact, in a 2004 mapReduce research paper the designer states that key-value pairs is a key choice in designing mapReduce. Provides in memory storage for cached RDD’s.

Data Analysis 40

Data Analysis Hadoop Datasets Scala 40

ProjectPro

JUNE 6, 2015

Solocal has taken big data to the next stage of BI by designing a novel vision of BI with the open source distributed computing framework Hadoop. It replaced its traditional BI structure by integrating big data and Hadoop."-April In BI – there is a need to use ETL on top of Hadoop as there is not much scripting.

Business Intelligence 40

Business Intelligence Big Data BI Hadoop 40

Knowledge Hut

DECEMBER 26, 2023

Facebook It is a social media platform created originally by Mark Zuckerberg for college students in 2004. Most of the Data engineers working in the field enroll themselves in several other training programs to learn an outside skill, such as Hadoop or Big Data querying, alongside their Master's degree and PhDs.

Data Engineering 98

Data Engineering Data Engineer Engineering Pipeline-centric 98