article thumbnail

Brief History of Data Engineering

Jesse Anderson

Doug Cutting took those papers and created Apache Hadoop in 2005. They were the first companies to commercialize open source big data technologies and pushed the marketing and commercialization of Hadoop. Hadoop was hard to program, and Apache Hive came along in 2010 to add SQL. They eventually merged in 2012.

article thumbnail

Hadoop 2.0 (YARN) Framework - The Gateway to Easier Programming for Hadoop Users

ProjectPro

Hadoop (Hadoop 1.0) has progressed from a more restricted processing model of batch oriented MapReduce jobs to developing specialized and interactive processing models (Hadoop 2.0). With the advent of Hadoop 2.0, In this piece of writing we provide the users an insight on the novel Hadoop 2.0 to Hadoop 2.0.

Hadoop 40
article thumbnail

20 Best Open Source Big Data Projects to Contribute on GitHub

ProjectPro

It even allows you to build a program that defines the data pipeline using open-source Beam SDKs (Software Development Kits) in any three programming languages: Java, Python, and Go. Apache Spark is also quite versatile, and it can run on a standalone cluster mode or Hadoop YARN , EC2, Mesos, Kubernetes, etc.