Sat.Feb 25, 2017 - Fri.Mar 03, 2017

article thumbnail

Apache Airflow and the Future of Data Engineering: A Q&A

Maxime Beauchemin

With a brief Introduction and Takeaway added by Taylor D. Edmiston Introduction Every once in a while I read a post about the future of tech that resonates with clarity. A few weeks ago it was The Rise of the Data Engineer by Maxime Beauchemin, a data engineer at Airbnb and creator of their data pipeline framework, Apache Airflow. At Astronomer, Apache Airflow is at the very core of our tech stack : our integration workflows are defined by data pipelines built in Apache Airflow as directed acycl

article thumbnail

Recap of Hadoop News for February 2017

ProjectPro

News on Hadoop-February 2017 Big data brings breast cancer research forwards by 'decades'. ScienceDaily.com, February 1, 2017. Researchers analysed data of more than 28000 different genes and millions of images of 300,000 breast cancer cells and found that any cell shape changes caused by physical pressures on the tumours are converted into gene activity.

Hadoop 40