Software Development

Apache Hadoop 2.4.0

The Apache community has voted to release Apache Hadoop 2.4.0, so the new release is now available and consists of important improvements. The improvements are related not only to HDFS but also to MapReduce.

The important improvement in HDFS is about NameNodes. Multiple independent Namenodes and Namespaces are now used that do not require coordination with each other. Datanodes are used as common storage for blocks by all Namenodes and each datanode registers with all Namenodes in the cluster. Heartbeats and block reports are sent from datanodes to Namenodes, that send back commands handled by the datanodes.

MapReduce changes are mostly about the ResourceManager. Since 0.23 version of Hadoop the two major functions of the JobTracker, resource management and job life-cycle management were seperated into separate components. In this release, the new ResourceManager manages the global assignment of resources to applications and the per-application ApplicationMaster manages the application‚ scheduling and coordination. The per-application ApplicationMaster is a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.

Interested to get started with Hadoop? Check out our complimentary whitepaper “Hadoop Illuminated”!

Theodora Fragkouli

Theodora has graduated from Computer Engineering and Informatics Department in the University of Patras. She also holds a Master degree in Economics from the National and Technical University of Athens. During her studies she has been involved with a large number of projects ranging from programming and software engineering to telecommunications, hardware design and analysis. She works as a junior Software Engineer in the telecommunications sector where she is mainly involved with projects based on Java and Big Data technologies.
Notify of

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Inline Feedbacks
View all comments
Back to top button