Hadoop architecture Not too complex!

In the Hadoop MapReduce system, there are several highly serviceable components which make this software framework so efficient. If you are working on this software framework you must be well aware of the benefits of this creation. Hadoop is a highly advanced and powerful suite of tools, this may not be readily apparent in first place but when you start using this software framework you will realize its benefits. If you want to see it before actually using it, check all those areas where it used extensively including the type of task and the situations. This way you will have a better understanding of the usability of the Hadoop MapReduce system.

If you want to have some idea about the Hadoop architecture, then read on. Here we are discussing some of the major aspects involved in the Hadoop architecture. One of the major components in its architecture is Hadoop Common, which provides access to the file systems supported by Hadoop. This is basically a package that contains the necessary JAR files and scripts which are needed to start Hadoop. With this, the Hadoop Common package also provides source code, documentation, and a contribution section. Now in the processing, if you want to have effective scheduling of work, every Hadoop-compatible file system needs to provide location awareness. What all elements are involved in delivering the location awareness is a big question? Well, it is not a huge list, simply the name of the rack where the worker node is. If you are little confused about the usage of this information, here is the answer. This information is best used by the Hadoop applications to run work on the node where the actual data is stored. This is used by the Hadoop Distributed File System (HDFS) while replicating the data as this file system tries to keep different copies of the data on different racks. What is the major purpose of this whole thing is to reduce the impact of rack power outage or switch failure. In the events when such things happen it is very difficult to manage data, but with this Hadoop architecture things are really manageable. You can easily read the data and perform the work effectively.

Additionally, the Hadoop architecture has a small Hadoop cluster which contains a single master node and multiple worker nodes. In the master node, there are several components namely, jobtracker, tasktracker, namenode, and datanode. And the worker node consists of a datanode and tasktracker.

Leave a Reply