Apache Hadoop Development Tools (HDT) is still in development phase. So, no official distribution of Hadoop 2.2.0 Eclipse Plugin is available now. But we can build the same using winghc/hadoop2x-eclipse-plugin. In this post, we'll build, install and configure the plugin with the Eclipse or any Eclipse based IDE (say, Spring Tool Suite) to ease the development activities using Hadoop framework.
The Hadoop Development Tools (HDT) is a set of plugins for the Eclipse IDE for developing against the Hadoop platform.
Currently we are in the process of porting the existing MapReduce tools from the Apache Hadoop project to allow working with multiple versions of Hadoop from within one IDE.
Come get involved as we move towards our first release!
This project is currently a member of the Apache Incubator, so check back for updates, or come join us dev@hdt.incubator.apache.org.
Tools and Technologies used in this article
1. Download
Download winghc/hadoop2x-eclipse-plugin zip.
2. Extract
Extract to a local directory (say, 'C:\hadoop2x-eclipse-plugin').
3. Build
- Open '
\src\contrib\eclipse-plugin' in the Command prompt. - Run ANT build
hadoop.home: Hadoop installation directory.
Note: Internet connection is required as 'ivy-2.1.0.jar' will be downloaded.
eclipse.home: Installation directory of Eclipse IDE.
4. Install
On successful build, 'hadoop-eclipse-plugin-2.2.0.jar' will be generated inside '
5. Configure
Restart the Eclipse IDE if already started. Otherwise start it.
- Goto Window --> Open Perspective --> Other and select 'Map/Reduce' perspective.
- Click 'New Hadoop location...' (Blue Elephant icon) and define Hadoop location to run MapReduce applications. Click 'Finish' button.
Map/Reduce(V2) Master: Address of the Map/Reduce master node (The Job Tracker).
DFS Master: Address of the Distributed FileSystem Master node (The Name Node).
To know the 'Port' numbers, start Hadoop and open http://localhost:8088/cluster in a browser. Click Tools --> Configuration and search for the following properties.
DFS Master Map/Reduce(V2) Master - Now we can browse the Hadoop file system and perform different files / folder operations using the GUI only.
Also, we can easily create Map/Reduce Project, Mapper, Reducer and MapReduce Driver using the wizard (File --> New --> Other... --> Map/Reduce) and jump into Hadoop programming.
Comments