관리 메뉴

HAMA 블로그

hadoop-1.2.1 이클랩스 import 하기 본문

Hadoop

hadoop-1.2.1 이클랩스 import 하기

[하마] 이승현 (wowlsh93@gmail.com) 2015. 4. 27. 11:14

hadoop-2.2.0 경우는 이클립스에서  maven  으로 import 하면 됩니다만


hadoop-1.2.1 은 이클립스에서 ant 로 import 할 경우 ivy-common (??)  에러나면서 안됩니다.

그럴 경우 아래 방법을  참고하셔서 import 하시면  됩니다.

Basic Environment Description:

     

OS:

     

Ubuntu 12.04

JAVA version:

     

jdk 1.6.0_27

Eclipse version:

     

Service Release 1


  Part 1: Building Hadoop Developing Environment

           

1) download hadoop source code.

  • For Hadoop release, download it here. I used Hadoop 1.2.0;
  • For Newest versions, it can be downloaded from Hadoop Version Control System;
  • Assume HADOOP_HOME is the directory where the source code is.
2) Install tools.
  • Run command: sudo apt-get install ant ivy automake libtool to install "ant ivy automake libtool".
  • Download forrest-0.8 and unzip it. Assume its directory is FORREST_HOME 
    Note that forrest-0.8 is not the latest version, Using the latest version may confront with some problems. In order to run forrest-0.8, we need java-1.5. Because we only need java-1.5 for forrest-0.8, we put java-1.5 in a separate directory. Assume it is JV5_HOME. The version I used is jdk1.5.0_22
3) Change configuration file of Hadoop
  • Open config file: $HADOOP_HOME/src/c++/task-controller/configure.ac 
    Find a line which contains AC_SYS_LARGEFILE, delete it.
  • Open config file: $HADOOP_HOME/build.xml 
    Find a line contains < target name="ant-eclipse-download", there is a line closely below, start with < get src= which indicates the source location of file "ant-eclipse.jar". The location in my version does not exist, change it to https://ant-eclipse.googlecode.com/files/ant-eclipse-1.0.bin.tar.bz2.
4) Build hadoop.
  • cd $HADOOP_HOME
  • ant compile
  • ant clean package -Djava5.home=$JV5_HOME -Dforrest.home=$FORREST_HOME
5) Create eclipse project files.
  • ant eclipse
6) Configure eclipse and import hadoop project.
  • Open eclipse and add class path variable. 
    The operation is "Window->preferences->java->Buiid path-> Classpath Variables" 
    Create new variable ANT_HOME=/user/share/ant
  • Import Hadoop project. 
    The operation is "File->Import->General->Existing Project Into Workspace". Goto $HADOOP_HOME and import the project
  • Configure Hadoop project. 
    Because Hadoop need to find classes for webapp but these .Class files are not included in project configuration, it cause runtime errors. 
    A simple solution is adding path "$HADOOP_HOME/build/classes" to CLASS_PATH of the project. The operation is "File->Properties->Java Build Path->Libraries->Add Class Folder"

7) Test your environment

  • There are many JUnit test examples in the project. You can choose one to test your environment. 
    If debugging Junit testcases add following arguments to your "VM Argument" in your "Debug configuration". 
    -Dhadoop.log.dir=build/test/logs -Dtest.build.data=build/test/data 
    -Dhadoop.log.file=hadoop.log 
    -Djava.net.preferIPv4Stack=true






java 5 sdk 설치방법 (1)

ubuntu의 기본 update repository에서는 sun-1.5-jdk를 제공하지 않으므로, 다음의 방법으로 추가 repository를 추가한다.

sudo add-apt-repository "deb http://us.archive.ubuntu.com/ubuntu/ hardy multiverse"
sudo add-apt-repository "deb http://us.archive.ubuntu.com/ubuntu/ hardy-updates multiverse"

aapt-get update

sudo apt-get install sun-java5-jdk



java 5 sdk 설치방법 (2) 

oracle 에서 5버전 bin 다운받은후 terminal 창에서 
sudo chmod +x jdk-1_5_0_22-linux-amd64.bin   (엔터)
sudo ./jdk-1_5_0_22-linux-amd64.bin   (엔터) 


Comments