compared with
Current by Matthias Rella
on Nov 07, 2012 20:17.

(show comment)
This line was removed.
This word was removed. This word was added.
This line was added.

Changes (3)

View Page History
h3. Evaluation

YARN, MapReduce v2 and HDFS were installed on an Eucalyptus machine image (emi-F64A14C5) on the cluster infrastructure of AIT using Cloudera 4 and deployed on three nodes (one master = Resource Manager, two slaves). Two example applications have been executed successfully:

* *org.apache.hadoop.yarn.applications.DistributedShell*
A non-MapReduce application simply executing a shell command or script on each node.
$ yarn jar /usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar org.apache.hadoop.yarn.applications.distributedshell.Client -jar /usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar -shell_command whoami
* *org.apache.hadoop.examples.WordCount*
The classic word count example counting words in files of a input directory on HDFS.
$ mkdir wordcount
$ echo "Hello World" > wordcount/hello.txt
$ echo "Hello Yarn" > wordcount/yarn.txt
$ hdfs dfs -put wordcount wordcount
$ yarn jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar wordcount wordcount/ wc-output/
$ hdfs dfs -cat wc-output/part-r-00000
Hello 2
World 1
Yarn 1

Existing MapReduce applications of the SCAPE project (eg. the toolwrapper) could not be tested for API incompatibility between the employed Hadoop version 0.20.203 and MapReduce v2. Furthermore the YARN framework is not recommended for production use at the current state of development and resource management is only supported for memory.