Overview

Hadoop 2.2.0 YARN & Mapreduce development platform Ubuntu Charm.

This Charm will assist Hadoop YARN and MapReduce application developers to focus on application development logic rather then spending long pain-full hours/days searching for hadoop packages, matching development tools, install/configure required packages, and finally build an isolate test/development platform. This charm will seamlessly build a fully functional Hadoop 2.2.0 development environment on an isolated Ubuntu Linux Container or a remote cloud environment in under 5 minutes.

This charm will automate following required steps: 1) Install all Hadoop 2.x binary and source packages 2) Install the latest openjdk 7 jdk 3) Fully configure and setup an operational hadoop on a pseudo-distributed environment (i.e. ssh) 4) Configure a mapreduce build environment. 5) Install other useful development packages maven, build-essential, autoconf, automake, libtool, cmak,e zlib1g-dev, pkg-config, libssl-dev, and snappy packages

Usage

How to Install and configure juju: https://juju.ubuntu.com/docs/getting-started.html

    From your charm home directory, execute "juju deploy <repository>:<series>/hadoop2-devel"
    Step #1 juju deploy local:trusty/hadoop2-devel

    Use "juju status" command to monitor "agent-state" of your deployment. "started" state is
    the indication that hadoop development platform is ready.
            Step #2 "watch juju status"

    Now that hadoop development environement is ready, simply ssh to it, enjoy..
            Step #3 "juju ssh hadoop2-devel/0"

Best source for Juju deployment information: https://juju.Ubuntu.com/docs/charms-deploying.html

Test your deployment

within ssh session from command-line type: 
    step #1 :start-dfs.sh
    step #2 :start-yarn.sh
    step #3 :jps
         you should get something similar to:
    Jps
    SecondaryNameNode
    DataNode
    NameNode
    NodeManager
    ResourceManager
    step #4 : run a mapreduce application
        hadoop jar hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar pi 2 5

Contact Information

amir sanjar amir.sanjar@canonical.com