hadoop2 devel #1
Description
Hadoop is a software platform that lets one easily write and
run MapReduce applications that process vast amounts of data.
This Charm aids MapReduce application developers to constructing a fully functional
and isolated hadoop development platform in a local Linux Container or a cloud environment in
less then 5 minutes.
- Tags:
- applications ›
Overview
Hadoop 2.2.0 YARN & Mapreduce development platform Ubuntu Charm.
This Charm will assist Hadoop YARN and MapReduce application developers to focus on application development logic rather then spending long pain-full hours/days searching for hadoop packages, matching development tools, install/configure required packages, and finally build an isolate test/development platform. This charm will seamlessly build a fully functional Hadoop 2.2.0 development environment on an isolated Ubuntu Linux Container or a remote cloud environment in under 5 minutes.
This charm will automate following required steps: 1) Install all Hadoop 2.x binary and source packages 2) Install the latest openjdk 7 jdk 3) Fully configure and setup an operational hadoop on a pseudo-distributed environment (i.e. ssh) 4) Configure a mapreduce build environment. 5) Install other useful development packages maven, build-essential, autoconf, automake, libtool, cmak,e zlib1g-dev, pkg-config, libssl-dev, and snappy packages
Usage
How to Install and configure juju: https://juju.ubuntu.com/docs/getting-started.html
From your charm home directory, execute "juju deploy <repository>:<series>/hadoop2-devel"
Step #1 juju deploy local:trusty/hadoop2-devel
Use "juju status" command to monitor "agent-state" of your deployment. "started" state is
the indication that hadoop development platform is ready.
Step #2 "watch juju status"
Now that hadoop development environement is ready, simply ssh to it, enjoy..
Step #3 "juju ssh hadoop2-devel/0"
Best source for Juju deployment information: https://juju.Ubuntu.com/docs/charms-deploying.html
Test your deployment
within ssh session from command-line type:
step #1 :start-dfs.sh
step #2 :start-yarn.sh
step #3 :jps
you should get something similar to:
Jps
SecondaryNameNode
DataNode
NameNode
NodeManager
ResourceManager
step #4 : run a mapreduce application
hadoop jar hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar pi 2 5
Contact Information
amir sanjar amir.sanjar@canonical.com
Configuration
- boolean-option
- (boolean) A short description of the configuration option