Installing Cloudera on Ubuntu 12.04 - the missing detailed instructions

Installing Cloudera on Ubuntu 12.04 - the missing detailed instructions

Originally posted in my bitbucket account repository.

https://bitbucket.org/barseghyanartur/simple-cloudera-install

Install Cloudera on Ubuntu 12.04 LTS

=============================================
Install SSH
---------------------------------------------
    $ sudo apt-get install ssh

    $ sudo apt-get install rsync

    $ sudo apt-get install openssh-server
    
    $ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

    $ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

    $ sudo chmod 600 ~/.ssh/id_dsa

    $ sudo chmod 600 ~/.ssh/id_dsa.pub

    $ sudo service ssh start

Clone the repository
---------------------------------------------
Install Mercurial:

    $ sudo apt-get install mercurial

Clone the repository:

    $ hg clone ssh://hg@bitbucket.org/barseghyanartur/simple-cloudera-install

Create cloudera user
---------------------------------------------
Create user cloudera

    $ sudo useradd cloudera

Set password for user cloudera

    $ sudo passwd cloudera

Run from `simple-cloudera-install` directory to create default dirs

    $ chmod +x scripts/create_dirs.sh

    $ sudo ./scripts/create_dirs.sh cloudera

Add user cloudera to sudoers group

    $ sudo adduser cloudera sudo

Cloudera requires to be installed on account with sudo rights not 
requiring a password. Thus, make a temporary change to sudoers 
group behaviour, but !!!MAKE SURE TO ROLL THIS CHANGE BACK AFTER 
INSTALLATION!!!

    !!!MAKE SURE TO ROLL THIS CHANGE BACK AFTER INSTALLATION!!!

    $ sudo nano /etc/sudoers
    
    And add replace the line starting
    
        %sudo  ALL=(ALL:ALL)
    
    with
    
        %sudo  ALL=(ALL:ALL) NOPASSWD: ALL

    !!!END MAKE SURE TO ROLL THIS CHANGE BACK AFTER INSTALLATION!!!

    $ sudo su cloudera

    $ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

    $ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

    $ cat ~/.ssh/id_dsa.pub|xclip -i

    $ sudo chmod 600 ~/.ssh/id_dsa

    $ sudo chmod 600 ~/.ssh/id_dsa.pub

    $ sudo service ssh restart

Use FQDN in /etc/hosts and /etc/hostname
---------------------------------------------
Copy the host address to clipboard from:

    $ ifconfig

Change the local address like `artur-VirtualBoxVM` to 
`hadoop.example.com` in files:

    $ sudo nano /etc/hosts

    > 217.21.196.178        hadoop.example.com        hdp

    $ sudo nano /etc/hostname/

    > hdp

Install python avro
---------------------------------------------
    $ sudo apt-get install python-virtualenv

    $ sudo apt-get install python-setuptools

    $ sudo easy_install virtualenv

    $ sudo easy_install avro

Stop the firewall if running
---------------------------------------------
    $ sudo service firewall stop

Make sure your hostname is resolvable
---------------------------------------------
    $ python -c 'import socket; print socket.getfqdn(), socket.gethostbyname(socket.getfqdn())'
    
    You should then get something like: hadoop.example.com

Run the cloudera automated installer
---------------------------------------------
    $ cd distrib/cloudera/

    $ chmod +x cloudera-manager-installer.bin

    $ sudo ./cloudera-manager-installer.bin

Before proceeding to installer web interface
---------------------------------------------
Run the following command:

    $ virtualenv /usr/lib/cmf/agent/build/env

    $ sudo su cloudera    

    $ ssh hadoop.example.com

    $ ssh hdp

Start the installer web interface
---------------------------------------------
Open the hadoop.example.com URL in your browser and log in using admin:admin credentials.

Database passwords
---------------------------------------------
You can always get the initially saved passwords in the file /etc/cloudera-scm-server/db.mnmt.properties
    
Uninstalling Cloudera on Ubuntu 12.04 LTS
=============================================
Instructions taken from https://ccp.cloudera.com/display/FREE373/Uninstalling+Cloudera+Manager+Free+Edition

Recovering from a failed install:

    $ sudo rm -Rf /usr/share/{cmf,hue} /var/lib/cloudera* /var/cache/yum/cloudera*

Step 1: Stop all of your services.

Step 2: Uninstall the Cloudera Manager Server and its embedded PostgreSQL database.

   sudo /usr/share/cmf/uninstall-cloudera-manager.sh

If uninstall-cloudera-manager.sh somehow doesn't work, do as follows:

    $ sudo service cloudera-scm-server stop

    $ sudo service cloudera-scm-server-db stop

Step 3: On all Agent hosts, uninstall CDH3 and the Cloudera Manager Agents.

    $ sudo service cloudera-scm-agent stop

Step 4: On all Agent hosts, remove all Cloudera Manager data.

    $ sudo rm -Rf /usr/share/{cmf,hue} /var/lib/cloudera* /var/cache/yum/cloudera*

Step 5: Remove the Cloudera Manager lock file.

    $ sudo rm /tmp/.scm_prepare_node.lock

Step 6: Finally (just to make sure nothing is left out)

    $ sudo service cloudera-scm-server stop

    $ sudo service cloudera-scm-server-db stop

    $ sudo apt-get remove  cloudera-manager-server cloudera-manager-server-db cloudera-manager-daemons

    $ sudo service cloudera-scm-agent hard_stop

    $ sudo apt-get remove cloudera-manager-agent cloudera-manager-daemons

    $ sudo rm -rf /usr/share/cmf