Reprint Please specify source: http://blog.csdn.net/beitiandijun/article/details/41802543
Ubuntu under Storm installation steps
Before installing storm, you first need to install some dependent libraries:
Zookeeper, JDK 6, python2.6.6, JZMQ, ZEROMQ
The dependent libraries required for these libraries are no longer one by one-sum.
The following are the specific installation procedures:
First, install the JDK
Zookeeper requires JDK 6 or later (currently the latest stable version is JDK8), but because Storm requires JDK 6 to be installed, you choose to install JDK 6.
Jdk:http://www.oracle.com/technetwork/java/javase/archive-139210.html
JDK installation process: Http://www.oracle.com/technetwork/java/javase/index-137561.html#linux
1. Download JDK6
(1) Open JDK:
(2) Select Java SE 6 and download
2. Installing the JDK
(1) Modify permissions:
chmod+x 6u <version> -linux-x64.bin
(2) Installation: "Java Core Programming", published by Sun, is recommended for installation in/usr, not/usr/local, for specific reasons.
sudo./ 6u <version>-linux-x64.bin installation directory
(3) Configuring environment variables
Export Java_home= installation directory
Export Jre_home= $JAVA _home/jre
Export classpath=.: $JAVA _home/lib: $JRE _home/lib
Export path= $JAVA _home/bin: $PATH
Note: Solve JDK version conflicts are modified by modifying these environment variables
Three different ways:
Mode one: Available only in the current shell:
The above statement in the current shell command line
Mode two: Available only in the current user:
Add the above statement at the end of the ~/.BASHRC
Enabled in the current shell:
SOURCE ~/.BASHRC
Method Three: All users are available:
Add the above statement at the end of the/etc/profile
Enabled in the current shell:
sudo source/ect/profile
In this example, mode three is used.
3. Test if Java is installed successfully
Java-version
Second, installation zookeeper
Reference Zookeeper official wiki:http://zookeeper.apache.org/doc/r3.3.3/zookeeperadmin.html
1, zookeeper system requirements:
Supported platforms include: Gnu/linux, Sun Solaris, FreeBSD, Win32, MAXOSX
Software Requirements: JDK 6 or higher is required, for zookeeper clusters, the official recommended minimum number of nodes is 3, and run on different machines; to avoid performance degradation due to data swap, increase the Java heap size as much as possible in memory, such as 4GB Memory Java heap size can be set to 3GB
2, the specific installation process
(1)
(2) Installing Zookeeper's Dependent Toolkit:
GCC, Cppunit, python-setuptools
I. GCC system usually comes with, do not need to install
II. Installation Python-setuptools:
Sudoapt-get Install Python-setuptools
III. Installing the Cppunit
Sudoapt-get Install Cppunit
(2) Configure Zookeeperconfiguration file:
Since there are 10 nodes in this cluster, the configuration
Third, install Python
Depending on storm requirements, installation of python2.6.6,python:http://www.python.org/ftp/python/2.6.6/python-2.6.6.tar.bz2 is required
1. Download Python 2.6.6
wget http://www.python.org/ftp/python/2.6.6/Python-2.6.6.tar.bz2
2. Compiling and installing Python 2.6.6
(1) Decompression: tar-jxvfpython-2.6.6.tar.bz2
(2) Compile and install:
CD Python
Sudo./configure--prefix=/usr/local/lib/python2.6.6
Sudomake
Several errors may occur during the installation process
Error One:
Based on the above error, the search results are due to the multiarch mechanism of Debian/ubuntu: even binaries compiled under different schemas can be run under the same system.
Reference Documentation:
Http://jisedai.vayn.de/2011/06/28/python-3-installation-note.html
http://bugs.python.org/issue9762
http://bugs.python.org/issue11715
Cause of error: The search paths for libraries under different schemas may not be consistent, so setup.py may not find the library files that are needed at compile time.
Workaround: Modify the setup.py to add the Multiarch path to the setup.py search path.
Specific Patch: https://hg.python.org/cpython/rev/bd0f73a9538e
Error Two:
After the error is resolved, make, still error:
The above error indicates that a dependent library is missing: Follow the prompts to indicate that the dependent libraries above can modify the Detect_modules () function in setup.py to cancel the dependency, but because it is not possible to anticipate which results are not installed, it is decided to install a total of 18 dependent libraries.
Install in sequence:
1) Install zlib (compression library)
Find Zlib Library
Installation: sudo apt-get install Zlib1g-dev
To see if the installation was successful, Zlib did not appear stating that the installation was successful, continue the installation
2) Install bz2
Find bz2 Library (compression library)
Installation: Sudoapt-get Install Libbz2-dev
3) Install GDBM (dbm is a database of Unix-like systems, use hash to save unstructured data, does not support SQL,GDBM is the GNU version of DBM)
Find gdbm and install:
Sudoapt-get Install Libgdbm-dev
Note: After installing GDBM, you do not need to install dbm again
4) Install ReadLine (GNU, for edit command line, Current User: Gdb,vi,emacs, etc.)
Find and install ReadLine:
Sudoapt-get Install Libreadline-dev
5) Install _sqlite3 (SQL Database engine C Library )
Installation: Libsqlite3-dev
6) Install _curses and _curses_panel
Installation: Libncurses5-dev
3, installation failure of the dependent library description
The following is a dependent library that did not install successfully: (does not mean that the installation failed, but that python2.6.6 cannot call the following libraries)
First, the installation steps are listed as follows:
1) Install _BSDDB (Pythoninterface for Berkeley DB)
Method One: unsuccessful
Search Apt-cachesearch BSDDB
But after installing PYTHON-BSDDB3 and python-bsddb3-dbg, execute Python's make discovery still hints that there is no _bsddb, so go to the Python website for BSDDB related information:
Https://pypi.python.org/pypi?%3Aaction=search&term=bsddb&submit=search
Method Two: unsuccessful
Still not, online search found that someone first installed Berkeley Db:libdb-dev
Method Three:
Online has said Bsddb is a very old Berkeley DB module that can not be installed.
By modifying the setup.py
disabled_module_list= ["bsddb185"]
Still fail
2) Install _hashlib (hash table) and _ssl (information security, data encryption)
Search and install: Sudoapt-get Install Libssl-dev
After installing Libssl-dev, the compiled Python still does not successfully identify the _ssl module.
Installing Openssl:sudo apt-get Install OpenSSL
The result still fails to successfully invoke the _ssl module, as shown in the following:
3 ) DL , Imageop , Linuxaudiodev , Ossaudiodev
DL is an obsolete module (replaced by cTYPES) on a 32-bit system;
Imageop is also an obsolete module on 32-bit systems, replaced by PIL;
Linuxaudiodev is an audio module;
Ossaudiodev is an audio module;
_tkinter is the interface module.
These modules do not need to be installed.
4. Resolving Python version conflicts
Originally there is python2.7 on the machine, now storm needs to use python2.6, how to solve the version conflict problem is need to solve.
Search the Internet and propose two solutions:
(1) Install Python version management tool Pythonbrew
: https://pypi.python.org/pypi/pythonbrew/1.3.5
Download Unzip, enter pythonbrew1.3.5
To read the Readme.rst, follow the recommended installation steps:
(1) Curl-klhttp://xrl.us/pythonbrewinstall | Bash
Need to install dependent libraries: curl
Sudoapt-get Install Curl
After you install curl, follow the steps (1) command line to display:
(2) According to the above output hint, the prompt command is written to the end of ~/.BASHRC, namely:
[-S "$HOME/.PYTHONBREW/ETC/BASHRC"]] && source "$HOME/.PYTHONBREW/ETC/BASHRC"
(3) Select the appropriate version number:
Pythonbrew Switch 2.6.6
Note: Pythonbrew software is the key recommendation , I hope you install pythonbrewbefore installing python , you can use pythonbrew Install the desired python version.
(2) using soft Links:
In this example, python2.6.6 is installed under/usr/local/lib, so a soft link can be:
sudo ln-s/usr/bin/python/usr/local/lib/python2.6.6/bin/python2.6
Iv. installation of ZEROMQ
1, installation Zeromq
(1) Download ZEROMQ source code, version number 2.1.7
wget http://download.zeromq.org/zeromq-2.1.7.tar.gz
Error, Missing Uuid-dev
So install the sudo apt-get install Uuid-dev
(2) Compile and install:
Make
Make install
V. Installation of JZMQ
(1) Download JZMQ source code
Https://github.com/zeromq/jzmq
Execute./autogen.sh, Error:
Need to install Pkg-config
sudo apt-get install Pkg-config
Still error:
Need to install Libtool
sudo apt-get install Libtool
Still error:
Need to install autoconf and Automake:
sudo apt-get install Automake autoconf
The dependent libraries are automatically installed during the installation M4
./autogen.sh execution succeeded, generating configure
./configure
Make error:
Workaround: http://my.oschina.net/mingdongcheng/blog/43009
1) A classdist_noinst.stamp empty file is generated first in the makefile path and the SRC directory.
2) to Src/org/zeromq directory: Javac *.java
3) to src directory: make-j24; Make install
4) to makefile under the path: make-j24; make install
Vi. storm Installation and configuration process:
1. Download the executable package:
Http://storm.incubator.apache.org/downloads.html
2. Startup process:
Refer to Storm official website:
Http://storm.incubator.apache.org/documentation/Setting-up-a-Storm-cluster.html
(1) Modify the Storm configuration file:
(2) Start Storm Nimbus:
Bin/storm Nimbus >/dev/null 2>&1 &
(3) Start Supervisor:
Bin/storm Supervisor >/dev/null 2>&1 &
(4) Start the UI:
Bin/storm UI >/dev/null 2>&1&
Attention:
UI node and Nimbus nodes are best located on the same node .
UI The default use port is 8080 , but this port has been TCP needs to be modified. Ui.port parameter to prevent port collisions.
(5) To view the Storm UI:
Storm document----Build your own storm cluster