Category List

python (6) airflow (2) docker (2) ssh (2) Ansible (1) Apache-storm (1) SMTP (1) Zookeeper (1) apache-airflow (1) argparse (1) argumentparser (1) containers (1) installing apache-storm (1) numpy (1) pycharm (1) pylint (1) python-flask (1) ssh keys (1) ssh-keygen (1) streamparse (1) unix-split command (1)
Showing posts with label installing apache-storm. Show all posts
Showing posts with label installing apache-storm. Show all posts

Saturday 26 May 2018

Install apache storm and Zookeeper

Requirements:


  • install java
  • install zookeeper
  • install storm

Installing Java:

    sudo apt-get install software-properties-common python-software-properties
    sudo add-apt-repository ppa:webupd8team/java
    sudo apt-get update
    sudo apt-get install oracle-java8-installer

    

 Check Java version after successful installation :


java -version

Setting Java path


Add following two lines into .bashrc file

Refer https://askubuntu.com/questions/175514/how-to-set-java-home-for-java

export JAVA_HOME =/usr/lib/jvm/java-8-oracle
export PATH=$PATH:$JAVA_HOME/bin
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home
export PATH=$PATH:$JAVA_HOME/bin


Installing zookeeper:

mv zookeeper-3.4.10/ zookeeper
cd zookeeper
mkdir data
cp conf/zoo_sample.cfg conf/zoo.cfg 
bin/zkServer.sh start


Installing Storm:

mv apache-storm-1.1.1/ apache-storm
cd apache-storm
mkdir data
cd apache-storm/bin/
chmod +x storm 

configuring storm.yaml:

vim apache-storm/conf/storm.yaml
storm.zookeeper.servers:     
    - "localhost"
nimbus.seeds: ["localhost"]

Adding Storm path:


Add following line into .bashrc file

export PATH=$PATH:/home/storm_user/storm-pixel/apache-storm/bin


check storm commands after successful installation:

  • storm version
  • storm nimbus 
  • storm supervisor
  • storm ui

Configuring streamparse:



Dependencies:

  • lein
  • storm

Install lein:

# refer http://leiningen.org/ cd /usr/bin wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein chmod +x lein run lein command. it will install packages to local lein version