JStorm的搭建文檔

一、下載jstorm的jar包
　　https://github.com/alibaba/jstorm/releases

二、解壓jstorm的包
　　tar -xvf jstorm-2.4.0.tgz
　　mv jstorm-2.4.0 jstorm
修改環境變量：
　　vi /etc/profile
　　　　export JSTORM_HOME=/usr/local/jstorm
　　　　export PATH=$PATH:$JSTORM_HOME/bin
　　生效 source /etc/profile

二、上傳並修改storm.yaml
　　mv storm.yaml storm.yaml.bak

　　vi storm.yaml
　　nimbus.host: 表示nimbus的地址， 填寫ip

storm.zookeeper.servers: - "zk-datanode-01"
     - "zk-datanode-02"
     - "zk-datanode-03" nimbus.host: "192.168.46.160" storm.zookeeper.root: "/jstorm" storm.local.dir: "/usr/local/jstorm/data" supervisor.slots.ports: - 6800
    - 6801
    - 6802
    - 6803

三、部署到其餘節點
　　scp -r jstorm zk-datanode-02:/usr/local/
　　scp -r jstorm zk-datanode-03:/usr/local/

四、啓動storm
　　在nimbus 節點上執行
　　nohup jstorm nimbus & 查看$JSTORM_HOME/logs/nimbus.log檢查有無錯誤

　　在supervisor節點上執行
　　nohup jstorm supervisor & 查看$JSTORM_HOME/logs/supervisor.log檢查有無錯誤
　　若沒有配置環境變量
　　nohup bin/jstorm supervisor &

五、webUI的部署
　　必須使用tomcat 7.x版本， 注意不要忘記拷貝 ~/.jstorm/storm.yaml
　　Web UI 能夠和Nimbus不在同一個節點

　　

mkdir ~/.jstorm cp -f $JSTORM_HOME/conf/storm.yaml ~/.jstorm cd /usr/local/tomcat cd webapps cp $JSTORM_HOME/jstorm-ui-2.4.0.war ./            
        mv ROOT ROOT.old ln -s jstorm-ui-2.4.0 ROOT cd ../bin ./startup.sh

訪問http://192.168.46.160:8080/ 便可查看到jstorm的界面

 

六、WebUI 增長新集羣
　　注意： web ui使用的版本必須和集羣中JStorm最高的版本一致

　　在運行Web UI的機器上， 修改~/.jstorm/storm.yaml
　　把默認的ui.clusters註釋給去掉， 補充

# UI MultiCluster # Following is an example of multicluster UI configuration ui.clusters: - { name: "jstorm.share", zkRoot: "/jstorm", zkServers: [ "zk-datanode-01", "zk-datanode-02", "zk-datanode-03"], zkPort: 2181, } - { name: "jstorm.bu1", zkRoot: "/jstorm.dw", zkServers: [ "10.125.100.101", "10.125.100.101", "10.125.100.101"], zkPort: 2181, } 解釋一下 - { name: "jstorm.bu1",    --- 這個集羣的名字， 每一個集羣的名字必須不同 zkRoot: "/jstorm.dw",  --- 這個集羣 zk的根節點，能夠參考$JSTORM_HOME/con/storm.yaml 中「storm.zookeeper.root」 字段 zkServers: [ "10.125.100.101", "10.125.100.101", "10.125.100.101"],   -- zk 機器列表 zkPort: 2181,                                                  -- zk 的客戶端端口 }

 

提交jstorm的任務
　　jstorm jar /usr/local/jstormpack/JstormTest-0.0.1-SNAPSHOT.jar com.dinpay.bdp.rcp.mywordcount.WordCountTopologMain WordCountTopolog

中止jstorm的任務
　　jstorm kill KTTAndNormalTopology

nimbus.childopts: "-Xms1g -Xmx1g -Xmn512m -XX:PermSize=256m -XX:SurvivorRatio=4 -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=100M -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000" worker.childopts: "-Xms512m -Xmx512m -Xmn256m -XX:SurvivorRatio=2 -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=65" ui.childopts: "-Xms768m -Xmx768m -Xmn256m -XX:PermSize=96m -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000" supervisor.childopts: "-Xms2g -Xmx2g -Xmn1g -XX:PermSize=256m -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=100M -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000"