開始實戰,肯定你已經安裝全部須要的軟件: java
==================== node
啓動zookeeper
shawny
@localhost :~/software/zookeeper-3.3.3/bin$ ./zkServer.sh start
JMX enabled by default
Using config: /home/shawny/software/zookeeper-3.3.3/bin/../conf/zoo.cfg
Starting zookeeper ...
STARTED
===========
啓動storm nimbus (若是啓動失敗則查看zookeeper上的storm znode是否沒有清除乾淨,若是非空則手動刪除storm znode下面的全部子節點)
shawny
@localhost :~/software/storm-0.5.5-SNAPSHOT$ ./bin/storm nimbus
Running: java -server -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Xmx1024m -Dlogfile.name=nimbus.log -Dlog4j.configuration=storm.log.properties -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/log4j:/home/shawny/software/storm-0.5.5-SNAPSHOT/conf backtype.storm.daemon.nimbus
===========
啓動storm supervisor
shawny
@localhost :~/software/storm-0.5.5-SNAPSHOT$ ./bin/storm supervisor
Running: java -server -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Xmx1024m -Dlogfile.name=supervisor.log -Dlog4j.configuration=storm.log.properties -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/log4j:/home/shawny/software/storm-0.5.5-SNAPSHOT/conf backtype.storm.daemon.supervisor
==========
啓動storm UI
shawny
@localhost :~/software/storm-0.5.5-SNAPSHOT$ bin/storm ui
Running: java -server -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Xmx768m -Dlogfile.name=ui.log -Dlog4j.configuration=storm.log.properties -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/log4j:/home/shawny/software/storm-0.5.5-SNAPSHOT:/home/shawny/software/storm-0.5.5-SNAPSHOT/conf backtype.storm.ui.core
=================
經過「http://localhost:8080/」查看storm信息
Cluster Summary
Nimbus uptime
Supervisors
Used slots
Free slots
Total slots
Running tasks
4m 10s
1
0
4
4
0
Topology summary
Name
Id
Status
Uptime
Num workers
Num tasks
Supervisor summary
Host
Uptime
Slots
Used slots
localhost
3m 1s
4
0
=============
建立Storm_test工程
shawny
@localhost :~/workspace/java/Storm_test$ tree
.
|-- bin
| |-- RandomSentenceSpout.class
| |-- resources
| | |-- splitsentence.py
| | |-- storm.py
| | `-- storm.rb
| |-- storm.yaml
| |-- WordCountTopology.class
| |-- WordCountTopology$SplitSentence.class
| `-- WordCountTopology$WordCount.class
|-- multilang
| `-- resources
| |-- splitsentence.py
| |-- storm.py
| `-- storm.rb
|-- src
| |-- RandomSentenceSpout.java
| `-- WordCountTopology.java
`-- storm_test.jar
NOTE:
1) 程序寫好後打成jar包,注意export後面的library handling選擇第二個
2) ~/.storm下面須要有storm.yaml, 裏面包含storm的配置信息
===========
提交jar包給storm
shawny
@localhost :~/software/storm-0.5.5-SNAPSHOT$ ./bin/storm jar ~/workspace/java/Storm_test/storm_test.jar WordCountTopology test
Running: export STORM_JAR=/home/shawny/workspace/java/Storm_test/storm_test.jar; java -client -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/workspace/java/Storm_test/storm_test.jar:/home/shawny/.storm:/home/shawny/software/storm-0.5.5-SNAPSHOT/bin WordCountTopology test
0 [main] INFO backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
25 [main] INFO backtype.storm.StormSubmitter - Uploading topology jar /home/shawny/workspace/java/Storm_test/storm_test.jar to assigned location: /tmp/storm/nimbus/inbox/stormjar-a37d88eb-f4c1-4fa0-8621-27c00a8beaf3.jar
472 [main] INFO backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: /tmp/storm/nimbus/inbox/stormjar-a37d88eb-f4c1-4fa0-8621-27c00a8beaf3.jar
473 [main] INFO backtype.storm.StormSubmitter - Submitting topology test in distributed mode with conf {"topology.workers":3,"topology.debug":true}
747 [main] INFO backtype.storm.StormSubmitter - Finished submitting topology: test
====================
查看storm logs
shawny
@localhost :~/software/storm-0.5.5-SNAPSHOT/logs$ tree
.
|-- nimbus.log
|-- supervisor.log
|-- ui.log
|-- worker-6700.log
|-- worker-6701.log
`-- worker-6702.log
===============
kill 提交的topology (test)
shawny
@localhost :~/software/storm-0.5.5-SNAPSHOT$ ./bin/storm kill test Running: java -client -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/.storm:/home/shawny/software/storm-0.5.5-SNAPSHOT/bin backtype.storm.command.kill_topology test 0 [main] INFO backtype.storm.command.kill-topology - Killed topology: test