AWS EC2 CentOS release 6.5 部署zookeeper、kafka、dubbo

AWS EC2 CentOS release 6.5 部署zookeeper、kafka、dubbo 參考： http://blog.csdn.net/yizezhong/article/details/47777625 一．安裝zookeeper 1)     下載zookeeper安裝包 可去官網下載 ，也可經過wget命令： [root@ip-172-31-46-4 ~]# wget http://apache.fayea.com/zookeeper/zookeeper-3.4.9/zookeeper-3.4.9.tar.gz 2)     解壓 執行命令： [root@ip-172-31-46-4 ~]# tar -zxvf zookeeper-3.4.9.tar.gz 3)     安裝 執行命令： [root@ip-172-31-46-4 local]# cd /usr/local/ [root@ip-172-31-46-4 local]# mv ~/zookeeper-3.4.9 ./ 修改配置項 [root@ip-172-31-46-4 local]# cd zookeeper-3.4.9/conf [root@ip-172-31-46-4 conf]# cp zoo_sample.cfg zoo.cfg [root@ip-172-31-46-4 conf]# vim zoo.cfg dataDir=/tmp/zookeeper dataLogDir=/tmp/zookeeper/log 建立文件夾 [root@ip-172-31-46-4 conf]# mkdir -p /tmp/zookeeper [root@ip-172-31-46-4 conf]# mkdir -p /tmp/zookeeper/log 導入環境變量 [root@ip-172-31-46-4 conf]# export ZOOKEEPER_INSTALL=/usr/local/zookeeper-3.4.9/ [root@ip-172-31-46-4 conf]# export PATH=$PATH:$ZOOKEEPER_INSTALL/bin 啓動Zookeeper [root@ip-172-31-46-4 conf]# cd /usr/local/zookeeper-3.4.9/bin/ [root@ip-172-31-46-4 bin]# ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper-3.4.9/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@ip-172-31-46-4 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper-3.4.9/bin/../conf/zoo.cfg Mode: standalone 啓動客戶端 [root@ip-172-31-46-4 bin]# ./zkCli.sh Connecting to localhost:2181 2017-01-12 02:02:35,075 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT 2017-01-12 02:02:35,080 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=ip-172-31-46-4.us-west-2.compute.internal 2017-01-12 02:02:35,080 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_111 2017-01-12 02:02:35,082 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2017-01-12 02:02:35,082 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-0.b15.el6_8.x86_64/jre 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper-3.4.9/bin/../build/classes:/usr/local/zookeeper-3.4.9/bin/../build/lib/*.jar:/usr/local/zookeeper-3.4.9/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper-3.4.9/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper-3.4.9/bin/../lib/netty-3.10.5.Final.jar:/usr/local/zookeeper-3.4.9/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper-3.4.9/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper-3.4.9/bin/../zookeeper-3.4.9.jar:/usr/local/zookeeper-3.4.9/bin/../src/java/lib/*.jar:/usr/local/zookeeper-3.4.9/bin/../conf: 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA> 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=2.6.32-573.26.1.el6.x86_64 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root 2017-01-12 02:02:35,083 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/usr/local/zookeeper-3.4.9/bin 2017-01-12 02:02:35,085 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@69d0a921 Welcome to ZooKeeper! 2017-01-12 02:02:35,123 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2017-01-12 02:02:35,275 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2017-01-12 02:02:35,317 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x159906873c60000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0]  二．安裝kafka 下載 [root@ip-172-31-46-4 tools]# wget http://www-us.apache.org/dist/kafka/0.10.1.1/kafka_2.10-0.10.1.1.tgz 解壓 [root@ip-172-31-46-4 tools]# tar -zxvf kafka_2.10-0.10.1.1.tgz 移動 [root@ip-172-31-46-4 tools]# mv kafka_2.10-0.10.1.1 /usr/local/ 配置文件 [root@ip-172-31-46-4 tools]# cd /usr/local/kafka_2.10-0.10.1.1/config/ [root@ip-172-31-46-4 config]# vi server.properties # The address the socket server listens on. It will get the value returned from # java.net.InetAddress.getCanonicalHostName() if not configured. #   FORMAT: #     listeners = security_protocol://host_name:port #   EXAMPLE: #     listeners = PLAINTEXT://your.host.name:9092 #listeners=PLAINTEXT://:9092 port=9092 # Zookeeper connection string (see zookeeper docs for details). # This is a comma separated host:port pairs, each corresponding to a zk # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002". # You can also append an optional chroot string to the urls to specify the # root directory for all kafka znodes. zookeeper.connect=35.160.181.*:2181 啓動中止 [root@ip-172-31-46-4 tools]# cd /usr/local/kafka_2.10-0.10.1.1/bin/ #啓動 [root@ip-172-31-46-4 bin]# sh bin/kafka-server-start.sh config/server.properties & #中止 [root@ip-172-31-46-4 bin]# sh bin/kafka-server-stop.sh 生產/消費消息 #建立topic bin/kafka-topics.sh --create --zookeeper 35.160.181.*:2181 --replication-factor 1 --partitions 1 --topic test1 #生產消息 bin/kafka-console-producer.sh --broker-list 35.160.181.*:9092 --topic test1 #消費消息 bin/kafka-console-consumer.sh --zookeeper 35.160.181.*:2181 --topic test1 --from-beginning 錯誤提示： Java Hotspot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 986513408 bytes for committing reserved memory. # An error report file with more information is saved as: # hs_err_pid5535.log 解決辦法： 將 kafka-server-start.sh的 export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G" 修改成 export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M" 由於個人內存是1G 詳細配置說明，請參考：http://www.cnblogs.com/rilley/p/5391268.html (固然也能夠使用kafka自帶的zookeeper，本文只是爲初學者提供教程) 三．dubbo 參考：http://www.tuicool.com/articles/fayqAvJ 使用上步安裝好的zookeeper便可 dubbo-admin-2.5.4.war http://download.csdn.net/detail/liweifengwf/7784901