容器化微服務

時間  2019-11-16
標籤 容器 微服 简体版
原文   原文鏈接

 本文是<Java Rest Service實戰>的容器化服務章節實驗記錄。使用的基礎環境ubuntu 16.04 LTS，實驗中的集羣都在一個虛擬機上，其實質是僞集羣，但對於瞭解搭建的基本方法已經知足基本要求了。html

1、構建Zookeeper容器集羣前端

1. 定義Dockerfilejava

FROM index.tenxcloud.com/docker_library/java
MAINTAINER HaHa
#事先下載好zookeeper
COPY zookeeper-3.4.8.tar.gz /tmp/
RUN tar -xzf /tmp/zookeeper-3.4.8.tar.gz -C /opt
RUN cp /opt/zookeeper-3.4.8/conf/zoo_sample.cfg /opt/zookeeper-3.4.8/conf/zoo.cfg
RUN mv /opt/zookeeper-3.4.8 /opt/zookeeper
RUN rm -f /tmp/zookeeper-3.4.8.tar.gz

ADD entrypoint.sh /opt/entrypoint.sh
RUN chmod 777 /opt/entrypoint.sh

EXPOSE 2181 2888 3888
WORKDIR /opt/zookeeper
VOLUME ["/opt/zookeeper/conf","/tmp/zookeeper"]
CMD ["/opt/entrypoint.sh"]

  2. 編寫entrypoint.shnode

#!/bin/sh
ZOO_CFG="/opt/zookeeper/conf/zoo.cfg"
echo "server id (myid): ${SERVER_ID}"
echo "${SERVER_ID}" > /tmp/zookeeper/myid

echo "${APPEND_1}" >> ${ZOO_CFG}
echo "${APPEND_2}" >> ${ZOO_CFG}
echo "${APPEND_3}" >> ${ZOO_CFG}
echo "${APPEND_4}" >> ${ZOO_CFG}
echo "${APPEND_5}" >> ${ZOO_CFG}
echo "${APPEND_6}" >> ${ZOO_CFG}
echo "${APPEND_7}" >> ${ZOO_CFG}
echo "${APPEND_8}" >> ${ZOO_CFG}
echo "${APPEND_9}" >> ${ZOO_CFG}
echo "${APPEND_10}" >> ${ZOO_CFG}
/opt/zookeeper/bin/zkServer.sh start-foreground

 3.構建鏡像linux

//工做目錄nginx

root@ubuntu:/home/zhl/zookeeper# ll
total 21756
drwxrwxr-x 2 zhl zhl 4096 Sep 14 22:17 ./
drwxr-xr-x 25 zhl zhl 4096 Sep 14 05:09 ../
-rw-rw-r-- 1 zhl zhl 512 Sep 14 20:25 Dockerfile
-rw-r--r-- 1 root root 508 Sep 14 22:17 entrypoint.sh
-rw-rw-r-- 1 zhl zhl 22261552 Sep 14 05:29 zookeeper-3.4.8.tar.gzweb

//在當前目錄下構建名爲zk的zookeeper鏡像。
root@ubuntu:/home/zhl/zookeeper#docker build -t zk .

Sending build context to Docker daemon 22.27MB
Step 1/13 : FROM index.tenxcloud.com/docker_library/java
---> 264282a59a95
Step 2/13 : MAINTAINER HaHa
---> Running in 79720c1fbb96
---> ae7eddae4e93
Removing intermediate container 79720c1fbb96
Step 3/13 : COPY zookeeper-3.4.8.tar.gz /tmp/
---> 245818bb5e48
Removing intermediate container 4f8a2919a235
Step 4/13 : RUN tar -xzf /tmp/zookeeper-3.4.8.tar.gz -C /opt
---> Running in b8302238ceb1
---> 00aea27e64e3
Removing intermediate container b8302238ceb1
Step 5/13 : RUN cp /opt/zookeeper-3.4.8/conf/zoo_sample.cfg /opt/zookeeper-3.4.8/conf/zoo.cfg
---> Running in 6278f1a9487c
---> 007e855798a5
Removing intermediate container 6278f1a9487c
Step 6/13 : RUN mv /opt/zookeeper-3.4.8 /opt/zookeeper
---> Running in 90bb30879ea4
---> 63d17dc7b863
Removing intermediate container 90bb30879ea4
Step 7/13 : RUN rm -f /tmp/zookeeper-3.4.8.tar.gz
---> Running in d7ea5b8a83f4
---> b59d11ed6bdd
Removing intermediate container d7ea5b8a83f4
Step 8/13 : ADD entrypoint.sh /opt/entrypoint.sh
---> 9576b2b0cebf
Removing intermediate container ef1bd65c7c80
Step 9/13 : RUN chmod 777 /opt/entrypoint.sh
---> Running in f9dd51fe02f6
---> 4ffadd4c1d60
Removing intermediate container f9dd51fe02f6
Step 10/13 : EXPOSE 2181 2888 3888
---> Running in e58393d692c1
---> e2c47dc22195
Removing intermediate container e58393d692c1
Step 11/13 : WORKDIR /opt/zookeeper
---> 1fac68fcb274
Removing intermediate container 851cde7114c4
Step 12/13 : VOLUME /opt/zookeeper/conf /tmp/zookeeper
---> Running in e395e1f1ef13
---> 77f2f7be2dd0
Removing intermediate container e395e1f1ef13
Step 13/13 : CMD /opt/entrypoint.sh
---> Running in 8bf6fa5a1079
---> 5c819179f3f8
Removing intermediate container 8bf6fa5a1079
Successfully built 5c819179f3f8
Successfully tagged zk:latestredis

4. 單主機啓動3個zookeeper實例 spring

//以守護態的形式啓動zookeeper 容器 zk1\zk2\zk3
root@ubuntu:/home/zhl/zookeeper# docker run -d --name=zk1 --net=host -e SERVER_ID=1  -e APPEND_1=server.1=192.168.225.128:2888:3888 -e APPEND_2=server.2=192.168.225.128:2889:3889 -e APPEND_3=server.3=192.168.225.128:2890:3890 -e APPEND_4=clientPort=2181 zk
c3990b9e7bdedd1fdf4e73848eb4370279d1da018a82cc767f9529d2f9f5f72b
root@ubuntu:/home/zhl/zookeeper# docker run -d --name=zk2 --net=host -e SERVER_ID=2  -e APPEND_1=server.1=192.168.225.128:2888:3888 -e APPEND_2=server.2=192.168.225.128:2889:3889 -e APPEND_3=server.3=192.168.225.128:2890:3890 -e APPEND_4=clientPort=2182 zk
df4c81b16c7de76d74145a06bc978959f0e11c8d4aa7d615f3a98053f8a5cd2d
root@ubuntu:/home/zhl/zookeeper# docker run -d --name=zk3 --net=host -e SERVER_ID=3  -e APPEND_1=server.1=192.168.225.128:2888:3888 -e APPEND_2=server.2=192.168.225.128:2889:3889 -e APPEND_3=server.3=192.168.225.128:2890:3890 -e APPEND_4=clientPort=2183 zk
bd7718052a8d8e1e9c37b326c704f92c120383ab4cbf6d5988cffc7cb13bc720

其中，192.168.225.128爲主機IP地址(echo $HOST_IP)

5. 查看zookeeper運行狀況docker

zoo.cfg內容(選擇其中一個)：

root@ubuntu:/# cat ./var/lib/docker/volumes/b38902183ee2684494ea1edba0c963635851660e5edc0424471a49935309d669/_data/zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=192.168.225.128:2888:3888
server.2=192.168.225.128:2889:3889
server.3=192.168.225.128:2890:3890
clientPort=2182

 

//查看全部的容器
root@ubuntu:/home/zhl/zookeeper# docker ps -a

 　　CONTAINER ID IMAGE COMMAND CREATED 　　　　 STATUS 　　　　 PORTS　　 NAMES
　　c1ab885b2770 zk "/opt/entrypoint.sh" 26 seconds ago Up 　　26 　　　 seconds zk3
　　b939bfa60ea2 zk "/opt/entrypoint.sh" 58 seconds ago Up 　　58 　　　 seconds zk2
　　bd161a246c28 zk "/opt/entrypoint.sh" 2 minutes ago Up 　　 2 　　　　minutes zk1

root@ubuntu:~# echo stat|nc 127.0.0.1 2181
Zookeeper version: 3.4.8--1, built on 02/06/2016 03:18 GMT
Clients:
/172.17.0.3:45638[1](queued=0,recved=1177,sent=1177)
/127.0.0.1:57130[0](queued=0,recved=1,sent=0)

 
 
Latency min/avg/max: 0/0/70
Received: 9114
Sent: 9121
Connections: 2
Outstanding: 0
Zxid: 0x50000005e
Mode: follower
Node count: 21
 
 
root@ubuntu:~# telnet 192.168.225.128 2183
Trying 192.168.225.128...
Connected to 192.168.225.128.
Escape character is '^]'.
stat
Zookeeper version: 3.4.8--1, built on 02/06/2016 03:18 GMT
Clients:
/192.168.225.128:40962[0](queued=0,recved=1,sent=0)
/172.17.0.2:43694[1](queued=0,recved=1801,sent=1801)
 
 
Latency min/avg/max: 0/0/187
Received: 18234
Sent: 18241
Connections: 2
Outstanding: 0
Zxid: 0x50000005e
Mode: follower
Node count: 21
Connection closed by foreign host.
root@ubuntu:~# jps
6888 ZooKeeperMain
27882 Jps
6955 ZooKeeperMain 
 

 

或者 查看zookeeper是否已啓動使用：ps -ef | grep zoo.cfg
 

 
 
root@ubuntu:~# ps -ef | grep zoo.cfg
root       6462   6439  0 14:59 ?        00:00:36 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.8.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/zookeeper/bin/../conf/zoo.cfg
root       6561   6540  0 14:59 ?        00:00:44 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.8.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/zookeeper/bin/../conf/zoo.cfg
root       6670   6649  0 14:59 ?        00:00:38 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.8.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/zookeeper/bin/../conf/zoo.cfg
root      28049   3295  0 23:54 pts/2    00:00:00 grep --color=auto zoo.cfg
 

 

 若是遇到問題，能夠經過docker logs c1ab885b2770 查看。 
 

 
 
//jps(Java Virtual Machine Process Status Tool)是JDK 1.5提供的一個顯示當前全部java進程pid的命令。
root@ubuntu:~# jps -m //-m 輸出傳遞給main 方法的參數
27665 Jps -m
6888 ZooKeeperMain -server 192.168.225.128:2181
6955 ZooKeeperMain -server 192.168.225.128:2182
root@ubuntu:~# echo stat|nc 127.0.0.1 2182
Zookeeper version: 3.4.8--1, built on 02/06/2016 03:18 GMT
Clients:
 /127.0.0.1:36988[0](queued=0,recved=1,sent=0)
 /172.17.0.4:60280[1](queued=0,recved=955,sent=955)

Latency min/avg/max: 0/0/288
Received: 18211
Sent: 18212
Connections: 2
Outstanding: 0
Zxid: 0x50000005e
Mode: leader
Node count: 21 
 

 

 6. 其它可能用到的命令或小知識
 

 
 
1. echo stat|nc 127.0.0.1 2181 查看哪一個節點被選擇做爲follower或者leader
2. echo ruok|nc 127.0.0.1 2181 測試是否啓動了該Server，若回覆imok表示已經啓動。
3. echo dump| nc 127.0.0.1 2181 列出未經處理的會話和臨時節點。
4. echo kill | nc 127.0.0.1 2181 關掉server
5. echo conf | nc 127.0.0.1 2181 輸出相關服務配置的詳細信息。
6. echo cons | nc 127.0.0.1 2181 列出全部鏈接到服務器的客戶端的徹底的鏈接 / 會話的詳細信息。
7. echo envi |nc 127.0.0.1 2181  輸出關於服務環境的詳細信息（區別於 conf 命令）。
8. echo reqs | nc 127.0.0.1 2181 列出未經處理的請求。
9. echo wchs | nc 127.0.0.1 2181 列出服務器 watch 的詳細信息。
10. echo wchc | nc 127.0.0.1 2181 經過 session 列出服務器 watch 的詳細信息，它的輸出是一個與 watch 相關的會話的列表。
11. echo wchp | nc 127.0.0.1 2181 經過路徑列出服務器 watch 的詳細信息。它輸出一個與 session 相關的路徑。 
 

 

 
 
//zk集羣有3個節點，爲什麼只能查詢出來2個節點
root@ubuntu:~# netstat -an | grep 2183
tcp6       0      0 :::2183                 :::*                    LISTEN     
tcp6       0      0 192.168.225.128:2183    172.17.0.2:43694        ESTABLISHED
unix  3      [ ]         STREAM     CONNECTED     21838    /run/systemd/journal/stdout
unix  3      [ ]         STREAM     CONNECTED     21837    
root@ubuntu:~# netstat -atunp 
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN      1023/dnsmasq    
tcp        0      0 172.17.0.1:44870        172.17.0.3:9092         CLOSE_WAIT  4886/docker-proxy
tcp        0      0 172.17.0.1:50278        172.17.0.2:9091         CLOSE_WAIT  4692/docker-proxy
tcp        0      0 172.17.0.1:41458        172.17.0.4:9093         CLOSE_WAIT  5041/docker-proxy
tcp6       0      0 :::9092                 :::*                    LISTEN      4886/docker-proxy
tcp6       0      0 :::2181                 :::*                    LISTEN      6462/java       
tcp6       0      0 :::9093                 :::*                    LISTEN      5041/docker-proxy
tcp6       0      0 :::2182                 :::*                    LISTEN      6561/java       
tcp6       0      0 :::2183                 :::*                    LISTEN      6670/java       
tcp6       0      0 192.168.225.128:2889    :::*                    LISTEN      6561/java       
tcp6       0      0 192.168.225.128:3888    :::*                    LISTEN      6462/java       
tcp6       0      0 192.168.225.128:3889    :::*                    LISTEN      6561/java       
tcp6       0      0 192.168.225.128:3890    :::*                    LISTEN      6670/java       
tcp6       0      0 :::34035                :::*                    LISTEN      6462/java       
tcp6       0      0 :::37529                :::*                    LISTEN      6670/java       
tcp6       0      0 :::37817                :::*                    LISTEN      6561/java       
tcp6       0      0 :::9091                 :::*                    LISTEN      4692/docker-proxy
tcp6       0      0 192.168.225.128:58368   192.168.225.128:2889    ESTABLISHED 6670/java       
tcp6       0      0 192.168.225.128:9091    172.17.0.4:39558        FIN_WAIT2   4692/docker-proxy
tcp6       0      0 192.168.225.128:42050   192.168.225.128:3888    ESTABLISHED 6670/java       
tcp6       1      0 192.168.225.128:49148   192.168.225.128:2181    CLOSE_WAIT  6888/java       
tcp6       0      0 192.168.225.128:3889    192.168.225.128:50582   ESTABLISHED 6561/java       
tcp6       0      0 192.168.225.128:42002   192.168.225.128:3888    ESTABLISHED 6561/java       
tcp6       0      0 192.168.225.128:3888    192.168.225.128:42050   ESTABLISHED 6462/java       
tcp6       0      0 192.168.225.128:9092    172.17.0.4:37246        FIN_WAIT2   4886/docker-proxy
tcp6       0      0 192.168.225.128:58374   192.168.225.128:2889    ESTABLISHED 6462/java       
tcp6       0      0 192.168.225.128:2182    172.17.0.4:60280        ESTABLISHED 6561/java       
tcp6       0      0 192.168.225.128:3888    192.168.225.128:42002   ESTABLISHED 6462/java       
tcp6       1      0 192.168.225.128:59822   192.168.225.128:2182    CLOSE_WAIT  6955/java       
tcp6       0      0 192.168.225.128:2889    192.168.225.128:58368   ESTABLISHED 6561/java       
tcp6       0      0 192.168.225.128:2889    192.168.225.128:58374   ESTABLISHED 6561/java       
tcp6       0      0 192.168.225.128:2181    172.17.0.3:45638        ESTABLISHED 6462/java       
tcp6       0      0 192.168.225.128:9093    172.17.0.4:60222        FIN_WAIT2   5041/docker-proxy
tcp6       0      0 192.168.225.128:2183    172.17.0.2:43694        ESTABLISHED 6670/java       
tcp6       0      0 192.168.225.128:50582   192.168.225.128:3889    ESTABLISHED 6670/java       
udp        0      0 0.0.0.0:631             0.0.0.0:*                           2688/cups-browsed
udp        0      0 0.0.0.0:60401           0.0.0.0:*                           1023/dnsmasq    
udp        0      0 127.0.1.1:53            0.0.0.0:*                           1023/dnsmasq    
udp        0      0 0.0.0.0:68              0.0.0.0:*                           26791/dhclient  
udp        0      0 0.0.0.0:5353            0.0.0.0:*                           818/avahi-daemon: r
udp        0      0 0.0.0.0:36101           0.0.0.0:*                           818/avahi-daemon: r
udp6       0      0 :::54133                :::*                                818/avahi-daemon: r
udp6       0      0 :::5353                 :::*                                818/avahi-daemon: r
root@ubuntu:~# lsof -i:2181
COMMAND  PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
java    6462 root   34u  IPv6  495094      0t0  TCP *:2181 (LISTEN)
java    6462 root   40u  IPv6 1615785      0t0  TCP 192.168.225.128:2181->172.17.0.3:45638 (ESTABLISHED)
java    6888 root   13u  IPv6  498296      0t0  TCP 192.168.225.128:49148->192.168.225.128:2181 (CLOSE_WAIT)
root@ubuntu:~# lsof -i:2182
COMMAND  PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
java    6561 root   34u  IPv6  495425      0t0  TCP *:2182 (LISTEN)
java    6561 root   41u  IPv6 1615791      0t0  TCP 192.168.225.128:2182->172.17.0.4:60280 (ESTABLISHED)
java    6955 root   13u  IPv6  500012      0t0  TCP 192.168.225.128:59822->192.168.225.128:2182 (CLOSE_WAIT)
root@ubuntu:~# lsof -i:2183
COMMAND  PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
java    6670 root   34u  IPv6  495813      0t0  TCP *:2183 (LISTEN)
java    6670 root   39u  IPv6 1615789      0t0  TCP 192.168.225.128:2183->172.17.0.2:43694 (ESTABLISHED)
 

 

 經常使用的三個狀態是：ESTABLISHED 表示正在通訊，TIME_WAIT 表示主動關閉，CLOSE_WAIT 表示被動關閉經常使用的三個狀態是：ESTABLISHED 表示正在通訊，TIME_WAIT 表示主動關閉，CLOSE_WAIT 表示被動關閉
 

 

 2、Kafka
 

 1. Kafka簡介：
 

 
 
Kafka是一種分佈式的，基於發佈/訂閱的消息系統。主要設計目標以下：

以時間複雜度爲O(1)的方式提供消息持久化能力，即便對TB級以上數據也能保證常數時間複雜度的訪問性能。高吞吐率。即便在很是廉價的商用機器上也能作到單機支持每秒100K條以上消息的傳輸。
支持Kafka Server間的消息分區，及分佈式消費，同時保證每一個Partition內的消息順序傳輸。同時支持離線數據處理和實時數據處理。Scale out：支持在線水平擴展。

經常使用Message Queue對比

RabbitMQ

RabbitMQ是使用Erlang編寫的一個開源的消息隊列，自己支持不少的協議：AMQP，XMPP, SMTP, STOMP，也正因如此，它很是重量級，更適合於企業級的開發。同時實現了Broker構架，
這意味着消息在發送給客戶端時先在中心隊列排隊。對路由，負載均衡或者數據持久化都有很好的支持。

Redis

Redis是一個基於Key-Value對的NoSQL數據庫，開發維護很活躍。雖然它是一個Key-Value數據庫存儲系統，但它自己支持MQ功能，因此徹底能夠當作一個輕量級的隊列服務來使用。
對於RabbitMQ和Redis的入隊和出隊操做，各執行100萬次，每10萬次記錄一次執行時間。測試數據分爲128Bytes、512Bytes、1K和10K四個不一樣大小的數據。實驗代表：入隊時，
當數據比較小時Redis的性能要高於RabbitMQ，而若是數據大小超過了10K，Redis則慢的沒法忍受；出隊時，不管數據大小，Redis都表現出很是好的性能，而RabbitMQ的出隊性能則遠低於Redis。

ZeroMQ

ZeroMQ號稱最快的消息隊列系統，尤爲針對大吞吐量的需求場景。ZeroMQ可以實現RabbitMQ不擅長的高級/複雜的隊列，可是開發人員須要本身組合多種技術框架，
技術上的複雜度是對這MQ可以應用成功的挑戰。ZeroMQ具備一個獨特的非中間件的模式，你不須要安裝和運行一個消息服務器或中間件，由於你的應用程序將扮演這個服務器角色。
你只須要簡單的引用ZeroMQ程序庫，可使用NuGet安裝，而後你就能夠愉快的在應用程序之間發送消息了。可是ZeroMQ僅提供非持久性的隊列，也就是說若是宕機，數據將會丟失。
其中，Twitter的Storm 0.9.0之前的版本中默認使用ZeroMQ做爲數據流的傳輸（Storm從0.9版本開始同時支持ZeroMQ和Netty做爲傳輸模塊）。

ActiveMQ

ActiveMQ是Apache下的一個子項目。 相似於ZeroMQ，它可以以代理人和點對點的技術實現隊列。同時相似於RabbitMQ，它少許代碼就能夠高效地實現高級應用場景。

Kafka/Jafka

Kafka是Apache下的一個子項目，是一個高性能跨語言分佈式發佈/訂閱消息隊列系統，而Jafka是在Kafka之上孵化而來的，即Kafka的一個升級版。
具備如下特性：快速持久化，能夠在O(1)的系統開銷下進行消息持久化；高吞吐，在一臺普通的服務器上既能夠達到10W/s的吞吐速率；
徹底的分佈式系統，Broker、Producer、Consumer都原生自動支持分佈式，自動實現負載均衡；支持Hadoop數據並行加載，對於像Hadoop的同樣的日誌數據和離線分析系統，
但又要求實時處理的限制，這是一個可行的解決方案。Kafka經過Hadoop的並行加載機制統一了在線和離線的消息處理。Apache Kafka相對於ActiveMQ是一個很是輕量級的消息系統，
除了性能很是好以外，仍是一個工做良好的分佈式系統。
 

 

 2. kafka架構
 

 
 
Terminology

Broker
Kafka集羣包含一個或多個服務器，這種服務器被稱爲broker

Topic
每條發佈到Kafka集羣的消息都有一個類別，這個類別被稱爲Topic。（物理上不一樣Topic的消息分開存儲，邏輯上一個Topic的消息雖然保存於一個或多個broker上但用戶只需指定消息的Topic便可生產或
消費數據而沒必要關心數據存於何處）

Partition
Parition是物理上的概念，每一個Topic包含一個或多個Partition.

Producer
負責發佈消息到Kafka broker

Consumer
消息消費者，向Kafka broker讀取消息的客戶端。

Consumer Group
每一個Consumer屬於一個特定的Consumer Group（可爲每一個Consumer指定group name，若不指定group name則屬於默認的group）。
 

 

 
 

 
 

 
 
一個典型的Kafka集羣中包含若干Producer（能夠是web前端產生的Page View，或者是服務器日誌，系統CPU、Memory等），若干broker（Kafka支持水平擴展，通常broker數量越多，集羣吞吐率越高），
若干Consumer Group，以及一個Zookeeper集羣。Kafka經過Zookeeper管理集羣配置，選舉leader，以及在Consumer Group發生變化時進行rebalance。Producer使用push模式將消息發佈到
broker，Consumer使用pull模式從broker訂閱並消費消息。
 

 

 3.kafka鏡像製做與構建容器集羣
 

(1). 編寫Dockerfile文件
 

 
 
root@ubuntu:/home/zhl/kafka# vi Dockerfile 
FROM index.tenxcloud.com/docker_library/java
MAINTAINER HaHa
COPY kafka_2.10-0.9.0.1.tgz /tmp/
RUN tar -xzf /tmp/kafka_2.10-0.9.0.1.tgz -C /opt
RUN mv /opt/kafka_2.10-0.9.0.1 /opt/kafka
RUN rm -f /tmp/zookeeper-3.4.8.tar.gz

ENV KAFKA_HOME /opt/kafka
ADD start-kafka.sh /usr/bin/start-kafka.sh
RUN chmod 777 /usr/bin/start-kafka.sh

CMD /usr/bin/start-kafka.sh
 

 

 (2). 編寫容器啓動腳本
 

 
 
root@ubuntu:/home/zhl/kafka# vi start-kafka.sh 
cp $KAFKA_HOME/config/server.properties $KAFKA_HOME/config/server.properties.bk
sed -r -i "s/(zookeeper.connect)=(.*)/\1=${ZK}/g" $KAFKA_HOME/config/server.properties
sed -r -i "s/(broker.id)=(.*)/\1=${BROKER_ID}/g" $KAFKA_HOME/config/server.properties
sed -r -i "s/(log.dirs)=(.*)/\1=\/tmp\/kafka-logs-${BROKER_ID}/g" $KAFKA_HOME/config/server.properties
sed -r -i "s/#(advertised.host.name)=(.*)/\1=${HOST_IP}/g" $KAFKA_HOME/config/server.properties
sed -r -i "s/#(port)=(.*)/\1=${PORT}/g" $KAFKA_HOME/config/server.properties
sed -r -i "s/(listeners)=(.*)/\1=PLAINTEXT:\/\/:${PORT}/g" $KAFKA_HOME/config/server.properties
if [ "$KAFKA_HEAP_OPTS" !=""]; then
sed -r -i "s/^(export KAFKA_HEAP_OPTS)=\"(.*)\"/\1=\"$KAFKA_HEAP_OPTS\"/g" $KAFKA_HOME/bin/kafka-server-start.sh
fi
$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties
 

 

 (3). 構建鏡像
 

 
 
docker build -t kafkatest .
 

 

 (4). 使用kafkatest鏡像，啓動3個kafka容器實例，
 

 
 
K_PORT=9091
docker run --name=k1 -p ${K_PORT}:${K_PORT} -e BROKER_ID=1 -e HOST_IP=${HOST_IP} -e PORT=${K_PORT} -e ZK='192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183' -d kafkatest
K_PORT=9092
docker run --name=k2 -p ${K_PORT}:${K_PORT} -e BROKER_ID=2 -e HOST_IP=${HOST_IP} -e PORT=${K_PORT} -e ZK='192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183' -d kafkatest
K_PORT=9093
docker run --name=k3 -p ${K_PORT}:${K_PORT} -e BROKER_ID=3 -e HOST_IP=${HOST_IP} -e PORT=${K_PORT} -e ZK='192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183' -d kafkatest   
 

 

 查看啓動結果：
 

 
 
root@ubuntu:/home/zhl/kafka# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
637beebec29e        kafkatest           "/bin/sh -c /usr/b..."   9 minutes ago       Up 9 minutes        0.0.0.0:9093->9093/tcp   k3
4bf4925e6f40        kafkatest           "/bin/sh -c /usr/b..."   9 minutes ago       Up 9 minutes        0.0.0.0:9092->9092/tcp   k2
988c32940785        kafkatest           "/bin/sh -c /usr/b..."   11 minutes ago      Up 11 minutes       0.0.0.0:9091->9091/tcp   k1
c1ab885b2770        zk                  "/opt/entrypoint.sh"     4 hours ago         Up 4 hours                                   zk3
b939bfa60ea2        zk                  "/opt/entrypoint.sh"     4 hours ago         Up 4 hours                                   zk2
bd161a246c28        zk                  "/opt/entrypoint.sh"     4 hours ago         Up 4 hours                                   zk1
 

 

 
 
root@ubuntu:~# ps -ef | grep config/server
root       4755   4742  0 13:24 ?        00:01:51 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/opt/kafka/bin/../logs -Dlog4j.configuration=file:/opt/kafka/bin/../config/log4j.properties -cp :/opt/kafka/bin/../libs/* kafka.Kafka /opt/kafka/config/server.properties
root       4954   4943  0 13:25 ?        00:01:55 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/opt/kafka/bin/../logs -Dlog4j.configuration=file:/opt/kafka/bin/../config/log4j.properties -cp :/opt/kafka/bin/../libs/* kafka.Kafka /opt/kafka/config/server.properties
root       5107   5095  0 13:25 ?        00:01:52 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/opt/kafka/bin/../logs -Dlog4j.configuration=file:/opt/kafka/bin/../config/log4j.properties -cp :/opt/kafka/bin/../libs/* kafka.Kafka /opt/kafka/config/server.properties
root      28057   3295  0 23:57 pts/2    00:00:00 grep --color=auto config/server
 

 

  (5). 經過命令行的方式啓動生產者和消費者進行測試(http://www.jianshu.com/p/dc4770fc34b6)        
 

 
 
//--進入到kafka目錄，建立「test5」topic主題：分區爲三、備份爲3
root@ubuntu:/var/lib/docker/aufs/diff/c097ed7784e8ef038c4d564ca566de3c554ff52813f642220fed7cad0705810e/opt/kafka/bin# ./kafka-topics.sh --create --zookeeper 192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183 --replication-factor 3 --partitions 3 --topic test5
Created topic "test5".
//--查看"test5"主題詳情
root@ubuntu:/var/lib/docker/aufs/diff/c097ed7784e8ef038c4d564ca566de3c554ff52813f642220fed7cad0705810e/opt/kafka/bin# ./kafka-topics.sh --describe --zookeeper  192.168.225.128:2181 --topic test5
Topic:test5    PartitionCount:3    ReplicationFactor:3    Configs:
    Topic: test5    Partition: 0    Leader: 1    Replicas: 1,2,3    Isr: 1,2,3
    Topic: test5    Partition: 1    Leader: 2    Replicas: 2,3,1    Isr: 2,3,1
    Topic: test5    Partition: 2    Leader: 3    Replicas: 3,1,2    Isr: 3,1,2
root@ubuntu:/var/lib/docker/aufs/diff/c097ed7784e8ef038c4d564ca566de3c554ff52813f642220fed7cad0705810e/opt/kafka/bin# ./kafka-console-producer.sh --broker-list 192.168.225.128:9092 --topic test5  //--broker-list : 值能夠爲broker集羣中的一個或多個節點
this is test
1
 

 

 
 

 
 
//新開啓一個窗口，啓動消費者
root@ubuntu:/var/lib/docker/aufs/diff/c097ed7784e8ef038c4d564ca566de3c554ff52813f642220fed7cad0705810e/opt/kafka/bin# ./kafka-console-consumer.sh --zookeeper 192.168.225.128 --topic test5 --from-beginning
this is test
1
 

 

 
 

 server.properties內容：
 

 
 
root@ubuntu:/# cat ./var/lib/docker/aufs/diff/6befb484b490a6f34d4a0e5ca00f2eda7c92bce972cabf510518ed4ce12b1fef/opt/kafka/config/server.properties
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# see kafka.server.KafkaConfig for additional details and defaults

############################# Server Basics #############################

# The id of the broker. This must be set to a unique integer for each broker.
# 每個broker在集羣中的惟一表示
broker.id=1

############################# Socket Server Settings #############################

listeners=PLAINTEXT://:9091

# The port the socket server listens on
port=9091

# Hostname the broker will bind to. If not set, the server will bind to all interfaces
#host.name=localhost

# Hostname the broker will advertise to producers and consumers. If not set, it uses the
# value for "host.name" if configured.  Otherwise, it will use the value returned from
# java.net.InetAddress.getCanonicalHostName().
advertised.host.name=192.168.225.128

# The port to publish to ZooKeeper for clients to use. If this is not set,
# it will publish the same port that the broker binds to.
#advertised.port=<port accessible by clients>

# The number of threads handling network requests
num.network.threads=3

# The number of threads doing disk I/O
num.io.threads=8

# The send buffer (SO_SNDBUF) used by the socket server
socket.send.buffer.bytes=102400

# The receive buffer (SO_RCVBUF) used by the socket server
socket.receive.buffer.bytes=102400

# The maximum size of a request that the socket server will accept (protection against OOM)
socket.request.max.bytes=104857600


############################# Log Basics #############################

# A comma seperated list of directories under which to store log files
log.dirs=/tmp/kafka-logs-1

# The default number of log partitions per topic. More partitions allow greater
# parallelism for consumption, but this will also result in more files across
# the brokers.
num.partitions=1

# The number of threads per data directory to be used for log recovery at startup and flushing at shutdown.
# This value is recommended to be increased for installations with data dirs located in RAID array.
num.recovery.threads.per.data.dir=1

############################# Log Flush Policy #############################

# Messages are immediately written to the filesystem but by default we only fsync() to sync
# the OS cache lazily. The following configurations control the flush of data to disk.
# There are a few important trade-offs here:
#    1. Durability: Unflushed data may be lost if you are not using replication.
#    2. Latency: Very large flush intervals may lead to latency spikes when the flush does occur as there will be a lot of data to flush.
#    3. Throughput: The flush is generally the most expensive operation, and a small flush interval may lead to exceessive seeks.
# The settings below allow one to configure the flush policy to flush data after a period of time or
# every N messages (or both). This can be done globally and overridden on a per-topic basis.

# The number of messages to accept before forcing a flush of data to disk
#log.flush.interval.messages=10000

# The maximum amount of time a message can sit in a log before we force a flush
#log.flush.interval.ms=1000

############################# Log Retention Policy #############################

# The following configurations control the disposal of log segments. The policy can
# be set to delete segments after a period of time, or after a given size has accumulated.
# A segment will be deleted whenever *either* of these criteria are met. Deletion always happens
# from the end of the log.

# The minimum age of a log file to be eligible for deletion
log.retention.hours=168

# A size-based retention policy for logs. Segments are pruned from the log as long as the remaining
# segments don't drop below log.retention.bytes.
#log.retention.bytes=1073741824

# The maximum size of a log segment file. When this size is reached a new log segment will be created.
log.segment.bytes=1073741824

# The interval at which log segments are checked to see if they can be deleted according
# to the retention policies
log.retention.check.interval.ms=300000

############################# Zookeeper #############################

# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
# zookeeper集羣的地址，能夠是多個，多個之間用逗號分割
zookeeper.connect=192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183

# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000
 

 

 
 

 
 

 
 

 三、微服務
 


1.使用微服務鏡像mstest,啓動3個實例
 

 
 
root@ubuntu:/home/zhl/mstest# PORT=18081
root@ubuntu:/home/zhl/mstest# docker run -d --name=kaka1 --hostname=kaka1 -p ${PORT}:8080 -e ZK='192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183' -e KAFKA='192.168.225.128:9091,192.168.225.128:9092,192.168.225.128:9093' mstest
3c11d508bf89d96c73e29747b79095225775c2daab88cf4075e81be567a9444b
root@ubuntu:/home/zhl/mstest# PORT=18080
root@ubuntu:/home/zhl/mstest# docker run -d --name=kaka2 --hostname=kaka2 -p ${PORT}:8080 -e ZK='192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183' -e KAFKA='192.168.225.128:9091,192.168.225.128:9092,192.168.225.128:9093' mstest
8531d77e485c1c722dc6f2f6fc2fce3e9e316040c1ea7b0f0fb3d991fa3b2e46
root@ubuntu:/home/zhl/mstest# PORT=18082
root@ubuntu:/home/zhl/mstest# docker run -d --name=kaka3 --hostname=kaka3 -p ${PORT}:8080 -e ZK='192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183' -e KAFKA='192.168.225.128:9091,192.168.225.128:9092,192.168.225.128:9093' mstest
 

 

2. 查看微服務運行狀況：
 

 
 
root@ubuntu:/home/zhl/mstest# docker ps -a
CONTAINER ID        IMAGE                                 COMMAND                  CREATED             STATUS                     PORTS                     NAMES
d64ef36d5919        mstest                                "/start.sh"              2 minutes ago       Up 2 minutes               0.0.0.0:18081->8080/tcp   kaka1
637beebec29e        kafkatest                             "/bin/sh -c /usr/b..."   3 days ago          Up 26 hours                0.0.0.0:9093->9093/tcp    k3
4bf4925e6f40        kafkatest                             "/bin/sh -c /usr/b..."   3 days ago          Up 26 hours                0.0.0.0:9092->9092/tcp    k2
988c32940785        kafkatest                             "/bin/sh -c /usr/b..."   3 days ago          Up 26 hours                0.0.0.0:9091->9091/tcp    k1
c1ab885b2770        zk                                    "/opt/entrypoint.sh"     4 days ago          Up 25 hours                                          zk3
b939bfa60ea2        zk                                    "/opt/entrypoint.sh"     4 days ago          Up 25 hours                                          zk2
bd161a246c28        zk                                    "/opt/entrypoint.sh"     4 days ago          Up 25 hours                                          zk1
 

 

  
 

 
 
root@ubuntu:/home/zhl/mstest# docker logs d6
ZK=192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183
KAFKA=192.168.225.128:9091,192.168.225.128:9092,192.168.225.128:9093
zk.kaka.properties:
====
spring.application.name=bootZKKafka
spring.cloud.config.enabled=false
spring.cloud.zookeeper.connectString=192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183
server.port=8080
endpoints.restart.enabled=true
endpoints.shutdown.enabled=true
endpoints.health.sensitive=false
logging.level.org.apache.zookeeper.ClientCnxn: WARN
zkConnect=192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183
kafkaServerList=192.168.225.128:9091,192.168.225.128:9092,192.168.225.128:9093
group=my_group
topic=my_story
key=my_dream
====
start micro services...
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/my_app.jar!/lib/logback-classic-1.1.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/my_app.jar!/lib/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2017-09-19 08:14:54.994  INFO 9 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@30f92868: startup date [Tue Sep 19 08:14:54 UTC 2017]; root of context hierarchy
2017-09-19 08:14:55.715  INFO 9 --- [           main] f.a.AutowiredAnnotationBeanPostProcessor : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2017-09-19 08:14:55.782  INFO 9 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'configurationPropertiesRebinderAutoConfiguration' of type [class org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$faa8dd1d] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2017-09-19 08:14:56.445  INFO 9 --- [           main] o.a.c.f.imps.CuratorFrameworkImpl        : Starting
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:host.name=kaka1
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:java.version=1.8.0_91
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:java.vendor=Oracle Corporation
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:java.class.path=/my_app.jar
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:java.io.tmpdir=/tmp
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:java.compiler=<NA>
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:os.name=Linux
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:os.arch=amd64
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:os.version=4.10.0-33-generic
2017-09-19 08:14:56.466  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:user.name=root
2017-09-19 08:14:56.467  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:user.home=/root
2017-09-19 08:14:56.467  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Client environment:user.dir=/
2017-09-19 08:14:56.470  INFO 9 --- [           main] org.apache.zookeeper.ZooKeeper           : Initiating client connection, connectString=192.168.225.128:2181,192.168.225.128:2182,192.168.225.128:2183 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@4487109a
2017-09-19 08:14:56.789  INFO 9 --- [ain-EventThread] o.a.c.f.state.ConnectionStateManager     : State change: CONNECTED

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.3.3.RELEASE)

2017-09-19 08:14:57.337  INFO 9 --- [           main] b.c.PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource [name='zookeeper', propertySources=[ZookeeperPropertySource [name='config/bootZKKafka'], ZookeeperPropertySource [name='config/application']]]
2017-09-19 08:14:57.447  INFO 9 --- [           main] com.example.KafkaApplication             : No active profile set, falling back to default profiles: default
2017-09-19 08:14:57.486  INFO 9 --- [           main] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@1c7faa8a: startup date [Tue Sep 19 08:14:57 UTC 2017]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@30f92868
2017-09-19 08:14:59.061  INFO 9 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Overriding bean definition for bean 'requestContextFilter' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.jersey.JerseyAutoConfiguration; factoryMethodName=requestContextFilter; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/jersey/JerseyAutoConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter; factoryMethodName=requestContextFilter; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter.class]]
2017-09-19 08:14:59.064  INFO 9 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Overriding bean definition for bean 'beanNameViewResolver' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter.class]]
2017-09-19 08:14:59.364  INFO 9 --- [           main] o.s.cloud.context.scope.GenericScope     : BeanFactory id=c693bc6c-904d-38f8-8ece-872daebe5dd1
2017-09-19 08:14:59.390  INFO 9 --- [           main] f.a.AutowiredAnnotationBeanPostProcessor : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2017-09-19 08:14:59.425  INFO 9 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [class org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$faa8dd1d] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2017-09-19 08:15:00.563  INFO 9 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 8080 (http)
2017-09-19 08:15:00.601  INFO 9 --- [           main] o.apache.catalina.core.StandardService   : Starting service Tomcat
2017-09-19 08:15:00.603  INFO 9 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/8.0.32
2017-09-19 08:15:00.816  INFO 9 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2017-09-19 08:15:00.816  INFO 9 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 3330 ms
2017-09-19 08:15:01.744  INFO 9 --- [ost-startStop-1] o.s.b.c.e.ServletRegistrationBean        : Mapping servlet: 'com.example.JerseyConfig' to [/*]
2017-09-19 08:15:01.748  INFO 9 --- [ost-startStop-1] o.s.b.c.e.ServletRegistrationBean        : Mapping servlet: 'dispatcherServlet' to [/]
2017-09-19 08:15:01.756  INFO 9 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'characterEncodingFilter' to: [/*]
2017-09-19 08:15:01.761  INFO 9 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
2017-09-19 08:15:01.761  INFO 9 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'httpPutFormContentFilter' to: [/*]
2017-09-19 08:15:01.762  INFO 9 --- [ost-startStop-1] o.s.b.c.embedded.FilterRegistrationBean  : Mapping filter: 'requestContextFilter' to: [/*]
2017-09-19 08:15:01.943  INFO 9 --- [           main] o.a.k.clients.producer.ProducerConfig    : ProducerConfig values: 
    compression.type = none
    metric.reporters = []
    metadata.max.age.ms = 300000
    metadata.fetch.timeout.ms = 60000
    reconnect.backoff.ms = 50
    sasl.kerberos.ticket.renew.window.factor = 0.8
    bootstrap.servers = [192.168.225.128:9091, 192.168.225.128:9092, 192.168.225.128:9093]
    retry.backoff.ms = 100
    sasl.kerberos.kinit.cmd = /usr/bin/kinit
    buffer.memory = 33554432
    timeout.ms = 30000
    key.serializer = class org.apache.kafka.common.serialization.StringSerializer
    sasl.kerberos.service.name = null
    sasl.kerberos.ticket.renew.jitter = 0.05
    ssl.keystore.type = JKS
    ssl.trustmanager.algorithm = PKIX
    block.on.buffer.full = false
    ssl.key.password = null
    max.block.ms = 60000
    sasl.kerberos.min.time.before.relogin = 60000
    connections.max.idle.ms = 540000
    ssl.truststore.password = null
    max.in.flight.requests.per.connection = 5
    metrics.num.samples = 2
    client.id = boost.zk.kafka
    ssl.endpoint.identification.algorithm = null
    ssl.protocol = TLS
    request.timeout.ms = 30000
    ssl.provider = null
    ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
    acks = 1
    batch.size = 16384
    ssl.keystore.location = null
    receive.buffer.bytes = 32768
    ssl.cipher.suites = null
    ssl.truststore.type = JKS
    security.protocol = PLAINTEXT
    retries = 0
    max.request.size = 1048576
    value.serializer = class org.apache.kafka.common.serialization.StringSerializer
    ssl.truststore.location = null
    ssl.keystore.password = null
    ssl.keymanager.algorithm = SunX509
    metrics.sample.window.ms = 30000
    partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
    send.buffer.bytes = 131072
    linger.ms = 0

2017-09-19 08:15:02.215  INFO 9 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka version : 0.9.0.1
2017-09-19 08:15:02.215  INFO 9 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka commitId : 23c69d62a0cabf06
2017-09-19 08:15:02.378  INFO 9 --- [           main] o.a.k.clients.consumer.ConsumerConfig    : ConsumerConfig values: 
    metric.reporters = []
    metadata.max.age.ms = 300000
    value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
    group.id = my_group
    partition.assignment.strategy = [org.apache.kafka.clients.consumer.RangeAssignor]
    reconnect.backoff.ms = 50
    sasl.kerberos.ticket.renew.window.factor = 0.8
    max.partition.fetch.bytes = 1048576
    bootstrap.servers = [192.168.225.128:9091, 192.168.225.128:9092, 192.168.225.128:9093]
    retry.backoff.ms = 100
    sasl.kerberos.kinit.cmd = /usr/bin/kinit
    sasl.kerberos.service.name = null
    sasl.kerberos.ticket.renew.jitter = 0.05
    ssl.keystore.type = JKS
    ssl.trustmanager.algorithm = PKIX
    enable.auto.commit = true
    ssl.key.password = null
    fetch.max.wait.ms = 500
    sasl.kerberos.min.time.before.relogin = 60000
    connections.max.idle.ms = 540000
    ssl.truststore.password = null
    session.timeout.ms = 30000
    metrics.num.samples = 2
    client.id = 
    ssl.endpoint.identification.algorithm = null
    key.deserializer = class org.apache.kafka.common.serialization.IntegerDeserializer
    ssl.protocol = TLS
    check.crcs = true
    request.timeout.ms = 40000
    ssl.provider = null
    ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
    ssl.keystore.location = null
    heartbeat.interval.ms = 3000
    auto.commit.interval.ms = 1000
    receive.buffer.bytes = 32768
    ssl.cipher.suites = null
    ssl.truststore.type = JKS
    security.protocol = PLAINTEXT
    ssl.truststore.location = null
    ssl.keystore.password = null
    ssl.keymanager.algorithm = SunX509
    metrics.sample.window.ms = 30000
    fetch.min.bytes = 1
    send.buffer.bytes = 131072
    auto.offset.reset = latest

2017-09-19 08:15:02.633  INFO 9 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka version : 0.9.0.1
2017-09-19 08:15:02.634  INFO 9 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka commitId : 23c69d62a0cabf06
2017-09-19 08:15:03.032  INFO 9 --- [              c] com.example.EagleService                 : [c], Starting 
2017-09-19 08:15:03.384  WARN 9 --- [              c] org.apache.kafka.clients.NetworkClient   : Error while fetching metadata with correlation id 1 : {my_story=LEADER_NOT_AVAILABLE}
2017-09-19 08:15:03.879  WARN 9 --- [              c] org.apache.kafka.clients.NetworkClient   : Error while fetching metadata with correlation id 3 : {my_story=LEADER_NOT_AVAILABLE}
2017-09-19 08:15:03.987  WARN 9 --- [              c] org.apache.kafka.clients.NetworkClient   : Error while fetching metadata with correlation id 5 : {my_story=LEADER_NOT_AVAILABLE}
2017-09-19 08:15:04.169  INFO 9 --- [           main] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@1c7faa8a: startup date [Tue Sep 19 08:14:57 UTC 2017]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@30f92868
2017-09-19 08:15:04.696  INFO 9 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2017-09-19 08:15:04.699  INFO 9 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
2017-09-19 08:15:04.869  INFO 9 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2017-09-19 08:15:04.871  INFO 9 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2017-09-19 08:15:05.057  INFO 9 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2017-09-19 08:15:05.654  WARN 9 --- [           main] c.n.c.sources.URLConfigurationSource     : No URLs will be polled as dynamic configuration sources.
2017-09-19 08:15:05.657  INFO 9 --- [           main] c.n.c.sources.URLConfigurationSource     : To enable URLs as dynamic configuration sources, define System property archaius.configurationSource.additionalUrls or make config.properties available on classpath.
2017-09-19 08:15:05.675  WARN 9 --- [           main] c.n.c.sources.URLConfigurationSource     : No URLs will be polled as dynamic configuration sources.
2017-09-19 08:15:05.676  INFO 9 --- [           main] c.n.c.sources.URLConfigurationSource     : To enable URLs as dynamic configuration sources, define System property archaius.configurationSource.additionalUrls or make config.properties available on classpath.
2017-09-19 08:15:05.895  INFO 9 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2017-09-19 08:15:05.938  INFO 9 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Bean with name 'configurationPropertiesRebinder' has been autodetected for JMX exposure
2017-09-19 08:15:05.944  INFO 9 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Bean with name 'refreshScope' has been autodetected for JMX exposure
2017-09-19 08:15:05.944  INFO 9 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Bean with name 'environmentManager' has been autodetected for JMX exposure
2017-09-19 08:15:05.956  INFO 9 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Located managed bean 'environmentManager': registering with JMX server as MBean [org.springframework.cloud.context.environment:name=environmentManager,type=EnvironmentManager]
2017-09-19 08:15:06.002  INFO 9 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Located managed bean 'refreshScope': registering with JMX server as MBean [org.springframework.cloud.context.scope.refresh:name=refreshScope,type=RefreshScope]
2017-09-19 08:15:06.042  INFO 9 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Located managed bean 'configurationPropertiesRebinder': registering with JMX server as MBean [org.springframework.cloud.context.properties:name=configurationPropertiesRebinder,context=1c7faa8a,type=ConfigurationPropertiesRebinder]
2017-09-19 08:15:06.353  INFO 9 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Starting beans in phase 0
2017-09-19 08:15:06.750  INFO 9 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
2017-09-19 08:15:07.490  WARN 9 --- [           main] org.apache.curator.utils.ZKPaths         : The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead.
2017-09-19 08:15:07.722  INFO 9 --- [           main] com.example.KafkaApplication             : Started KafkaApplication in 14.542 seconds (JVM running for 16.038)
 

 

 3. 遇到的問題：
 

 在運行微服務實例時，曾出現以下問題：
 

 
 
 [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
 

 

 後經排查，發現源文件中spring.cloud.zookeeperconnectString=ZK，而實際上應該爲：spring.cloud.zookeeper.connectString=ZK，少了一個點
 

4. 測試
 

 
 

 
 

4、使用Nginx進行負載均衡
 

1. 新建並編輯/tmp/nginx/nginx.conf文件   
 

 
 
worker_processes 2;

events {
  worker_connections  1024;
}

http {
  upstream my_service {
    server 172.17.0.5:8080;server 172.17.0.6:8080;server 172.17.0.7:8080;
 
 
}
 server { 
　　listen 8000; 
　　server_name localhost; 
　　location / { 
　　　　proxy_pass http://my_service;  
      proxy_redirect off; 
    }
 } 
}
 

 

 
 

2. 構建鏡像：    
 

　　使用網易docker鏡像：https://c.163.com/hub#/m/repository/?repoId=2967
　　運行：docker pull hub.c.163.com/library/nginx:latest
 

   下載完畢後，能夠查詢：docker ps -a   
 

 
 
root@ubuntu:/tmp# docker images
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
mstest                                    latest              7ae56ffdd866        2 hours ago         769MB
kafkatest                                 latest              eaa35ba12fe6        4 days ago          762MB
zk                                        latest              5c819179f3f8        4 days ago          778MB
ubuntu                                    xenial              8b72bba4485f        6 days ago          120MB
hub.c.163.com/library/nginx               latest              46102226f2fd        4 months ago        109MB
index.tenxcloud.com/docker_library/java   latest              264282a59a95        15 months ago       669MB
index.tenxcloud.com/tenxcloud/redis       latest              c9dc199d1b1c        2 years ago         190MB
 

 

 
 

3. 運行容器
 

 
 
root@ubuntu:/tmp# docker run -d --name kaka-ng -v /tmp/website:/usr/share/nginx/html -v /tmp/nginx:/etc/nginx:ro -p 18000:8000 hub.c.163.com/library/nginx
393cedc53742e537100672b012346169d26f342efeb236437a1412446c7edc27
 

 

 
 

4. 測試Nginx負載均衡(輪詢)
 

 
 
root@ubuntu:/tmp/nginx# http --body localhost:18000/hi
172.17.0.5:8080

root@ubuntu:/tmp/nginx# http --body localhost:18000/hi
172.17.0.6:8080

root@ubuntu:/tmp/nginx# http --body localhost:18000/hi
172.17.0.7:8080

root@ubuntu:/tmp/nginx# http --body localhost:18000/hi
172.17.0.5:8080

root@ubuntu:/tmp/nginx# http --body localhost:18000/hi
172.17.0.6:8080

root@ubuntu:/tmp/nginx# http --body localhost:18000/hi
172.17.0.7:8080

root@ubuntu:/tmp/nginx# 
 



        

            

        	





    

		

            
相關文章

            

                

                

                    

                

            


            
相關標籤/搜索

            
        


    

        

            

        

        

        
        

        

        

    




	

    





    

    	

    

        每日一句
    

    
    
    	每一个你不满意的现在，都有一个你没有努力的曾经。
    







    

    


    



    

        本站公眾號
    

    

           歡迎關注本站公眾號,獲取更多信息

        
    




    

    




	


	







    

        聯繫我們
        最近搜索
        最新文章
        
        沪ICP备13005482号-10

            MyBatis教程
            SQL 教程
            MySQL教程
            Java 教程
            Thymeleaf 教程
            Hibernate教程
            Spring教程 
            Redis教程