Go語言之高級篇beego框架之日誌收集系統
1、日誌收集系統架構設計
圖1java
圖2git
2、開發環境
一、安裝jdkgithub
jdk-8u51-windows-x64.exeapache
安裝目錄：C:\Program Files\jdk8bootstrap
二、安裝zookeeperwindows
zookeeper-3.3.6.tar.gzsession
安裝目錄： C:\Program Files\zookeeper-3.3.6架構
配置：app
C:\Program Files\zookeeper-3.3.6\confdom
把配置文件：zoo_sample.cfg，更名成：zoo.cfg
修改部署路徑爲你安裝的路徑。
配置環境變量：
C:\Program Files\zookeeper-3.3.6\bin
#啓動服務：
進行目錄，直接啓動服務：C:\Program Files\zookeeper-3.3.6\bin
直接運行服務名稱: zkServer.cmd
h had sessionid 0x16908b8a4920000 2019-02-20 10:52:39,153 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53568 2019-02-20 10:52:39,154 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53568 2019-02-20 10:52:39,155 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53568 2019-02-20 10:52:39,158 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:39,161 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:39,163 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53568 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:41,214 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53570 2019-02-20 10:52:41,216 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53570 2019-02-20 10:52:41,217 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53570 2019-02-20 10:52:41,220 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:41,224 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:41,226 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53570 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:42,438 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53571 2019-02-20 10:52:42,439 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53571 2019-02-20 10:52:42,441 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53571 2019-02-20 10:52:42,444 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:42,448 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:42,449 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53571 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:44,219 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53572 2019-02-20 10:52:44,220 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53572 2019-02-20 10:52:44,221 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53572 2019-02-20 10:52:44,223 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:44,227 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:44,229 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53572 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:45,462 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53573 2019-02-20 10:52:45,464 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0 :0:0:0:1:53573 2019-02-20 10:52:45,465 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /0:0:0:0:0:0:0:1:53573 2019-02-20 10:52:45,468 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:45,473 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:45,475 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53573 whic h had sessionid 0x16908b8a4920000 2019-02-20 10:52:46,769 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53574 2019-02-20 10:52:46,770 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53574 2019-02-20 10:52:46,772 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53574 2019-02-20 10:52:46,775 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:46,779 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:46,781 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53574 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:48,284 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53575 2019-02-20 10:52:48,285 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53575 2019-02-20 10:52:48,286 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53575 2019-02-20 10:52:48,289 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:48,293 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:48,295 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53575 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:50,069 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53576 2019-02-20 10:52:50,071 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53576 2019-02-20 10:52:50,072 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53576 2019-02-20 10:52:50,075 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:50,080 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:50,081 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53576 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:51,547 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53577 2019-02-20 10:52:51,548 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53577 2019-02-20 10:52:51,550 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53577 2019-02-20 10:52:51,552 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:51,557 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:51,559 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53577 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:53,315 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53579 2019-02-20 10:52:53,316 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0 :0:0:0:1:53579 2019-02-20 10:52:53,318 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /0:0:0:0:0:0:0:1:53579 2019-02-20 10:52:53,321 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:53,325 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java .io.IOException: 遠程主機強迫關閉了一個現有的鏈接。 2019-02-20 10:52:53,326 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53579 whic h had sessionid 0x16908b8a4920000 2019-02-20 10:52:54,537 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53580 2019-02-20 10:52:54,539 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0 :0:0:0:1:53580 2019-02-20 10:52:54,540 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /0:0:0:0:0:0:0:1:53580 2019-02-20 10:52:54,545 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:54,551 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@634] - EndOfStreamException: Unable to read additional data from client sessionid 0x16908b8a4920000, likely client has closed socket 2019-02-20 10:52:54,552 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53580 whic h had sessionid 0x16908b8a4920000 2019-02-20 10:52:56,153 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53581 2019-02-20 10:52:56,154 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53581 2019-02-20 10:52:56,156 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53581 2019-02-20 10:52:56,159 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:56,163 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@634] - EndOfStreamException: Unable to read additional data from client sessionid 0x16908b8a4920000, likely client has closed socket 2019-02-20 10:52:56,164 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53581 which had sessionid 0x16908b8a4920000 2019-02-20 10:52:58,023 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53582 2019-02-20 10:52:58,024 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0 :0:0:0:1:53582 2019-02-20 10:52:58,026 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /0:0:0:0:0:0:0:1:53582 2019-02-20 10:52:58,028 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:58,032 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@634] - EndOfStreamException: Unable to read additional data from client sessionid 0x16908b8a4920000, likely client has closed socket 2019-02-20 10:52:58,034 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53582 whic h had sessionid 0x16908b8a4920000 2019-02-20 10:52:59,424 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53584 2019-02-20 10:52:59,425 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53584 2019-02-20 10:52:59,425 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53584 2019-02-20 10:52:59,428 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:52:59,431 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@634] - EndOfStreamException: Unable to read additional data from client sessionid 0x16908b8a4920000, likely client has closed socket 2019-02-20 10:52:59,431 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53584 which had sessionid 0x16908b8a4920000 2019-02-20 10:53:01,246 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53585 2019-02-20 10:53:01,247 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53585 2019-02-20 10:53:01,249 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53585 2019-02-20 10:53:01,252 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:53:01,259 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@634] - EndOfStreamException: Unable to read additional data from client sessionid 0x16908b8a4920000, likely client has closed socket 2019-02-20 10:53:01,260 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53585 which had sessionid 0x16908b8a4920000 2019-02-20 10:53:02,677 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53587 2019-02-20 10:53:02,678 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0 :0:0:0:1:53587 2019-02-20 10:53:02,679 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /0:0:0:0:0:0:0:1:53587 2019-02-20 10:53:02,682 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:53:02,685 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@634] - EndOfStreamException: Unable to read additional data from client sessionid 0x16908b8a4920000, likely client has closed socket 2019-02-20 10:53:02,686 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53587 whic h had sessionid 0x16908b8a4920000 2019-02-20 10:53:04,384 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53588 2019-02-20 10:53:04,385 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53588 2019-02-20 10:53:04,387 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53588 2019-02-20 10:53:04,390 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:53:04,399 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@634] - EndOfStreamException: Unable to read additional data from client sessionid 0x16908b8a4920000, likely client has closed socket 2019-02-20 10:53:04,400 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53588 which had sessionid 0x16908b8a4920000 2019-02-20 10:53:06,078 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53589 2019-02-20 10:53:06,079 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0 :0:0:0:1:53589 2019-02-20 10:53:06,081 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /0:0:0:0:0:0:0:1:53589 2019-02-20 10:53:06,084 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14 2019-02-20 10:53:06,089 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@634] - EndOfStreamException: Unable to read additional data from client sessionid 0x16908b8a4920000, likely client has closed socket 2019-02-20 10:53:06,090 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53589 whic h had sessionid 0x16908b8a4920000 2019-02-20 10:53:07,834 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53590 2019-02-20 10:53:07,835 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0 .1:53590 2019-02-20 10:53:07,836 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6 000 for client /127.0.0.1:53590 2019-02-20 10:53:07,838 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe eperServer@593] - Dropping packet at server of type 14
三、kafka
kafka_2.12-2.1.1.tgz
安裝目錄：D:\run_file\kafka_2.12-2.1.1 （不能這在C:\Program Files\ 下面啓動服務會報錯。）
#修改配置文件
D:\run_file\kafka_2.12-2.1.1\config
修改成本機IP地址：
advertised.host.name=172.28.5.25
#配置日誌地址：
########### Log Basics ############# # A comma separated list of directories under which to store log files #log.dirs=/tmp/kafka-logs log.dirs=D:/run_file/kafka_2.12-2.1.1/kafka-logs #增長這行代碼
啓動服務：
進入安裝目錄中：
在命令行下： cd D:\run_file\kafka_2.12-2.1.1 ./bin/windows/kafka-server-start.bat ./config/server.preperties
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSock
NIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1145
[2019-02-20 10:53:06,087] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,089] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:06,189] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,190] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:06,190] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,191] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:07,832] INFO Opening socket connection to server localhost/1
.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.
ache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,834] INFO Socket connection established to localhost/127.
0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,836] WARN Connected to an old server; r-o mode will be un
ailable (org.apache.zookeeper.ClientCnxnSocket)
[2019-02-20 10:53:07,837] INFO Session establishment complete on server localh
t/127.0.0.1:2181, sessionid = 0x16908b8a4920000, negotiated timeout = 6000 (or
apache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,844] INFO [SocketServer brokerId=0] Started processors fo
1 acceptors (kafka.network.SocketServer)
[2019-02-20 10:53:07,853] INFO Kafka version : 2.1.1 (org.apache.kafka.common.
ils.AppInfoParser)
[2019-02-20 10:53:07,853] INFO Kafka commitId : 21234bee31165527 (org.apache.k
ka.common.utils.AppInfoParser)
[2019-02-20 10:53:07,856] INFO [KafkaServer id=0] started (kafka.server.KafkaS
ver)
[2019-02-20 10:53:12,993] INFO [GroupMetadataManager brokerId=0] Removed 0 exp
ed offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
2、tail
在cmd中安裝：
go get github.com/hpcloud/tail
#代碼：
建立一個my.log的文件
路徑：
D:\GoFiles\my.log
示例：
vi my.log
############################# Producer Basics ############################# # list of brokers used for bootstrapping knowledge about the rest of the cluster # format: host1:port1,host2:port2 ... bootstrap.servers=localhost:9092 # specify the compression codec for all data generated: none, gzip, snappy, lz4, zstd compression.type=none # name of the partitioner class for partitioning events; default partition spreads data randomly #partitioner.class= # the maximum amount of time the client will wait for the response of a request #request.timeout.ms= # how long `KafkaProducer.send` and `KafkaProducer.partitionsFor` will block for #max.block.ms= # the producer will wait for up to the given delay to allow other records to be sent so that the sends can be batched together #linger.ms= # the maximum size of a request in bytes #max.request.size= # the default batch size in bytes when batching multiple records sent to a partition #batch.size= # the total bytes of memory the producer can use to buffer records waiting to be sent to the server #buffer.memory=
示例：把文件中內容讀出來
vi main.go
package main
import (
"fmt"
"time"
"github.com/hpcloud/tail"
)
func main() {
filename := "D:\\GoFiles\\my.log"
tails, err := tail.TailFile(filename, tail.Config{
ReOpen: true,
Follow: true,
//Location: &tail.SeekInfo{Offset: 0, Whence: 2},
MustExist: false,
Poll: true,
})
if err != nil {
fmt.Println("tail file err:", err)
return
}
var msg *tail.Line
var ok bool
for true {
msg, ok = <-tails.Lines
if !ok {
fmt.Printf("tail file close reopen,filename:%s\n", tails, filename)
time.Sleep(100 * time.Millisecond)
continue
}
fmt.Println("msg:", msg.Text)
}
}
執行結果：
#編譯
#在終端中輸入： D:\GoFiles\src\log_collect\tail>go build D:\GoFiles\src\log_collect\tail #運行 D:\GoFiles\src\log_collect\tail>tail.exe #返回結果： ############################# Producer Basics ############################# msg: msg: # list of brokers used for bootstrapping knowledge about the rest of the cl uster msg: # format: host1:port1,host2:port2 ... msg: bootstrap.servers=localhost:9092 msg: msg: # specify the compression codec for all data generated: none, gzip, snappy, lz4, zstd msg: compression.type=none msg: msg: # name of the partitioner class for partitioning events; default partition spreads data randomly msg: #partitioner.class= msg: msg: # the maximum amount of time the client will wait for the response of a req uest msg: #request.timeout.ms= msg: msg: # how long `KafkaProducer.send` and `KafkaProducer.partitionsFor` will bloc k for msg: #max.block.ms= msg: msg: # the producer will wait for up to the given delay to allow other records t o be sent so that the sends can be batched together msg: #linger.ms= msg: msg: # the maximum size of a request in bytes msg: #max.request.size= msg: msg: # the default batch size in bytes when batching multiple records sent to a partition msg: #batch.size= msg: msg: # the total bytes of memory the producer can use to buffer records waiting to be sent to the server msg: #buffer.memory=
#
2、els安裝
一、安裝Elasticsearch
elasticsearch-5.5.0.msi
安裝目錄
C:\Program Files\Elastic\Elasticsearch
#啓動目錄
C:\Program Files\Elastic\Elasticsearch\bin
運行：
elasticsearch.exe
#訪問： http://127.0.0.1:9200/
二、kafka（windows版本安裝）
軟件部署：
kibana-5.5.1-windows-x86.zip
解壓到下面目錄中：
C:\Program Files (x86)\kibana
#修改配置文件
elasticsearch.url: "http://127.0.0.1:9200" #指向es的啓動頁面
#啓動kibana
C:\Program Files (x86)\kibana\bin
運行：
kibana.bat
示例：
運行程序的前提條件：
先安裝elastic.v2
安裝包：
go get gopkg.in/olivere/elastic.v2
使用方法：
import "gopkg.in/olivere/elastic.v2"
目錄：
代碼：
vi main.go
package main
import (
"fmt"
elastic "gopkg.in/olivere/elastic.v2"
)
type Tweet struct {
User string
Message string
}
func main() {
client, err := elastic.NewClient(elastic.SetSniff(false), elastic.SetURL("http://127.0.0.1:9200/"))
if err != nil {
fmt.Println("connect es error", err)
return
}
fmt.Println("conn es succ")
for i := 0; i < 10000; i++ {
tweet := Tweet{User: "olivere", Message: "Take Five"}
_, err = client.Index().
Index("twitter").
Type("tweet").
Id(fmt.Sprintf("%d", i)).
BodyJson(tweet).
Do()
if err != nil {
// Handle error
panic(err)
return
}
}
fmt.Println("insert success")
}
#打開頁面：
http://localhost:5601
輸入： twitter --->再點Discover就能夠看到下面頁面。
三、
相關文章
- 1. Go語言之高級篇beego框架之controller調用model
- 2. Go語言之高級篇beego框架之cookie與session
- 3. go語言日誌收集系統項目--配置文件
- 4. go學習之Beego框架
- 5. 日誌收集之kafka篇
- 6. Go語言及Web框架Beego環境
- 7. go語言beego速成框架開發
- 8. 海量日誌實時收集系統架構設計與go語言實現
- 9. Go語言實現日誌系統
- 10. go 語言 beego
- 更多相關文章...
- • XML DOM 高級 - XML 教程
- • Markdown 高級技巧 - Markdown 教程
- • Docker容器實戰(七) - 容器眼光下的文件系統
- • 互聯網組織的未來：剖析GitHub員工的任性之源
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
- 1. 字節跳動21屆秋招運營兩輪面試經驗分享
- 2. Java 3 年，25K 多嗎？
- 3. mysql安裝部署
- 4. web前端開發中父鏈和子鏈方式實現通信
- 5. 3.1.6 spark體系之分佈式計算-scala編程-scala中trait特性
- 6. dataframe2
- 7. ThinkFree在線
- 8. 在線畫圖
- 9. devtools熱部署
- 10. 編譯和鏈接
歡迎關注本站公眾號,獲取更多信息
