Graylog+elasticsearch+mongodb集羣+nginx負載均衡前端

 

網上有張圖畫的很好，搜索有關它的配置文章，google裏有幾篇英文的，都是依靠haproxy等或別的什麼實現，沒有純粹的Graylog+elasticsearch+mongodb集羣，項目須要，只有本身摸索一下了，搞好發現其實也很簡單。

 

 

 

 

整個結構大概就是上圖樣子，nginx的負載均衡能夠參考我前面的文章。將後端的web集羣換成graylog集羣地址就能夠了。

 

下面是4臺集羣的配置效果圖，完成後的效果是隨便訪問那個IP，均可以訪問到全部的日誌，節點能夠根據需求任意添加，不管graylog，es,仍是mongodb均可以。

 

 

配置過程以下，難點在於graylog的集羣，安裝軟件的過程能夠參考前面的安裝過程。

========================================elasticsearch=================================

#vim /etc/elasticsearch/elasticsearch.yml

node.master: true    #兩臺機器設置成master，其他節點此項不寫

network.bind_host: 192.168.1.231  #綁定實際的服務器IP地址，不是集羣的IP

cluster.name: graylog        #與/etc/graylog/server/server.conf 中的elasticsearch_cluster_name 集羣名稱必須相同

discovery.zen.minimum_master_nodes: 2  #4臺機器就2個夠了

discovery.zen.ping.multicast.enabled: false

discovery.zen.ping.unicast.hosts: ["192.168.1.231:9300","192.168.1.232:9300","192.168.1.233:9300","192.168.1.234:9300"]

 

=================================================mongodb========================================

 

vim /etc/mongod.conf   #全部節點配置文件相同

 

replication: 

   replSetName: graylog

systemLog:

  destination: file

  logAppend: true

  path: /var/log/mongodb/mongod.log

storage:

  dbPath: /var/lib/mongo

  journal:

    enabled: true

processManagement:

  fork: true  # fork and run in background

  pidFilePath: /var/run/mongodb/mongod.pid  # location of pidfile

net:

  port: 27017

 

 

 

#mongo//進入數據庫配置

rs.initiate()

rs.add("192.168.1.231:27017");

rs.add("192.168.1.232:27017");

rs.addArb("192.168.1.233:27017");

rs.add("192.168.1.234:27017");

graylog:PRIMARY>  rs.config();   rs.config(); 

{

        "_id" : "graylog",

        "version" : 4,

        "protocolVersion" : NumberLong(1),

        "members" : [

                {

                        "_id" : 0,

                        "host" : "node231:27017",

                        "arbiterOnly" : false,

                        "buildIndexes" : true,

                        "hidden" : false,

                        "priority" : 1,

                        "tags" : {

 

                        },

                        "slaveDelay" : NumberLong(0),

                        "votes" : 1

                },

                {

                        "_id" : 1,

                        "host" : "192.168.1.232:27017",

                        "arbiterOnly" : false,

                        "buildIndexes" : true,

                        "hidden" : false,

                        "priority" : 1,

                        "tags" : {

 

                        },

                        "slaveDelay" : NumberLong(0),

                        "votes" : 1

                },

                {

                        "_id" : 2,

                        "host" : "192.168.1.234:27017",

                        "arbiterOnly" : false,

                        "buildIndexes" : true,

                        "hidden" : false,

                        "priority" : 1,

                        "tags" : {

 

                        },

                        "slaveDelay" : NumberLong(0),

                        "votes" : 1

                },

                {

                        "_id" : 3,

                        "host" : "192.168.1.233:27017",

                        "arbiterOnly" : true,

                        "buildIndexes" : true,

                        "hidden" : false,

                        "priority" : 1,

                        "tags" : {

 

                        },

                        "slaveDelay" : NumberLong(0),

                        "votes" : 1

                }

        ],

        "settings" : {

                "chainingAllowed" : true,

                "heartbeatIntervalMillis" : 2000,

                "heartbeatTimeoutSecs" : 10,

                "electionTimeoutMillis" : 10000,

                "getLastErrorModes" : {

 

                },

                "getLastErrorDefaults" : {

                        "w" : 1,

                        "wtimeout" : 0

                },

                "replicaSetId" : ObjectId("578c49834bd6994271fcfad7")

        }

}

============================graylog===================================

#vim /etc/graylog/server/server.conf

password_secret =0b4e7a0e5fe84ad35fb5f95b9ceeac790b4e7a0e5fe84ad35fb5f95b9ceeac79

root_password_sha2 =ed02457b5c41d964dbd2f2a609d63fe1bb7528dbe55e1abf5b52c249cd735797

elasticsearch_cluster_name = graylog 

elasticsearch_discovery_zen_ping_multicast_enabled = false 

elasticsearch_discovery_zen_ping_unicast_hosts = 192.168.1.231:9300,192.168.1.232:9300,192.168.1.233:9300,192.168.1.234:9300

mongodb_uri = mongodb://192.168.1.231:27017,192.168.1.232:27017,192.168.1.233:27017,192.168.1.234:27017/graylog

elasticsearch_shards = 4

elasticsearch_replicas = 1

mongodb_useauth = false       #默認爲false,可沒必要添加，生產環境必須

 

 

=====================測試=============================

 

使用curl命令操做elasticsearch

# curl -X GET 'http://192.168.1.231:9200/_cat/nodes'

192.168.1.234 192.168.1.234  7 13 0.06 d * Marrina                                     

192.168.1.233 192.168.1.233  7 21 0.12 d m Banshee                                      

192.168.1.231 192.168.1.231  8 23 0.43 d m Katie Power                                 

192.168.1.232 192.168.1.232  7 22 0.15 d m Dark Phoenix                                

192.168.1.231 192.168.1.231 20 23 0.43 c - graylog-8562414f-0f9b-48d9-b0c6-9faab367832e

]# curl -X GET 'http://192.168.1.231:9200/_cluster/health?pretty=true'

{

  "cluster_name" : "graylog",

  "status" : "green",

  "timed_out" : false,

  "number_of_nodes" : 4,

  "number_of_data_nodes" : 3,

  "active_primary_shards" : 4,

  "active_shards" : 4,

  "relocating_shards" : 0,

  "initializing_shards" : 0,

  "unassigned_shards" : 0,

  "delayed_unassigned_shards" : 0,

  "number_of_pending_tasks" : 0,

  "number_of_in_flight_fetch" : 0,

  "task_max_waiting_in_queue_millis" : 0,

  "active_shards_percent_as_number" : 100.0

}

 

mongodb狀態查詢

graylog:PRIMARY> rs.status()

{

        "set" : "graylog",

        "date" : ISODate("2016-07-18T09:14:33.419Z"),

        "myState" : 1,

        "term" : NumberLong(2),

        "heartbeatIntervalMillis" : NumberLong(2000),

        "members" : [

                {

                        "_id" : 0,

                        "name" : "node231:27017",

                        "health" : 1,

                        "state" : 1,

                        "stateStr" : "PRIMARY",

                        "uptime" : 19532,

                        "optime" : {

                                "ts" : Timestamp(1468833273, 2),

                                "t" : NumberLong(2)

                        },

                        "optimeDate" : ISODate("2016-07-18T09:14:33Z"),

                        "electionTime" : Timestamp(1468813741, 1),

                        "electionDate" : ISODate("2016-07-18T03:49:01Z"),

                        "configVersion" : 5,

                        "self" : true

                },

                {

                        "_id" : 1,

                        "name" : "192.168.1.232:27017",

                        "health" : 1,

                        "state" : 2,

                        "stateStr" : "SECONDARY",

                        "uptime" : 8272,

                        "optime" : {

                                "ts" : Timestamp(1468833271, 4),

                                "t" : NumberLong(2)

                        },

                        "optimeDate" : ISODate("2016-07-18T09:14:31Z"),

                        "lastHeartbeat" : ISODate("2016-07-18T09:14:31.802Z"),

                        "lastHeartbeatRecv" : ISODate("2016-07-18T09:14:31.765Z"),

                        "pingMs" : NumberLong(0),

                        "syncingTo" : "node231:27017",

                        "configVersion" : 5

                },

                {

                        "_id" : 2,

                        "name" : "192.168.1.234:27017",

                        "health" : 1,

                        "state" : 2,

                        "stateStr" : "SECONDARY",

                        "uptime" : 8257,

                        "optime" : {

                                "ts" : Timestamp(1468833271, 4),

                                "t" : NumberLong(2)

                        },

                        "optimeDate" : ISODate("2016-07-18T09:14:31Z"),

                        "lastHeartbeat" : ISODate("2016-07-18T09:14:31.736Z"),

                        "lastHeartbeatRecv" : ISODate("2016-07-18T09:14:31.802Z"),

                        "pingMs" : NumberLong(0),

                        "syncingTo" : "192.168.1.232:27017",

                        "configVersion" : 5

                },

                {

                        "_id" : 3,

                        "name" : "192.168.1.233:27017",

                        "health" : 1,

                        "state" : 7,

                        "stateStr" : "ARBITER",

                        "uptime" : 8092,

                        "lastHeartbeat" : ISODate("2016-07-18T09:14:31.802Z"),

                        "lastHeartbeatRecv" : ISODate("2016-07-18T09:14:32.548Z"),

                        "pingMs" : NumberLong(0),

                        "configVersion" : 5

                }

        ],

        "ok" : 1

}

graylog:PRIMARY>