用cAdvisor InfluxDB Grafana監控docker容器的TcpState

問題

搭建完cAdvisor InfluxDB Grafana監控集羣后, 發現沒有tcp相關的數據.

源碼版本:

https://github.com/google/cad...
git commit hash:9db8c7dee20a0c41627b208977ab192a0411bf93

搭建cAdvisor InfluxDB Grafana參考

https://botleg.com/stories/mo...

定位過程

是否cadvisor沒有記錄tcp state?

容易搜索到, 由於cadvisor的高cpu佔用, 須要--disable_metrics=""
https://github.com/google/cad...
實際上並不是如此.
不帶任何參數狀況下, 本地啓動cadvisor.
~/gopath/src/github.com/google/cadvisor(master*) » sudo ./cadvisor -logtostderr
在瀏覽器中打開 http://127.0.0.1:8080/containers/ 能夠看到response中, 帶有TcpState.

是否寫入了influxdb?

1. 打開influx db shell

InfluxDB shell 0.9.6.1
> show databases
name: databases
---------------
name
_internal
mydb
cadvisor
> use cadvisor
Using database cadvisor
> show tag keys
name: cpu_usage_system
----------------------
tagKey
container_name
machine

能夠看到, 這些tagKey對應grafana中的select column.
那麼, 是否cadvisor沒有寫入influxdb呢?
cadvisor/storage/influxdb/influxdb.go:174

func (self *influxdbStorage) containerStatsToPoints(
    cInfo *info.ContainerInfo,
    stats *info.ContainerStats,
) (points []*influxdb.Point) {
    // CPU usage: Total usage in nanoseconds
    points = append(points, makePoint(serCpuUsageTotal, stats.Cpu.Usage.Total))

    // CPU usage: Time spend in system space (in nanoseconds)
    points = append(points, makePoint(serCpuUsageSystem, stats.Cpu.Usage.System))

    // CPU usage: Time spent in user space (in nanoseconds)
    points = append(points, makePoint(serCpuUsageUser, stats.Cpu.Usage.User))

    // CPU usage per CPU
    for i := 0; i < len(stats.Cpu.Usage.PerCpu); i++ {
        point := makePoint(serCpuUsagePerCpu, stats.Cpu.Usage.PerCpu[i])
        tags := map[string]string{"instance": fmt.Sprintf("%v", i)}
        addTagsToPoint(point, tags)

        points = append(points, point)
    }

    // Load Average
    points = append(points, makePoint(serLoadAverage, stats.Cpu.LoadAverage))

    // Memory Usage
    points = append(points, makePoint(serMemoryUsage, stats.Memory.Usage))

    // Working Set Size
    points = append(points, makePoint(serMemoryWorkingSet, stats.Memory.WorkingSet))

    // Network Stats
    points = append(points, makePoint(serRxBytes, stats.Network.RxBytes))
    points = append(points, makePoint(serRxErrors, stats.Network.RxErrors))
    points = append(points, makePoint(serTxBytes, stats.Network.TxBytes))
    points = append(points, makePoint(serTxErrors, stats.Network.TxErrors))

    self.tagPoints(cInfo, stats, points)

    return points
}

結論

須要修改cadvisor代碼, 將本身須要的metrics加上.