簡單的應用服務器性能監控

因爲項目上線，緊急應急需求，話說回來，公司CTO無能，如此重大的項目，竟然沒有早早規劃應用系統的監控準備工做，還他媽的要求每10分鐘彙報一次，愚蠢的很。。。暫且不吐槽了。

 

項目主要要求監控服務器的CPU利用率，磁盤利用率，以及tomcat的平均鏈接數（可能一臺服務器上有多個tomcat）。

 

關於上述指標，我採用shell腳本實現相關數據的抓取，可是呢，上面領導的無能，總不能致使咱們項目實際負責人也跟着無能的讓團隊受苦吧，就算不能百分百解放團隊成員，至少能作到一些自動化，也是技術提高生產力，釋放人類雙手的一種價值所在吧。個人方案，就是經過shell抓取實際待監控的數據，而後用python程序做爲agent，裝在全部帶監控的服務器上，以服務的形式存在在。另外，在其中一臺上面，安裝一個watchdog的程序，每隔一段時間，就「汪汪」一下，向主人提醒，有新的數據到來了。。。

另外，爲了方便查看數據，統計的數據配合EChart.js展現，將上述監控數據在這個watchdog程序裏面彙總，而後經過rsync將結果文件同步到一個網站（nginx）上，經過網絡訪問該監控數據web頁面，作到隨時隨地的看監控。雖然不是高大上的系統，至少也算是一種簡單有效的方案，我崇尚簡單就是美！

 

下面簡單看看，這個解決方案的架構圖：

如上面已經介紹過，具體的性能數據的獲取，是經過shell實現的。當watchdog這個定時任務執行的時候，將會向全部的待監測服務器發送指令（python程序實現的，基於TCP發送指令），待監控的機器上的python編寫的agent程序在收到指令後，即執行shell腳本，實現數據的監控獲取。而後以TCP的方式返回給watchdog程序。watchdog程序，將全部的檢測數據，按照必定的格式，固然是方便EChat的展現了，寫入一個文件，最終經過rsync推送到nginx的web服務器上，這樣一來，須要看這個數據的人，就能夠經過訪問網址的形式，獲得相關的監測結果。

 

思路和實現過程都不是很複雜，就不作過多的描述。下面，將相關的shell腳本以及python的agent程序以及watchdog程序，附在這裏，供有須要的參考！

一、 shell腳本

#!/bin/bash

function cpuSts(){
   content=`top -n 1| grep "Cpu(s)"`
   ##echo $content
   cpup=`echo ${content#*:}`
   cpuu=`echo ${cpup%us*}`
   cpuss=`echo ${cpup#*,}`
   cpus=`echo ${cpuss%sy*}`
   us=`echo "$cpuu" | sed 's/^[ \t]*//g'`
   sy=`echo "$cpus" | sed 's/^[ \t]*//g'`
   #echo $us
   ius=`echo $us |awk -F ' ' '{print $2}'`
   #echo $ius
   #echo $sy
   isy=`echo $sy |awk -F ' ' '{print $2}'`
   #echo $isy

   fius=`echo $ius |awk -F '%' '{print $1}'`
   fisy=`echo $isy |awk -F '%' '{print $1}'`
   total=`echo $fius+$fisy | bc`
   #echo "us:$ius, sy:$isy, total:0$total%"
   echo "$total%"
}
cpus=`cpuSts`
#echo "cpu:$cpus"

function diskSts() {
   content=`df -h | grep "/dev" | grep -v "tmpfs" | awk '{print $5}'`
   echo "$content"
}
disks=`diskSts`
#echo "disk:$disks"

function tomcatSts() {
   pids=`ps -ef|grep tomcat | grep "org.apache.catalina.startup.Bootstrap start" |grep -v "grep" | awk '{print $2}'`
   cnt=0
   idx=0
   for pid in $pids; do
       #echo "$pid"
       tct=`netstat -alptn | grep $pid/java | grep ESTABLISHED | wc -l`
       let cnt+=tct
       let idx+=1
   done
   echo "establishedCount:$cnt,tomcatCount:$idx"
}
tomcats=`tomcatSts`
echo "cpu:$cpus,disk:$disks,$tomcats"

 

2. watchdog的python編寫的cron的任務代碼（TCP客戶端）

#!/usr/bin/env python
# -*- coding:utf-8 -*-

# #給server端發送命令
import socket, sys, traceback
import threading
import time
import subprocess
host1 = '10.130.207.210'
host2 = '10.130.207.211'
port = 23153
curDate =  str(time.strftime('%Y-%m-%d',time.localtime(time.time())))
filepath = "/home/tomcat/weibao/robot/%s.txt"%(curDate)
filepathjs = "/home/tomcat/weibao/robot/%s.js"%(curDate)

def trans(info):
   keypair = info.split(",")
   res=""
   est=0
   tmc=1
   connc=0
   for item in keypair:
       key,val = item.split(":")
       if key == "cpu" or key == "disk":
           lent = len(val)
           val = val[0:lent-1]
           val = float(val)
       if key == "establishedCount":
           est = val
       elif key == "tomcatCount":
           if val > 0:
               tmc = val
       else:
           res += '"%s":"%s",' % (key,val)

   connc = int(est) / int(tmc)
   res += '"estcon":"%s"' % connc
   if res[-1] == ",":
       return res[0:len(res)-1]
   else:
       return res

def client(host):
   s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
   try:
      s.connect((host, port))
   except Exception, e:
      msg = traceback.format_exc()
      print '鏈接錯誤:', msg

   s.send("weibaost")
   buff = s.recv(4096)
   if not len(buff):
      print 'buff is empty !'
   else :
     # sys.stdout.write(buff)
      print 'currentDate:%s'%(curDate)
      print 'host:%s,%s'%(host,str(buff))

      curTime = str(time.strftime('%H:%M',time.localtime(time.time())))
      with open(filepath,"a") as f:
          f.write('{\"time\":\"%s\",\"host\":\"%s\",\"qa\":\"12\",%s},'%(curTime,host,trans(str(buff))))

def filetrans():
    with open(filepath,"r") as f:
        content = f.read()
        print content
        content = content[0:len(content)-1]
        print content
        content = "json=[" +content + "]"
        print content
    with open(filepathjs,"w")as f:
        f.write(content)

threads = []
t1 = threading.Thread(target=client,args=(host1,))
threads.append(t1)
t2 = threading.Thread(target=client,args=(host2,))
threads.append(t2)
if __name__ == "__main__":
   for t in threads:
   #    print 'thread start....'
       t.start()
       t.join()

   filetrans()
   handler = subprocess.Popen("/home/tomcat/rsyncwb.sh", shell=True, stdout=subprocess.PIPE)
   output = handler.stdout.readlines()
   print output
   with open("/home/tomcat/listenLog.log","a") as f:
       f.write(str(output))# client(host1)

上述代碼中rsyncwb.sh就是執行rsync指令，實現將最終處理好的數據推送到nginx的web服務器上的某個指定路徑下，方便客戶經過瀏覽器訪問

 

3. python的agent程序（TCP服務端）

#!/usr/bin/env python
# -*- coding:utf-8 -*-

# #執行客戶端發送過來的命令，並把執行結果返回給客戶端
import socket, traceback, subprocess

host = '0.0.0.0'
port = 23153

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

s.bind((host, port))
s.listen(10)

while 1:
    try:
        client_socket, client_addr = s.accept()
    except Exception, e:
        traceback.print_exc()
        continue

    try:
        print 'From host:', client_socket.getpeername()

        command = client_socket.recv(4096)
        if not len(command):
            break
        print client_socket.getpeername()[0] + ':' + str(command)
        scommand = str(command)
        if scommand == "weibaost":
            # 執行客戶端傳遞過來的命令
            handler = subprocess.Popen("/home/tomcat/weibaoSt.sh", shell=True, stdout=subprocess.PIPE)
            output = handler.stdout.readlines()
            if output is None:
                output = []
            for one_line in output:
                client_socket.sendall(one_line)
               # client_socket.sendall("\n")
        else:
            print "Not supported command!!!"

    except Exception, e:
        traceback.print_exc()
        client_socket.sendall("command error")

    try:
        client_socket.close()
    except Exception, e:
        traceback.print_exc()

上述代碼中的weibaoSt.sh，就是第一小節貼出的shell代碼內容。

 

最後，EChart展現相關的，請查考百度的官網，本身研究，不是這裏的重點。這裏，將關鍵的配置和展現部分的代碼貼出來：

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>機器人系統監控</title>
<link href="css/statistics.css" rel="stylesheet">

<!-- 引入 ECharts 文件 -->
<script src="js/echarts.js"></script>
<script src="js/jquery-2.1.1.min.js"></script>

</head>

<body>
    
    
    <div class="col-sm-9 col-sm-offset-3 col-lg-10 col-lg-offset-2 main main-plane title-box"  id="reportTab" style="margin-left:12%;width:100%">
        
        <div class="chart-container">
        <!--問題開始 -->
            <div style="text-align:center;;margin-left: 4%">
                <h2>問答數</h2>
                <div>
                    <!-- 爲ECharts準備一個具有大小（寬高）的Dom -->
                    <div id="r01" style="width:1200px;height:500px;"></div>
                </div>
            </div>
        <!--問題結束-->
        
        
      
        </div> 

         <div class="chart-container">
        <!--CPU開始 -->
            <div style="text-align:center;;margin-left: 4%">
                <h2>CPU</h2>
                <div>
                    <!-- 爲ECharts準備一個具有大小（寬高）的Dom -->
                    <div id="r02" style="width:1200px;height:500px;"></div>
                </div>
            </div>
        <!--CPU結束-->
        
        
      
        </div> 

         <div class="chart-container">
        <!--DISK開始 -->
            <div style="text-align:center;;margin-left: 4%">
                <h2>DISK</h2>
                <div>
                    <!-- 爲ECharts準備一個具有大小（寬高）的Dom -->
                    <div id="r03" style="width:1200px;height:500px;"></div>
                </div>
            </div>
        <!--DISK結束-->
        
        
      
        </div> 

         <div class="chart-container">
        <!--TOMCAT開始 -->
            <div style="text-align:center;;margin-left: 4%">
                <h2>TOMCAT</h2>
                <div>
                    <!-- 爲ECharts準備一個具有大小（寬高）的Dom -->
                    <div id="r04" style="width:1200px;height:500px;"></div>
                </div>
            </div>
        <!--TOMCAT結束-->
        
        </div> 
        
    </div>
    <script src="js/example1.js"></script>

</body>

</html>

View Code

$(function() {

    Date.prototype.Format = function (fmt) { //author: meizz 
    var o = {
        "M+": this.getMonth() + 1, //月份 
        "d+": this.getDate(), //日 
        "h+": this.getHours(), //小時 
        "m+": this.getMinutes(), //分 
        "s+": this.getSeconds(), //秒 
        "q+": Math.floor((this.getMonth() + 3) / 3), //季度 
        "S": this.getMilliseconds() //毫秒 
    };
    if (/(y+)/.test(fmt)) fmt = fmt.replace(RegExp.$1, (this.getFullYear() + "").substr(4 - RegExp.$1.length));
    for (var k in o)
    if (new RegExp("(" + k + ")").test(fmt)) fmt = fmt.replace(RegExp.$1, (RegExp.$1.length == 1) ? (o[k]) : (("00" + o[k]).substr(("" + o[k]).length)));
    return fmt;
    }


    var jsName = new Date().Format("yyyy-MM-dd")+".js";
    var jsUrl = "http://ecuat.taikang.com/whcms/weibao/robot/"+jsName;
    $.getScript(jsUrl,function(){
        loadData();
    });

    /********************************問題統計 start ********************************************/

    var R01Chart = echarts.init(document.getElementById('r01'));
    // 顯示標題，圖例和空的座標軸
    R01Chart.setOption({
         tooltip : {
        trigger: 'axis'
    },
    legend: {
        
    },
    toolbox: {
        show : true,
        feature : {
            mark : {show: true},
            dataView : {show: true, readOnly: false},
            magicType : {show: true, type: ['line', 'bar', 'stack', 'tiled']},
            restore : {show: true},
            saveAsImage : {show: true}
        }
    },
    calculable : true,
    xAxis : [
        {
            type : 'category',
            boundaryGap : false,
            data : []
        }
    ],
    yAxis : [
        {
            type : 'value'
        }
    ],
    series : [
        
    ]
    });
    //R01Chart.showLoading(); // 數據加載完以前先顯示一段簡單的loading動畫

    /********************************問題統計 end ********************************************/


    /********************************CPU start ********************************************/

    var R02Chart = echarts.init(document.getElementById('r02'));
    // 顯示標題，圖例和空的座標軸
    R02Chart.setOption({
         tooltip : {
        trigger: 'axis'
    },
    legend: {
        
    },
    toolbox: {
        show : true,
        feature : {
            mark : {show: true},
            dataView : {show: true, readOnly: false},
            magicType : {show: true, type: ['line', 'bar', 'stack', 'tiled']},
            restore : {show: true},
            saveAsImage : {show: true}
        }
    },
    calculable : true,
    xAxis : [
        {
            type : 'category',
            boundaryGap : false,
            data : []
        }
    ],
    yAxis : [
        {
            type : 'value'
        }
    ],
    series : [
        
    ]
    });


    /********************************CPU end ********************************************/


    /********************************DISK start ********************************************/


var R03Chart = echarts.init(document.getElementById('r03'));
    // 顯示標題，圖例和空的座標軸
    R03Chart.setOption({
         tooltip : {
        trigger: 'axis'
    },
    legend: {
        
    },
    toolbox: {
        show : true,
        feature : {
            mark : {show: true},
            dataView : {show: true, readOnly: false},
            magicType : {show: true, type: ['line', 'bar', 'stack', 'tiled']},
            restore : {show: true},
            saveAsImage : {show: true}
        }
    },
    calculable : true,
    xAxis : [
        {
            type : 'category',
            boundaryGap : false,
            data : []
        }
    ],
    yAxis : [
        {
            type : 'value'
        }
    ],
    series : [
        
    ]
    });


    /********************************DISK end ********************************************/


    /********************************TOMCAT start ********************************************/


var R04Chart = echarts.init(document.getElementById('r04'));
    // 顯示標題，圖例和空的座標軸
    R04Chart.setOption({
         tooltip : {
        trigger: 'axis'
    },
    legend: {
        
    },
    toolbox: {
        show : true,
        feature : {
            mark : {show: true},
            dataView : {show: true, readOnly: false},
            magicType : {show: true, type: ['line', 'bar', 'stack', 'tiled']},
            restore : {show: true},
            saveAsImage : {show: true}
        }
    },
    calculable : true,
    xAxis : [
        {
            type : 'category',
            boundaryGap : true,
            data : []
        }
    ],
    yAxis : [
        {
            type : 'value'
        }
    ],
    series : [
        
    ]
    });


    /********************************TOMCAT end ********************************************/

    function loadData() {

        var xAxisData = new Set();
        var IPDATA = new Object();
        var CPUDATA = new Object();
        var DISKDATA = new Object();
        var TOMCATDATA = new Object();

        var legendSet = new Set();
        $.each(json,function(i,n){
            var IP = n.host;
            var QA = n.qa;
            var CPU = n.cpu;
            var DISK = n.disk;
            var TOMCAT = n.estcon;
            var TIME = n.time;

        if(!IPDATA.hasOwnProperty(IP)){
            IPDATA[IP]=new Object();
            IPDATA[IP].name=IP;
            IPDATA[IP].type='line';
            //IPDATA[IP].stack= '總量';
            IPDATA[IP].data=new Array();
            IPDATA[IP].xb=0;

            CPUDATA[IP]=new Object();
            CPUDATA[IP].name=IP;
            CPUDATA[IP].type='line';
            //CPUDATA[IP].stack= '總量';
            CPUDATA[IP].data=new Array();
            CPUDATA[IP].xb=0;

            DISKDATA[IP]=new Object();
            DISKDATA[IP].name=IP;
            DISKDATA[IP].type='line';
            //DISKDATA[IP].stack= '總量';
            DISKDATA[IP].data=new Array();
            DISKDATA[IP].xb=0;

            TOMCATDATA[IP]=new Object();
            TOMCATDATA[IP].name=IP;
            TOMCATDATA[IP].type='line';
            //TOMCATDATA[IP].stack= '總量';
            TOMCATDATA[IP].data=new Array();
            TOMCATDATA[IP].xb=0;
        }

            var kk=IPDATA[IP].xb;
            IPDATA[IP].data[kk]=QA;
            IPDATA[IP].xb=kk+1;

            CPUDATA[IP].data[kk]=CPU;
            CPUDATA[IP].xb=kk+1;

            DISKDATA[IP].data[kk]=DISK;
            DISKDATA[IP].xb=kk+1;

            TOMCATDATA[IP].data[kk]=TOMCAT;
            TOMCATDATA[IP].xb=kk+1;

            xAxisData.add(TIME);
            legendSet.add(IP);
            



        });

            var legendArr = new Array();
            var ii = 0;
            legendSet.forEach(function (item) {
                legendArr[ii] = item;
                ii++;
            });

            var tt=0;
            var series=new Array();
            for(item in  IPDATA){
                series[tt]=IPDATA[item];
                tt++;
            }

            var tt2=0;
            var series2=new Array();
            for(item in  CPUDATA){
                series2[tt2]=CPUDATA[item];
                tt2++;
            }

            var tt3=0;
            var series3=new Array();
            for(item in  DISKDATA){
                series3[tt3]=DISKDATA[item];
                tt3++;
            }

            var tt4=0;
            var series4=new Array();
            for(item in  TOMCATDATA){
                series4[tt4]=TOMCATDATA[item];
                tt4++;
            }

            var xAxisDataArr = new Array();
            var iii = 0;
            xAxisData.forEach(function (item) {
                xAxisDataArr[iii] = item;
                iii++;
            });

    console.log(series);
    console.log(series2);
    console.log(series3);
    console.log(series4);
        
        /*******問題數據加載*****/
            R01Chart.setOption({
                legend : {
                    data : legendArr
                },
                xAxis : [
                {
                    data : xAxisDataArr
                }    
                ],
                series : series
            });

        /*******CPU數據加載*****/
            R02Chart.setOption({
                legend : {
                    data : legendArr
                },
                xAxis : [
                {
                    data : xAxisDataArr
                }    
                ],
                series : series2
            });

        /*******DISK數據加載*****/
            R03Chart.setOption({
                legend : {
                    data : legendArr
                },
                xAxis : [
                {
                    data : xAxisDataArr
                }    
                ],
                series : series3
            });

        /*******TOMCAT數據加載*****/
            R04Chart.setOption({
                legend : {
                    data : legendArr
                },
                xAxis : [
                {
                    data : xAxisDataArr
                }    
                ],
                series : series4
            });

    }
});

View Code

另外，附帶一下相關的測試數據，以js爲後綴名，時間爲文件名的數據文件：

json=[{"time":"08:58","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"08:58","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"08:59","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"08:59","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:00","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:00","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:01","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:01","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:02","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:02","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:03","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:03","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:04","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:04","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:05","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:05","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:06","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:06","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:07","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:07","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:08","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:08","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:09","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:09","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:10","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:10","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:11","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:11","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:12","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:12","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:13","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:13","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:14","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:14","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:15","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:15","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:16","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:16","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:17","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:17","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:18","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:18","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:19","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:19","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:20","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:20","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:21","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:21","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:22","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:22","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:23","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"30"},{"time":"09:23","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"19"},{"time":"09:24","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"31"},{"time":"09:24","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"20"},{"time":"09:25","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"31"},{"time":"09:25","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"20"},{"time":"09:26","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:26","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"21"},{"time":"09:27","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:27","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"21"},{"time":"09:28","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:28","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"26"},{"time":"09:28","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:28","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"21"},{"time":"09:29","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:29","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"22"},{"time":"09:30","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:30","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"22"},{"time":"09:31","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:31","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"22"},{"time":"09:32","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:32","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"22"},{"time":"09:33","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:33","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"22"},{"time":"09:34","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:34","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"22"},{"time":"09:35","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:35","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"22"},{"time":"09:36","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:36","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"21"},{"time":"09:37","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:37","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"21"},{"time":"09:38","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:38","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"21"},{"time":"09:39","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"09:39","host":"10.130.207.211","qa":"12","cpu":"3.7","disk":"31.0","estcon":"21"},{"time":"09:40","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:40","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:41","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:41","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:42","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:42","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:43","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:43","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:44","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:44","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:45","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:45","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:46","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:46","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:47","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:47","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:48","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:48","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:49","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:49","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"09:50","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:50","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"23"},{"time":"09:51","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:51","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"23"},{"time":"09:52","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:52","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"23"},{"time":"09:53","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:53","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"23"},{"time":"09:54","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:54","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"23"},{"time":"09:55","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:55","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"23"},{"time":"09:56","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:56","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"09:57","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:57","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"09:58","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:58","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"09:59","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"09:59","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:00","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"34"},{"time":"10:00","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:01","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"34"},{"time":"10:01","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:02","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"34"},{"time":"10:02","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:03","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"34"},{"time":"10:03","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:04","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"34"},{"time":"10:04","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:05","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"34"},{"time":"10:05","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:06","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:06","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"10:07","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"10:07","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:08","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"10:08","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:09","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"10:09","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:10","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"10:10","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:11","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"10:11","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:12","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:12","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:13","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:13","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:14","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:14","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:15","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:15","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:16","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:16","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:17","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:17","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:18","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:18","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:19","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:19","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"10:20","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:20","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"21"},{"time":"10:21","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:21","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"22"},{"time":"10:22","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:22","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"23"},{"time":"10:23","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:23","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:24","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:24","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:25","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:25","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:26","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:26","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"26"},{"time":"10:27","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:27","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"26"},{"time":"10:28","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:28","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"26"},{"time":"10:29","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:29","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:30","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:30","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:31","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"5"},{"time":"10:31","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:32","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"31"},{"time":"10:32","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:33","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"34"},{"time":"10:33","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:34","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"35"},{"time":"10:34","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:35","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"35"},{"time":"10:35","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:36","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"36"},{"time":"10:36","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:37","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"35"},{"time":"10:37","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:38","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"35"},{"time":"10:38","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:39","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"33"},{"time":"10:39","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"25"},{"time":"10:40","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:40","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:41","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:41","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:42","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:42","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:43","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"31"},{"time":"10:43","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:44","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:44","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:45","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:45","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:46","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:46","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:47","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:47","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"},{"time":"10:48","host":"10.130.207.210","qa":"12","cpu":"3.8","disk":"45.0","estcon":"32"},{"time":"10:48","host":"10.130.207.211","qa":"12","cpu":"3.8","disk":"31.0","estcon":"24"}]

View Code

 

上面的測試案例中，有個指標，問答數，是hardcode寫的，由於腳本尚未完成，這個博文只是說明，這個簡單有效的監控方案，是應急的時候的行之有效的方案。下面給出一個tomcat鏈接數的監控截圖：

 

監控是互聯網系統的很是重要的內容，須要很是的重視，即使你再牛逼，也不能輕視這個方面的投入，最後的防線預警機！