性能測試-Hbase Hadoop監控

之前作過一個性能測試項目，基於nginx+Hbase+Hadoop，第一次接觸開源的東西，遇到各類問題，印象深的是Hbase和Hadoop的監控，當時也搜索到能夠用開源的監控工具或寫代碼經過JMX取JVM的信息。在摸索的過程當中發現一種更簡單方便的辦法，不用監控工具和寫大量代碼，直接用loadrunner腳本（該辦法可能不專業，可是能用，可讓項目儘快進行，僅供參考）。

咱們知道Hbase開放了60010和60030端口，Hadoop開放的50070端口，以web的方式查看master、region和hadoop的性能情況（好比查看master只須要在瀏覽器輸入http://IP:60010/），能夠查看進程的heap、request、queue使用狀況，這些指標正是性能測試須要關注的。而這些信息是經過JMX傳遞，在瀏覽器中輸入http://IP:port/jmx，會以json的格式返回此刻進程JVM的狀況，效果如截圖。

master信息

json信息

看到這些信息以後，個人第一反應就是這些返回的信息可用loadrunner的web_reg_save_param函數保存起來，而後寫到txt裏面。

因此決定Hbase和Hadoop的資源監控流程：

一、loadrunner定時獲取json

二、講關鍵指標保存到txt

三、用excel生成趨勢圖

loadrunner腳本：

Action()
{
int filename;
int i;
int len;
char * item;

//hbase region 
lr_think_time(10);
lr_start_transaction("heap");
    web_reg_save_param_ex(
        "ParamName=regionHeap",
        "LB= : ",
        "RB=\n    },",
        SEARCH_FILTERS,
        "Scope=Body",
        "IgnoreRedirections=Yes",
        "RequestUrl=*/jmx*",
        LAST);

    web_url("jmx",
        "URL=http://IP:60030/jmx",
        "Resource=1",
        "RecContentType=application/json",
        "Referer=",
        "Snapshot=t1.inf",
        LAST);
    lr_error_message("region is %s", lr_eval_string("{regionHeap}"));
//hadoop namenode
    web_reg_save_param_ex(
        "ParamName=nameNodeHeap",
        "LB= : ",
        "RB=\n    },",
        SEARCH_FILTERS,
        "Scope=Body",
        "IgnoreRedirections=Yes",
        "RequestUrl=*/jmx*",
        LAST);

    web_url("jmx",
        "URL=http://IP:50070/jmx",
        "Resource=1",
        "RecContentType=application/json",
        "Referer=",
        "Snapshot=t1.inf",
        LAST);
    lr_error_message("nameNodeHeap is %s", lr_eval_string("{nameNodeHeap}"));
//hbase master
    web_reg_save_param_ex(
        "ParamName=masterHeap",
        "LB= : ",
        "RB=\n    },",
        SEARCH_FILTERS,
        "Scope=Body",
        "IgnoreRedirections=Yes",
        "RequestUrl=*/jmx*",
        LAST);
    web_url("jmx",
        "URL=http://IP:60010/jmx",
        "Resource=1",
        "RecContentType=application/json",
        "Referer=",
        "Snapshot=t1.inf",
        LAST);
    lr_error_message("master is %s", lr_eval_string("{regionHeap}"));
lr_end_transaction("heap",LR_AUTO);

//write text
filename=fopen("E:\\cowen\\B2C\\monitor\\master.txt","a");//w-write,r-read,a-append
fprintf(filename,"%s",lr_eval_string("{masterHeap}"));
    fprintf(filename,"%s","\n");
    fclose(filename);

filename=fopen("E:\\cowen\\B2C\\monitor\\region.txt","a");//w-write,r-read,a-append
fprintf(filename,"%s",lr_eval_string("{regionHeap}"));
    fprintf(filename,"%s","\n");
    fclose(filename);

filename=fopen("E:\\cowen\\B2C\\monitor\\namenode.txt","a");//w-write,r-read,a-append
fprintf(filename,"%s",lr_eval_string("{nameNodeHeap}"));
    fprintf(filename,"%s","\n");
    fclose(filename);
return 0;
}

效果圖