使用nGrinder執行socket.io應用負載測試

原文 ：  Using nGrinder to perform load test for a socket.io app   by  Mavlarn 

    nGrinder不只能夠用來測試一般的Web應用程序，也能夠用於JDBC，Web服務或者像socket.io所提供的這樣的實時應用。
    socket.io旨在幫助咱們在各類瀏覽器與移動設備上實現實時app功能。如今，基於瀏覽器和移動設備的應用愈來愈多，而咱們也能夠使用nGrinder對這些應用進行性能測試。由於咱們能夠使用擴展的java包來擴展測試腳本，因此，咱們藉助socket.io的java客戶端，就能夠實現對於基於socket.io的應用的性能測試。關於socket.io的java客戶端，咱們使用socket.io-java-client，有關這個包的使用，請參考github上的說明，其代碼中也有實例。

    可是，這個java客戶端使用了異步方式來發送請求和接收響應。而咱們在進行測試的時候，須要記錄每個請求的處理時間，因此咱們須要將對SocketIO類作一些修改，來實現同步的目的。

    其主要思想是，使用SocketIO對象建立一個與應用服務器的鏈接，藉助java的同步機制，在發送請求以後，就等待返回值。在這個例子中，我使用了Java Lock和Condition來達到這一目標。咱們建立一個SocketIO類的子類BlockingSocketIO，添加一個發送消息並等待返回結果的方法。下面是BlockingSocketIO類的源代碼：    

package my;
 
import io.socket.IOAcknowledge;
import io.socket.IOCallback;
import io.socket.SocketIO;
import io.socket.SocketIOException;
 
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;
 
import org.json.JSONObject;
 
/**
 * Class description.
 *
 * @author Mavlarn
 * @since
 */
public class BlockingSocketIO implements IOCallback {
     
    private SocketIO socketIO;
    private ReentrantLock transportLock;
    private Condition responseCondition;
    private String respMsg;
     
    public BlockingSocketIO (String url) {
        try {
            transportLock = new ReentrantLock();
            responseCondition = transportLock.newCondition();
            socketIO = new SocketIO(url, this);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
     
    public String sendAndRcv (final String message) {
        try {
            transportLock.lock();
            socketIO.send(message);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }
     
    public String sendAndRcv(final JSONObject json) {
        try {
            transportLock.lock();
            socketIO.send(json);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }
 
    public String emitAndRcv(String event, final Object args) {
        try {
            transportLock.lock();
            socketIO.emit(event, args);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }
 
    @Override
    public void onMessage(JSONObject json, IOAcknowledge ack) {
        setResponse(json.toString());
    }
 
    @Override
    public void onMessage(String data, IOAcknowledge ack) {
        setResponse(data);
    }
 
    private void setResponse(String data) {
        try {
            transportLock.lock();
            respMsg = data;
            responseCondition.signal();
            System.out.println("Server said:" + data);
        } finally {
            transportLock.unlock();
        }
    }
 
    @Override
    public void onError(SocketIOException socketIOException) {
        System.out.println("an Error occured");
        socketIOException.printStackTrace();
    }
 
    @Override
    public void onDisconnect() {
        System.out.println("Connection terminated.");
    }
 
    @Override
    public void onConnect() {
        System.out.println("Connection established");
    }
 
    @Override
    public void on(String event, IOAcknowledge ack, Object... args) {
        System.out.println("Server triggered event '" + event + "'");
        setResponse(args<a href="/wiki_ngrinder/entry/0" class="notexist">0</a>.toString());
    }
 
}

    咱們須要把這個類打成jar包並上傳到nGrinder的lib文件夾中。同時，也要上傳socketio.jar及它所依賴的庫WebSocket.jar和Json-org.jar。

    接下來，咱們須要在nGrinder中執行測試場景的Python腳本。以下所示：

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
 
from org.json import JSONObject
from my import BlockingSocketIO
 
test1 = Test(1, "Test1")
 
class TestRunner:
 
    def testSocketIO(self):
        json = JSONObject()
        user = "Thread-%s" % grinder.threadNumber
        json.putOpt("user", user)
        msg = "test message<%s>." % user
        json.putOpt("message", msg)
        grinder.logger.info("msg:" + json.toString())
        respMsg = self.socketIO.emitAndRcv("user message", json)
        return respMsg
     
    def __init__(self):
        grinder.statistics.delayReports=True
        #init socket io
        #create socket io object in thread init function. Then every thread will use its own socket.io connection.
        self.socketIO = BlockingSocketIO("http://127.0.0.1:3000")
         
        #send socket.io server to init user
        json = JSONObject()
        user = "Thread-%s" % grinder.threadNumber
        json.putOpt("username", user)
        self.socketIO.emitAndRcv("user", json)
 
    # test method       
    def __call__(self):
        resp = self.testSocketIO()
 
        if "test message" in resp :
            grinder.statistics.forLastTest.success = 1
        else :
            grinder.statistics.forLastTest.success = 0
 
test1.record(TestRunner.testSocketIO)

    在這個腳本中，在測試對象TestRunner的init函數中，咱們建立了一個socket.io鏈接對象，而後這個線程的全部測試將使用相同的鏈接。這對基於socket.io的長鏈接池的應用是很是重要的。由於基於socket.io應用中，每一個用戶跟服務器之間是一直保持鏈接的。
    而後，在這個init函數中，一個包含「user」事件的消息被髮送到服務器，至關於在服務器端作用戶登錄之類的初始化。 再而後，在每個測試函數中，咱們都會發送一個包含「「user message」」事件的消息。
    接下來，咱們須要一個支持這個客戶端腳本的服務器端的應用程序。服務器端使用node.js，安裝socket.io模塊。
    而後再寫一個名爲server.js腳本，內容以下：

var http = require('http'), io = require('socket.io');
 
var app = http.createServer();
app.listen(3000);
 
console.log('Server running at http://127.0.0.1:3000/');
 
// Socket.IO server
var io = io.listen(app);
 
io.sockets.on('connection', function (socket) {
  console.log("new connection from" + socket); get and log connection
  socket.on('user message', function (msg) {  //accept a request with 「user message」 event
    socket.emit('user message processed', {user: msg.user, message: msg.message});
  });
 
  socket.on('user', function (userMsg) { //accept a request with 「user」 event, like user login.
    socket.user = userMsg.username;
    socket.emit('user processed', {user: userMsg.user, message: "New user come in."});
  });
 
  socket.on('disconnect', function () {
    if (!socket.user) return;
    socket.emit('announcement', {user: socket.user, action: 'disconected'});
  });
});

    用下面的語句運行這個模擬服務器：

node server.js

    你應該可以看到一條日誌說這個服務器正運行在http://127.0.0.1:3000/。
    而後，在nGrinder的腳本編輯頁面中驗證這個腳本以確認它可以正常運行。驗證結果應該是這樣的：

2013-03-11 13:13:08,844 INFO  elapsed time is 17 ms
2013-03-11 13:13:08,844 INFO  Final statistics for this process:
2013-03-11 13:13:08,854 INFO 
             Tests        Errors       Mean Test    Test Time    TPS         
                                       Time (ms)    Standard                 
                                                    Deviation                
                                                    (ms)                     
 
Test 1       1            0            3.00         0.00         58.82         "Test1"
 
Totals       1            0            3.00         0.00         58.82       
 
  Tests resulting in error only contribute to the Errors column.         
  Statistics for individual tests can be found in the data file, including
  (possibly incomplete) statistics for erroneous tests. Composite tests  
  are marked with () and not included in the totals.                     
 
 
……
2013-03-11 13:13:08,750 INFO  validation-0: starting threads
Mar 11, 2013 1:13:08 PM io.socket.IOConnection sendPlain
INFO: > 5:::{"args":<a href="/wiki_ngrinder/entry/usernamethread-0" class="notexist">{"username":"Thread-0"}</a>,"name":"user"}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 1::
Connection established
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 5:::{"name":"user processed","args":<a href="/wiki_ngrinder/entry/messagenew-user-come-in" class="notexist">{"message":"New user come in."}</a>}
Server triggered event 'user processed'
Server said:{"message":"New user come in."}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection sendPlain
INFO: > 5:::{"args":<a href="/wiki_ngrinder/entry/messagetest-messagethread-0-userthread-0" class="notexist">{"message":"test message<Thread-0>.","user":"Thread-0"}</a>,"name":"user message"}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 5:::{"name":"user message processed","args":<a href="/wiki_ngrinder/entry/userthread-0messagetest-messagethread-0" class="notexist">{"user":"Thread-0","message":"test message<Thread-0>."}</a>}
Server triggered event 'user message processed'
Server said:{"message":"test message<Thread-0>.","user":"Thread-0"}
2013-03-11 13:13:08,855 INFO  validation-0: finished

    從結果的信息咱們能夠看出這個測試是成功的，並且服務器處理了2個請求，一個是「user」，另外一個是「user message」。而在用戶的名字方面，我使用「線程－<線程號>」。 若是咱們想要使用多個Vuser（虛擬用戶）來進行測試，就要使用不一樣的名字。
    服務器端日誌應與下面的內容相似：

debug - client authorized
info - handshake authorized gr0AYzAn7sAKTE_XsORt
debug - setting request GET /socket.io/1/websocket/gr0AYzAn7sAKTE_XsORt
debug - set heartbeat interval for client gr0AYzAn7sAKTE_XsORt
debug - client authorized for
debug - websocket writing 1::
new connection from<a href="/wiki_ngrinder/entry/object-object" class="notexist">object Object</a>
debug - websocket writing 5:::{"name":"user processed","args":<a href="/wiki_ngrinder/entry/messagenew-user-come-in" class="notexist">{"message":"New user come in."}</a>}
debug - websocket writing 5:::{"name":"user message processed","args":<a href="/wiki_ngrinder/entry/userthread-0messagetest-messagethread-0" class="notexist">{"user":"Thread-0","message":"test message<Thread-0>."}</a>}
info - transport end (socket end)
debug - set close timeout for client gr0AYzAn7sAKTE_XsORt
debug - cleared close timeout for client gr0AYzAn7sAKTE_XsORt
debug - cleared heartbeat interval for client gr0AYzAn7sAKTE_XsORt
debug - discarding transport

    這些服務器日誌說的是，它收到了一個客戶端鏈接並握手成功，而後處理了2個請求。最後，客戶端斷開。 接下來，咱們能夠用這個測試腳本建立一個nGrinder測試。

    下面是最終的報告：

最後，別忘了檢查服務器端日誌：

......
info  - transport end (socket end)
debug - set close timeout for client JFrRHYoO3__jN4pdsOSi
debug - cleared close timeout for client JFrRHYoO3__jN4pdsOSi
debug - cleared heartbeat interval for client JFrRHYoO3__jN4pdsOSi
debug - discarding transport
info  - transport end (socket end)
debug - set close timeout for client pDHSiLJhTXaqVR5osOSk
debug - cleared close timeout for client pDHSiLJhTXaqVR5osOSk
debug - cleared heartbeat interval for client pDHSiLJhTXaqVR5osOSk
debug - discarding transport
info  - transport end (socket end)
debug - set close timeout for client 7u_rypQFSZ2vcTGKsOSj
debug - cleared close timeout for client 7u_rypQFSZ2vcTGKsOSj
debug - cleared heartbeat interval for client 7u_rypQFSZ2vcTGKsOSj
debug - discarding transport
info  - transport end (socket end)
debug - set close timeout for client fmxnHFQ_U-wsCmdMsOSg
debug - cleared close timeout for client fmxnHFQ_U-wsCmdMsOSg
debug - cleared heartbeat interval for client fmxnHFQ_U-wsCmdMsOSg
debug - discarding transport

    在服務器的日誌中，應該會有一些鏈接被丟棄（discarded）的日誌。在這個測試中，Vuser是10，因此在日誌中應該出現10次「discarding transport」。這意味着，一個是Vuser模擬一個建立了一個鏈接的真實用戶。若是你不想用一個線程對應一個鏈接，你能夠將下面的代碼移動到TestRunner以前：

socketIO = BlockingSocketIO("http://127.0.0.1:3000")

那麼在這個處理過程當中的全部線程將使用同一鏈接。

    順便說一下，node服務器是行在我我的的筆記本上。從TPS和平均時間，咱們能夠看到socket.io服務器的性能是很是不錯的。請在附件中查看這個test所使用的須要上傳到lib目錄的Java包。    

   附件1： report.png   附件2： socket.io-test-libs.zip