使用Gearman+Calabash並行測試手機APP

摘要：使用任務分發系統Gearman分佈式執行Calabash的自動化測試用例，能夠達到並行測試手機APP的目的。

背景介紹

Gearman

Gearman是一個分發任務的程序框架，能夠用在各類場合，與Hadoop相比，Gearman更偏向於任務分發功能。它的 任務分佈很是簡單，簡單得能夠只須要用腳本便可完成。

Calabash

Calabash-android是支持android的UI自動化測試框架，PC端使用了cucumber框架，經過http和json與模擬器和真機上安裝的測試apk通訊，測試apk調用robotium的方法來進行UI自動化測試，支持webview操做。

calabash腳本以 使用calabash測試開源中國Android客戶端 爲例。

Gearman安裝和執行測試

Ubuntu上安裝Gearman

$ sudo apt-get install gearman-job-server
$ gearmand -V

gearmand 1.0.6 - https://bugs.launchpad.net/gearmand 

$ sudo apt-get install gearman-tools
$ gearman -H

運行Gearman併發執行Calabash測試用例

啓動gearman job server，做爲後臺服務運行:

$ gearmand -d

啓動兩個worker，每一個woker表明一個手機測試設備(tester)，woker始終不退出，且沒有任何打印:

$ gearman -w -f tester -- ./test-case.sh NEXUS-7 testdata-aa.bashrc
$ gearman -w -f tester -- ./test-case.sh HUAHEI-8860 testdata-bb.bashrc

分發測試用例到各個worker，收集到全部測試打印後，即完成測試：

./run-suite.sh @BVT

實時查看測試結果，任何一個worker測試完一個用例，測試結果會當即打印出來：

tail -f /tmp/test-result

併發測試的效果： 理論上若是順序執行測試全部用例要10個小時，那麼只要準備10個手機，每一個手機對應啓動一個worker:tester，測試時間將縮短到最少1個小時。

下面的腳本是在同一臺Ubuntu電腦上，利用多個USB口鏈接多個手機來併發測試， 若是要在多臺電腦上執行併發測試，那要考慮這多臺電腦如何獲取同一份apk和calabash腳本，能夠考慮從一個公共的url去wget。

Gearman測試腳本

test-case.sh

從stdin接收到的calabash測試腳本文件路徑，調用calabash-android完成測試，再做爲client調用printer

#!/bin/bash
if [ x$2 == x ]
then
  echo "usage: $0 device-id testdata-xx.bashrc"
  echo "      device-id get from: adb devices"
  exit 1
fi
read line
echo $0 $1 $2
uuid=`uuidgen`
d1=`date +%T`
suite_path=$HOME/git/oschina/android-app/calabash
apk_path=$HOME/git/oschina/android-app/bin/oschina-android-app.apk
echo "test result in /tmp/${uuid}"
echo $d1 at `hostname`:$1 >> /tmp/${uuid}
mkdir -p $HOME/$1
cd $HOME/$1
export ADB_DEVICE_ARG=$1
. $src/$2
calabash-android run $apk_path -r $suite_path/features/ $line 2>&1 | tee -a /tmp/${uuid}
d2=`date +%T`
echo $d2 at `hostname`:$1 >> /tmp/${uuid}
failed=`grep "Failing Scenarios" /tmp/${uuid} | wc -l`
if [ $failed == 0 ]
then
  echo "===PASS=== : $line" >> /tmp/${uuid}
else
  echo "===FAIL=== : $line" >> /tmp/${uuid}
fi
gearman -f printer < /tmp/${uuid}
rm -f /tmp/${uuid}

print-result.sh

從stdin接收測試結果，並添加到文件/tmp/test-result

#!/bin/bash
while read line 
do
  echo "$line" >> /tmp/test-result
done
case_count=`cat /tmp/case_count`
finished=`egrep "===(PASS|FAIL)===" /tmp/test-result | wc -l`
echo -n "Progress:" $finished "/ $case_count  [" >> /tmp/test-result

for ((i=0; i<$finished; i++ ))
do
  echo -n ">" >> /tmp/test-result
done
let unfinished=$case_count-$finished
for ((i=0; i<$unfinished; i++ ))
do
  echo -n "." >> /tmp/test-result
done
echo -en "]\n" >> /tmp/test-result

run-suite.sh

做爲client，後臺一次分發全部calabash腳本到tester，並啓動woker: printer，直到收到全部測試結果打印才退出

#!/bin/bash
if [ x$1 == x ]
then
  echo "usage:" $0 "@BVT|@nightly|all|failed"
  exit 1
fi

suite_path=$HOME/git/oschina/android-app/calabash
uuid=`uuidgen`

if [ "$1" == "all" ]
then
  find $suite_path -name "*.feature" > /tmp/${uuid}
elif [ "$1" == "failed" ]
then
  if [ -f /tmp/failed ]
  then
    cat /tmp/failed > /tmp/${uuid}
  else
    touch /tmp/${uuid}
  fi
elif [[ "$1" == @* ]]
then
  grep $1 $suite_path -rl | grep ".feature$" > /tmp/${uuid}
fi

case_count=`cat /tmp/${uuid} | wc -l`
echo $case_count > /tmp/case_count
if [ $case_count == 0 ]
then
  echo "NO case to run, exit." | tee /tmp/test-result
  rm -f /tmp/${uuid}
  exit 1
fi

echo "total $case_count cases." | tee /tmp/test-result

i=0
while read line 
do
  let i+=1
  echo "send No.$i case --- $line"
  echo $line | gearman -b -f runcase
done < /tmp/${uuid}
rm -f /tmp/${uuid}

gearman -w -c $case_count -f printer -- ./print-result.sh $case_count
cat /tmp/test-result
grep "===FAIL===" /tmp/test-result | awk '{print $3}' > /tmp/failed
echo "" > /tmp/summary
echo "=================test result summary================" 2>&1 | tee -a /tmp/summary
grep "===PASS===" /tmp/test-result 2>&1 | tee -a /tmp/summary
grep "===FAIL===" /tmp/test-result 2>&1 | tee -a /tmp/summary
fails=`grep "===FAIL===" /tmp/test-result | wc -l`
if [ $fails == 0 ] 
then
  echo -e "\033[32;1;7mPASS\033[0m :" `grep "===PASS===" /tmp/test-result | wc -l` 2>&1 | tee -a /tmp/summary
else
  echo -e "\033[31;1;7mFAIL\033[0m :" `grep "===FAIL===" /tmp/test-result | wc -l` 2>&1 | tee -a /tmp/summary
fi
echo "TOTAL: $case_count" 2>&1 | tee -a /tmp/summary
cat /tmp/summary >> /tmp/test-result