Magellan: A Simple xUnit Test Framework in Modern C++11

Magellan：開啓新的征程

靈感

Magellan是一個簡單的、可擴展的、使用C\+\+11實現的xUnit測試框架。Magellan設計靈感來自於Java社區著名的測試框架JUnit。

安裝

GitHub

地址：https://github.com/horance/magellan
做者：劉光聰
Email：horance@outlook.com

編譯環境

支持的平臺:

• [MAC OS X] supported

• [Linux] supported

• [Windows] not supported

支持的編譯器:

• [CLANG] 3.4 or later.

• [GCC] 4.8 or later.

• [MSVC] not supported.

安裝CMake

CMake的下載地址：http://www.cmake.org。

安裝Magellan

$ git clone https://gitlab.com/horance/magellan.git
$ cd magellan
$ git submodule init
$ git submodule update
$ mkdir build
$ cd build
$ cmake ..
$ make
$ sudo make install

測試Magellan

$ cd magellan/build
$ cmake -DENABLE_TEST=on ..
$ make
$ test/magellan-test
$ lib/l0-infra/l0-infra-test
$ lib/hamcrest/hamcrest-test

使用Rake

使用Rake可簡化Magelan的構建和測試過程，而且使得Magellan自我測試變成可能。

$ rake           # build, install, and test using clang
$ rake clang     # build, install, and test using clang
$ rake gcc       # build, install, and test using gcc
$ rake clean     # remove temp directory, and uninstall magellan
$ rake uninstall # uninstall magellan only

破冰之旅

物理目錄

quantity
├── include
│   └── quantity
├── src
│   └── quantity
└── test
│   ├── main.cpp
└── CMakeLists.txt

main函數

#include "magellan/magellan.hpp"


int main(int argc, char** argv)
{
    return magellan::run_all_tests(argc, argv);
}

CMakeList腳本

project(quantity)

cmake_minimum_required(VERSION 2.8)

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x")

include_directories(${CMAKE_CURRENT_SOURCE_DIR}/include)

file(GLOB_RECURSE all_files
src/*.cpp
src/*.cc
src/*.c
test/*.cpp
test/*.cc
test/*.c)

add_executable(quantity-test ${all_files})

target_link_libraries(quantity-test magellan hamcrest l0-infra)

構建

$ mkdir build
$ cd build
$ cmake ..
$ make

運行

$ ./quantity-test

[==========] Running 0 test cases.
[----------] 0 tests from All Tests
[----------] 0 tests from All Tests

[==========] 0 test cases ran.
[  TOTAL   ] PASS: 0  FAILURE: 0  ERROR: 0  TIME: 0 us

體驗Magellan

第一個用例

#include <magellan/magellan.hpp>

#include "quantity/Length.h"

USING_HAMCREST_NS

FIXTURE(LengthTest)
{
    TEST("1 FEET should equal to 12 INCH")
    {
        ASSERT_THAT(Length(1, FEET), eq(Length(12, INCH)));
    }
};

使用 Magellan,只須要包含 magellan.hpp 一個頭文件便可。Magellan 使用 Hamcrest 的斷言機制,
使得斷言更加統1、天然,且具備良好的擴展性;使用 USING_HAMCREST_NS,從而可使用 eq 代
替 hamcrest::eq,簡短明確;除非出現名字衝突,不然推薦使用簡寫的 eq。

Length實現

// quantity/Length.h
#include "quantity/Amount.h"

enum LengthUnit
{
    INCH = 1,
    FEET = 12 * INCH,
};

struct Length
{
    Length(Amount amount, LengthUnit unit);

    bool operator==(const Length& rhs) const;
    bool operator!=(const Length& rhs) const;

private:
    const Amount amountInBaseUnit;
};

// quantity/Length.cpp
#include "quantity/Length.h"

Length::Length(Amount amount, LengthUnit unit)
  : amountInBaseUnit(unit * amount)
{
}

bool Length::operator==(const Length& rhs) const
{
    return amountInBaseUnit == rhs.amountInBaseUnit;
}

bool Length::operator!=(const Length& rhs) const
{
    return !(*this == rhs);
}

構建

$ mkdir build
$ cd build
$ cmake ..
$ make

運行

$ ./quantity-test

[==========] Running 1 test cases.
[----------] 1 tests from All Tests
[----------] 1 tests from LengthTest
[ RUN      ] LengthTest::1 FEET should equal to 12 INCH
[       OK ] LengthTest::1 FEET should equal to 12 INCH(13 us)
[----------] 1 tests from LengthTest

[----------] 1 tests from All Tests

[==========] 1 test cases ran.
[  TOTAL   ] PASS: 1  FAILURE: 0  ERROR: 0  TIME: 13 us

Fixture

FIXTURE的參數能夠是任意的C\+\+標識符。通常而言，將其命名爲CUT(Class Under Test)的名字便可。根據做用域的大小，Fixture可分爲三個類別：獨立的Fixture，共享的Fixture，全局的Fixture。

支持BDD風格

xUnit	BDD
FIXTURE	CONTEXT
SETUP	BEFORE
TEARDOWN	AFTER
ASSERT_THAT	EXPECT

獨立的Fixture

#include <magellan/magellan.hpp>

FIXTURE(LengthTest)
{
    Length length;

    SETUP()
    {}

    TEARDOWN()
    {}

    TEST("length test1")
    {}

    TEST("length test2")
    {}
};

執行序列爲：

1. Length 構造函數

2. SETUP

3. TEST("length test1")

4. TEARDOWN

5. Length 析構函數

6. Length 構造函數

7. SETUP

8. TEST("length test2")

9. TEARDOWN

10. Length 析構函數

共享的Fixture

#include <magellan/magellan.hpp>

FIXTURE(LengthTest)
{
    Length length;

    BEFORE_CLASS()
    {}

    AFTER_CLASS()
    {}

    BEFORE()
    {}

    AFTER()
    {}

    TEST("length test1")
    {}

    TEST("length test2")
    {}
};

執行序列爲：

1. BEFORE_CLASS

2. Length 構造函數

3. BEFORE

4. TEST("length test1")

5. AFTER

6. Length 析構函數

7. Length 構造函數

8. BEFORE

9. TEST("length test2")

10. AFTER

11. Length 析構函數

12. AFTER_CLASS

全局的Fixture

有時候須要在全部用例啓動以前完成一次性的全局性的配置，在全部用例運行完成以後完成一次性的清理工做。Magellan則使用BEFORE_ALL和AFTER_ALL兩個關鍵字來支持這樣的特性。

#include <magellan/magellan.hpp>

BEFORE_ALL("before all 1")
{
}

BEFORE_ALL("before all 2")
{
}

AFTER_ALL("after all 1")
{
}

AFTER_ALL("after all 2")
{
}

BEFORE_ALL和AFTER_ALL向系統註冊Hook便可，Magellan便能自動地發現它們，並執行它們。猶如C\+\+不能保證各源文件中全局變量初始化的順序同樣，避免在源文件之間的BEFORE_ALL和AFTER_ALL設計不合理的依賴關係。

#include <magellan/magellan.hpp>

FIXTURE(LengthTest)
{
    Length length;

    BEFORE_CLASS()
    {}

    AFTER_CLASS()
    {}

    BEFORE()
    {}

    AFTER()
    {}

    TEST("length test1")
    {}

    TEST("length test2")
    {}
};

#include <magellan/magellan.hpp>

FIXTURE(VolumeTest)
{
    Volume volume;

    BEFORE_CLASS()
    {}

    AFTER_CLASS()
    {}

    BEFORE()
    {}

    AFTER()
    {}

    TEST("volume test1")
    {}

    TEST("volume test1")
    {}
};

Magellan可能的一個執行序列爲：

1. BEFORE_ALL("before all 1")

2. BEFORE_ALL("before all 2")

3. LengthTest::BEFORE_CLASS

4. Length構造函數

5. LengthTest::BEFORE

6. TEST("length test1")

7. LengthTest::AFTER

8. Length析構函數

9. Length構造函數

10. LengthTest::BEFORE

11. TEST("length test2")

12. LengthTest::AFTER

13. Length析構函數

14. LengthTest::AFTER_CLASS

15. VolumeTest::BEFORE_CLASS

16. Volume構造函數

17. LengthTest::BEFORE

18. TEST("volume test1")

19. LengthTest::AFTER

20. Volume析構函數

21. Volume構造函數

22. LengthTest::BEFORE

23. TEST("volume test2")

24. LengthTest::AFTER

25. Volume析構函數

26. VolumeTest::AFTER_CLASS

27. AFTER_ALL("after all 2")

28. AFTER_ALL("after all 1")

用例設計

自動標識

Magellan可以自動地實現測試用例的標識功能，用戶可使用字符串來解釋說明測試用例的意圖，使得用戶在描述用例時更加天然和方便。

#include <magellan/magellan.hpp>
#include "quantity/length/Length.h"

USING_HAMCREST_NS

FIXTURE(LengthTest)
{
    TEST("1 FEET should equal to 12 INCH")
    {
        ASSERT_THAT(Length(1, FEET), eq(Length(12, INCH)));
    }

    TEST("1 YARD should equal to 3 FEET")
    {
        ASSERT_THAT(Length(1, YARD), eq(Length(3, FEET)));
    }

    TEST("1 MILE should equal to 1760 YARD")
    {
        ASSERT_THAT(Length(1, MILE), eq(Length(1760, YARD)));
    }
};

面向對象

Magellan實現xUnit時很是巧妙，使得用戶設計用例時更加面向對象。RobotCleaner robot在每一個用例執行時都將獲取一個獨立的、全新的實例。

#include "magellan/magellan.hpp"
#include "robot-cleaner/RobotCleaner.h"
#include "robot-cleaner/Position.h"
#include "robot-cleaner/Instructions.h"

USING_HAMCREST_NS

FIXTURE(RobotCleanerTest)
{
    RobotCleaner robot;

    TEST("at the beginning, the robot should be in at the initial position")
    {
        ASSERT_THAT(robot.getPosition(), is(Position(0, 0, NORTH)));
    }

    TEST("left instruction: 1-times")
    {
        robot.exec(left());
        ASSERT_THAT(robot.getPosition(), is(Position(0, 0, WEST)));
    }

    TEST("left instruction: 2-times")
    {
        robot.exec(left());
        robot.exec(left());
        ASSERT_THAT(robot.getPosition(), is(Position(0, 0, SOUTH)));
    }
};

函數提取

提取的相關子函數，能夠直接放在Fixture的內部，使得用例與其的距離最近，更加體現類做用域的概念。

#include "magellan/magellan.hpp"
#include "robot-cleaner/RobotCleaner.h"
#include "robot-cleaner/Position.h"
#include "robot-cleaner/Instructions.h"

USING_HAMCREST_NS

FIXTURE(RobotCleanerTest)
{
    RobotCleaner robot;

    void WHEN_I_send_instruction(Instruction* instruction)
    {
        robot.exec(instruction);
    }

    void AND_I_send_instruction(Instruction* instruction)
    {
        WHEN_I_send_instruction(instruction);
    }

    void THEN_the_robot_cleaner_should_be_in(const Position& position)
    {
        ASSERT_THAT(robot.getPosition(), is(position));
    }

    TEST("at the beginning")
    {
        THEN_the_robot_cleaner_should_be_in(Position(0, 0, NORTH));
    }

    TEST("left instruction: 1-times")
    {
        WHEN_I_send_instruction(left());
        THEN_the_robot_cleaner_should_be_in(Position(0, 0, WEST));
    }

    TEST("left instruction: 2-times")
    {
        WHEN_I_send_instruction(repeat(left(), 2));
        THEN_the_robot_cleaner_should_be_in(Position(0, 0, SOUTH));
    }

    TEST("left instruction: 3-times")
    {
        WHEN_I_send_instruction(repeat(left(), 3));
        THEN_the_robot_cleaner_should_be_in(Position(0, 0, EAST));
    }

    TEST("left instruction: 4-times")
    {
        WHEN_I_send_instruction(repeat(left(), 4));
        THEN_the_robot_cleaner_should_be_in(Position(0, 0, NORTH));
    }
};

斷言

ASSERT_THAT

Magellan只支持一種斷言原語：ASSERT_THAT, 從而避免用戶在選擇ASSERT_EQ/ASSERT_NE, ASSERT_TRUE/ASSERT_FALSE時的困擾，使其斷言更加具備統一性，一致性。

此外，ASSERT_THAT使得斷言更加具備表達力，它將實際值放在左邊，指望值放在右邊，更加符合英語習慣。

#include <magellan/magellan.hpp>

FIXTURE(CloseToTest)
{
    TEST("close to double")
    {
        ASSERT_THAT(1.0, close_to(1.0, 0.5));
        ASSERT_THAT(0.5, close_to(1.0, 0.5));
        ASSERT_THAT(1.5, close_to(1.0, 0.5));
    }
};

Hamcrest

Hamcrest是Java社區一個輕量級的，可擴展的Matcher框架，曾被Kent Beck引入到JUnit框架中，用於加強斷言的機制。Magellan引入了Hamcrest的設計，實現了一個C\+\+移植版本的Hamcrest，使得Magellang的斷言更加具備擴展性和可讀性。

anything

匹配器	說明
anything	老是匹配
_	anything語法糖

#include <magellan/magellan.hpp>

USING_HAMCREST_NS

FIXTURE(AnythingTest)
{
    TEST("should always be matched")
    {
        ASSERT_THAT(1, anything<int>());
        ASSERT_THAT(1u, anything<unsigned int>());
        ASSERT_THAT(1.0, anything<double>());
        ASSERT_THAT(1.0f, anything<float>());
        ASSERT_THAT(false, anything<bool>());
        ASSERT_THAT(true, anything<bool>());
        ASSERT_THAT(nullptr, anything<std::nullptr_t>());
    }

    TEST("should support _ as syntactic sugar")
    {
        ASSERT_THAT(1u, _(int));
        ASSERT_THAT(1.0f, _(float));
        ASSERT_THAT(false, _(int));
        ASSERT_THAT(nullptr, _(std::nullptr_t));
    }
};

比較器

匹配器	說明
eq	相等
ne	不相等
lt	小於
gt	大於
le	小於或等於
ge	大於或等於

#include <magellan/magellan.hpp>

USING_HAMCREST_NS

FIXTURE(EqualToTest)
{
    TEST("should allow compare to integer")
    {
        ASSERT_THAT(0xFF, eq(0xFF));
        ASSERT_THAT(0xFF, is(eq(0xFF)));

        ASSERT_THAT(0xFF, is(0xFF));
        ASSERT_THAT(0xFF == 0xFF, is(true));
    }

    TEST("should allow compare to bool")
    {
        ASSERT_THAT(true, eq(true));
        ASSERT_THAT(false, eq(false));
    }

    TEST("should allow compare to string")
    {
        ASSERT_THAT("hello", eq("hello"));
        ASSERT_THAT("hello", eq(std::string("hello")));
        ASSERT_THAT(std::string("hello"), eq(std::string("hello")));
    }
};

FIXTURE(NotEqualToTest)
{
    TEST("should allow compare to integer")
    {
        ASSERT_THAT(0xFF, ne(0xEE));

        ASSERT_THAT(0xFF, is_not(0xEE));
        ASSERT_THAT(0xFF, is_not(eq(0xEE)));
        ASSERT_THAT(0xFF != 0xEE, is(true));
    }

    TEST("should allow compare to boolean")
    {
        ASSERT_THAT(true, ne(false));
        ASSERT_THAT(false, ne(true));
    }

    TEST("should allow compare to string")
    {
        ASSERT_THAT("hello", ne("world"));
        ASSERT_THAT("hello", ne(std::string("world")));
        ASSERT_THAT(std::string("hello"), ne(std::string("world")));
    }
};

修飾器

匹配器	說明
is	可讀性裝飾器
is_not	可讀性裝飾器

#include <magellan/magellan.hpp>

USING_HAMCREST_NS

FIXTURE(IsNotTest)
{
    TEST("integer")
    {
        ASSERT_THAT(0xFF, is_not(0xEE));
        ASSERT_THAT(0xFF, is_not(eq(0xEE)));
    }

    TEST("string")
    {
        ASSERT_THAT("hello", is_not("world"));
        ASSERT_THAT("hello", is_not(eq("world")));

        ASSERT_THAT("hello", is_not(std::string("world")));
        ASSERT_THAT(std::string("hello"), is_not(std::string("world")));
    }
};

空指針

匹配器	說明
nil	空指針

#include <magellan/magellan.hpp>

USING_HAMCREST_NS

FIXTURE(NilTest)
{
    TEST("equal_to")
    {
        ASSERT_THAT(nullptr, eq(nullptr));
        ASSERT_THAT(0, eq(NULL));
        ASSERT_THAT(NULL, eq(NULL));
        ASSERT_THAT(NULL, eq(0));
    }

    TEST("is")
    {
        ASSERT_THAT(nullptr, is(nullptr));
        ASSERT_THAT(nullptr, is(eq(nullptr)));

        ASSERT_THAT(0, is(0));
        ASSERT_THAT(NULL, is(NULL));
        ASSERT_THAT(0, is(NULL));
        ASSERT_THAT(NULL, is(0));
    }

    TEST("nil")
    {
        ASSERT_THAT((void*)NULL, nil());
        ASSERT_THAT((void*)0, nil());
        ASSERT_THAT(nullptr, nil());
    }
};

字符串

匹配器	說明
contains_string	斷言是否包含子串
contains_string_ignoring_case	忽略大小寫，斷言是否包含子
starts_with	斷言是否以該子串開頭
starts_with_ignoring_case	忽略大小寫，斷言是否以該子串開頭
ends_with	斷言是否以該子串結尾
ends_with_ignoring_case	忽略大小寫，斷言是否以該子串結尾

#include <magellan/magellan.hpp>

USING_HAMCREST_NS

FIXTURE(StartsWithTest)
{
    TEST("case sensitive")
    {
        ASSERT_THAT("ruby-cpp", starts_with("ruby"));
        ASSERT_THAT("ruby-cpp", is(starts_with("ruby")));

        ASSERT_THAT(std::string("ruby-cpp"), starts_with("ruby"));
        ASSERT_THAT("ruby-cpp", starts_with(std::string("ruby")));
        ASSERT_THAT(std::string("ruby-cpp"), starts_with(std::string("ruby")));
    }

    TEST("ignoring case")
    {
        ASSERT_THAT("ruby-cpp", starts_with_ignoring_case("Ruby"));
        ASSERT_THAT("ruby-cpp", is(starts_with_ignoring_case("Ruby")));

        ASSERT_THAT(std::string("ruby-cpp"), starts_with_ignoring_case("RUBY"));
        ASSERT_THAT("Ruby-Cpp", starts_with_ignoring_case(std::string("rUBY")));
        ASSERT_THAT(std::string("RUBY-CPP"), starts_with_ignoring_case(std::string("ruby")));
    }
};

浮點數

匹配器	說明
close_to	斷言浮點數近似等於
nan	斷言浮點數不是一個數字

#include <magellan/magellan.hpp>
#include <math.h>

USING_HAMCREST_NS

FIXTURE(IsNanTest)
{
    TEST("double")
    {
        ASSERT_THAT(sqrt(-1.0), nan());
        ASSERT_THAT(sqrt(-1.0), is(nan()));

        ASSERT_THAT(1.0/0.0,  is_not(nan()));
        ASSERT_THAT(-1.0/0.0, is_not(nan()));
    }
};

程序選項

TestOptions::TestOptions() : desc("magellan")
{
    desc.add({
        {"help,     h",   "help message"},
        {"filter,   f",   "--filter=pattern"},
        {"color,    c",   "--color=[yes|no]"},
        {"xml,      x",   "print test result into XML file"},
        {"list,     l",   "list all tests without running them"},
        {"progress, p",   "print test result in progress bar"},
        {"verbose,  v",   "verbosely list tests processed"},
        {"repeat,   r",   "how many times to repeat each test"}
    });
    
    // default value
    options["color"]  = "yes";
    options["repeat"] = "1";
}

設計與實現

核心領域

Magellan總體的結構實際上是一棵樹，用於用例的組織和管理。

struct TestResult;

DEFINE_ROLE(Test)
{
    ABSTRACT(const std::string& getName () const);
    ABSTRACT(int countTestCases() const);
    ABSTRACT(int countChildTests() const);
    ABSTRACT(void run(TestResult&));
};

適配

如何讓FIXTURE中一個普通的成員函數TEST在運行時表現爲一個TestCase呢？在C++的實現中，彷佛變得很是困難。Magellan的設計很是簡單，將TEST的元信息在編譯時註冊到框架，簡單地使用了C++元編程的技術，及其C++11的一些特性保證，從而解決了C++社區一直未解決此問題的關鍵。

TEST的運行時信息由TestMethod的概念表示，其表明FIXTURE中一個普通的成員函數TEST，它們都具備一樣的函數原型: void Fixture::*)(); TestMethod是一個泛型類，泛型參數是Fixture；形式化地描述爲：

template <typename Fixture>
struct TestMethod
{    
    using Method = void(Fixture::*)();
};

TestCaller也是一個泛型類，它將一個TestMethod適配爲一個普通的TestCase。

template <typename Fixture>
struct TestCaller : TestCase
{    
    using Method = void(Fixture::*)();

    TestCaller(const std::string& name, Method method)
        : TestCase(name), fixture(0), method(method)
    {}

private:
    OVERRIDE(void setUp())
    {
        fixture = new Fixture;
        fixture->setUp();
    }

    OVERRIDE(void tearDown())
    {
        fixture->tearDown();        
        delete fixture;
        fixture = 0;
    }

    OVERRIDE(void runTest())
    {
        (fixture->*method)();
    }

private:
    Fixture* fixture;
    Method method;
};

裝飾

TestDecorator實際上是對Magellan核心領域的一個擴展，從而保證核心領域的不變性，而使其具備最大的可擴展性和靈活性。

工廠

在編譯時經過測試用例TEST的元信息的註冊，使用TestFactory很天然地將這些用例自動生成出來了。由於Magallan組織用例是一刻樹，TestFactory也被設計爲一棵樹，從而使得其與框架核心領域保持高度的一致性，更加天然、漂亮。

監聽狀態

Magellan經過TestListener對運行時的狀態變化進行監控，從而實現了Magellan不一樣格式報表打印的變化。