最新教程html
自動化測試框架pytest教程python
這裏是python測試開發的平常記錄,主要涉及單元測試、安全測試、python基礎、性能測試等內容。mysql
項目地址:https://bitbucket.org/xurongzhong/small_python_daily_toolslinux
代碼位置: db/oracle 文件:config.py transfer_db.pygit
測試環境:CentOS release 6.6 (Final) 64 python2.7github
運行依賴: python外部庫cx_Oracle、mysql-connector-pythonweb
功能:遷移oracle數據到mysql,須要先行在mysql建立表。特別注意,有清空mysql表的動做。sql
簡介:使用python批量清空數據庫,基於mysql python api和python標準庫configparser,實現可配置地批量清空mysql表。數據庫
支持版本:python2.7/3.*api
後期規劃:經過argparse支持指定配置文件,增長delete、truncate、rename等選項。
代碼 db/mysql del_db.py db.conf
簡介:在oracle中插入假數據
測試環境:CentOS release 6.6 (Final) 64 python2.7
代碼 db/oracle oracle_add.py
pytest是強大的python單元測試框架。因爲python的膠水語言特性,實質上pytest在接口測試、自動化測試等方面也普遍使用。
成熟全功能的Python測試工具
靈活
適合簡單的單元測試到複雜的功能測試
集成
擴展插件和定製系統:
本節來源:http://pytest.org/latest/
交流:python開發自動化測試羣291184506 PythonJava單元白盒單元測試羣144081101 商務合做微信:pythontesting
英文原版書籍下載:https://bitbucket.org/xurongzhong/python-chinese-library/downloadss。
精品文章推薦:
性能測試藝術
Java單元測試之模擬利器-使用PowerMock進行Mock測試
python -m pytest調用:
獲取版本,選項名,環境變量
失敗後中止執行
執行選擇用例
調試輸出:
失敗時調用PDB (Python Debugger):
Python帶有一個內置的Python調試器稱爲PDB。pytest能夠在命令行選項指定調用:
py.test --pdb
這將每次失敗時調用Python調試器。
py.test -x --pdb # 失敗時調用pdb,而後退出測試。 py.test --pdb - maxfail=3# 前3次失敗調用pdb。
異常信息保存在sys.last_value
, sys.last_type和``sys.last_traceback,交互式狀況能夠調試:
>>> import sys >>> sys.last_traceback.tb_lineno 42 >>> sys.last_value AssertionError('assert result == "ok"',)
import pytest def test_function(): ... pytest.set_trace() # invoke PDB debugger and tracing
2.0.0之前的版本中只有經過py.test-s禁用命令行捕捉才能夠進入pdb調試。2.0.0及之後版本在進入pdb調試時自動禁用輸出捕捉。
2.4.0 支持使用importpdb;pdb.set_trace()進入pdb。
py.test --durations=10
建立Jenkins或其餘持續集成服務器的結果文件:
py.test --junitxml=path
2.8新增
在xml文件中記錄額外信息
def test_function(record_xml_property): record_xml_property("example_key", 1) assert 0
效果以下:
<testcase classname="test_function" file="test_function.py" line="0" name="test_function" time="0.0009"> <properties> <property name="example_key" value="1" /> </properties> </testcase>
試驗功能,與pytest-xdist不兼容,還有可能破壞模式驗證,與一些CI集成可能會有問題。
要建立純文本的機器可讀的結果文件,用於PyPy-testweb展現等。
py.test --resultlog=path
bpaste能夠爲你的文本生成url鏈接,下面爲建立每一個測試失敗建立一個url:
py.test --pastebin=failed
py.test --pastebin=all py.test --pastebin=failed -x # 只發送一次
目前只支持:http://bpaste.net
禁用插件
py.test -p no:doctest
在python代碼中調用pytest
2.0新增 用exitcode代替了SystemExit。 pytest.main([’-x’, ’mytestdir’]) pytest.main("-x mytestdir")
# 指定插件
# content of myinvoke.py import pytest class MyPlugin: def pytest_sessionfinish(self): print("*** test run reporting finishing") pytest.main("-qq", plugins=[MyPlugin()])
執行結果:
$ python myinvoke.py *** test run reporting finishing
#virtualenv . New python executable in ./bin/python Installing setuptools, pip...done. root@AutoTest:[/data/code/python/pytest]#source bin/activate (pytest)root@AutoTest:[/data/code/python/pytest]#pip install pytest Downloading/unpacking pytest Downloading pytest-2.5.2.tar.gz (608kB): 608kB downloaded Running setup.py (path:/data/code/python/pytest/build/pytest/setup.py) egg_info for package pytest
Downloading/unpacking py>=1.4.20 (from pytest) Downloading py-1.4.22.tar.gz (189kB): 189kB downloaded Running setup.py (path:/data/code/python/pytest/build/py/setup.py) egg_info for package py
Installing collected packages: pytest, py Running setup.py install for pytest
Installing py.test-2.7 script to /data/code/python/pytest/bin Installing py.test script to /data/code/python/pytest/bin
Running setup.py install for py
Successfully installed pytest py Cleaning up...
測試佈局的方法有2種。一爲放置在應用代碼以外,適用於有不少功能測試等狀況。
setup.py # your distutils/setuptools Python package metadata mypkg/ __init__.py appmodule.py tests/ test_app.py ...
二爲嵌入測試目錄到應用,當(單元)測試和應用之間的有直接關係,並想一塊兒發佈時有用:
setup.py # your distutils/setuptools Python package metadata mypkg/ __init__.py appmodule.py ... test/ test_app.py
http://stackoverflow.com/search?q=pytest 上有不少例子。
# content of test_assert1.py def f(): return 3 def test_function(): assert f() == 4
執行結果:
$ py.test test_assert1.py =================================================================================== test session starts ==================================================================================== platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /home/andrew/tmp, inifile: plugins: xdist-1.14 collected 1 items test_assert1.py F ========================================================================================= FAILURES ========================================================================================= ______________________________________________________________________________________ test_function _______________________________________________________________________________________ def test_function(): > assert f() == 4 E assert 3 == 4 E + where 3 = f() test_assert1.py:6: AssertionError ================================================================================= 1 failed in 0.00 seconds ================================================================================= $
import pytest def test_zero_division(): with pytest.raises(ZeroDivisionError): 1 / 0
獲取異常信息:
def test_recursion_depth(): with pytest.raises(RuntimeError) as excinfo: def f(): f() f() assert 'maximum recursion' in str(excinfo.value)
xfail中也能夠指定異常類型:
import pytest @pytest.mark.xfail(raises=NameError) def test_f(): f()
pytest.raises適用於故意產生異常,@pytest.mark.xfail適用於處理bug。
# content of test_assert2.py def test_set_comparison(): set1 = set("1308") set2 = set("8035") assert set1 == set2
執行結果:
# py.test test_assert2.py ================================================================================================= test session starts ========================================================================= platform linux2 -- Python 2.7.5 -- py-1.4.23 -- pytest-2.6.1 collected 1 items
test_assert2.py F
============================================================================== FAILURES ============================================================================== ________________________________________________________________________ test_set_comparison _________________________________________________________________________
def test\_set\_comparison(): set1 = set("1308") set2 = set("8035")
> assert set1 == set2 E assert set(['0', '1', '3', '8']) == set(['0', '3', '5', '8']) E Extra items in the left set: E '1' E Extra items in the right set: E '5'
test_assert2.py:5: AssertionError ====================================================================== 1 failed in 0.01 seconds ======================================================================
conftest.py
# content of conftest.py from test_foocompare import Foo def pytest_assertrepr_compare(op, left, right): if isinstance(left, Foo) and isinstance(right, Foo) and op == "==": return ['Comparing Foo instances:', 'vals: %s != %s' % (left.val, right.val)]
test_foocompare.py
# content of test_foocompare.py class Foo: def __init__(self, val): self.val = val
def test_compare(): f1 = Foo(1) f2 = Foo(2) assert f1 == f2
執行結果:
# py.test -q test_foocompare.py F ================================================================================================================ FAILURES ================================================================================================================= ______________________________________________________________________________________________________________ test_compare _______________________________________________________________________________________________________________
def test_compare(): f1 = Foo(1) f2 = Foo(2)
> assert f1 == f2 E assert Comparing Foo instances: E vals: 1 != 2
test_foocompare.py:9: AssertionError 1 failed in 0.01 seconds
暫略,後面補上
failure_demo.py 結合斷言的使用,展現了多種錯誤報告:
https://github.com/alfredodeza/pytest/blob/master/doc/en/example/assertion/failure_demo.py
from py.test import raises import py def otherfunc(a,b): assert a==b def somefunc(x,y): otherfunc(x,y) def otherfunc_multi(a,b): assert (a == b) def test_generative(param1, param2): assert param1 * 2 < param2 def pytest_generate_tests(metafunc): if 'param1' in metafunc.fixturenames: metafunc.addcall(funcargs=dict(param1=3, param2=6)) class TestFailing(object): def test_simple(self): def f(): return 42 def g(): return 43 assert f() == g() def test_simple_multiline(self): otherfunc_multi( 42, 6*9) def test_not(self): def f(): return 42 assert not f() class TestSpecialisedExplanations(object): def test_eq_text(self): assert 'spam' == 'eggs' def test_eq_similar_text(self): assert 'foo 1 bar' == 'foo 2 bar' def test_eq_multiline_text(self): assert 'foo\nspam\nbar' == 'foo\neggs\nbar' def test_eq_long_text(self): a = '1'*100 + 'a' + '2'*100 b = '1'*100 + 'b' + '2'*100 assert a == b def test_eq_long_text_multiline(self): a = '1\n'*100 + 'a' + '2\n'*100 b = '1\n'*100 + 'b' + '2\n'*100 assert a == b def test_eq_list(self): assert [0, 1, 2] == [0, 1, 3] def test_eq_list_long(self): a = [0]*100 + [1] + [3]*100 b = [0]*100 + [2] + [3]*100 assert a == b def test_eq_dict(self): assert {'a': 0, 'b': 1, 'c': 0} == {'a': 0, 'b': 2, 'd': 0} def test_eq_set(self): assert set([0, 10, 11, 12]) == set([0, 20, 21]) def test_eq_longer_list(self): assert [1,2] == [1,2,3] def test_in_list(self): assert 1 in [0, 2, 3, 4, 5] def test_not_in_text_multiline(self): text = 'some multiline\ntext\nwhich\nincludes foo\nand a\ntail' assert 'foo' not in text def test_not_in_text_single(self): text = 'single foo line' assert 'foo' not in text def test_not_in_text_single_long(self): text = 'head ' * 50 + 'foo ' + 'tail ' * 20 assert 'foo' not in text def test_not_in_text_single_long_term(self): text = 'head ' * 50 + 'f'*70 + 'tail ' * 20 assert 'f'*70 not in text def test_attribute(): class Foo(object): b = 1 i = Foo() assert i.b == 2 def test_attribute_instance(): class Foo(object): b = 1 assert Foo().b == 2 def test_attribute_failure(): class Foo(object): def _get_b(self): raise Exception('Failed to get attrib') b = property(_get_b) i = Foo() assert i.b == 2 def test_attribute_multiple(): class Foo(object): b = 1 class Bar(object): b = 2 assert Foo().b == Bar().b def globf(x): return x+1 class TestRaises: def test_raises(self): s = 'qwe' raises(TypeError, "int(s)") def test_raises_doesnt(self): raises(IOError, "int('3')") def test_raise(self): raise ValueError("demo error") def test_tupleerror(self): a,b = [1] def test_reinterpret_fails_with_print_for_the_fun_of_it(self): l = [1,2,3] print ("l is %r" % l) a,b = l.pop() def test_some_error(self): if namenotexi: pass def func1(self): assert 41 == 42 # thanks to Matthew Scott for this test def test_dynamic_compile_shows_nicely(): src = 'def foo():\n assert 1 == 0\n' name = 'abc-123' module = py.std.imp.new_module(name) code = py.code.compile(src, name, 'exec') py.builtin.exec_(code, module.__dict__) py.std.sys.modules[name] = module module.foo() class TestMoreErrors: def test_complex_error(self): def f(): return 44 def g(): return 43 somefunc(f(), g()) def test_z1_unpack_error(self): l = [] a,b = l def test_z2_type_error(self): l = 3 a,b = l def test_startswith(self): s = "123" g = "456" assert s.startswith(g) def test_startswith_nested(self): def f(): return "123" def g(): return "456" assert f().startswith(g()) def test_global_func(self): assert isinstance(globf(42), float) def test_instance(self): self.x = 6*7 assert self.x != 42 def test_compare(self): assert globf(10) < 5 def test_try_finally(self): x = 1 try: assert x == 0 finally: x = 0
執行:
$ py.test test_failures.py =================================================================================== test session starts ==================================================================================== platform linux -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /iscsi_data1/data/code/python/pytest/doc/en, inifile: pytest.ini collected 1 items test_failures.py . ================================================================================= 1 passed in 0.56 seconds ================================================================================= [root@public01 assertion]# py.test failure_demo.py =================================================================================== test session starts ==================================================================================== platform linux -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /iscsi_data1/data/code/python/pytest/doc/en, inifile: pytest.ini collected 39 items failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF ========================================================================================= FAILURES ========================================================================================= ____________________________________________________________________________________ test_generative[0] ____________________________________________________________________________________ param1 = 3, param2 = 6 def test_generative(param1, param2): > assert param1 * 2 < param2 E assert (3 * 2) < 6 failure_demo.py:15: AssertionError
餘下部分不作介紹,請以實際執行結果爲準。
下面函數中的cmdopt,須要從命令行讀取:
# content of test_sample.py def test_answer(cmdopt): if cmdopt == "type1": print ("first") elif cmdopt == "type2": print ("second") assert 0 # to see what was printed
設置_conftest.py_
# content of conftest.py import pytest def pytest_addoption(parser): parser.addoption("--cmdopt", action="store", default="type1", help="my option: type1 or type2") @pytest.fixture def cmdopt(request): return request.config.getoption("--cmdopt")
執行:
$ py.test -q --cmdopt=type2 F ========================================================================================= FAILURES ========================================================================================= _______________________________________________________________________________________ test_answer ________________________________________________________________________________________ cmdopt = 'type2' def test_answer(cmdopt): if cmdopt == "type1": print ("first") elif cmdopt == "type2": print ("second") > assert 0 # to see what was printed E assert 0 test_sample.py:7: AssertionError ----------------------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------------------- second 1 failed in 0.00 seconds
# content of conftest.py import sys def pytest_cmdline_preparse(args): if 'xdist' in sys.modules: # pytest-xdist plugin import multiprocessing num = max(multiprocessing.cpu_count() / 2, 1) args[:] = ["-n", str(num)] + args
下面用例,若是打了slow標籤,則會須要添加選項--runslow才能執行。
配置
# content of conftest.py import pytest def pytest_addoption(parser): parser.addoption("--runslow", action="store_true", help="run slow tests")
用例:
# content of test_module.py import pytest slow = pytest.mark.skipif( not pytest.config.getoption("--runslow"), reason="need --runslow option to run" ) def test_func_fast(): pass @slow def test_func_slow(): pass
默認執行:
$ py.test -rs =================================================================================== test session starts ==================================================================================== platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /home/andrew/tmp, inifile: plugins: xdist-1.14 collected 2 items test_module.py .s ================================================================================= short test summary info ================================================================================== SKIP [1] test_module.py:15: need --runslow option to run =========================================================================== 1 passed, 1 skipped in 0.01 seconds ============================================================================ $
指定標籤執行
$ py.test --runslow =================================================================================== test session starts ==================================================================================== platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /home/andrew/tmp, inifile: plugins: xdist-1.14 collected 2 items test_module.py .. ================================================================================= 2 passed in 0.01 seconds ================================================================================= $
python中經典的xunit風格的setup是2005年首先出如今pytest,後面被nose和unittest採用。建議優先使用fixture,由於它有依賴注入,更強大。
def setup_module(module): """ setup any state specific to the execution of the given module.""" def teardown_module(module): """ teardown any state that was previously setup with a setup_module method. """
@classmethod def setup_class(cls): """ setup any state specific to the execution of the given class (which usually contains tests). """ @classmethod def teardown_class(cls): """ teardown any state that was previously setup with a call to setup_class. """
@classmethod def setup_class(cls): """ setup any state specific to the execution of the given class (which usually contains tests). """ @classmethod def teardown_class(cls): """ teardown any state that was previously setup with a call to setup_class. """
def setup_function(function): """ setup any state tied to the execution of the given function. Invoked for every test function in the module. """ def teardown_function(function): """ teardown any state that was previously setup with a setup_function call. """
tmpdir是py.path.local對象,提供了os.path等的方法。
test_tmpdir.py
# content of test_tmpdir.py import os def test_create_file(tmpdir): p = tmpdir.mkdir("sub").join("hello.txt") p.write("content") assert p.read() == "content" assert len(tmpdir.listdir()) == 1 assert 0
執行結果:
$ py.test test_tmpdir.py ======================================================================= test session starts ======================================================================= platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /home/andrew/tmp/py_test, inifile: collected 1 items test_tmpdir.py F ============================================================================ FAILURES ============================================================================= ________________________________________________________________________ test_create_file _________________________________________________________________________ tmpdir = local('/tmp/pytest-of-andrew/pytest-0/test_create_file0') def test_create_file(tmpdir): p = tmpdir.mkdir("sub").join("hello.txt") p.write("content") assert p.read() == "content" assert len(tmpdir.listdir()) == 1 > assert 0 E assert 0 test_tmpdir.py:8: AssertionError ==================================================================== 1 failed in 0.06 seconds ===================================================================== $
生成的目錄通常在系統臨時目錄下面,格式爲pytest-NUM,好比/tmp/pytest-of-andrew/pytest-0/test_create_file0,每次測試執行時NUM會增長,3個之前的目錄會清除。以下方式會指定爲其餘目錄:py.test --basetemp=mydir。
2.8新增
tmpdir_factory基於session,可建立任意臨時目錄。
例如,假設test suite須要大文件:
# contents of conftest.py import pytest @pytest.fixture(scope='session') def image_file(tmpdir_factory): img = compute_expensive_image() fn = tmpdir_factory.mktemp('data').join('img.png') img.save(str(fn)) return fn # contents of test_image.py def test_histogram(image_file): img = load_image(image_file) # compute and test histogram
相關方法:
TempdirFactory.``mktemp
(basename, numbered=True)[source]
TempdirFactory.``getbasetemp
()[source]
默認捕捉stdout/stderr,若是test或者setup方法失敗,traceback會打印相關內容。
stdin爲null,讀取會報錯,由於自動化測試鮮有交互式輸入。
默認捕捉爲寫入到底層文件,能夠捕捉print語言輸出到test中的subprocess輸出。
默認捕捉方式爲file descriptor (FD)級捕捉。捕捉全部到操做系統的1,2輸出。
syslevel級捕捉只捕捉python的sys.stdout和sys.stderr。
py.test -s # disable all capturing py.test --capture=sys # replace sys.stdout/stderr with in-mem files py.test --capture=fd # also point filedescriptors 1 and 2 to temp file
默認捕捉stdout/stderr:
# content of test_module.py def setup_function(function): print ("setting up %s" % function) def test_func1(): assert True def test_func2(): assert False
執行結果:
$ py.test test_module.py ======================================================================= test session starts ======================================================================= platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /home/andrew/tmp/py_test, inifile: collected 2 items test_module.py .F ============================================================================ FAILURES ============================================================================= ___________________________________________________________________________ test_func2 ____________________________________________________________________________ def test_func2(): > assert False E assert False test_module.py:10: AssertionError ---------------------------------------------------------------------- Captured stdout setup ---------------------------------------------------------------------- setting up <function test_func2 at 0x7fdc31ed6048> =============================================================== 1 failed, 1 passed in 0.11 seconds ================================================================ $