Introduction

I thought that graph based logic represent is more human friendly than code based one that means graph based test automation is more efficient to develop and ease maintain. I create an all in one graphical test automation tool to verify this idea. It is named Gate. Gate include all functions required for test automation. All execution logic was represent by a directed vector graph which we called model.html

Implementation:

Gate was implemented by combine JMeter and JGraphX together. It use JGraphX to create a test logic flow and it also use many parts of JMeter. It looks and use like JMeter but it is not just a copy of JMeter. Gate is for function test.java

Major features:

User defined variables.
Functions.
Package reusable test flows and include the package to test cases.
Test fixtures and test case dependency.
CSV data provider for test case.
Mutual-thread test execution in test case level.
Selenium and HTTP/HTTPS request.
Run in GUI and CMD mode. GUI for develop and CMD for CICD.

Build package:

Require jdk 8 to build and maven 3 to run it. Clone the master then enter the folder. Use "mvn package" to assemble the package. The package could be find in the "dest" folder.node

Thanks to contributor of JMeter and JGraphX. I will never be here without their great works. Gate is avaiable : https://github.com/Carl-Hou/gategit

Using Gate

Gate is graph based automation function test development environment. It include all functions required for test automation. It will generate a JUnit like xml report which is used to integrate with CICD pipeline. All execution logic was represent by a directed vector graph which we called model. github

Start Gate

Run 「mvn package」 with java8 and mvn 3. There have a 「dist\gate-0.0.1」 folder after build with 「mvn package」. Run the gate.bat the GUI will show up. Command args of gate.batapache

name	descriptions
-h	print usage information and command line options and exit
-n	run Gate in non-gui mode
-G	Define Gate properties e.g. -GvarName=value -GsomePort=8081
-t	Gate test file location
-S	Filter test suites by name prefix. Separate with comma. Command line only
-C	Filter test case by name prefix. Separate with comma. Command line only

Add test case

Add test suite and test case

create a test suite then add test case to it. Change the name of elements by edit the name panel then press enter. Double click on the test case to open the model editor. dom

Add vertex to model

Right click on the model. Move curse to add -> controller-> start then click the start to add a "start" to the editor. Test case will start to execute from it. Add a debug by add->sampler->debug. Add a "success" by add->link->success. maven

Link vertex with condtion edges

Connect the 「start」 and the 「debug」 by the 「success」 . Drag the arrow of link connect to the 「debug」 when connect the 「start」 and the 「debug」. That means execute the 「debug」 after 「start」 run successfully. ide

key board shortcutsui

The model editor have keyboard shortcuts 「ctrl +a」, 「ctrl +c」, 「ctrl +v」, 「ctrl+x」 for edit and 「ctr+x」, 「ctrl + y」 for undo/redo. The keyboard short cuts is not implement on the test tree yet. You can do copy, remove, cut on wit the right click menu .

Drag and drop

Nodes on the test tree to order them. This drag and drop function is used to support the test case dependency.

Start and stop test

Start test

Select the test case which just add above.Click the green arrow on the tool bar to run the test. Check the 「result」 of the tab on the bottom. The results list one the result tree. Click the node on the result tree. The result details of the graph element show on the right. Use key board shortcuts "ctrl + r"to start

Stop test

Click the the red 「round」 button on the tool box to stop the test. Let’s add a sleep to the model to make the stop button enabled.

Open the test case which create on above steps.
Add a timer by right click on the model editor add->timer-> constant timer. Modify the sleep time of it.
Connect the debug and this timer with a success link. Run the test case then click the red button. The test case will be stopped. Use key board shortcuts "ctrl + comma"to start

Select test cases to run

Add more test case (don’t forget to add the start). Select some of it them to run the test. Only select test case will be execute.Select the test suite will run all test case in the suite. Select the test suites will run all the test cases in the test cases.

Enable and Disable test suite/case

Test suites and test cases can be enable/disable by click the enable/disable on the right click manual. The disabled test cases will be ignore when start the test even when it was selected. Take care of this when run test case which depend on other.

Test case dependency

Test case run in concurrency in default. Test case dependency is used to constraint some test case run after other specified test case run successfully.

Set the 「gate.engine.test.runner.number」 in the gate.properties to 1 will make the test run with single executor. The test suite and test case will be execute in same order as they list on the tree. The dependency is not needed with this setting.

Click the test case node which need to run after some test cases. Add the test cases names which required to run before it. Separate them with comma if more than one test cases need to be dependent.

Run test case multi-times

There have option to make a test case run multi-times. Set the "invocation count" or add a "data provider" to test case.

Invocation count

Click on the test case node. Set the "invocation count" field to number which bigger than 1. The test will repeat. It will reset the configuration and variable same as the first time model run for each repeat . 「user defined variable」 can be use in the "invocationCount" field. Test case result will display with a suffix which indicate how may test has been executed before.

Invocation count	Test case suffix

Data provider

Test case will be iterated after add "csv data provider to it". It will use data defined in the csv as value of variables configured in "csv data provider".

Add file e.g example.csv with a absolute path to the "file name" field with content like bellowing.

11,12,13
21,22,23

Config variables name in "variables name" field. The delimiter is "," in default which can be change to what you like. We will use "v1,v2,v3" in this case.
Select the test case then run it. Check the result then we will find the variable and its value change in each iteration.

Configuration	Result

Test Fixtures

There have two type of test fixtures the setup and teardown. The setup always execute before the any other node on the test tree. The teardown fixtures always execute after other node on the test tree. They will be execute in same orders with the test tree.

Add before/teardown fixtures on the suites/suite by right click on the node then add->fixture->setup/teardown.
Double click the fixture to open the model editor. Add graph element 「start」「debug」 then link them. This is same as create the test case.
Run the case and check the result.
Add more setup/teardown drag them to different place.

Use Actions

Gate support package reusable test logic flow to an model which is called action.The action model could be used in the test case or another action. Add action to itself is not allowed. The actions use same context with the test case which refer to it.

Click the 「Lib」 tab of the tree panel to add an action. Right click on the action node then add->action.

Double click the action which just added the add the 「start」「debug」 and link them.

Open the test case which need to refer to the action

Keep the model editor open for the test case. Click the 「Lib」 tab of the tree panel. Right click the action which we created above. Click the "add to editor".

Keep the model editor open for the test case. Link the "Debug" and the "Action".

Run the test and check the result.

Function and variables

Function and variable is inherit from JMeter. They are almost same with JMeter implementation . Refer to JMeter doc for more details https://jmeter.apache.org/usermanual/functions.html.

Variables

Variable could be use in all type of model it also could be used in suites, suite, case config. Use ${variableName} refer to runtime values. Variables can be defined by 「User define variables」 or Call「vars.put(「name」,」value」) in JSR223 Sampler. It will be inherit and override in order suites -> suite -> case -> model.

Add user define variable in test tree. Right click on test suite/test case node. Right Click on the model editor then add -> config -> 「user defined variable」.

Click the "user define variable". Check the properties On the bottom of the Gui. Click add then input the name-value pairs to create variable. press "enter" after input the value to the field. Run the test then check the result of the debug sampler.

Add user defined variable inside model. Open the model editor then add-> config -> "user defined variables". Add name-value pairs to the "user defined variables" vertex of the test case model. run the test case then check the result.

Functions

Functions can be use anywhere in side the model. It also works in the user define variable. A function call looks like this: ${__functionName(var1,var2,var3)}. List implement function bellowing. Detail about how to use them refer to JMeter and JMeter plugin function document. They are almost same. Some of the function have a variable name as parameter which use to return value. Take care about the white space of the variable name. Here is an example to use __javaScript.

Add a "user defined variable" in the model. Add variable n1=1.
Add a "Decide" by Right click -> add -> controller -> Decide. Set condition to ${__javaScript(var a=${n1};a >1? true : false;)}
Add a debug then link success with it. Add a variable assert. Set variable name field to "n1". Set the "Pattern to Test" to 0.
Run the test then check the result. This example means test will failed if ${n1} <=1 && ${n1}!=0

Details of bellowing function refer for to https://jmeter.apache.org/usermanual/functions.html

__time;__Random;__log;__logn;__UUID;__groovy;__javaScript;__property;__setProperty;__split;__eval;__evalVar;__V;__char;__escapeHtml;__unescape;__unescapeHtml;__urlencode;__urldecode;__dateTimeConvert __digest;__isVarDefined;__isPropDefined;__changeCase;__FileToString;__StringToFile;__XPath

Details of bellowing function refer to https://jmeter-plugins.org/wiki/Functions

__base64Encode;__base64Decode;__env;__if;__strLen;__substring;__strReplace

Comments and Analyze

Add a comments vertex to model by add -> comments -> comment. Write comments in the editor. Gate support "note" link vertex after this link which will never be execute. This is used for link comments vertex to other vertex. Gate have a function to check whether all vertex is connected by a link. The "note" link is used at this time. Click Option -> Analyze -> "Is Connected" on the tool bar to check whether all vertex is connect.

Properties file

There have two properties file in the $GATE_HOME/bin folder. The gate.properties and system.properties. This two properties file load on Gate start up. system.properties load properties in JVM scope. It is a batch java "-D" or System.setProperty. gate.properties load properties used in Gate scope. almost all of them are Gate's config.

Introduction to Gate