XStudio fat client comes with a tool called XContinuousIntegration.
This is even more convenient when we speak about continuous integration. In this case, each time a commit will occur on the versioning system, a build will be generated and the tests executed.
This command will run XContinuousIntegration in synchronous mode (it returns only when the session completed).
If at least one of the agents
the
When the session is completed, an HTML report named
In this session, the test
The linux version is also provided and use the same syntax except that the binary is a shell script. You can also run XContinuousIntegration by interpreting directly the jar file.
If you need to execute XContinuousIntegration remotely on a linux server for example, you can call it within an ssh session using a command similar to:
Of course you can also ensure you have the right certificates stored to avoid using the
You may also need to download/upload some files from/to the remote server.
In this case, you can use scp a similar way.
For instance, to upload some files:
To download some files:
However, the continuous integration system may even better integrate with XStudio if it can retrieve the results.
Jenkins can also parse any test output that conforms to the standard JUnit XML format so you can use
Please refer to Jenkins documentation on how to include the XContinuousIntegration command line in the process.
Basically, in the build step click on Execute a shell script and type the command:
To get some nice graphics and trend charts showing the evolution of your test results you can add a step in your build plan:
In Post-build Actions, click on Add post-build action and select Publish JUnit test result report. In the Test report XMLs field, specify the directory where all your xml reports are getting generated and you're done.
If you click on Test Result it will take you to the generated report. Here you will be able to see:
Bamboo (and others) can parse any test output that conforms to the standard JUnit XML format. You have just to specify the option
You can refer to Atlassion's tutorial on how to configure an external command in Bamboo.
Atlassian also have a page describing the JUnit XML format if you're interested.
Hudson can also parse any test output that conforms to the standard JUnit XML format so you can use
Please refer to Hudson documentation on how to include the XContinuousIntegration command line in the process.
TeamCity can also parse any test output that conforms to the standard JUnit XML format so you can use
Please refer to TeamCity documentation on how to include the XContinuousIntegration command line in the process.
- On Windows platforms, you can find it in the
xstudiofolder. The executable is namedxcontinuousintegration_console.exe - On MacOSX and linux platforms, the executable script is named
xcontinuousintegration.sh
Goal
XContinuousIntegration is aimed at being called by automatic build machines. Once a build has been run successfully, the build server can automatically trigger the creation and the start of a campaign session in XStudio. This way, after your overnight build has completed you can automatically run your automated test suites without having to attend.This is even more convenient when we speak about continuous integration. In this case, each time a commit will occur on the versioning system, a build will be generated and the tests executed.
Usage
To trigger the creation and the execution of a campaign session, the build server will just have to call the XContinuousIntegration executable with the appropriate arguments which are...
Usage: xcontinuousintegration_console.exe --campaignId <campaignId>
--agents <agentId>:<nbInstances>:<sync>,...
--sutId <sutId>
--configurations <categoryId>:<configId>,...
Options:
--sessionName <sessionName>
--operatorId <userId>
--agentsPick <agentsPick>
--ccEmailUserIds <userId>,...
--monitoringAgentId <agentId>
--monitoringConfId <monitoringConfId>
--ifParentFailed <action>
--ifParentUnknown <action>
--checkAgentOnline
--synchronous
--synchronousTimeout <timeout> (only with '--synchronous')
--generateReport (only with '--synchronous')
--reportTransform <xsltName> (only with '--synchronous' and '--generateReport')
--reportStyle <styleName> (only with '--synchronous' and '--generateReport')
--reportOutputPath <path> (only with '--synchronous' and '--generateReport')
--reportName <reportName> (only with '--synchronous' and '--generateReport')
--reportIncludesAttachments
--overloadBooleanAttributes <testId>[<attributeId>:<attributeValue>;...],...
--overloadIntegerAttributes <testId>[<attributeId>:<attributeValue>;...],...
--overloadStringAttributes <testId>[<attributeId>:<attributeValue>;...],...
--overloadBooleanAttributes <paramId>:<paramValue>,...
--overloadIntegerAttributes <paramId>:<paramValue>,...
--overloadStringAttributes <paramId>:<paramValue>,...
Where:
<sync> can take the following values: 0=None,
1=Synchronization on tests,
2=Synchronization on test cases,
3=In a row
<agentsPick> can take the following values: 0=Any agent in the pool,
1=All agents in the pool (default value),
2=first available agent in the pool
<action> can take the following values: 'EXECUTE',
'SET_AS_FAILED'
'DO_NOT_EXECUTE'
<timeout> is a timeout given in seconds (if the session is not executed after this delay,
XContinuousIntegration exits and no report is generated)
<path> is a local path to an existing folder (i.e. 'C:/Users/eric')
Where do I find all those Ids
As you see, this command line needs many ids. Hopefully you can get them pretty easily:| Id | Where to find it |
| Mandatory | |
| campaignId | From the campaign tree, select the campaign and check the header on the right panel |
| agentId | From the agent tree, select the agent and check the header on the right panel |
| sutId | From the SUT tree, select the sut and check the header on the right panel |
| categoryId | From the test tree, select the category and check the header on the right panel |
| configurationId | From the campaign tree, select a session that has already been executed using this configuration, select the Configuration tab and check for each category the id of the configuration |
| Optional | |
| operatorId | From the user tree, select the user and check the header on the right panel |
| userId | From the user tree, select the user and check the header on the right panel |
| monitoringConfId | From the campaign tree, select a session that has already been executed using this monitoring configuration, select the Monitoring / Monitored features tab and check the id of the configuration |
| testId | From the test tree, select the test and check the header on the right panel |
| attributeId | Directly from the database in the table_attr_* |
| paramId | Directly from the database in the table_param_* |
IMPORTANT: all the ids must be provided WITHOUT their prefix!
For instance, if you have a campaign with the id
For instance, if you have a campaign with the id
C_1234, you just need to provide to XContinuousIntegration the value 1234.
Example
xcontinuousintegration_console.exe --campaignId 1090 --agents 16:1:0;17:2:2 --sutId 89
--configurations 80:42;81:13 --checkAgentOnline --synchronous --synchronousTimeout 14400
--generateReport --reportTransform HTML_Complete_with_stats --reportStyle XQual
--reportOutputPath E:/tmp --reportName my_last_report --reportIncludesAttachments
--overloadBooleanAttributes 123[12:0;22:1],134[12:1]
This command will run XContinuousIntegration in synchronous mode (it returns only when the session completed).
If at least one of the agents
AG_16 and AG_17 are offline, the program exits immediately because ofthe
--checkAgentOnline argument).
The program will return anyway if the session is not completed after 4 hours (14400 seconds).When the session is completed, an HTML report named
my_last_report.html is generated locally in E:/tmp based on the XSLT reportHTML_Complete_with_stats.xslt (presented in XStudio as (HTML) Complete_with_stats in the drop-down menu).
A my_last_report.zip file is also created that contains all the resources (including attachments etc.) to have a fully indexed/emailable/attachable report.
In this session, the test
T_123 will be overloaded with a boolean attribute 12 set to false and the attribute 22 set to true.
The test T_134 will be overloaded with the attribute 12 set to true.
The linux version is also provided and use the same syntax except that the binary is a shell script. You can also run XContinuousIntegration by interpreting directly the jar file.
If you need to execute XContinuousIntegration remotely on a linux server for example, you can call it within an ssh session using a command similar to:
ssh -o StrictHostKeyChecking=no <account> "cd <path to xcontinuousintegration>;
java -jar xcontinuousintegration.jar <options as above>"
Of course you can also ensure you have the right certificates stored to avoid using the
StrictHostKeyChecking=no option that is unsecure.
You may also need to download/upload some files from/to the remote server.
In this case, you can use scp a similar way.
For instance, to upload some files:
scp -o StrictHostKeyChecking=no * <account>:<destination path>
To download some files:
scp -o StrictHostKeyChecking=no <account>:<source path> .
Integration with third-party systems
XStudio can be integrated in ANY continuous integration environment as it just consists in executing a simple command after a build is completed.However, the continuous integration system may even better integrate with XStudio if it can retrieve the results.
Jenkins can also parse any test output that conforms to the standard JUnit XML format so you can use
--reportTransform XML_JUnit as well with Jenkins.
Please refer to Jenkins documentation on how to include the XContinuousIntegration command line in the process.
Basically, in the build step click on Execute a shell script and type the command:
java -jar xcontinuousiontegration.jar with all the appropriate options.To get some nice graphics and trend charts showing the evolution of your test results you can add a step in your build plan:
In Post-build Actions, click on Add post-build action and select Publish JUnit test result report. In the Test report XMLs field, specify the directory where all your xml reports are getting generated and you're done.
If you click on Test Result it will take you to the generated report. Here you will be able to see:
- Test Results showing number of test failures out of whole tests
- All Failed Tests list down the name of all the test methods which got failed
- All Tests will show the duration of execution, number of test failed, number of tests skipped, number of test passed and total number of tests
Bamboo (and others) can parse any test output that conforms to the standard JUnit XML format. You have just to specify the option
--reportTransform XML_JUnit in the command line and a
JUnit XML report will be additionally generated so that Bamboo can parse it and retrieve the results.
You can refer to Atlassion's tutorial on how to configure an external command in Bamboo.
Atlassian also have a page describing the JUnit XML format if you're interested.
Hudson can also parse any test output that conforms to the standard JUnit XML format so you can use
--reportTransform XML_JUnit as well with Hudson.
Please refer to Hudson documentation on how to include the XContinuousIntegration command line in the process.
TeamCity can also parse any test output that conforms to the standard JUnit XML format so you can use
--reportTransform XML_JUnit as well with TeamCity.
Please refer to TeamCity documentation on how to include the XContinuousIntegration command line in the process.