Manual Offline Execution

I some circumstances, you may need to test some systems on client side and you'll not have any internet access (or your XStudio is behind a firewall and not accessible from the outside).

In that case, you can reference and document your tests in XStudio then prepare a campaign as if you were going to execute it locally.
Then just generate a "offline_manual_testing" Excel report on that campaign. It will contain all the descriptions and procedures of all the tests and testcases in that campaign.

The tester can then safely go on customer site, perform his manual testing and report the results directly in the Excel file.

When the tester is back at the office, he can just import the results by providing the modified Excel file.

The complete process is shown in the schema below:

Asynchronous execution

In some circumstances, mainly when you're having a lot of automated tests that are existing and performed by a proprietary mechanism or you're using continuous integration based processes, the test execution can be triggered by the third-party itself.

There are 2 possibilities then, either the tests are executed by the third-party or they can be executed by XStudio.

You'll find below the 5 main situations you can encounter and how to deal with them with XStudio.
Breaking news: XStudio can adapt itself to any of them smoothly!

The test execution is triggered by the continuous integration system

This is the most common way to handle automated tests when they must be triggered by a continuous Integration system (such as Jenkins, Bamboo, TeamCity etc.) or any other third-party system.

In that case, the Continuous Integration system will start the XContinuousIntegration process on a specific agent.
This will automatically generate a new session and start it.
Then, an agent (probably the one on which we started XContinuousIntegration) will run the automated tests through the appropriate launchers and will store the results, logs etc. in XStudio's database.

For more details, pleaser refer to the dedicated section in the XContinuousIntegration documentation.

The complete process is shown in the schema below:

The third-party uses the unitary methods from the XStudio's REST API

When some automated tests must be executed by a continuous Integration system (such as Jenkins, Bamboo, TeamCity etc.) or any other third-party system you can also develop your own plugin to upload results, logs etc to XStudio's database.

Your plugin will have to rely on our REST API.

The main methods you may need to call are:

• authenticate
• getSessionInstanceId
• getSessionWebClientInstanceId
• updateSessionTestcaseResult
• updateSessionStartDateTime (Opt.)
• updateSessionStopDateTime (Opt.)
• updateSessionStatus (Opt.)
• uploadTestcaseExecutionAttachment (Opt.)

The complete process is shown in the schema below:

The third-party uses the REST API to import a full batch of results

When some automated tests must be executed by a continuous Integration system (such as Jenkins, Bamboo, TeamCity etc.) or any other third-party system you can also develop your own plugin to upload results from an Excel file.

Your plugin will have to rely on our REST API.

Only one call is necessary:

• authenticate
• importResults

The complete process is shown in the schema below:

The third-party uses the Results Publisher

If your tests must be executed directly by a third-party system and you don't want to develop a dedicated plugin, you can just use XResultsPublisher.

XResultsPublisher is a simple command line tool (Java program so working on ANY OS) easy to use. You just need to provide your credentials to connect and the results/comments to push to XStudio for a specific testcase in a session
More details in the Results Publisher documentation.

The complete process is shown in the schema below:

The third-party generates a parsable report and import it using the offline_generic launcher

If your tests must be executed directly by a third-party system and you don't want to develop a dedicated plugin, you can configure XStudio to manage the whole process for you.

In that case, the third-party system will execute the tests and generate a report. It will then copy the report on a XStudio instance.
The report must be parsable. different types are currently supported but it's easy to add new ones:

• JUnit
• TestNg
• Mockito
• EasyMock
• Tabular (test-result)
• Tabular (test-testcase-result)
• Tabular (test-error#)
• Tabular (test-testcase-error#)
• GitLab Alt.1
• GitLab Alt.2
• other formats can be easily added on demand

The third-party system will then start the XContinuousIntegration process on this specific agent that has access to the report.
This will automatically generate a new session and start it.
The agent will then run the automated tests through the "offline_generic.jar" launcher (which is aimed at only parsing a file in one of the above specific formats) and will store the results, logs etc. in XStudio's database.

The complete process is shown in the schema below: