The SUTs

Coverage Metrics
Searching
Quality Metrics
The root node
- Tabs
  - Coverage
  - Custom fields
- Toolbar
  - Create a company
  - Create a report

The SUT nodes
- Tabs
- Toolbar

Here is a typical SUTs tree. It displays a number of useful pieces of information:

the version of each SUT
the functional coverage of each SUT/folder/company (by requirements, by specifications and by tests)
the number of links determining the functional coverage
the quality index of each SUT/folder/company
the number of path leading to a successful test case/total number of path to testcases determining the quality index
the total number of SUTs
the number of SUTs in each company/folder

A SUT is an abstract object representing the target we want to test. It can be a hardware device, a software component etc. The SUT must be detailed enough so that we can identify it easily. A SUT should be uniquely defined through its name and version.
It is possible to group/organize the SUTs by creating a structure of companies, folders and sub-folders hosting the SUTs.

Coverage Metrics

In order to have a quantifiable follow-up of the coverage, the SUT (system under test) tree includes a coverage column showing at a glance the coverage of each SUT in the tree and the number of requirements/specifications/tests covering the SUT. The coverage is also consolidated at the folder/company level.

The coverage indicator provides the estimation of completeness in terms of requirements, specifications or tests. The figure on the right-hand side of the coverage progress bar indicates how many links are linking the SUT to some requirements, specifications or tests. If the SUT is linked several times to the same requirement, specification or test through different path, it is counted as different links.

Coverage column

Depending on the type of coverage requested (through the Coverage drop-down menu), the tree will include different kinds of coverage:

By requirements

The SUTs are directly covered by requirements. To view the coverage by requirements:

click on the drop-down list Coverage and select
the SUT coverage and the number of requirements linked to this SUT are displayed

The coverage of each SUT in the tree is directly taken from the value in its Details tab.

By default, when a SUT is being created, a 0% value will be assigned to it. As soon as one requirement is linked to the current SUT, the coverage indicator of the current SUT will switch automatically to 100%. However, you can edit it manually refining the coverage value between 0% and 100%. For instance, if you think that you will need approximately 10 requirements to cover the SUT and that you already have linked 2 of them, just set the coverage to 20%.

NOTE: This feature is what will allow you to get truely reliable coverage metrics.

NOTE: If no requirement is linked to the current SUT, you cannot edit manually the coverage indicator. If you try to edit it anyway, the following message will popup:

To edit the coverage indicator

select a SUT
click on Details menu
set the value of Coverage by requirements (%) field
click on submit
the SUT coverage is modified in the tree

By specifications

The SUTs are indirectly covered by specifications. To view the coverage by specifications:

click on the drop-down list Coverage and select
the SUT coverage and the number of specifications linked to this SUT are displayed

The coverage of each SUT is calculated based on:

its coverage by requirements,
the coverage by specifications and the weigth of each requirement linked to the SUT.

The weigth of each requirement being calculated from its priority using this simple mapping:

Priority	Weigth
Low	1
Normal	2
High	3

By tests

The SUTs are indirectly covered by tests. To view the coverage by tests:

click on the drop-down list Coverage and select
the SUT coverage and the number of tests linked to this SUT are displayed

The coverage of each SUT is calculated based on:

its coverage by requirements,
the coverage by specifications and the weigth of each requirement linked to the SUT,
the coverage by tests and the weigth of each specification linked to the previous requirements.

The weigth of each specification being calculated from its priority using this simple mapping:

Priority	Weigth
Low	1
Normal	2
High	3

Coverage aggregation

For each folder the coverage is calculated by aggregation, based on the coverage of its underlying objects.

For instance, the coverage by requirements of a folder will be calculated using the formula:

Quality Metrics

In order to have a quantifiable follow-up of the quality, the SUTs tree includes a coverage column showing at a glance the estimated quality of each SUT in the tree.
The quality is also consolidated at the folder/company level.

The quality indicator provides the estimation of the percentage of success of the latest executions of the tests functionally covering the SUT. The figure on the right-hand side of the quality progress bar indicates how many links are linking the SUT to its successful/total test cases. If the SUT is linked several times to the same test/test case through different path, it is counted as different links.

Coverage column

For each node of the corresponding trees (folders), quality will be calculated taking into account the share of successful related tests.
The quality indicator will be an integer with possible values from 0 to 100.

0% value means no test linked to the current object is successful.
100% value means that coverage is optimal and all linked tests are successful.

The generic quality formula for an object is:

Searching

For more details, refer to the Searching object chapter.

The root node

Tabs

Coverage

The Coverage tab will display the coverage details of the root folder in terms of Requirements, specifications (if enabled) and tests.
In addition, some advises are given to improve the coverage of the root folder:

You can customize the SUT by creating some new custom fields that will appear in the Details tab of the SUT.
Custom fields can have different types: text fields, drop-down menu, checkbox, formatted text field. They can also be marked as mandatory or not and are positionned at the index you specify.
The Custom fields tab allows one to create, edit and delete custom fields.

Create custom field

Here is the process to create a custom field for a SUT:

select the root folder in the tree
on the right panel, click on the create custom field button
enter the name, the type, the position, if it's mandatory or not and the default value of the custom field and submit
the custom field will immediately appears in the panel
the new custom field is added to all the SUTs assigned with its default value

Edit custom field

Here is the process to edit a custom field for a SUT:

select the root folder in the tree
on the right panel, select the custom field
click on the edit custom field button
change the name, the type, the position, if it's mandatory or not and/or the default value of the custom field and submit
the custom field immediately appears in the panel with new values
the custom field is updated on all the SUTs (except its value)

Delete custom field

Here is the process to delete a custom field for a SUT:

select the root folder in the tree
on the right panel, select the custom field
click on the delete custom field button
confirm and submit
the custom field immediately disappears from the panel
the custom field is removed from all the SUTs

Toolbar

Create a company

A company can be created only under the root node of the tree.

Here is the process to create a new company:

select the root folder in the tree
on the right panel, click on the create company button
enter the name of the company and submit
the company immediately appears in the tree

Create a report

It may be interesting to export or to print a global report on all the SUTS you have.
For more details, refer to the SUTs report chapter.

The company nodes