The SUTs

Here is a typical SUTs tree. It displays a number of useful information:
  • the version of each SUT
  • the functional coverage of each SUT/folder/company (by requirements, by specifications and by tests)
  • the number of links determining the functional coverage
  • the quality index of each SUT/folder/company
  • the number of successful results/total test cases determining the quality index
  • the total number of SUTs
  • the number of SUTs in each company/folder

Highslide JS

A SUT is an abstract object representing the target we want to test. It can be a hardware device, a software component etc. The SUT must be detailed enough so that we can identify it easily. A SUT should be uniquely defined through its name and version.
It is possible to group/organize the SUTs by creating a structure of companies, folders and sub-folders hosting the SUTs.

Coverage Metrics

In order to have a quantifiable follow-up of the coverage, the SUT (system under test) tree includes a coverage column showing at a glance the coverage of each SUT in the tree and the number of requirements/specifications/tests covering the SUT. The coverage is also consolidated at the folder/company level.


The coverage indicator provides the estimation of completeness in terms of requirements, specifications or tests. The figure on the right-hand side of the coverage progress bar indicates how many links are linking the SUT to some requirements, specifications or tests. If the SUT is linked several times to the same requirement, specification or test through different path, it is counted as different links.


Coverage column



Depending on the type of coverage requested (through the Coverage drop-down menu), the tree will include different kinds of coverage:

By requirements

The SUTs are directly covered by requirements. To view the coverage by requirements:
  • click on the drop-down list Coverage and select
  • the SUT coverage and the number of requirements linked to this SUT are displayed


Highslide JS

The coverage of each SUT in the tree is directly taken from the value in its Details tab.


By default, when a SUT is being created, a 0% value will be assigned to it. As soon as one requirement is linked to the current SUT, the coverage indicator of the current SUT will switch automatically to 100%. However, you can edit it manually refining the coverage value between 0% and 100%. For instance, if you think that you will need approximately 10 requirements to cover the SUT and that you already have linked 2 of them, just set the coverage to 20%.


NOTE: This feature is what will allow you to get truely reliable coverage metrics.


NOTE: If no requirement is linked to the current SUT, you cannot edit manually the coverage indicator. If you try to edit it anyway, the following message will popup:

Highslide JS


To edit the coverage indicator
  • select a SUT
  • click on Details menu
  • set the value of Coverage by requirements (%) field
  • click on submit
  • the SUT coverage is modified in the tree


Highslide JS

By specifications

The SUTs are indirectly covered by specifications. To view the coverage by specifications:
  • click on the drop-down list Coverage and select
  • the SUT coverage and the number of specifications linked to this SUT are displayed


Highslide JS

The coverage of each SUT is calculated based on:
  • its coverage by requirements,
  • the coverage by specifications and the weigth of each requirement linked to the SUT.





The weigth of each requirement being calculated from its priority using this simple mapping:
PriorityWeigth
Low1
Normal2
High3


By tests

The SUTs are indirectly covered by tests. To view the coverage by tests:
  • click on the drop-down list Coverage and select
  • the SUT coverage and the number of tests linked to this SUT are displayed


Highslide JS

The coverage of each SUT is calculated based on:
  • its coverage by requirements,
  • the coverage by specifications and the weigth of each requirement linked to the SUT,
  • the coverage by tests and the weigth of each specification linked to the previous requirements.





The weigth of each specification being calculated from its priority using this simple mapping:
PriorityWeigth
Low1
Normal2
High3


Coverage aggregation

For each folder the coverage is calculated by aggregation, based on the coverage of its underlying objects.


For instance, the coverage by requirements of a folder will be calculated using the formula:




Quality Metrics

In order to have a quantifiable follow-up of the quality, the SUTs tree includes a coverage column showing at a glance the estimated quality of each SUT in the tree.
The quality is also consolidated at the folder/company level.


The quality indicator provides the estimation of the percentage of success of the latest executions of the tests functionally covering the SUT. The figure on the right-hand side of the quality progress bar indicates how many links are linking the SUT to its successful/total test cases. If the SUT is linked several times to the same test/test case through different path, it is counted as different links.



Coverage column

For each node of the corresponding trees (folders), quality will be calculated taking into account the share of successful related tests.

The quality indicator will be an integer with possible values from 0 to 100.
  • 0% value means no test linked to the current object is successful.
  • 100% value means that coverage is optimal and all linked tests are successful.


The generic quality formula for an object is:



Searching

By name

All the trees in XStudio include an indexing system allowing to search real-time an entity in the tree by his name. To experiment it by yourself just start entering some text in the Search field of the tree

Highslide JS


To automatically select a searched item in the tree, you can move using the arrows keys within the suggestion list and validate by pressing Enter or just click on the item in the list.

By Id

The data model gives a unique identifier for each entity managed by XStudio. The Ids are on the form:

Template Represents
SU_<id> SUT


Ids are also available from XStudio’s GUI (in the Details tab there is an Identifier field that corresponds to the <id> in the template.

To search for an entity from its Id, just type the Id (i.e. SU_123) in the Search Id field and validate.

The root node

Tabs

Custom fields

You can customize the SUT by creating some new custom fields that will appear in the Details tab of the SUT.
Custom fields can have different types: text fields, drop-down menu, checkbox, formatted text field. They can also be marked as mandatory or not and are positionned at the index you specify.

The Custom fields tab allows you to create, edit and delete custom fields.
Highslide JS

Create custom field

Here is the process to create a custom field for a SUT:
  • select the root folder in the tree
  • on the right panel, click on the create custom field button
  • enter the name, the type, the position, if it's mandatory or not and the default value of the custom field and submit
    Highslide JS
  • the custom field will immediately appears in the panel
  • the new custom field is added to all the SUTs assigned with its default value


Highslide JS Highslide JS

Edit custom field

Here is the process to edit a custom field for a SUT:
  • select the root folder in the tree
  • on the right panel, select the custom field
  • click on the edit custom field button
  • change the name, the type, the position, if it's mandatory or not and/or the default value of the custom field and submit
    Highslide JS
  • the custom field immediately appears in the panel with new values
  • the custom field is updated on all the SUTs (except its value)


Delete custom field

Here is the process to delete a custom field for a SUT:
  • select the root folder in the tree
  • on the right panel, select the custom field
  • click on the delete custom field button
  • confirm and submit
    Highslide JS
  • the custom field immediately disappears from the panel
  • the custom field is removed from all the SUTs


Toolbar

Create a company

A company can be created only under the root node of the tree.


Here is the process to create a new company:
  • select the root folder in the tree
  • on the right panel, click on the create company button
  • enter the name of the company and submit
  • the company immediately appears in the tree


Create a report

It may be interesting to export or to print a global report on all the SUTS you have.

To do so:
  • select the root folder in the tree
  • click on the Create report button
  • select the destination folder for the report
    Highslide JS
  • select the extension of the report (HTML or XML).
    If you choose to generate the report in HTML you can select one report template (Type drop-down menu).
    To each template corresponds one specific XSLT transform (one physical .xslt file).
    By default, one XSLT is provided along with XStudio:
    Complete_With_stats
    Highslide JS
  • provide the file name for the report (Save as text field)
  • submit
  • the report is automatically opened:
Highslide JS

TIP: Custom reports
You can customize the SUT reports.
To do that, just edit the following .xslt files located in <XStudio install dir>/export/xsl/ folder (or if you use the web package in /export/xsl/ of the export.zip archive on the server):
  • sutHTML_Lib.xslt: SUT XSLT library
  • sutHTML_Complete_with_stats.xslt: Complete report on the SUTs with statistics
You can also add your own template if you follow the naming convention of the xslt file.


The company nodes

Tabs

Details

The company details tab allows you to specify the name of the company


Attachments

You have the ability to attach some files (whatever the format is) to some entities managed by XStudio.
This is an extremely powerful way of centralizing/sharing documents in a given context.
The GUI is extremely simple:

Highslide JS

Two panels are displayed: one for attachments directly attached to the entity and another one for the attachments inherited from the ancestor nodes. This is allowing to avoid duplication and share some document among several objects. It is possible to move directly to one of these parents by clicking on the anchors.

Add an attachment
To add an attachment:
  • click on the Create attachment button
  • a dialog box is displayed
  • pick a file and submit


Download/Open an attachment
To download/open an attachment:
  • select an item in the list
  • click on the Download attachment button or Open attachment button


Open and lock an attachment
When a user needs to modify a document, nobody on the network should be able to modify it at the same time.
This is the role of the locking system. When the user locks a document, the file is still accessible and readable by the other users but it cannot be modified. Hence, the locking mechanism prevents any editing conflicts.

To lock and modify an attachment:
  • select an item in the list
  • click on the Open and lock attachment button
  • the file open using the default program associated to the file extension
  • the file can be edited and saved (it's saved only on the local disk in a temporary folder at this point)


Commit and unlock an attachment
When you're done with your modification on a document. You just need to save it and commit it to the repository. In addition, you'll need to unlock it so that someone else can eventually edit it later on.

To commit and unlock an attachment:
  • select an item in the list
  • click on the Commit and unlock attachment button
  • the file being edited (and which has been saved locally) is commited to the XStudio database


Toolbar

Delete a company

You can delete a company by just clicking this button and confirming the deletion.
WARNING: deleting a company will delete all objects linked to this company (i.e. underlying folders and SUTs, sessions, users etc.)


Create a folder

A folder can be created only under a company node or another folder node.


Here is the process to create a new folder:
  • select a company node in the tree
  • on the right panel, click on the create folder button
  • enter the name of the folder and submit
  • the folder immediately appears in the tree


Create a report

It may be interesting to export or to print a report on all the SUTS belonging to a company.

For more information check the process to generate a report chapter from the root node section.

The folder nodes

Tabs

Details

The folder details tab allows you to specify the name and the description of the folder


Highslide JS

Attachments

Attachments on SUT are handled similarly to the attachments on the companies.

Toolbar

Create a folder

A folder can be created only under a company node or another folder node.


Here is the process to create a new folder:
  • select a company node in the tree
  • on the right panel, click on the create folder button
  • enter the name of the folder and submit
  • the folder immediately appears in the tree


Delete a folder

You can delete a folder by just clicking this button and confirming the deletion.
WARNING: deleting a folder will delete all objects linked to this folder (i.e. underlying folders and SUTs, sessions etc.)


Empty a folder

You can empty a folder by just clicking this button and confirming the emptying (all child nodes will be deleted).
WARNING: emptying a folder will delete all objects linked to its childs (i.e. underlying folders and SUTs, sessions etc.)


Move a folder

You can move a folder by just clicking this button, selecting the destination folder and submitting.


Create a SUT

A SUT can be created only under a folder node.
Here is the process to create a new SUT:
  • select a folder in the tree (create one if necessary)
  • on the right panel, click on the create sut button
  • enter the name and the version of the SUT
  • (opt.) enter the value of all the custom fields
  • submit
  • the SUT immediately appears in the tree


Create a SUT Inheriting requirements from another SUT

When you create a new version of a SUT, it's very likely that it will be connected to almost the same requirements than the previous version. Here is the process to create a new SUT inheriting requirements from another SUT:
  • select a folder in the tree(create one if necessary)
  • on the right panel, click on the create sut button
  • enter the name and the version of the SUT
  • select the Requirements tab
  • click the Preset import from SUT settings button
  • select the reference SUT and submit
  • click the Preset import from SUT button
  • the relevant requirements are by default pre-selected
  • (opt.) select/unselect some requirements if required
  • submit
  • the SUT immediately appears in the tree

Highslide JS Highslide JS

Create a report

It may be interesting to export or to print a report on all the SUTS under a specific folder.

For more information check the process to generate a report chapter from the root node section.

The SUT nodes

Tabs

Details

The SUT Details tab allows you to specify the name, the version of the SUT
If some custom fields have been configured, they will appear here as well (positionned as specified).


Highslide JS

Coverage

The SUT Coverage tab will display the coverage details of the selected SUT in terms of requirements, specifications (if enabled) and tests.
In addition, some advises are given to improve the coverage of the SUT:

Highslide JS Highslide JS

To modify the list of requirements linked to the current SUT, just un-toggle the tree filter (to show the currently not selected requirements), select/unselect requirements from the tree and submit.

WARNING: You can't directly edit the specifications or tests linked to this SUT (these trees are displayed read-only). This must be done through the requirements and the specifications intermediates.


Bugs found

This tab shows all the bugs that have been found on the selected SUT.
Bugs may have been found on this SUT from two different ways:
  • either they have been submitted independently and manually from the bugs tree,
  • or they may have been found through a failed test and link to this test execution.


Highslide JS Highslide JS

Bugs fixed

This tab shows all the bugs that have been marked as fixed on the selected SUT.


Highslide JS

Attachments

Attachments on SUT are handled similarly to the attachments on the companies.

Revisions (versioning)

This tab shows all the revisions of the selected SUT.
You can open a specific revision of the SUT by selecting a revision in the list and clicking on the open button.

Highslide JS

Changes (audit log)

XStudio allows tracking any changes on a particular object.


To check the history of a SUT:
  • select the SUT in the tree
  • open the Changes tab on the right panel
  • the panel shows a table gathering all the dated changes.


On a SUT you will get a panel similar to this one:

Highslide JS

TIP: You can edit directly the details of a user by clicking on his name in the changes table.


Toolbar

Delete a SUT

You can delete a SUT by just clicking this button and confirming the deletion.
WARNING: deleting a SUT will delete all objects linked to this SUT (i.e. sessions etc.)


Move a SUT

You can move a SUT by just clicking this button, selecting the destination folder and submitting.


Create a report

It may be interesting to export or to print a report on just one specific SUT.

For more information check the process to generate a report chapter from the root node section.