Smarter kpis drive smarter decisions - part 2
Specifiability & Testability
Keywords:
Specifiability
Testability
Smart KPIs
Risk Management
Decisions
Specifiability
Testability
Smart KPIs
Risk Management
Decisions
Abstract: This paper addresses the need for
smarter metrics for managers to make the
right decisions during a Sprint, a project or a
program, but also to make the call when to ship
a new release in production or to consumers.
The vast majority of organizations still make such decisions based on SWAG 1 rather than documented metrics that matter. XStudio brings a set of automated and real time KPIs 2 to help in this.
This paper explains how this works using XStudio's specifiability (weighted specification coverage) and testability (weighted test coverage).
All this builds on the scope notion addressed in the first paper of this series (part 1) and describes the major advantages that this brings. Please ensure reading this paper first.
The third paper of this series (part 3) will expand the notion of scope, specifiability and testability to the quality aspects.
The vast majority of organizations still make such decisions based on SWAG 1 rather than documented metrics that matter. XStudio brings a set of automated and real time KPIs 2 to help in this.
This paper explains how this works using XStudio's specifiability (weighted specification coverage) and testability (weighted test coverage).
All this builds on the scope notion addressed in the first paper of this series (part 1) and describes the major advantages that this brings. Please ensure reading this paper first.
The third paper of this series (part 3) will expand the notion of scope, specifiability and testability to the quality aspects.
Summary
Introduction
In the the first paper of this series (part 1), we introduced the reasons why managers need to know more than just "are we
progressing fast enough?". We also explained in detailed how the weighted scope coverage KPI can tremendously help managers to make decisions during the project or sprint lifecycle.
In this paper, we address and extend this notion to cover the Specifiability and Testability KPIs.
In this paper, we address and extend this notion to cover the Specifiability and Testability KPIs.
Specifiability
XStudio does not impose to manage specifications. Although this is quite useful for many
projects, including those that are defined as "Agile".
When you get a User Story, you often need to define some aspects of the story: GUI
wireframing, basic workflows, data in and out, computation or decision rules...
You capture those using a large array of tools, from simple pen and paper to advanced modeling tools.
Many organizations still use full textual natural language documents.
Whatever your context and practices are, you may benefit from managing those as part of
your ALM and managing the traceability from Requirements to Specifications and Tests to
tests results.
Testability
This same issue exists when calculating the SUT's test coverage. As for the requirements coverage, XStudio handles this using:- specification's priority
- specification's status
- test's priority
- test's executability
Figure 1 : Cross coverage from requirement by specification
Describing the full detailed algorithm is beyond the scope of this paper but you get a basic idea in the previous paper.
The important information here is that for each SUT and SUT Folder you get 3 KPIs:
- Weighted Requirement Coverage (also named Scope)
- Weighted Specification Coverage (also named Specifiability)
- Weighted Test Coverage (also named Testability)
Figure 5: Weighted requirement coverage consolidation 1
Figure 6: Weighted specification coverage consolidation 2
Figure 7: Weighted test coverage consolidation 3
Using those three KPIs together, you can get a much better view of the projects progress status. As explained in the previous paper, being able to know where your coverage levels stand at any time during the project allows taking smarter decision. For example, here, if you are in a context where each functionality, each component must be tested, the following diagram (Figure 2) would alert you very rapidly... you are not quite there yet!
Figure 2: Test coverage progress
Therefore, in such case, you clearly must decide either to reduce the scope or to lengthen the project duration.
Figure 3: Reduced scope
Conclusion:
WE'VE SEEN 3 DIFFERENT BUT RELATED KPIS IN THE TWO FIRST PAPER OF THIS SERIES:
AT ANY TIME, YOU CAN ANALYZE THE TRUE STATUS OF YOUR PROJECT AND MAKE SMART DECISIONS. AT THE END OF ANY CYCLE (SPRINT, RELEASE, PROJECT), YOU HAVE THE TOOLS TO DECIDE WHETHER YOU WANT TO PUSH THIS TO PRODUCTION OR SHIPPING IT TO YOUR CUSTOMERS.
THOSE KPI ARE CALCULATED REAL-TIME AND THE BASIC PARAMETERS ARE ALL CONFIGURABLE, MAKING IT EASY TO ADAPT TO ANY ORGANIZATION. WE'LL SEE IN THE NEXT WHITE-PAPER HOW WE CAN BENEFIT FROM THOSE METRICS TO GENERATE A NEW ONE: THE QUALITY SCORE.
- SCOPE
- SPECIFIABILITY
- TESTABILITY
AT ANY TIME, YOU CAN ANALYZE THE TRUE STATUS OF YOUR PROJECT AND MAKE SMART DECISIONS. AT THE END OF ANY CYCLE (SPRINT, RELEASE, PROJECT), YOU HAVE THE TOOLS TO DECIDE WHETHER YOU WANT TO PUSH THIS TO PRODUCTION OR SHIPPING IT TO YOUR CUSTOMERS.
THOSE KPI ARE CALCULATED REAL-TIME AND THE BASIC PARAMETERS ARE ALL CONFIGURABLE, MAKING IT EASY TO ADAPT TO ANY ORGANIZATION. WE'LL SEE IN THE NEXT WHITE-PAPER HOW WE CAN BENEFIT FROM THOSE METRICS TO GENERATE A NEW ONE: THE QUALITY SCORE.
References:
1 SWAG: https://en.wikipedia.org/wiki/Scientific_wild-ass_guess
2 KPI: https://en.wikipedia.org/wiki/KPI
1 SWAG: https://en.wikipedia.org/wiki/Scientific_wild-ass_guess
2 KPI: https://en.wikipedia.org/wiki/KPI
Figures:
Figure 1: Cross coverage from requirement by specification
Figure 2: Test coverage progress
Figure 3: Reduced_scope
Figure 1: Cross coverage from requirement by specification
Figure 2: Test coverage progress
Figure 3: Reduced_scope