There is often a lack of traceability in many of today’s organizations and the reality is, inadequate traceability can be one of the major factors to software project failures and budget overruns. Because of this, there has been a flood of research and thought leadership across the industry on the subject of traceability, and many testing organizations have been striving to improve their traceability practices.
In this article, we examine what exactly is traceability, why is it important to managers and testers, along with ways QA/testing organizations can work to improve traceability.
What exactly is traceability?
To start, traceability is linking all test assets from one stage to another as you progress through your testing project. Looking below, a typical testing process usually contains these important steps:
Gather and analyze requirements. Perform a feasibility study and decide which of these requirements can or can’t be tested and lastly, finalizing the testing scope.
Chart the tentative scenarios for each requirement.
Develop test cases by adding detailed test steps, data and expected results for each scenario.
Following the design, the test cases are run on the AUT and test execution status is captured. Defects are logged for failed test cases.
Linking all of these stages to keep track of the originating requirements all the way to tests and defects is a tester’s conscious efforts towards traceability.
Why is this important to you?
If you’re a tester:
Establishing traceability will ensure that each requirement is tested at least once. It helps show gaps in test design that can be addressed.
Makes sure that no test element in missed when transitioning between requirements to test scenarios to test cases.
Extending the traceability all the way to execution will confirm if the requirements have been tested completely or not.
When further extended until defects, they help us trace the defect all the way back to the originating requirement, thus keeping the application in perspective while dealing with individual bugs.
If you’re a manager:
From a managerial stand point, understanding and measuring progress of the project & quality of the product is critical. Here are a few ways in which traceability helps:
Traceability is the only tangible way to be confident that the product would be or in fact is tested enough.
Measuring Quality & Progress: Metrics can sometimes be elusive. For example, when 90% pass rate is reported for a said testing engagement, it often means the product is faring well. But let’s say for the most critical requirement the pass rate is merely 40%, that would then indicate poor quality. It is possible to arrive at this sort of per requirement quality statistics by setting up traceability for each of the requirements in scope.
Similar assessments of progress also become easier.
Defect density per requirements also would be apparent when we leverage traceability all the way back to the specifications.
How to improve traceability?
Improving traceability can be accomplished in many ways, but here are a few easy way to get started:
For projects that don’t use test/application lifecycle management tools, a simple table/matrix can be drawn (may be in MS excel) with requirements, Test scenarios, Test cases, Status, defect ID as the columns and the data filled correspondingly.
Projects could both start their testing process with a traceability matrix and use it as an index from which everything else would stem out. Or design as normal and consolidate all test assets into a matrix at the end. The former approach is called Forward traceability and the later of course, backward traceability. Either way, the resulting traceability matrix is the same and helps check for 100% test coverage and tracks the defects back to the functional elements of the application.
When a test management/application life cycle management tool is used, traceability becomes an integral part of creating & managing the requirements, adding tests corresponding to them, execution and defect reporting. Tools also provide requirement coverage analytics that makes it easier to identify orphan requirements or test cases if any.
Perhaps one of the biggest takeaways is that traceability improves confidence on our test design and aligns our testing efforts to reach the 100% requirement coverage goal. The advantages being abound, building traceability is a simple yet beneficial aspect of effective test management. A few minutes of effort could equate to tons of confidence and clarity.