QASymphony / Blog / Where qTrace can save your team’s effort and how to measure its saving
Where qTrace can save your team’s effort and how to measure its saving
The decision on acquiring a software productivity tool depends on many factors such as its adequate functionality, ease of use, vendor support, and performance. But in the end, the most important factor perhaps is whether it can help improve the overall productivity of the software development team, i.e., whether it can save time when using versus without using it. If a tool does not help improve the productivity of the current practice, then it is hard to justify its use.
qTrace is designed to be used mainly by software testers. But it does not only help save the time spent by software testers alone. The clarity of its bug reports increases the effectiveness and efficiency of team communications, which results in the time saving for software developers and team leads who investigate, fix, and manage bugs. Thus, qTrace users may overlook its benefits if they evaluate the tool with only software testers.
Let’s consider a typical bug lifecycle that starts when a bug is found until it is closed. And let’s consider fine-grained activities taken during this lifecycle. The flow chart of this lifecycle is shown in Figure 1 and described in Table 1. The first column in Table 1 lists activities performed; the second column represents roles involved in each activity; and the last column indicates the percentage of time reduction that qTrace is expected to produce as compared to the manual practice. These percentages are based on my experience after performing several controlled experiments to evaluate an experimental version of qTrace.
While qTrace is used to record and submit bugs, it is expected to reduce the time spent on this activity. The time reduction is expected to be between 30% and 60%, the highest percentages among the bug lifecycle activities. As the tool can help improve the quality of bug information, it indirectly decreases the time needed to verify, review, investigate, reproduce, and elaborate the bug. Developers are expected to spend less time on investigating and reproducing the bug, given that the bug report generated by qTrace captures exactly what has happened with the bug. Time reduction on activities #1, #3, #5, and #8 is none to marginal as these activities have little to do with the bug report generated by qTrace.
Figure 1 Flow Chart of a Typical Bug Lifecycle
Table 1 Bug Lifecycle Activities
Measuring Time Saving and Productivity Gain
To measure the time saving and productivity gain when using qTrace in your organization, you can either collect historical effort data of completed projects or perform a controlled experiment to generate data for comparison. In the first method, effort data is collected by using timesheets which include actual time distributed into different activities of the bug lifecycle. The productivity can be computed as the number of resolved bug divided by the total time spent on the bug lifecycle activities to handle these bugs. You can then compare the productivities of the projects using qTrace and the projects not using qTrace. As this method requires actual data from your projects, you need to use qTrace for an extended period of time in order to collect sufficient data. However, its advantage is that it produces highly accurate results.
The second method, which is faster than the first method to obtain the time saving and productivity gain, is to set up a controlled experiment. Two groups of three or more testers and developers are formed. One group does the tasks using qTrace, the other not using qTrace. Each group is asked to record, submit, and fix the same set of bugs – the activities taken by the participants should be the same as those of your actual projects. The capability of the participants on the two groups should be balanced. The time spent by each group is recorded. Although this method is fast, it requires careful design and execution to obtain reliable results.
It is essential to evaluate a software tool before using it on your software projects. The evaluation methods used may range from subjective reviews of software’s attributes such as ease of use, performance, reliability, and robustness to more formal approaches such as surveys and measurement. Regardless of which evaluation methods are used, evaluating the productivity gain from a productivity tool like qTrace is mistakable to neglect.