QASymphony / Blog / What testers look for and expect from a test management solution – Part II
What testers look for and expect from a test management solution – Part II
Read part II of this series and hear what real world testing practitioners have to say about their needs.
Our conversation with real world testing practitioners are ongoing. It’s embedded in our DNA to stay tied in to the audience that we serve to ensure that we are continuing to bring them value. Through our conversations we have collected even more insights into what testers deem as critical to the testing solutions they use.
1. Ability to integrate with third party defect tracking or ALM solutions
Anyone who has worked on the implementation of new testing tools such as test management or defect tracking tools knows how difficult it can be to demonstrate to your development team the value of those tools and convincing them to accept them. Development needs to see the direct impact/value to their daily work of designing and coding the software. If development is comfortable with their existing defect tracking solution, the likelihood they will want to change is low. Without the ability to integrate with an external defect tracking or ALM solution, the test team is limited which makes the need for seamless integration into the systems used by the development side of the house critical for maximum collaboration. Integration ensures that the testing team’s testing efficacy is not impacted and that they can perform full test management while still collaborating with their development team on defects that are detected.
2. Ability to integrate with test automation tools or automation frameworks
Automation/performance testing is a critical part of quality assurance processes so when a company is evaluating a new test management tool one critical selection criteria is the ability of tool to integrate with their existing (or soon-to-be-built) test automation solution. There are several key benefits, that were brought up in conversations, for a test management tool that supports automation integration:
Test Setup – manual and automated tests need to be able to be setup on the same tool. This allows QA to quickly identify which test cases are automated so they can easily identify which areas still need to be tested and easily assess the level of effort that remains for manual testing.
Test Execution – Once automated test cases are identified in a test run, testers need the ability to enable the kick-off of the automated tests from within the test management solution. They then need the ability to interact with the automation framework to execute tests on a specific environment and then provide the live test results back into the test run.
Test Coverage Analysis – With manual and automated tests being managed in a single tool, the test team can quickly analyze the test coverage and make the determination of whether or not testing for a build or specific feature has been performed at the desired coverage level.
Reuse Reporting Feature – With an integration to the test management tool, the test automation framework needs to be able to leverage the reporting, metrics, dashboard functionality of the test management tool. The consolidated view of reporting is particularly beneficial to the QA tester and also keeps them from having to develop a custom reporting view.
3. Visualization to support decision making
In the movement to Agile, testers are under even higher pressure to deliver a high quality product on time. In a lot of situations teams have very narrow timelines to review all the reports and metrics provided by their test management tool which makes visual tools even more important. Here is an example of when more is not necessarily better. Almost all commercial testing solutions offer some form of visual metrics and reports, however, there are big gaps between tools. Some tools provide visualization and charts for every metric. This doesn’t lessen the burden on the tester and doesn’t enable them to make quick, yet informed decisions. In the end it is still the same amount of review time and analysis that needs to be performed and in many cases ends up being confusing. From the point of view of the testers we spoke to, testers don’t really need dozens of metrics and reports; they would benefit more from one consolidated report that provides them insights into the current status of the release, cycle, test runs and defects.
With the continued movement from waterfall to agile-based development practices, there are higher demands for not only the skill sets of testers but also the creative features of the tools they use. Testing organizations are looking for tools that help them become more productive in their job and best fit the organization’s processes and practices. Today’s testing tools require not only the capabilities of traceability, defect tracking and test case management but also extensibility to seamlessly integrate with external tools such as defect tracking systems, ALMs and automation tools – both commercial and open source. Testing is part of a cycle and as such testers demand that the testing tools they use help them easily collaborate and work with other departments within their organization.