We love product webinars, and the release of qTest Insights 2.0 is one of our biggest launches of the 2016 year, and we still have more to come! In this webinar, Ryan Yackel, QASymphony’s Senior Product Engineer will demonstrated the following feature metrics in software testing with qTest Insights 2.0:
Customized Dashboards for cross project reporting
Heat map visualizations to visualize risk
Interactive charts to simplify data drill downs
Shareable reporting for executive level visibility
Software testing metrics are hard to come by. That’s the general consensus we got from our Software Testing Metrics with qTest Insights 2.0 webinar. We had a feeling what the response was going to be when we asked the audience “What is your biggest challenge with software testing metrics”? While some said that reports were hard to create, and the data, by large response most checked the box for “All of the Above.”
This is why we have listened to our customers to provide out-of-the-box reports around test quality, coverage and velocity.
Core Metrics in Software Testing
The core areas of analysis that QA directors, team leads and project managers care about for software quality assurance are quality, coverage, and velocity. All three provide different insights into the application health of the systems under test, but they also build upon one another. qTest Insights 2.0 reporting comes standard with these three analysis reports. Learn more about each report.
Quality Analysis shows a results driven reporting model that can gauge the quality of the application at different stages in a testing project (by project, by release, by cycle, by application area etc.). QA directors need to know the quality of their pre-release software to feel confident about releasing the product into production.
qTest Insights provides real-time, test run results and current defect status’ at these different stages in the application life cycle to insure this confidence. Also, qTest Insights 2.0 will be rolling up session and exploratory testing data collected through qTest eXplorer.
Once results are summarized and tracked, the next question is “What areas of my application are being covered by these test results?” For example, I have new product features that need to be tested along with an entire stack of regression tests that need to be executed. There is never enough time to exhaustively test every part of your application before a production release, and there is a good change some bugs will leak into production (think back to your latest operating system update on any device).
With qTest Insights 2.0 Coverage Analysis report, directors and test teams can get heat map visualization on risky application areas and pinpoint what areas have NOT been tested. This allows them to quickly reallocate resources to troubled areas of the application or divert testing from areas that are being over-tested.
Even if we are getting good quality results and covering our most risky areas of the application, the rate at which we do this still may be poor and less optimal. For example, our test teams could have ironed out ten bugs and passed 100 tests in the last four days. However, we still have 100 more tests left that we scoped and only one day to complete them.
Therefore, a Velocity Analysis is needed to measure the speed at which our teams test the covered areas of the application. Velocity analysis reports, at the team and individual level, help identify if the executed tasks will meet the scoped expectations. qTest Insights 2.0 can forecast completion dates for test teams and estimated defects still not found based off embedded predictive analytics.
Webinar Questions Recap
On the Test Run Logs Data Drilldown page, is it possible to click on a failed test run vs. the test case to be redirected to the actual failed run?
Yes. Since qTest Test Case Management allows you to have multiple test runs associated with a test case, you can use insights to see total runs and total run logs through exploring test data.
How would I create a custom report? Is that even possible with qTest Insights 2.0?
Custom reports can be created by exploring data on the left navigation panel based of the data you want to see. Once you filter, sort and group you data, you can then create formulas, charts and crosstabs that can be added to your shared or personal dashboard.
If I have JIRA issue types tied into your qTest test case management, how are these bugs displayed in qTest Insights 2.0?
If you are connected with JIRA, VersionOne, or CA Agile central, you can see visualize all your externally tracked defects through reports labeled with the (linked) identification. Below you can see I have a (linked) defect report that contains by JIRA status’ and links externally based of my JIRA key.