I’ll never forget having to work on death march projects. I went in every morning ready to do good work, and only found a never-ending pile of old and out of date test scripts. Each test told me exactly what to click, and to check for, and when I was done. I was not engaged in the work. Those exciting moments where I actually did find important bugs usually had nothing to do with the script I was half-heartedly working through. It takes time to get into a problem like this; it also takes a lot of test documentation.
How can we spend less time documenting our work, and more time actually testing?
Shifting From Test Cases
Most people dislike abrupt change. Pitching the idea of working with test cases all day, every day most likely won’t go well. I know that from experience. What does work well is small changes; those changes result in better testing and, ultimately, a better product.
When working with detailed test cases, the first place to start is to generalize the test scripts. Instead of writing a test case for each possible way of handling a date field — good date, future date, past date, bad date — I wrote one script with a note for the types of data I’d recommend using. Doing this prevented test scripts from ever existing and gave some creative license to the people who used them later.
Once that style caught on, I took it one step further by using session notes. These notes come at the end of testing and describe the work that was actually done rather than prescribing the steps ahead of time like a test script. Instead of step by step descriptions, I ended up with something like this:
“I spent 30 minutes testing the new date field. I used future and past dates, good dates, misformatted dates, and dates that were in the acceptable range. I noticed that dates in the future were accepted when they shouldn’t be (bug ID #12345) and that a misformatted date caused a javascipt error to be thrown in the UI (bug ID #5678). I ran out of time, but would like to see how this value is used in other parts of the software.”
My preference is to make documentation as fast and painless as possible. It’s easy to get focused on the idea that test documentation lives in Excel spreadsheets, Word documents, and heavyweight test case repositories, but there are plenty of other options out there.
If you want something in digital format that can be saved on a network drive or emailed around, a mindmap might do the trick. QASymphonys qTest Insights has several good is a great example of a tool using mindmaps to work through test ideas. I have used mindmaps more than once as a way to keep track of bugs, figure out what test coverage looks like, and collaborate with others and figure out what important things are missing from the test plan.
If mindmaps aren’t your style, there is always the classic pen and paper.
What About Bugs?
After test cases, bug documentation is probably the next largest time expenditure. Bug reports are a gamble. I can spend an hour carefully researching and isolating the problem. looking through log files, and carefully describing how I made the bug happen, only to create a bloated report which no one wants to read. Alternately, I can keep it light and quick and risk leaving out critical information. That leaves us with a report that moves back and forth between development and test while we try to get the right information.
Reporting is an art. Practicing and getting feedback makes perfect.
On some teams, you can avoid the ticketing system altogether. When possible, instead of collecting data into a bug tracking system, I’ll see if a developer has a minute to watch me demonstrate the bug. If he or she can fix it right away, we leave it at that and move on.
Every minute you spend writing test cases, test ideas or bug reports is time you could have used in testing and finding the next problem.
What types of documentation will you drop in order to spend more time testing?