This is a question I often hear testers ask. In this post, I will share 2 rules for testers to consider when determining the amount of details their test scripts should have.
Rule 1: The level of detail of test scripts depends on the context of the project
Some might be disappointed that I call this a “rule”, but I’m afraid there is no hard and fast rule that can be applied regardless of the context of your project. So, what are the kinds of things you need to consider? First and foremost, the audiences. You need to ask yourself who the audiences of your test scripts are and what they are going to do with them.
Most of these audiences are likely other testers who need to execute these scripts. Test scripts should generally contain more details if they are to be executed by testers who lack of knowledge and experience in testing, the application and domain in question. Equipped with insufficient information and guidelines, these testers will struggle performing the tests and spending more time than necessary to figure something out or communicate back and forth for clarification. The net effect is fewer tests are executed and fewer bugs are found. Worse, these testers will miss the opportunity to learn more about the system and acquire testing skills from the script designers, e.g. how to setup test environment, what to do to effectively test a certain function, what test data to use in a particular scenario etc. should more instructions were included in the scripts in the first place.
On the other hand, if the targeted audiences are more experienced and knowledgeable testers, excessive details are nothing but waste. The time you spent adding unnecessary details to the test scripts is the time you could have spent on other value-added activities. Worse, this is the waste you have to maintain because you have to keep these scripts up to date with future changes in the system. Another side effect of excessive details is that they tend to discourage experienced testers from making judgment and improvising upon emerging discoveries and therefore foregoing the many advantages of exploratory testing.
One implication of the previous paragraph is that requirement changeability is another factor to be consider when documenting your test scripts. The more likely the system will change in part or whole, the less detailed you should aim your test scripts to be in order to avoid producing something which will be obsolete soon.
Besides targeted audiences and requirement changeability, there are other factors which might drive the details of test scripts such as regulation, contract compliance, recordings keeping demand etc. These are pretty straightforward so I’m not going to elaborate further. But even when you factor all these in, it is not always clear as to whether you are producing the right amount of details. Enter Rule #2.
Rule 2: When in doubt, better do less than more
Why? Well, unless you run out of value-added things to do for your project, why spend time doing something you’re uncertain about its value instead of something you are more certain about? Yes, you might be wrong and the testers will struggle with insufficient elaborated test scripts, but I would argue being wrong on this end is better than the other. Think about if, if you don’t have enough details, the testers executing the scripts will likely commit mistakes, produce little result and/or ask a lot of questions. In any case, the problem should manifest itself quite clearly and you can fix it as soon as it is known.
On the other hand, excessive elaborated test scripts, just as any other artifact, tend to go unnoticed or be ignored. After all, the executors have few reasons to complain that the scripts contain more instructions than they need and even if they do, the designers tend not to spend more time removing things they already produced (in the short term it feels like adding loss to a loss). By starting with less and continuously retrospecting on your process, you can refine the level of details on the needed basis while assuring that whatever things you do are the things adding the most value to the project. But then, you can still be wrong about how little you might need. You might think you’re doing less, but it’s still more than necessary. In that case, my advice is that the next time you have to update it, trim the fat so that you don’t have to bother maintaining the unnecessary the next time.
In summary, my advice to testers designing test scripts is: know your needs, start simple and tune often. As a matter of fact, that is my philosophy about the creation of other artifacts such as design documents.