With all they hype around TDD, BDD and ATDD, let’s examine a few common challenges that both small and enterprise testing teams face as they transition and scale into broader adoption of these methodologies.
Challenge: Transitioning traditional test cases to a gherkin format
Problem: Unless you are starting with a brand new application, odds are there are hundreds, if not thousands, of existing manual or automated test cases that are incompatible with BDD and ATDD approaches.
Solution: Don’t let the amount of existing test cases needed to convert overwhelm you and prevent you from getting started.
The first step is to truly evaluate which of your existing test cases are valid. If they are manual test cases, are they unique? Do they provide significant coverage? Do they regularly detect bugs? If they are automated, do they still run reliably? Quickly? If not, make sure you clear these test cases out before continuing any further.
The next step is to prioritize what is left based on the coverage it provides or defects it exposes, and begin the process by:
Prioritizing the test cases
Determining what, if any, changes should be made to the test cases (i.e. adding negative test coverage, refactoring, splitting into smaller pieces)
Assigning approximate automation effort to each of the test cases
Reprioritize test cases based on effort required to
Rewriting the test cases in a gherkin format
Completing the automation rewrite/update
Validating the test runs successfully and completely
We have found that the above process is often run best in a more iterative (i.e. agile) way. For example, the top X% of test cases are evaluated, estimated, reprioritized, rewritten, and automated in short 3 week sprints. Kanban boards are particularly useful for communicating progress back to the team and keeping morale high.
Challenge: Building/Scaling an “Automate First” process
Problem: Many teams struggle to adapt to the “automate first” philosophy that typically goes hand in hand with most BDD and ATDD processes.
Solution: Don’t overemphasize the tool – focus more attention on your people and process and understand that change will take time.
Many organizations falsely believe that popular BDD and ATDD tools provide a silver bullet that will drastically change the way that they are able to define, build, and maintain their automated tests. However, we must understand that these tools are simply an abstraction layer that allows us to better define our necessary tests up front and minimize redundancies in our tests. It will not fix problems like:
Not having enough testers with the necessary knowledge to build automation
Unclear understanding of how the product works, and what is should do
Inability to build robust and complete test coverage (automated or manual)
Applications with low testability (i.e. lacking API’s for service level tests, improperly named objects, old frameworks like Flash)
Lack of attention or bandwidth dedicated to failed/failing or slow tests
Thus, teams looking to succeed with BDD and ATDD approaches should ensure that they are spending effort to get their current people and process in a workable state before embarking on this new shift. Otherwise, they will quickly realize that their same problems still haunt them long after their new tools are implemented.
Challeng: Taking the BDD/ATDD Approach Enterprise Wide
Problem: While many teams can get started with a BDD/ATDD process in pockets of their organization, they hit a wall when they try to expand the usage across the entire organization.
Solution: Take an ROI based approach using a formally defined pilot, and document everything in a playbook for BDD/ATDD success.
While the benefits of a “test first” approach may be second nature to many testers, business analysts, and developers, many executive still have not heard of the concept, let alone understand why they should consider adopting it.
Development and IT executives are typically measured on their ability to deliver on time and budget, and minimize risk. So, it only makes sense to educate these executives on the real benefits of a “test first” approach in terms that will clearly align with their most important goals.
To start, teams must understand their current process first and identify areas where there are significant losses in time or money, or increase in risk. If these cannot be identified, then logic would question whether or not it is in the organization’s best interest for the company to implement the change. Significant focus should be spent in this area, as the bottlenecks and pain points identified here will be the baseline for the new process to be measured against.
When selecting a team for the pilot, it is wise to consider a fairly average to above average team and not one that is extremely high or low performing. This will help mitigate the risk of underselling the value of the transition (and potentially being unsuccessful) as well as the threat of overselling the value (and upsetting the executive who approves or sponsors it.)
At the end of the pilot, the team should have two solid deliverables:
A business case for making the BDD/ATDD transition across the enterprise, with clearly forecasted savings measured in time, dollars, and risk as well as the corresponding costs if the transition is not made
A success playbook that documents in detail the successes and failures experienced by the pilot team in implementing the process, and reflective advice about how to best go about the process on other teams.
With these two items in place, teams will be much more empowered to sell these “test first” approaches to their executives and management, and deliver greater than expected savings by implementing them.