Test automation can deliver huge benefits in terms of time-to-market, quality, productivity, and schedule confidence to almost any team creating or deploying software.
In working on hundreds of successful automation projects over 10+ years, TestPlant has worked with a leading global news publisher to reduce their app update cycle from three weeks to two days, a major US bank to reduce post-release defects by 65%.
Additionally, they’re one of the world’s top five retailers to double the number of apps they are delivering without increasing the size of their test team. These stories are not uncommon – test automation has the ability to deliver amazing benefits to lots of companies when properly implemented.
Unfortunately, there are also a large number of teams which have been trying to rollout test automation for years and not achieved any tangible benefits, often having invested significant effort.
TestPlant has worked closely with hundreds of companies to successfully deploy test automation, and has analyzed many failed test automation efforts. Based on this experience, we have identified two very simple, critical success factors that are common in successful efforts and missing in failed efforts.
Successful test automation deployments:
Are properly project managed
Include day-to-day someone who has successfully deployed test automation before.
These critical success factors are obvious when you read them, but the fact is that many (and possibly most) new test automation deployment efforts going on in the world today fail both these criteria, and as a result most likely will not achieve their objectives.
Going to the next level, what do successful test automation projects have in their plan? The key point is that it’s not only writing scripts! You need to setup the right infrastructure and context.
6 Key Elements of Any Test Automation Project
Set up your test environment – An inadequate test environment is possibly the most common cause of wasted time and unreliable test results. It is essential that you design and set up a reliable test environment before you start significant test script creation and execution. Note that setting up a reliable test environment is not difficult if planned and resourced properly, but if it is done in an ad-hoc manner it usually causes problems.
Set up your test management – A test management tool such as qTest is essential for both effective and efficient test automation. qTest makes test execution efficient and reliable, ensures that test results are clear and easily accessed, and most importantly helps you relate your test results back to your original objectives – the user stories or requirements. Without solid test management, we find that teams end up wasting a lot of time chasing ghosts in terms of script maintenance, spend hours a day on test execution and report generation, and don’t have confidence in their test coverage and risk of releasing into production.
Define and document your test architecture, conventions, and framework – Before you start creating automated test scripts for your test-cases you should define your test architecture, conventions, and framework.
Set up your relationships with other teams – A test team does not work in isolation. They work with the development team to get new versions of the application-under-test and raise defects. They most likely work with other teams such as the marketing team to understand requirements or the IT team to set up their test environment. It is important to set up your relationships with these other teams, meet the key people you will be working with, understand how they work, explain how you intend to work, understand the infrastructure each other are using, and discuss how to make the interfaces productive. Many a test automation project has failed despite great test coverage, simply because the results were not being fed back to the development team effectively and therefore had no impact. Integration to popular agile defect tracking solutions like JIRA is a key feature to consider in your testing solution stack, so developers can easily see which tests are failing and the underlying reasons for failure.
Set up your configuration management – You should setup a configuration management tool (SCM) such as Git or Subversion to store your test scripts; and automated test execution should always use a defined set of scripts from the SCM. Using an SCM facilitates sharing, ensures all script changes are logged, and ensures all scripts are frequently backed up. Additionally, as teams move towards TDD and DevOps, it helps to manage the frequent versioning and branching of code that will occur.
Train the team – Ensure all testers have been properly trained on the testing tools being used, the test environment, configuration management, the test architecture, and so on.
So, that’s it! Clearly there’s a bit more to a test automation project than simply asking someone to start writing some scripts, but none of this has to be complicated or take more than a few day’s work if you do it properly at the beginning. If you follow these rules, you will achieve great success with test automation.