While automated testing isn’t anything new, it is quickly becoming the industry standard. A recent study found an 85 percent increase in the use of test automation between 2014 and 2016 across several different industries. Why all the growth? In a word—agile.
If your organization has adopted agile methodologies and DevOps practices, odds are management is pushing you to automate testing—and rightly so. Automated testing is imperative in an agile environment. Not only does it speed time to market, it also enables continuous feedback to DevOps teams, finds bugs earlier on, reduces regression test cycle time and saves costs.
However, automating each and every build isn’t possible, nor is it always the most cost-effective option. Automated testing is not as simple as setting up a test once and having it run on its own after that. It requires resources and human input—time for development, equipment to run the tests and someone with the right skills to develop the test scripts, maintain the source codes and operate automated testing tools. Put simply, automated testing can save time, but it is not a “set it and forget it” scenario.
Like any worthwhile initiative, automated testing requires time, effort and strategy to be effective. The following are our top five tips for smarter automated testing:
1. Start by Automating Unit Tests
One mistake many organizations make is to begin automated testing at the user interface (UI) level. Because UI testing takes a lot of time and effort, this strategy offers very little return on investment. To maximize automation, organizations should reference Mike Cohn’s test automation pyramid, which calls for automating at three distinct levels. Organizations following this method should start at the lowest level of the pyramid—the unit level—and then move up to the integration or API level. This provides the best return. Ideally, organizations will use a continuous integration (CI) tool like Jenkins to continuously run unit and API level tests. This is a standard practice for teams doing CI because it provides better feedback and helps ensure that the code is stable upon code commits. However, it is important to note that not all functional tests, such as Selenium and Appium, should be run through CI.
2. Time Your Tests Strategically Functional
UI tests can take a long time to run to validate new features or code that has been deployed. Since CI is all about getting rapid feedback, it’s best to separate UI tests from the unit and integration tests that run through CI in order to optimize your test run planning. Running nightly builds for functional regression and end-to-end automation can be a good strategy for avoiding clogs in the the CI process. The overarching goal is to time tests so that you are never waiting on results or holding up development. Running low-level tests overnight, for example, allows teams to take action the next day if needed. This also frees up time for more complex, manual testing.
3. Have a Clean and Organized Testing Environment
An inadequate test environment is one of the most common causes of wasted time and unreliable test results. To avoid possible interference and errors, it is imperative to have a clean test environment. Ideally, teams should build and tear down environments before every run. This is usually best accomplished by leveraging cloud-based hosting. Environment fidelity is also critical. For test results to be meaningful, they should run in an environment that is as close to production as possible. Finally, organization is an important part of maintaining a successful and manageable test environment. Having a central automation hub, for example, can help teams understand which test machines are ready for execution, which ones are currently running tests and which ones have future automation jobs already scheduled.
4.Use BDD to Increase Efficiency
One methodology that is especially beneficial in an agile testing environment is Behavior Driven Development (BDD). Considered an extension of Test Driven Development (TDD), BDD uses a collaborative approach to identify user behaviors and then bases test scripts on those scenarios. This establishes a common language between product owners, testers, developers and any other stakeholders. It clearly defines the critical behaviors expected of an application so that every member of the development process understands the actual deliverables. This helps drives development and provides structure and focus, especially in automated testing. Instead of simply developing a bunch of automated tests, organizations can use BDD to create automated tests that provide real value and trace back to the original requirements set by the entire team.
5. Level Set Expectations
Here’s the truth: You can’t automate everything. Saying that you can automate everything assumes that you can test every single piece of code, which isn’t possible. And, while automated testing offers tremendous benefits, it has its limitations. There will always be instances when manual testing is necessary or even the better option. For example, with UI testing, automation can determine if an element renders, but only a human can determine if it appears in the right place and looks good. Some tests will need a human perspective to be fully accurate. Because automated testing can’t always mirror the user experience, it is important for organizations to focus on the goals of their test cases when determining whether or not to take an automated approach. Automated testing can be valuable, but it needs to be planned strategically and revisited regularly.
There is no question that automated testing is a critical part of agile development. However, it is not a blanket solution. Organizations that want to get the most out of automated testing need to be sure they are be both realistic and tactical in their approach. With a few best practices in place, testers can leverage automation to save time, save money and deliver higher-quality software.
We just need a little info from you.