For testers, this shift can be worrisome. After all, DevOps represents a significant change to when and even how testing occurs by calling for testing to occur more often, frequently at the hands of developers and in a more automated fashion.
Arguably the biggest change that results from this increased responsibility is the need to pay closer attention to what happens to software in production. In a typical waterfall or even agile scenario, production is very much a black box for testers. Once code gets sent to production, another team looks at the error rates and checks calls from customers about issues. That team then logs those issues as tickets for testers. But this system creates a long feedback cycle that simply doesn’t work in a fast-paced DevOps environment.
Meanwhile, when you adopt DevOps, one of the first things you often do is measure your cycle time and look for ways to reduce the build and test times. Given the need to resolve bugs faster, it’s essential to include the feedback cycle in that measurement as well. And the best way to shorten the feedback cycle is to involve testers in production. Specifically, instead of having your testers hand off code to production and walk away, ask them to start paying attention to what happens in production by looking for errors and working to fix them before they become issues reported by users. This approach not only shortens the feedback cycle, but it also reduces defect rates and minimizes errors found by users — an all around win.
2) Don’t Overlook the Importance of Soft Skills
One of the biggest challenges that testers tend to face when adopting DevOps centers around soft skills. Because DevOps requires testers to be involved in more than just one siloed testing phase in the software development lifecycle, it means they must flex communication and collaboration skills that they haven’t necessarily had to use in the past.
For example, in a DevOps world, testers must be able to attend a planning session and communicate effectively with developers about what will get tested, how it will get tested, who will do the testing and when the testing will occur. This need to communicate across teams more frequently can take many testers out of their comfort zones, but it’s a necessary leap in order to make a smooth transition to DevOps. That’s because if you don’t break down silos between testers and developers by increasing communication and collaboration, you’ll never achieve the efficiency you want, regardless of how well everything runs from a technical standpoint.
As part of this effort, testers must be able to check build logs, ensure automated tests run properly and understand how the software works on a much deeper level than they’ve had to previously. Additionally, testers must get proactively involved in the delivery process to help move software through the development lifecycle faster and produce a higher quality end product. Fulfilling those imperatives requires basic coding skills.
As a result, testers need to shift from reporting on these types of test results to reporting on a more strategic level that conveys how testing is progressing. For example, try reporting on how automation efforts contribute to bugs found in testing, the team’s progress toward automating more tests, how testing efficiency improves by release, how development and test teams are collaborating to reduce the turnaround time to fix issues and confirm fixes and how the testing team is increasing coverage (and thereby reducing risk).
5) Focus on Test Optimization, Not Test Automation
Test automation is absolutely necessary in a fast-moving DevOps environment, but it’s important to think about the bigger picture too, because test automation alone will not solve all of your manual process problems. Instead of focusing solely on how to achieve your desired level of test automation, try thinking about how to optimize your whole testing strategy.
Think of it this way: If you buy a Ferrari to go fast but end up taking only backroads instead of the highway, you won’t get to your destination any sooner. Focusing solely on test automation without taking steps to optimize your DevOps testing strategy creates a similar situation because you end up using an efficiency-driven tool in the least efficient way possible.
In contrast, taking on test optimization efforts should help increase efficiency by finding the right mix of unit, integration and functional automation alongside manual and exploratory testing. To find this mix, testers must identify and report on areas of risk and take the time to review test plans, including test creation, test execution and amount of testing.
6) Shift Left for Increased Automation…
Everyone knows that the longer you put off a problem, the worse it’s going to get in the long run. And in the continuous delivery-driven DevOps world, letting a problem fester means letting the release bottleneck grow. This situation happens most often when you run too many tests manually at the end of a cycle instead of automating them.
Now, we’ve always said testing in production isn’t ideal. After all, finding a bug in production is extremely costly. However, testing in production has become important since the frequent releases that come with operating in a DevOps environment can cause bugs to make it through the pre-production phases of the software development lifecycle.
As a result, testers should not only be prepared to test earlier in the software development lifecycle, but they should also make plans to test in production. One way to accomplish this shift right is to implement monitoring tools that can detect where errors occur and how users are actually using the software. This setup allows end users to inform the business about new test scenarios that require automation. From there, testers can add those tests to their library to make sure they get worked into the continuous pipeline.