We all know the story: DevOps is quickly becoming the methodology of choice for software development teams (it hit 74% adoption in 2017) and it’s enabling those who adopt it to release new software faster and more often (McKinsey finds it can help teams reduce the time to move from code to production by as much as 74 days). And as CIOs face increased pressure to reduce time to market, the efficiency that DevOps brings becomes extremely valuable (IDC predicts that in 2018, CIOs will take agility to the next level using DevOps).
But what about quality? As often as the DevOps story gets told, the question of where QA activities fit in gets asked just as often. It makes sense — with development cycles moving faster than ever, how do you fit in QA, something that’s typically viewed as a bottleneck?
As you dig deeper into what DevOps truly entails, you’ll find that it actually prioritizes quality and, when approached correctly, QA can help organizations both improve software quality and reduce time to market in a DevOps environment.
Five Major Challenges to DevOps Adoption
Although it seems everyone today is on the DevOps bandwagon, the truth is there are several hurdles that teams must clear before they can operate in a true DevOps environment. Key among those hurdles are the following five challenges:
Lack of test automation coverage:
First, many teams don’t have enough test automation coverage, but getting that coverage in place is important because the frequent release cycles of DevOps require automation in order to confirm software quality quickly and easily. One way to solve this problem is to introduce behavior driven development (BDD), which is a test-first methodology that builds on test-driven development (TDD) and encourages collaboration among testers, developers and product owners. The goals of BDD include better understanding business and user needs and making user experience testing more efficient and effective. Because of its collaborative nature, BDD makes it possible determine testing needs upfront and improve traceability in order to improve coverage.
Lack of visibility:
Second, we have a lack of visibility, which often becomes an even more glaring challenge as teams introduce more test automation. That’s because as automation increases, teams typically have less visibility into what all of the tests do and what the results mean. The solution to this problem lies in airtight documentation and reporting, that way it’s clear to everyone what tests do, how they function, and the results they’re delivering.
Aligning tests with traceability can also prove difficult in the speed of a typical DevOps environment, as code changes so often and gets tested so frequently. QA teams need to make sure that they are not only tracing their tests back to user stories, but also to the code that’s being developed for that user story. One way to overcome this challenge is to version tests with code that’s being developed, so that everyone can see how tests evolve alongside code changes. Using this approach, QA activities become part of development, rather than a separate task.
Inability to maintain a single source of truth:
When it comes to testing, no matter what methodology you follow, you need a single source of truth to keep everyone up to date and ensure that all team members work off of the most recent information. Once again, the speed of DevOps, along with the number of people touching the code in such a short timeframe, complicate this imperative. Therefore, it’s important for the teams to rally and clarify sources of truth.
Difficulty tying value to the pipeline:
Finally, many teams face challenges tying business value to what’s in the pipeline. However, assigning that value is important to getting approval, and when teams get this right, they typically move faster. Resolving this problem comes down to pairing what’s known as “agile upstream” activities, including Scrum, Kanban,, daily standups, and backlog planning, with “agile downstream” activities, including continuous integration,,continuous deployment, TDD and automated environment configurations. Without these two sides working in tandem, one group can outpace the other. And when that happens, it becomes like upgrading your car engine without changing the tires to match. But when you do get the combination right, it creates a fantastic flow powered by a feedback loop about what’s in progress, how releases are performing, and more. In turn, that feedback loop makes it possible to prove how you can pay for certain releases and to determine what does and doesn’t make sense to do.
Solving for DevOps Problems: Prioritizing Quality and Accelerating Speed to Market
Once you overcome the five challenges listed above, you’ll be well on your way to running a smooth DevOps environment in which testing occurs regularly. That said, you need to continue to stress the importance of quality on a regular basis.
Think of it this way: If you miss out on any QA activities or make quality an afterthought, then all of the testing that needs to take place gets piled up toward the end of the software development lifecycle. This pile up not only creates a bottleneck in which testing holds up everything else, but it can also end up adding even more time should you uncover any major issues.
On the other hand, if you prioritize quality, the situation never needs to get that far. Talking about quality from the very beginning and then running quality checks as often as possible helps both prevent issues and catch any issues that do arise much earlier on. In turn, this early detection means that the issue doesn’t have a chance to create a domino effect and can get fixed much faster and easier. As a result, this prioritization of quality is a win-win, since it helps reduce testing efforts, lower costs, and, most importantly, speed time to market. And given these benefits, it’s not surprising that injecting testing earlier in the software development lifecycle is a priority for 30 percent of executives surveyed in the 2017-18 World Quality Report.
The Positive Impact of Prioritizing Quality in Action
So how exactly does all of this look in action? Consider the case of Dolby Laboratories, which implements audio and video technologies across a myriad of products. The Dolby team took a strategic approach to DevOps and made quality a priority by developing a clear process for development that includes regular testing efforts and strong documentation requirements. Since doing so, the team increased their test runs from 45 test runs per minute to 4,000-7,000 test runs per minute while reducing the time required for reporting on test cycles from one to five hours to under two minutes.
And Dolby isn’t alone. When GE Digital needed a more efficient way to bring continuous integration and continuous delivery to its development team, it found that prioritizing quality could help. Specifically, the team introduced a QA process that injects testing throughout the development lifecycle (including at the very beginning of every sprint) as well as a handful of best in breed tools to make those activities possible and help maintain a single source of truth. Together, these changes resulted in improved test coverage, the ability to easily track versions of test cases, and reduced time to market for new releases.
We just need a little info from you.