QASymphony / Blog / Tips to Start Continuous Testing | Whiteboard Friday
Tips to Start Continuous Testing | Whiteboard Friday
Welcome to episode 4 of our Whiteboard Friday series, “Winning at DevOps with Continuous Testing”. In today’s episode I am discussing best practices to get started with Continuous Testing. These best practices will address important questions such as:
How should DevOps teams be structured?
What’s the optimal team size?
How to the best DevOp teams approach their testing?
In this episode you will learn about best practice approaches to getting started with Continuous Testing
Pair Up — A constant feedback loop is essential and pairing up a tester with a developer ensures the team knows the details. When this pairing happens, the tester and the developer are able to tie any results back to the source code being tested.
Minimal Team Sizes — Communication is often lost as a team grows. A general rule of thumb is the “two pizzas” approach. When working in teams, they should be no larger than what two pizzas can feed. This keeps communications streamlined and focused.
Automate First — An automation-first approach influences test prioritization and risk assessment in your application.
Think Small — Working within smaller parts with incremental measurements is at the core of DevOps. You need to be able to move rapidly with testing on codes that’s easily deployable.
Get Smart About Your Metrics — DevOps requires smart test metrics that close the loop in the feedback circle. Consider triggers that automatically update based on the build status or where errors lie. Look at how developer might be alerted based on the results of manual and automated testing. Smart metrics that transfer across integrated tools help keep small teams moving faster with better outcomes.
Full Transcript Below
Hey everyone, welcome back to Whiteboard Friday. We’re continuing with our series on how to win in DevOps with continuous testing. And last time around, we talked about how continuous testing fits into a DevOps pipeline. And today we’re going to talk about some tips and best practices of how to actually get started with continuous testing. But we’re not going to talk about tools today. We’re gonna talk about tools next time around. So be sure to come back for our last series on this, where we talk about different tools you’ll need to actually get up and running with continuous testing.
So let’s go ahead and talk about some best practices and get started here. First thing is, you want to look at is, you can pair up. And we talk a lot about this. And pairing up means that you pair up with a tester to a developer, where you’re getting a constant feedback loop of what actually is going on. This helps the developer and the tester communicate on what source code is being tested. How this source code relates back to a requirement or user story, and then also the tester can give feedback to the developer on, okay, what testing to be automated and what tests are actually affecting that source code for that feature that’s being released, or merged into a master branch for validation.
Continuing with kind of pairing up, is the two pizza approach. This is something that Amazon AWS does and what this really refers to is communication between teams. And this goes a little bit further than just pairing up with the developer and tester. What this says is, hey, communication’s great, but communication gets lost when you get more and more people a part of your team. So what do you do? Well, you take a two pizza approach. What that means is you say, okay, when we are working in teams, our teams will be no larger than what two pizzas can actually feed that team. So if you have two pizzas, and per person, that means you know, there’s about eight slices in a pizza. That refers to around eight people that you can actually feed if you got two pizzas in there.
So don’t get your team larger than how much a pizza can feed two people. And that just cuts down the communication. And you think about, you’re in a family environment, you’re around a dinner table. It’s much easier to communicate and talk and have intimate conversations with people at a dinner table than if you were to go to a loud, rowdy college bar. All right, that kinda makes sense.
The other thing, and this is kind of one of the core things, is that you want to automate first. This doesn’t mean that you just get rid of all your manual tasks. You’re probably not going to be able to. It’s probably not even wise to do that, with exploratory testing, and also testing that can’t be automated, such as usability testing and just really complex type testing that can’t be automated. You’re not gonna want to automate everything, but you want to take an automated first approach. And we actually have a webinar that talks about risk-based testing using JIRA. And what we do, you talk about a scientific and a heuristic approach to risk-based testing. And that really comes into test prioritization and assessing risk in your application, and figure out what things you need to automate, which things you don’t need to. So definitely check out that webinar as well. That’s on our resources page.
But again, you know what to automate first, maybe based off of the high areas that are going to be continually ran over and over again. And if you can’t automate first, then that’s okay. But we’ll look at that.
Next thing is think small. And again, we’re getting back to the paired approach and also the two pizza model, which is, okay, you first had all these blocks of work that you had to do which were very, very large. If you look from a waterfall and even a agile scrum fall type approach, right, you had all these things where you’re designing, coding, testing and deploying. Well, with a think small approach, what you’re doing is you’re breaking these down into incremental measurements for you to test after the code’s been completed and after the design has been done with that. So think small, break things down easier. And what that does is allows you to automate the tests that you need to, but it allows you to work in measurements that are easily deployable, that kind of feeds back in that DevOps model. So make sure you’re thinking small when you’re getting into this.
And the last one is get smart about your metrics. And what that is, is, when we get into the concept of trying to automate everything, we lose track of what we’re actually automating. So for example, this could be setting up triggers in place for pull requests. It can be triggers that are in for if a build’s complete, or if it’s not complete, or where the errors are lying. What areas of the code are affected? Can we alert the developer of the code that’s been affected by a manual test or an automated test. And then, of course, if you’re integrating with tools like JIRA or a user story tracking, a bug tracking, that also plays into the mix there. So get smart about this testing. Get smart about that even if you’re a small team. Metrics still matter in this continuous testing with DevOps.
So that kind of concludes our series today on just kinda talking about some best practices and tips. And come back next time, because we’re gonna be talking about, now that we’ve got a lot of this foundation in place, what are the tools? What are the bare minimum tools I need to get up and running with continuous testing in DevOps?