Today I’m talking more about how Continuous Testing is an essential part of the Continuous Integration model. Further, I explain how proper CI with Continuous Testing enables Continuous Delivery and Continuous Deployment.
In this episode you will learn about Continuous Testing in DevOps
Why having an ‘automated first’ first approach to continuous testing will help your team drive more continuous test automation, but also set realistic expectations for manual and exploratory testing.
How Continuous Testing looks to run automated tests immediately after build to determine the quality of the software and if it’s ready for more manual testing.
How Continuous Integration often suffers during the Continuous Testing phase because automated, manual and exploratory tests are often not attached to the source code.
And how Continuous Testing enables Continuous Deployment – iteratively planning, developing, and testing means that we can always be confident in the quality of the product if we chose to deploy at an on-demand pace.
Full Transcript Below
Hey, everyone. Welcome back to Whiteboard Friday. We’re continuing our series on how to win at DevOps with continuous testing. We’ve had two series so far, two Whiteboard Fridays. One was on DevOps, and the next one was on different types of DevOps. We talked about continuous integration, we talked about continuous delivery, and also continuous deployment. What we learned is that continuous integration is something that actually fuels continuous delivery and continuous deployment. It all started there in that continuous integration model. So we’re going to be talking more about that today on the topic of continuous testing in DevOps.
So let’s actually take a step back and look at what continuous integration is, and how that fits into the overall testing that’s going on. So with continuous integration, the whole point is, when you’re developing features, you’re merging that into a code branch. The code branch is then, whenever something gets merged, it’s going to be triggering a build for continuous integration to occur. What’s happening is you’ve got a CI tool like Jenkins or Teamcity, or something like that, that’s looking and going, “Oh, okay. There’s changes to this code. I need to do something. So I’m going to run a build. I’m going to deploy the build. If something fails, I alert back.”
So I’m constantly getting that feedback loop of, “Oh, this code may have broken,” or, “It’s actually working decently.” Then, it moves to a different stage. It moves into after maybe some unit tests have actually occurred into an area where you’re actually running tests that may have been deployed to a QA environment, or they still may be in the dev environment. This is where you’re getting into different types of testing. So as we blow open this look here, what’s actually going on here?
Well, what’s going on, the first thing is that you have an automation first approach to continuous testing. But there’s other things that are also going on. So what that means is that you look to automate things that should be automatable right away. Maybe there’s smoke test validation for functional UI testing. Or maybe they’re getting to a point where you can’t automate something, so you’re not going to force that in. So you do need to take an automated first approach so that when it hits here, and hopefully CI is also running these tests, when these run you’re running regression tests and functional tests as well, but you’re also doing manual testing and exploratory testing.
You can get triggered that, “Yeah. This build is ready to go. You need to go off and test this,” by whether or not the environment is ready for you to actually run these tests in. The key thing to keep in mind, though, is that since you’re wanting to connect this all to this pipeline over here, you want to make sure… I’ve drew a line kind of that goes all the way back here. What you want to make sure is that all the tests that you’re doing, whether exploratory, manual, automation, what you want to make them do is that if anything happens over here, you want to alert this pipeline of things that are broken.
That means, a lot of times, these tests should be actually attached to the actual source code that’s attached to the CI. A lot of times, those are detached. You have test cases they’re not actually attached to. Maybe some of the automated unit tests are actually going on in that CI build. So you want to make sure that if you can at all costs… Not at all costs, but at all reasons to do things, is that if you have an automation first approach, you want to make sure you can tie that back. What I like to do is kind of end this talking about an analogy of how this actually works.
So with CI and with continuous testing, what you’re doing is you’re iteratively doing something so that anytime you could actually deploy that piece of product out into production based on the quality that you’ve actually done.
So if I take a look at, or an example, like a deck, everyone goes out, has great parties on decks in summers. I just had one the other day, even though it’s really cold here in Georgia. But what you do is, you have a deck, and then you want to stain it. The whole point is, when you’re sanding a deck and you’re staining the deck, you’re iteratively putting on coats to that deck.
At any point in time, if I need to do three coats, if I just had one coat on it, I technically could say, “Hey, if you wanted room for a party, it’s still usable.” I can still use that deck. It may not lead to a finished product, but I’m still testing it, and sanding it, and staining it to a point where it’s available to be used. As I put more layers on it, I’m staining it more and more, I’m iteratively making it better and better, and better. So the final product, when I get to that point, I can then actually deploy that package into production. So that’s a quick example of what we’re trying to do here with continuous testing in this model.
The key points to kind of take away is that make sure that your tests can be tied back to source code and you’re taking an automation first approach. Not necessarily automating everything, but you’re first looking at what things can be automated first.
So that does it today for Whiteboard Friday. Hope to see you next time as we continue our series on DevOps and continuous testing.