QASymphony / Blog / Going Agile? Don’t Forget About Testing – Webinar Recap
Going Agile? Don’t Forget About Testing – Webinar Recap
Last week we joined forces with Gallop Solutions to host a webinar on software testing during the agile transformation. In this webinar, Sujit Nair, Associate Director with the Enterprise Solutions Group at Gallop Solutions Inc. and Kevin Dunne, VP of Business Development and Strategy for QASymphony explored the testing trends around people, process and technology that global enterprises are adopting in their agile software development.
We ran out of time during the webinar and only answered a couple of the questions so I asked Kevin and Sujit to take a moment to answer a handful of other questions. Which you can find the answers to below the recording and slides. Watch the on-demand webinar here.
How long can the stakeholder wait without the potential release in Agile?
Kevin Dunne: Agile is not necessarily about releasing software out in to production frequently – it is about creating working software more rapidly, whether that makes it to the customer or not. Focusing on creating something that creates accountability and urgency for the team. As far as Agile sprint length and what is the ideal length for a sprint, I think that will depend greatly from one organization to another and will not be consistent across all companies. Some factors to consider are how frequently you desire to release into production, what is the planning/meeting cadence of other complementary departments (i.e. marketing, sales, etc.), and the frequency of releases for associated vendors/products and competitors.
How do we encourage our development team who are doing test driven development to push more frequent deployments?
Kevin Dunne: I think more frequent deployments will come with increased confidence in the testing being performed within the build pipeline. Whether the tests are automated or manual, I think it is imperative that the tests are efficient, accurate, and offer good coverage. Without that, there will be a need may be frequent false positives requiring investigation, defect leakage, or unnecessary manual double checking of the application. All of these things add manual overhead to a release and provide negative incentives for organizations to release more quickly. If teams try to deploy more regularly without fixing this problem, it requires more and more effort to be paid to the deployment and associated problems, meaning less time can be spent coding new features and testing those features for release.
Are testers a separate role, or part of another role? BA/QA? Dev/QA?
Kevin Dunne: We are seeing that testers are taking on a lot of hybrid roles within agile teams. Many testers are typically under worked early in a sprint, and over worked as the sprint nears completion and the majority of code has been delivered. This opens the way for testers to take on other complementary responsibilities, which might include planning or being a scrum master, contributing to requirements or business analysis (especially in TDD/BDD), or becoming a more technical “software design engineer in test” or also contributing to the operations piece of deploying test and staging environments.
Do we really introduce Bouncer role in all Agile based frameworks?
Sujit Nair: Typically the scrum-master is the manager in agile teams but a full time scrum master role is not always necessary. A development manager can act as a bouncer; preventing direct access to developers while they’re busy coding and filtering the urgency of all requests.
In Agile, do developers execute unit level testing on their own code or are testers expected to take that on along with all other types of traditional testing responsibilities, functional, performance, end-to-end, etc can you provide examples of story pinning and decision matrices just mentioned?
Sujit Nair: Yes, in Agile or Waterfall, developers write and execute unit tests. It is a difficult ask to have testers review the code at component level and write unit tests. Let’s take an example user story: As a sales rep, I want the appointments page to store several days of appointments and display it while offline, so that I can see appointments even when my phone is offline. Now you will first want to slice this requirement into specific slices: Create a new UI element that displays appointments (don’t need to actually have an offline mode yet). When the phone has no service, add “Currently Offline” to the UI element. Maintain the appointment data from the last sync while offline. From several days you may change the slice to store five days of weather data. You keep refining the slices till you get to the smallest possible chunk that can be developed independent of other slices. This way if you underestimate the effort you can always thin the story by developing a subset of the slices and pushing the rest out to the next sprint.
In Agile, are developers expected to write design documentation on their features?
Sujit Nair: Yes, developers are expected to write design documentation on their features. Keep in mind that the best approach is to update the design documents only after the code for the story is delivered as opposed to producing design documents after preliminary work on stories.
Who writes technical requirement documentation?
Sujit Nair: In most case product owners write technical requirements documentation. But ideal they should do so in collaboration with the design / development team. For example bringing in members from the design / development team when conducting customer interviews.
I did not understand the difference between QA and Quality Eng. Regression Automation vs Automated test case Gen…..this sound to be all the same things expect different name…what is the big difference between those 2 methods? How do you define/use RAI?! How do the Estimation of Story points takes place in Agile? What is gallop requirements framework ?
Sujit Nair: Quality engineering is a infrastructure building activity. Quality engineering establishes the kind of environment that supports quality in both the process and the product. Quality assurance on the other hand assures or verifies quality in the product. Think of Quality engineering as before and during the process and Quality Assurance as after the process. Regression automation refers to developing automated scripts for the regression suite so that the execution can carried out in an automated manner whereas automated test case generation is for generating test cases that then can either be executed manually or automated. Requirements Ambiguity is about validating requirements against four key parameters namely clarity, consistency, completeness and testability. As for estimating user stories, the primary input comes from the testers who are part of the planning poker. Again the user story’s priority, complexity, associated use cases, types of testing needed are the primary determinants of the testing effort estimate.
How do we handle changing requirements in Agile?
Kevin Dunne: Agile is made for changing requirements – the nice part about working in shorter sprints (i.e. 2-3 weeks) is that we don’t have to push user stories into scope until they are ready. In waterfall, we might know we need to have something by the end of quarter or end of year, but aren’t sure about it yet. So we are forced into defining these requirements before we are all on the same page. In agile, we can consistently groom the requirements in the backlog every sprint until we know we are aligned on a requirement and push it into the sprint plan. As far as changing requirements within a given sprint, I think that can be done but should be something that happens rather infrequently. The close collaboration of agile teams means that this can usually be achieved with less risk than in waterfall, but on the flip side, things need to move so quickly in Agile that its possible that the tester has already started writing the test case, or the developer has already started coding when the requirement is changed. So, the cost of changing requirements is not zero and should be avoided when possible, although there are always times where it may be unavoidable. Agile is often explained as throwing a dart into a 1″ bullseye from 1′ away, where as waterfall is throwing a dart into a 10″ bullseye from 10′ away. The idea is that the target is so close to you it should be very hard to miss, so if you are changing your requirements very often within Agile, I think it demands further investigation.
Do you have any suggestions for the QA Engineer who wants to migrate their skill-set from those used in manual testing to those used in agile testing? The learning curve is steep and daunting.
Kevin Dunne: Yes, the time is better now than ever as the automation tools and development languages are becoming more and more accessible. I would highly recommend learning a more basic, object oriented programming language first, like Python or Ruby. There are many great free courses on the internet from places like Udemy, and also in classroom training and code bootcamps for those in between roles or who can negotiate more time away from the office. Even for those that struggle to learn the basics of a programming language, there are tools like Selenium IDE that can allow you to record tests and investigate the results to reverse engineer how to build tests. My one key recommendation is not to overspecialize in one particular framework. Things are changing constantly in Agile and in all likelihood the automation tools we use today will be obsolete in 5-10 years time. So invest heavily in skills that are timeless like defining good tests, debugging, utilizing data driven testing, etc. and be careful to not over invest in specifics functions tied to individual frameworks.