The Next Great Debate: The Role of AI in Software Testing

With Accelerate SF now less than 2 weeks away, we’re excited to announce the return of  the now notorious “Next Great Debate in Software Testing.” The topic this time around is “AI in Software Testing: Truth? You Can’t Handle the Truth!”

Wolfgang Platz (Tricentis Founder and Chief Product Officer), Jeff Wilkinson (Managing Director of Accenture), and Ramesh Pai (Global Head of NextGen QA at Wipro) will debate hot-button AI issues such as:

  • Are AI-based testing technologies a dream or nightmare for professional software testers?
  • How much testing can (and should) be delegated to AI?
  • What are the true benefits…and risks?

More good news: you can be a part of the debate even if you’re not in San Francisco. We’ll be livestreaming the debate as it occurs on May 22—live and uncensored.

[REGISTER FOR THE LIVESTREAM]

To whet your appetite, here’s Tricentis founder and Chief Product Officer Wolfgang Platz’s recap of the last Great Debate at Accelerate SF (originally published in InfoWorld):

“Boring as Sh*t” or “Sexy”? — The Evolving Role of Software Testing

Imagine this all-too-common scenario: Software testers are asked to check new application functionality after developers have deemed it “done.” They have a list of actions to perform, and they manually click through each and every step—trying to faithfully follow all instructions and methodically document the outcome of each step in case something goes awry. They don’t really understand the business or the application’s users, and they rarely interact with the developers or stakeholders.  Their primary goal is entering X and checking that the outcome is Y and not Z.

This is undeniably boring—but I don’t think it’s truly testing.  It’s certainly not the type of software testing that’s required to deliver exceptional customer experiences faster than your competitors.

The “digital transformation” sea change is impacting virtually every department at every business today, and few QA departments will remain untouched.  The resulting wave will likely wipe out the “boring as sh*t” manual verification I described above. But it could also elevate testing to a “sexy” discipline where testers become the primary stewards of the customer experience.

That’s probably the most memorable takeaway from “The Great Software Testing Debate”: a recent panel I participated on with Jeff Wilkinson (Managing Director at Accenture) and Anders Wallgren (CTO at Electric Cloud). As we debated the evolution of testing, we touched upon topics like the emerging role of SDETs and the future of TCoEs. We all agreed that the disruption we’re facing today presents a great opportunity to reposition testing as a discipline that’s driving innovation rather than dragging it down.  However, the devil is in the details. What exactly needs to change, and how do we get there?

Here’s a recap of some of the most interesting discussion points …

One team, one fight

Anders introduced a great mantra that captures how quality has become everyone’s responsibility: “When customers see a bug, they don’t blame the testers or the developers. They blame the company that released the crappy software.”

Continuing on this theme, Anders also shared an interesting analogy for how to approach quality as a process: “Think of the School House Rock on how a bill becomes a law. This is how we need to think about our code. We need to consider all the different things that happen to our code as it passes along the conveyer belt of our software delivery factory, all the different people that are involved in the process. Once we have this mapped out, that’s when we can really start optimizing the process…If a bug escapes through that process and reaches the customer, this means that there’s a larger organizational failure that must be identified, analyzed, and addressed.”

The need for diversity

Jeff is a passionate champion of diversity. In past conferences, I’ve witnessed his moving presentations about how including employees from a broad mix of backgrounds not only benefits specific testers…it also enhances the quality of your overall testing.

In this panel, Jeff focused on how the shift to DevOps is opening up a variety of different options to accommodate different skillsets.  For example, former manual testers can take a number of different paths to contribute to the automation effort. Those who want to continue along the testing path might learn model-based test automation and/or Selenium to remain marketable in the software testing field. Some might decide to focus on test data management.  Others will be more inclined to branch off and master DevOps practices and platforms. There are 40,000 testers at Accenture. They don’t all have the same personalities and skillsets…and Accenture is a much stronger company as a result.

The importance of unit testing

Anders is a staunch proponent of unit testing. He requires engineers to write unit tests for their code because it’s so much faster and cheaper to expose errors at that level. In his experience, many issues found in production could (and should) have been stopped with better unit tests.

I agree that unit testing is both valuable and necessary. As Anders noted, it forms the stable foundation of the Agile test pyramid. However, I’m less confident that unit testing will catch the issues that really impact the end-user experience. Especially in large enterprises, I’ve seen unit tests decay over time…to the point where nobody pays attention to them. I’ve also found that most of the issues reported by end users could not be found at the unit level; they would have required integration testing, system testing, or end-to-end testing by a professional tester who really understood the domain.

This ended up being one of the few discussion points where we had to agree to disagree.

Developers test, but aren’t testers

I stressed that unless you’re a GAFA (Google, Apple, Facebook, Amazon), you probably won’t have the luxury of hiring SDETs: developers who will devote themselves to testing. However, even if you could get developers to test, they’re probably not your best option for protecting the end user experience.

Yes, of course developers should be checking their code with static analysis, unit testing, peer code reviews, and so on. As Anders said, the earlier and cheaper you can find a problem, the better. However, also consider that the person who’s great at creating things isn’t always the best at breaking things. These are distinctly different disciplines. If you’re moving to a new skyscraper, would you want the final building inspection to be completed by the architect or by a professional building inspector?

Jeff put it this way: “QA is a mindset. To be really good at QA, you need to be passionate about driving quality in.”

Anders added: “The testing mindset is ‘I’m going to figure out how to break this.’ This is great because it gets developers thinking: ‘I’m not going to let them break this’…and the software is much stronger as a result.”

Shift (even farther) left

More and more organizations—including Accenture, Electric Cloud, and many Tricentis customers—are reinventing “testing” as “quality engineering.” The change is more than skin deep. Testing is reactive, while quality engineering is proactive. You don’t just catch defects, you prevent them from entering the code base in the first place.

We all agreed that quality engineering means addressing quality much earlier in the process than most organizations do today. But the solution isn’t simply to adopt the techniques commonly branded as “shift left testing.” It could also involve more early stage reviews, with testers participating in those reviews. Even farther left, it could involve inviting testers to the table at design sessions—where they can strategize on ways to build in quality and testability from the start.

With this shift, testers are elevated to the “sexy” role of trusted advisors, ambassadors of quality…and the “boring as sh*t” work becomes a relic of the past.

Leave a Reply

Your email address will not be published. Required fields are marked *

More Great Content

Get Started with qTest