Big Design Up Front and Exploratory Testing

Many years ago when I started practicing agile development, one of the practices, or more appropriately a mindset, that I embraced was No Big Design Up Front (no-BDUF).  Even in the agile community itself, there is debate as to whether the BDUF or no-BDUF mindset should be embraced.  For people in non-agile camps, like those practicing RUP or CMMi, no-BDUF is anything but orthodox.  A quick search on the internet and you’ll easily come across all kinds of arguments for and against BDUF.  Many of the arguments are, in my opinion, fruitless for people don’t take the time to understand the concept properly.  This post talks about no-BDUF and relates this value to Exploratory Testing.

Chance is that you have joined one or more projects in which there was an architect creating some sort of Software Architecture Document (aka SAD) and a couple of Detailed Design Documents (aka DDDs) out of some corporate templates.  These documents were meant to capture every single design decision, ranging from technologies selection to layer architecture, detailed classes and methods design, object collaboration, inter-processes/thread communication and deployment configurations etc.  While they were also meant to be live documents and evolved as the project proceeds, more often than not, in an effort of not leaving too many sections of those templates unaddressed, the architect ended up with documents each with dozens of pages.  These documents were then handed to those junior engineers whose were commanded to “just follow the design”…  But to everyone’s surprise, those documents very quickly became forgotten.  The structure of the software’s code wasn’t even close to what were documented.  Nobody could hardly recognize any class or method in those documents anymore.  And there were numerous reasons.  The customers had requested features that the architect never thought the system would have to deal with, like interfacing with those funny 3rd-party lab systems.  They had also requested changes to the software which if not implemented will make the software totally irrelevant to their targeted users.  Then some of the things the architect had speculated turned out to be wrong.  The system didn’t really need to sit in 3 different tiers with a couple of load balancers sitting in front of them.  And those singleton services he had baked into the design caused so many problems that the team basically had to throw away most of them by refactoring your code significantly .  Not to mention those hidden business logic that even the best business analyst in the team hadn’t figured out even a fraction of it during the design phase.  And nobody had the time to update those design documents which would be outdated the next day anyway.  It was when everyone in team realized that design documents weren’t nearly as valuable as they thought.

Is this story familiar to you?  It is to me.  I was there.  I was that “architect”.  The lesson was so profound that when I approached the agile movement, I picked up the concept of no-BDUF very naturally — because it was already inside me, it just needed a name.  To me, no-BDUF is not about doing no or little design or documentation.  To me, it means you should design up front only as much as you need to while pushing off the rest of the design activity to the last responsible moment, when you have the most up to date information about what you need to solve and the constraints around your solution.  You can pick technologies beforehand and outline a rough skeleton of the software architecture (probably in form of a whiteboard sketch, not a 30-page document) based on what you already know and use it as a frame of reference for more detailed design and discussion with the team, but most of the design happen when you actually sit in front of your monitor beginning to code.  Simply put, you design when you code.  More often than not, this on-the-fly designing process gives you feedback to update the software architecture, which by now you have realized so volatile that you needn’t to document it (or at least, a large portion of it) in the first place.  The code is now the live design artifact.  The code is the design.

How does this have anything to do with Exploratory Testing?  A great deal.  For one, exploratory testing is about designing test cases/steps as you execute them, instead of designing a bunch of test cases up front (and typically, documenting them down) before you ever execute a single test.  By delaying test design activity to the last responsible moment, you can take advantage of the most up to date understanding of the problem at hand and allow yourself to act upon emerging discoveries about the functionality under test.  After all, that’s the only sensical thing to do if the software you are testing now isn’t exactly the software you thought it would be when designing many test cases up front, so why not delaying test case creation until when you know about the software a bit better?  Not to mention, in many cases it is not even the software you think it currently is and that’s how acting upon emerging discoveries helps you to know more about the software and its (mis)behaviors.

But exploratory is more than just a testing practice or approach.  Like no-BDUF, exploratory testing is also a mindset which testers can adopt to perform various testing activities.  For example, testers can apply scripted testing by performing a certain level of up front test design, should they have to do so due to regulation, team process or any other reasons applicable to their context.  At the same time they can allow themselves to be adventurous and willing to explore unknown paths instead of rigidly exercising predefined test steps.  After all, there’s no point in rigidly following the instructions of the less experienced and knowledgeable tester that was you one month ago?  After all, why even rigidly following the instructions of a more experienced tester who is not you but doesn’t have the knowledge that you are having now, when exercising the most current software features and understand them a bit more.

Having said all this, I do realize in certain circumstances, you might have to design your architecture or test a little bit more upfront.  It might be because you are building a mission-critical software whose every requirement must be specified to the smallest detail and design must be created and reviewed carefully.  It might be because you work in a highly distributed team which calls for the need to communicating design ideas in written forms for ease of distribution.  Or it might simply be because what you’re doing works for you.  But then, unless you can foresee all changes or have the power to reject them all, which is unlikely for most software projects, adopting the emphasis on emergent design, learning and discoveries of no-BDUF and exploratory testing will certainly benefit you.

5 comments on Big Design Up Front and Exploratory Testing

  1. Savita says:

    Hi Buu,

    I agree with your point. Exploratory approach to test is to modify your test as & when required with new details.

    Exploration is the way of doing research. Basic nature Science discovery, medical devices testing is Exploratory.

    Very nice information!

  2. Buu Nguyen says:

    @Savita: thanks for your comment. I’m glad you find the post useful!

  3. Thang Mai says:

    No-BDUF is not new to me but Exploratory Testing is. Thanks

  4. Hi Buu

    Great article, and if its a good testing method so special tools will be develop to maintain this working method.
    see tools for Exploratory testing at http://www.qatestingtools.com/exploratory-testing-method
    if you know more tools that have special features that support this way of working, please update me. thanks

  5. Buu Nguyen says:

    @Yuval: we developed a tool that support exploratory testing: qTrace https://www.qasymphony.com/v2/exploratory-testing-with-qtrace.html. Please feel free to include it in your list.

Leave a Reply

Your email address will not be published. Required fields are marked *

More Great Content

Get Started with QASymphony