Exploratory Testing Undressed — How I Changed My Mind About It?

Exploratory testing is most probably the first type of testing a person interacting with a software is doing, even if it is a tester, a developer or a business analyst.

Cem Kaner defines it in 1984 as:

“a style of software testing that emphasises the personal freedom and responsibility of the individual tester to continually optimise the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”

Exploratory testing has no preset rules to follow, no written test cases and its precise target is to find bugs. At least this was my definition for it.

For years, I’ve seen the exploratory testing as a complementary activity, something to be done when you’re out of any other ideas, when the functional testing is done, when all the integration, regression and scripted testing are completed.

I have always seen the exploratory testing as being that unscripted testing activity which has no specific purpose, which is hard to estimate, to quantify and to track and no planning is done for it. What does the following say to you?

“I did yesterday some exploratory testing on the users page”.

For me means almost nothing. Ok, some testing was performed, I get it, but what scenarios were covered? Were the happy flows covered? Also negative flows, edge cases? Which was the purpose of testing the users page? Just clicking around? Looking for certain behaviours? Were any bugs found? Do you recall the conditions under you found the bugs?

This kind of questions made me think for a long period of time that exploratory testing is just a nice to have. Something to be added when there is some free time and there is no other way to be spent. I used to follow the 1+1=2 pattern when conducting my testing activities, needing to know exactly which is the entry data and which is the exit one.

Yes, you might be laughing. Or maybe you are on the same boat.

But “how does the exploratory testing got undressed though?” you may ask.

I recently read a book that made me change my thoughts about exploratory testing. Here is the name and the author of the book:

“Explore It! Reduce Risk and Increase Confidence with Exploratory Testing” by Elisabeth Hendrickson.

Elisabeth Hendrickson defines in her book a bunch of approaches, some were new for me, some were not, but definitely the following ones are the ones who have captured my attention.

1. Exploratory testing charters

A charter template is looking like this:

Explore (target)
with (resources)
to discover (information)

I personally find this approach very useful in helping you set a specific target to your exploratory session, other than only finding bugs. It also helps documenting the exploratory testing work, by writing the resources used for testing (tools, methods, input data, conditions, constraints etc.) and the discoveries (for example, if a path has the expected behaviour).

Let’s have an example:

Explore Login page
with Introducing incorrect username and/or password
to discover Appropriate error messages are displayed

In this way you can develop as many charters as you want, the only limit being your imagination.

2. The nightmare exercise

Oh, this one is so good!

When you’re feeling stuck or not able to produce new ideas for your charters, the author propose using the following exercise: imagine you just woken up, got out of bed and turned on the tv and suddenly some breaking news are jumping in. Now, think to the worst thing that can happen to your application that worth it be on the news. Maybe your payment system got a crash and the company is losing a lot of money or maybe your bank system just got hacked with a simple SQL injection and tones of sensitive data is revealed.

Don’t feel bad! It is just a dream. At least for now.

Coming back to reality, what you can do is start thinking such scenarios to be investigated during the exploratory testing, scenarios that can lead you to situations when your nightmare might become true.

3. Find the “always” and “never” rules

Every application that we are using has some always and never rules. One kind of rule that comes to my mind is this: an user should always be logged in with valid username and password and should never be logged in when providing an incorrect password.

Try and find the limits of your application, its always and never rules, and start investigate those rules during exploratory testing.

4. Define the nouns and verbs in your application

This is a good exercise when you know well the system you are testing, but gets more valuable when you know little about it and you start searching for meaning by finding the actions that can be performed with certain objects.

Make a list of objects (nouns) and one for actions (verbs). Thinking to an e-commerce website, we can think to those list as follows:

Objects (nouns):
product
category
price
size

Actions (verbs):
add to cart
search
review
add to favourites
share
filter

Now think to the most common scenarios that an user can do: “add product to cart” or “search for product”. How about “search for category”?If you start making combinations between the objects that you have defined and the actions, a lot of new and interesting scenarios can be revealed.

Take away!

Try to make exploratory testing part of your testing routine and work a bit in reinventing the old strategies that you were using.

Exploratory testing can be a worthless job, but can become a valuable weapon when using the right approaches to do it. Obvious things are easy to be spotted and when you want to dig further investigating large and complex systems or when little is known about them, you have now some new ways in doing the exploration.

Happy exploring!

Photo by Christopher Burns on Unsplash.