What is Exploratory Testing

"Exploratory software testing is a powerful approach, yet widely

misunderstood. In my experience, it can be orders of magnitude

more productive than scripted testing. All testers who create tests

at all practice some form of exploratory testing, yet many don't

even realize it. Few of us study this approach, and it doesn't get

much respect in our field. This attitude is beginning to change

as companies seek ever more agile and cost effective methods of

developing software."

James Bach

Exploratory testing (ET) is an approach to test software where the tester does not need to follow a specific test design. But rather, ET should facilitate the tester in testing the complete system comprehensively. ET is seen by some, as a way to conduct simultaneous learning, test design and execution of tests simultaneously. Today, ET is defined by most researchers as an activity where a developer/tester simultaneously learns, design and execute the tests. To summarise this, it means that the tester is exploring the software, learning its functionality and performing test execution on the basis of her intuition. No specific systematic approach is followed in terms of following a scripted test case document that leads the tester to execute the tests on a step by step basis. The tester himself controls the design of the tests while executing and learning more about the software. This helps her in building tests effectively while exploring the undiscovered parts of the software.

Exploratory testing is done with a view that software is bound to have defects in it. The tester has to somehow identify these defects using any possible combination of techniques that forces the software to behave in an undesirable manner. That approach is termed as destructive testing; where the original aim is to destroy the system by specific steps and then check if the system can withstand these destructive techniques. 

Exist various definitions of exploratory testing:

Exploratory testing is simultaneous learning about the software, designing the tests, executing the tests and using feedback from the last test to inform next.

Exploratory testing is an approach to testing that emphasizes the personal freedom and responsibility of each tester to optimize the value of his work. This is done by treating learning, test design, test execution, and test result evaluation as mutually supportive activities that run in parallel throughout the project. 

In other words, exploratory testing is any testing to the extent that the tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests. But don't forget what Exploratory Testing is not a testing technique. It's a way of thinking about testing.

Defining Exploratory Testing (let's call it ET) is easy, just as you might say that juggling is easy to define. Doing it well is more difficult.

We do many, many things in an exploratory way. Driving a car is exploratory. You don't drive by reading explicit instructions for every single movement you make. Having a conversation is exploratory. You don't go around with a prepared speech for every single occasion, do you? Much of cooking is exploratory. Most game play is exploratory. How long would you like this list to be? If you're human, then you are a creature with and exploring nature. Unleash your inner explorer and put it to work for testing. 

What Exploratory Testing Is Not?

Touring

Touring is one way of structuring exploratory testing, but exploratory testing is not necessarily touring, and touring is not necessarily exploratory.

Touring a program can be done in a more scripted or more exploratory way, just as touring a city can be done in a more scripted or more exploratory way. A tourist has many options. Before going on a trip, a tourist might study what is already known about a particular destination. To prepare, she might supply herself with maps and travel guides, and some ideas about destinations of interest. Upon arrival, she might choose a set of walking tours from a guidebook and follow the routes closely, eating only at the restaurants identified in the guidebook, noting buildings and artifacts and other objects of interest by matching them with the descriptions and illustrations. At a given site, she might listen to a prepared audio guide that directs her observations very specifically. She might spend all of her time in the presence of a tour guide who tells her what to observe and how to interpret it. She might choose to accept everything the tour guide told her as the complete story, and refrain from asking questions. Even though the experience would be new to her, and she might learn something from it, she would not likely add much to what is already known. We call that activity touring, but it isn't very exploratory, and a report on such a tour would largely recapitulate the guidebook. Is your testing like that?

On the other hand, rather than touring like a tourist, she might cover a territory as a historian, or a social scientist, or a travel writer. In that kind of role, she would have a research goal based on the idea of obtaining new knowledge. Learning something new and imparting it to other people requires a more open agenda than sitting on the bus while someone or something else directs your attention. Our researcher might make her way directly to particular destinations or landmarks and begin her research there, or she might amble through neighbourhoods or historical sites to discover new things about them. She could choose to focus on specific aspects of what's there to observe, or she could choose to let the observations come to her—and, of course, she might do both. She might work with descriptions that she had been given with the intention of adding to them, or she might work from a set of questions that haven't been asked before. Depending on her mission, she might choose to look for specific patterns or problems, or she might seek deeper understanding that would help her to identify or refine what kind of patterns or problems to look for. Even though the mission to discover new information might come from someone else, she remains in control of the specifics of the itinerary and of each activity from one moment to the next. Is your testing more like that?

One of the hallmarks of exploratory activity is the extent to which it is guided and structured by the person performing that activity. Another hallmark is the extent to which new knowledge feeds into choice of which action to perform next. Touring is not equivalent to exploration; touring can be done is a scripted way or an exploratory way.

After-Everything-Else Testing

Exploratory testing is not "after-everything-else-is-done" testing. Exploratory testing can (and does) take place at any stage of testing or development.

Exploratory testing is not a stage or phase of testing to be performed after you've performed your other test techniques. Exploratory testing is not an "other" test technique, because it's not a technique at all. Exploratory testing is not a thing that you do, but rather a way that you work (and think, and act), the hallmarks being who (or what) is in control, and the extent to which your activity is part of a loop, rather than a straight line. Any test technique can be applied in a scripted way or in an exploratory way. Do not say "we do exploratory testing after our acceptance tests are all running green".

Tool-Free Testing

People often make a distinction between "automated" and "exploratory" testing. This is like the distinction between "red" cars and "family" cars. That is, "red" (colour) and "family" (some notion of purpose) are in orthogonal categories. A car can be one colour or another irrespective of its purpose, and a car can be used for a particular purpose irrespective of its colour. Testing, whether exploratory or not, can make heavy or light use of tools. Testing, whether it entails the use of tools or not, can be highly scripted or highly exploratory.

Quick Tests

Quick testing is another approach to testing that can be done in a scripted way or an exploratory way. A tester using a highly exploratory approach is likely to perform many quick tests, and quick tests are often key elements in an exploratory approach. Nonetheless, quick testing and exploratory testing aren't the same.

by Andriy Skop, QA Project Lead