Quality Assurance and Illusion

This is the second in a series of posts looking at Illusion and testing. In the last post we put forward the idea that we do not see the world as it is. Instead, we see the world as it is useful for us to see it. We continuously take in an enormous amount of information and our brain filters and interprets this to help us make sense of the world around us and make decisions. Problems can arise when the brain is presented with conflicting information.

The Stroop Effect

To illustrate this, let’s try a few tests. The first is the Stroop test, which you can try here (opens in a new window)

How did you get on? The second word set is clearly much more difficult to process than the first set. When the word and colour match (known as congruent) it’s easy. You are simply being shown the same piece of information twice. But where the word and colour do not match (incongruent) a conflict arises as the brain processes the word and the colour separately and then inhibits one of the answers to arrive at its response. This requires more time and more attention.

And it’s not just colour and words that can cause this conflict. You can try these other related tests: Directional test and Animal test. In all these cases the conflicting information slows down our response time and increases the likelihood of getting it wrong.

Finally let’s take a look at this test on the Acutest website http://acutest.co.uk/acutest/aptitude-for-testing. The interference here is, once again, caused by reading the words instead of looking for the symbol "f". We’ve collected a lot of data on this test over the years and the vast majority of people get this wrong.

What to do

As professional testers we observe behaviour and report on what happens in the context of what we expect to happen. We are under time pressure to conduct these tests quickly and our reports are used to make decisions about the system under test. We need to be fast and accurate. So is there anything we can do to combat this problem? I would suggest two things.

Remove the conflicting information

In all the examples we used here, if we didn’t automatically read the words then there would be no conflict. So, can we make the words difficult to read? For example, if the words are on paper why not try turning the paper with the words on upside down. Or perhaps we could make the text too small to read. If you wear glasses, would removing them render the words unintelligible?

Remove the process that causes the conflict

By this, I mean remove your brain. Not physically, of course. Just remove your brain from the decision process. This can be done by writing an automated test which reports the answer back to you. No interference will arise because the test will only be looking for one type of information, such as colour or number of "f" symbols.

If neither of these is possible, I have one more suggestion. We looked at attention in the last post in connection with the Invisible Gorilla. Attentiveness requires effort and the longer we concentrate the more tired we become. The more tired we are, the more mistakes we make. So if you have to run tests that involve interference and incongruent information, how you maintain attention and avoid attention fatigue is critical. But that is the subject for another post.

Contact acutest