Trends in testing (part I)
Author: Danny E.
Elements was invited to join Testnet‘s fall event named Trends in Testing on October 14. Our test engineer Danny E. went to Nieuwegein to attend the talks and workshops, learned a lot and wrote down his experiences for all of us.
Test Better
The first workshop I attended was titled “Test Better” and was held by Robert Shaffer, a SaaS web tester, and David Colter, a mobile tester both working at BBC Digital. It was a super interesting interactive session which gave me some new insights about testing. After an pretty unusual introduction, which included name shouting, dinosaurs and a love declaration to the person next to you, two methods of testing were discussed: Socratic, which is basically hypothesis elimination (disproving) and Scientific which is the confirmation of a hypothesis (proofing). These were illustrated by a few games.
Jenga
A Jenga tower was built and it symbolized a new software application. Each block represented an application feature and had a unique number. Randomly six blocks with “application bugs” had to be removed from the software tower, to be later re-added (on top of the tower) without the bugs. This made the tower quite unstable. Before the second tower was build, all “feature” blocks were laying on the table. As the tower wasn’t built yet, three blocks with application bugs could already be removed. Now the tower could be built and only three bug blocks remained. The three bug blocks were removed and the tower (software) was way more stable.
As a tester involved early in the development process, most problems in software development are already addressed before they can happen. The tester needs to be the Negative Guy during story writing and sprint planning.
This was a really nice way to symbolize the effectiveness of having a tester throughout a whole project (Agile) instead of only testing the software once it has been completed (traditional).
Eleusis card game
During the workshop also a card game called Eleusis was demonstrated.
The rules of the game are:
- Dealer makes up a secret rule;
- Other players get 12 cards each;
- Other players ask question to dealer “can I play this card?”;
- Dealer says “yes/no” to the question;
- Correct cards at top row, wrong cards at bottom row;
- Other players need to find the secret rule by analyzing the patterns in the correct row;
- First player that guesses the secret rule correctly wins.
During this game, we learned about proving and disproving rules. It was a way to make you think about how to think what needs to be tested. Are you trying to disprove a certain rule and test that? Or are you trying to proof a certain case with your tests? It was a very interesting way that really made me think about testing in general.
Mind maps
Mind maps can be used to keep track of testing issues and features with meaningful icons and different styles. Topics to think about while working on mind maps are:
- High or low priorities;
- Good or bad design features;
- Linking directly to documentation of specific features;
- Quality criteria of an application (capability, usability, appeal, security, scalability, compatibility, performance, installation and continued Development);
- Elements of the product (structure, function, data, interfaces, platforms, operations and time).
Wason 2–4–6
The presenters also discussed a principle called Wason 2–4–6 which demonstrates confirmation bias. They showed four cards on the screen with letters and numbers on them: A-N-3–4. Afterwards a statement was given that needs to be proved or disproved with the fewest amount of card flips. The statement is: Every card with a vowel has an even number on the other side.
The reason the fewest amount of card turnovers had to be done is because each card flip:
- Costs € 10,000;
- Takes 6 months to process;
- Might get you fired if the test fails.
A big discussion started among the audience which card flip would give the highest chance of proving or disproving the statement.
Once the presenter started to click on the first card, the letter “A” turned into the number “2”, so a vowel had an even number on the back, it seemed. He then started to click a couple of more times and the “2” became an A again (he flipped the backside of the card back) and the “A” turned into a… “7”? Yes, the whole point he was trying to make is that you should never take the information you get as a tester for granted to perform a proper test. The lesson learned here is that a “card” on a screen does not behave like an actual real world card which always shows the same information if you keep flipping. It’s a computer program, so anything can happen. In other words: don’t expect anything, but in stead try everything. The fact that each card flip costs € 10K doesn’t help either, but don’t try to limit your tests or thinking about testing something properly by limitations like this. Instead, try to question the rule, maybe there is a much faster and cheaper way to have way more card flips. Try to change the rules, but be nice about it. Try to ask the questions no one else is asking and try to obtain as much information as you can by looking at the information you don’t yet have.
Overall all it was a great workshop and one of the best of this event.
In a new blog post I will write about the other the talks I saw at the event.
Originally published at www.elements.nl on October 15, 2015.
