Feedback from Nordic Testing Days conference — Audience perspective

Software testing conferences have spread up very quickly those last years. With nearly every country in Europe having its own, it can become difficult to make a choice when attending one.

After hearing a very good feedback from the Nordic Testing Days conference in Estonia from one of my previous managers, which compared it to a dozen of others, I decided to follow his recommendation: well, I was not disappointed at all. Let me present you here a summary of what happened.

It is 9 AM on Thursday, 2nd of June, in Tallinn, the capital of Estonia: the public is gathered in the biggest room and waits for the conference to start. The chairwoman comes up on stage, starts her speech, and within the next five minutes she makes a striking announcement: ‘We have plenty of food for the testers gathered here, because we don’t like hungry testers: they may start breaking software!”.

The audience laughs. The context and tone is set up. It sounds promising.

Different audience

Nordic Testing Days is interesting as it attracts a very different audience from the western Europe conferences: most testers are from the Baltic countries — which include Lithuania, Latvia and Estonia — and from the Scandinavian countries. It is an excellent opportunity to meet new faces and broaden the social network, as well as approaching testing from completely different perspectives since some of the keynotes and talks were focused on those people’s experience.

There is also what I call the usual crew from the United Kingdom, which includes testers who are accustomed to speaking at the various conferences around Europe: I can name for example Bill Matthews, Dan Billing and Stephen Janaway.

What came out of the talks?

Monopoly should not prevent quality

Estonia is nicknamed “e-Estonia” as it describes the country’s emergence as one of the most advanced e-societies in the world. The keynote introducing the conference highlighted that: it was presented by Andres Kütt, a digital architect working for the Estonian government, who is collaborating in providing e-services for the vital sectors to interconnect with, such as healthcare, or law institutions. This obviously infers that you have to consider those sectors perspective in order to provide the most useful solutions.

Andres insisted on considering, when looking for improving quality, not only the services they were providing, but the whole ecosystem using it: the users, the sectors, the different requirements. As being the only provider for such services, he warned about the dangerous monopoly way of thinking, which includes the easy drifting towards a lock-in attitude and the denial of quality improvement regarding their solutions, because as he said “people don’t have the choice, they have to use it anyway”.

Instead of that, he advocated for reducing dependency, increasing flexibility by being as transparent as possible towards their subsidiaries: documenting APIs, considering their mandatory requirements to insure system safety. A very efficient ‘documentation’ that they can provide in order to explain how to extract data through their service was done via the delivery of a tool rather than written specification: with a set of easy input configured by the users, the latter can extract the data they want fast. Finally, he also reminded us that quality was hard to define, and being able to ‘fail fast’ helped a lot in getting valuable feedback to improve it.

Security is your job

Mikko Hypponen, a very well-known and reputated security expert, presented a brilliant talk that resulted in reminding us that security was absolutely part of our job. He highlighted that all companies were software one, whether they provide software solutions or not, since all the data nowadays is digitalised and available online.

To demonstrate how important it was for us to be aware of security, he took as an example some of the biggest ‘attacks’ that occurred, such as the hack of Ukraines power grid link in December 2015 (for more info you can read this very interesting and detailed article from Wired). This attack was initially made possible through the reactions of few operators regarding a phishing campaign they had received, containing a malicious Word document attached: they ‘simply clicked on a button’ to enable the malware, exploiting an old-school vulnerability from Microsoft solutions.

Mikko also explained that the state of hacking evolved from random individuals in the 90’s to online organised crime nowadays, competing for the same ‘customer market’.

Why ‘Bad challenges’ in testing are dangerous

Lloyd Roden from the United Kingdom presented us his talk about the “Top 10 testing challenges that we face today”: starting with an energising eye-of-the-tiger soundtrack video showing that challenges are very different and relative to each one of us, he highlighted the danger of ‘bad’ challenges which led to negative effects on us included bad stress.

Within those dangerous challenges lied the incorrect interpretation of metrics by management, or how quantity very often took over quality. Using the example of test case execution count to illustrate that reality: tester A executes 30 test cases per day, tester B only 2. As an incorrect interpretation and shortcut of the mind, tester A could be considered as being more productive, therefore jeopardizing tester B reputation. Now, if we start questioning those metrics, and going more into details, we may find out in fact that tester A had only 5-steps test case with easy and fast pre-requisites to set up, whereas tester B had 40-steps test case with lots of test data to prepare for each test, with no easy automated way to do it.

I personnally remember, to backup M. Roden’s talk, the ‘Pass or Fail obsession’ track talk from Rikard Edgren at a previous session of Eurostar in 2012, where the management often focused much more on the overview of ‘Green or red’ color of test execution, rather than the outcomes of the tests, even if they were shown as ‘green’. By outcomes, I mean new questions, issues seen in areas of the software which were not covered by that specific test, etc.

Workshop ‘Testing smart algorithms’:

Track talks are good but we can easily forget that the best ways to learn come with practice: the conference workshops were made for that purpose, and I was able to attend Bill Matthew’s one, whose goal was to approach testing for those complex algorithms which we frequently come across in our daily lives.

One of the exercices used for that workshop was about testing a recommendation system.

Those systems can be seen on many websites, such as Amazon recommending its customers more articles to buy, or a TV channel suggesting other TV shows to watch, which may — or may not — be similar to the one you previously watched. The — may not — is part of the challenge when you’re building that type of feature, since you may do a ‘bad’ or a ‘good’ recommendation; Unfortunately, there is no specific rule to predict that, since it strongly depends on how customers interpret it: for example, recommending meat food to a vegetarian customer can be perceived as a bad recommendation.

With that in mind, we gathered in teams of five to elaborate and discuss possible testing strategies. The difficulty, again with that type of functionality, is that there is no specific rule, which means, there is no defined and clear behaviour to expect, and injecting two identicals sets of same data into the same test with the same steps may lead to a completely different outcome.

Interesting ideas came out of the discussion, including:

• In order to determine how the recommendation system behaves, test with very small sets of data and use the same steps for different sets to observe the outcome: the recommendation system may work from various criteria for each article, such as colour, size, etc, and the high number of variables makes it too hard to understand the behaviour with complex tests,
• Growing the sets of data with the sets generated in production environment by real customers, in order to reinject it in the test environment and gather more information,
• Since ‘bad’ and ‘good’ recommendations are very subjective, it may be a good idea to determine various customer profiles; You could do that by collecting and analysing plenty of data, or through beta testing.

Those were some of the main keynotes and workshops, but there were plenty of others focused on various topics, such as improving testability, or easying up the environmental issues. You can view the detailed program here.

Lightning talks: interesting content by newcomers

For those amongst you who are not familiar with that concept, it is a 5-minutes talk format followed by a short discussion which aims at encouraging non-speakers to go ‘on stage’ in front of an audience. The session was held in the evening after everyone has had a few drinks to relax, probably to keep it informal and reassure the shyest participants.

Very interesting content also came out of those talks, which were either related to a brief bit of personal experience, either brought up by recent thoughts and reflexion. Here are few examples of the topics covered:

Anticipate bugs by reading code

Have you had a moment when you were waiting for a new software version delivery, and you were wondering what else you could do in order to do more testing beforehand? Even if you don’t understand the code written — specific development language you don’t know about, or you are not familiar with development –, it is still very easy to spot for example the ‘if’ conditions and the user error messages written. The participant explained he had found several cases which had not been dealt with by just looking at those, preventing a delivery which consequently would not have passed the tests.

Leaving testing

Have you already had this weird feeling when one of your fellow tester colleagues announced he was ‘leaving’ his tester position to try on a new role? You may have felt that testing positions are disappearing and people are considering them less and less… unless their unique vision towards quality that they acquired as a tester can be spread in a more efficient way at a different position? The participant here tried to tell us that it is not necessarily negative for the future of testing that people go on and try other roles, and that reinforcing collaboration with developers, product owners with a testing background may have more benefits and positive impacts than it seems at first.

Specialisation versus generalization

Testers may have to acquire so many different skills across their career in order to solve problems and be able to test softwares… Some of them even become specialists in a specific domain: automation, database testing, performance, etc. The participant gave us food for thoughts by challenging both concepts: Are you not narrowing your focus and biasing yourself if you become too much of a specialist? On the other hand, are you not risking to be seen as not technical enough if you don’t have any speciality, jeopardizing future job positions? Reminding us however that generality and speciality are still compatible since the latter always derives from a general core concept, he compared to other jobs, such as being a nurse — general skill — versus being a nurse specialised in attending people having lung cancers: your core skill and objective does not change.

Why going so far away for a testing conference?

There are so many testing conferences in western Europe that you may wonder why I decided to attend a conference so far away from France…

In fact, since no company was sponsoring my tickets, there is only one reason: I was selected as a speaker! It was therefore an excellent opportunity to combine learning and sharing knowledge with other testers.

I will very soon publish another article where you will learn about this conference from a completely different angle, which is the speaker perspective, so stay posted ;)

Why should you attend this conference?

In 2012 in Amsterdam I attended Eurostar software testing, which was the only other ‘big’ software testing conference I have ever attended. Nordic Testing Days may be a bit far away to attend if you are living in Western Europe, and may look similar to others, but I have personally had a much better feeling about it, here is why:

• The organisers were very helpful, well organised and arranging with issues that may occur;
• Through various games and sponsors activities, I constantly felt active and entertained even outside of the talks: treasure hunt, powerpoint karaoke, sponsor puzzles, etc
• Good food and free drinks, even during the party on Thursday evening, that is important after a long day of attention!
• As a speaker — I will come back to it in another article –, I felt very valued through the organisation of my talk, and really felt I was bringing a positive contribution.

Overall I would say this conference was less formal than for example Eurostar, which obviously by its size attracts a bigger range of sponsors and professionals.

Summary

Without any doubt I would recommend this conference to other people working in IT, whether they are testers or not. Very well structured and organised, keeping it fun and social, one of the least formal conferences to attend with such rich content and famous professionals.