Attending the world’s largest test festival
Lectures, workshops, tutorials, didactic games. A time to share good practices, new automation tools, social gatherings and meet professionals from all over the world. Time to exchange ideas and, especially, to be in contact with the greatest representatives of agility. This is Agile Testing Days, the world’s largest agile test event, which after 9 years of existence, has come to the United States to see its first edition outside the European continent.
Between June 25th and 29th, me and Marcelle Araujo had the opportunity to attend the Agile Testing Days, in Boston, which had over 600 participants from various European countries, Asian and Latin countries, such as Uruguay, Mexico and Brazil, as well as the American hosts. This diversity of cultures was one of the multiple flags raised: according to José Diaz (@jdiaz_berlin), spanish businessman and organizer of the event, this year’s event had the insertion of different cultures and ethnicities representatives, as well as the opening for women speakers were focal concerns of the entire organizing team during planning.
On the first day, we were immersed in one of the 7 tutorials available. On other days, the challenge was to choose from more than 40 lecture and workshop sessions.
First day: time to specialize
Me, Livyson, was tasked with absorbing all the possible knowledge provided by the Mob Programming tutorial, which was taught by Woody Zuill (@woodyzuill), a professional with more than 30 years of programming experience and who today works as an independent developer, agile coach and trainer.
The tutorial, which began at 9 am, was extended until lunch, with a lot of theory, but without becoming tedious, since the speaker used fictional stories to convey concepts.
In the afternoon, Zuill carried out programming exercises that needed to be performed within the target software development approach: Mob Programming, an approach that aims to put all the participants of a team together in the same activity, an evolution of the pair programming. You can understand more about this trend that has been gaining space in large organizations through the slides made available by the lecturer:
In an eight-person class (three test engineers, three programmers, a project manager, and a team leader), divided into two groups of four people, the rule was that everyone worked on the same code at the same time, and on a single computer.
Soon after these exercises, we were faced with challenges unrelated to coding and we had to work together to develop other activities relevant to the software development team: set test objectives, manually test, and finally present the content produced — a kind of code review.
Throughout the day, we had the opportunity to explain how we work in our organization, detailing some good practices already adopted and often bringing, through them, the solutions to problems that other organizations presented. At that time, there was a noticeable cultural shock, marked by the surprise of many companies representatives, some already renowned in the global market, who were faced with processes not yet adopted by their organizations and that could solve some of their still expensive issues.
One of the most interesting aspects of the dynamics was that all team members were active agents of development, whether they had in-depth knowledge or not in the running activities, because the idea of Mob Programming is precisely to discover better ways to perform your work, while helping others to become capable of doing the same. The atmosphere was one of total collaboration and empathy, with the appreciation of all the questions, without belittling the contributions of each one: everyone was committed to developing a better product, but also concerned with the development of the whole team.
Unforgettable points learned about Mob Programming:
- Mob Programming is not the same as Pair Programming, it is an evolution of the latter;
- The whole team works on the same thing at the same time;
- All act in the same space;
- Everyone uses the same computer;
- It can be applied to other project activities in addition to coding activities.
“We are uncovering better ways of developing software by doing it and helping others do it.” (Woody Zuill, 2018).
Me, Marcelle, attended this tutorial guided by the experts Lisa Crispin (@lisacrispin) and Janet Gregory (@janetgregoryca), authors of the books Agile Testing (2009) and More Agile Testing (2014). Both work on Agile teams since 2000.
This was the tutorial with most audience, around 40 participants, organized in table groups. The majority were testers, some with less than 6 months of experience on agile projects, most of them with less than 2 years and only 2 participants with over 10 years of work experience in the area.
The tutorial started reinforcing the agile term definition, where test is no longer only a code validation task — a phase at the end of the project — to be an activity that permeates the whole development cycle. Both testers and developers need to have a mindset change.
Testers: instead of “We are here to find bugs…” or “ensure requirements are met…” or “to break the product”, think: “How do we help the team succeed? How can we prevent bugs defects from occurring in the first place?”.
Developers: instead of “we’re here to code and throw it over the wall and then fix bugs”, think: “What can we do to help create testable code and deliver the product successfully?”.
Initially the participants were asked to list the issues experienced by their own teams, then to identify which agile testing skills could be used to address each one. Issues such as mindset change, roles and responsibilities in agile environment and technical knowledge for automation were raised. It was an interesting exercise to show that testers can work on different abilities such as thinking skills, technical awareness and domain knowledge to solve different kinds of problems in projects as whole.
The next topic was test activities planning, where it was discussed the importance of understanding the context of the project, who is on the team, the domain and the level of planning needed.
We were then introduced to the Agile Testing Quadrants followed by a very interesting practice using a fake website which consisted on deciding in which quadrant each test applies. Other practices experienced were: exploring requirements with examples, which is, examples are created from a business rule and then turned into executable tests to guide development. Technics such as ATDD (Acceptance test-driven development), BDD (behaviour-driven development, Dan North) or SBE (Specification by Example, Gojko Adzic) follow the same premise. The format of those tests can be tabular, scenario or given/then/when.
Another area covered in the tutorial was Automation, reinforcing its importance on regression tests, as a way of getting fast feedback on continuous integration and releasing testers for “new” test.
The whole team collaboration, including testers, developers and devops are the key for success.
Finally, they presented exploratory tests and its variations such as personas, risks, tours and journeys.
It was a very rich tutorial with dynamics and experience exchanging between different realities.
Second day: many opportunities for a single day
The day starts with the famous social activity Early Morning Lean Coffee, leaded by the well known Lisa Crispin and Janet Gregory. During the breakfast, people from different cultures sit together talking about different topics including agile. A great opportunity for people to break into a new field and learn from others
At 9:30 am, after an honorable announcement from José Diaz, Selena Delesie (@selenadelesie) was in the main stage to present Evoke the Soul of Agile, which brought inspiration to combat the current model of excessive request that many organizations and the market have been adopting, imposing competitiveness without respecting individuality. Here, the speaker devoted her time to showing us wrong practices that we have committed every day in the face of pressure and stress, and the price paid for professional relations and the internal image of the organization. Delesie was able to show that work does not have to be hard and painful, but it can be engaging, interesting and a great place to create good connections. Better yet: it has shown that this can all occur by using agile movement as a tool.
“Stop focusing on DOING better… Focus on BEING better. Engage the soul of people, of teams, and your company for Agile to support your success.” (Selena Delesie, 2018).
As already mentioned, the other days would be of several lectures and workshops occurring, simultaneously, being difficult to choose from which to participate. Well, here are the ones we choose:
API Testing Fundamentals
In this practical workshop, Dan Gilkerson (@dangilkerson) and JoEllen Carter (@testacious) gave us an overview of what can be tested in APIs. We used the Postman tool and a test weather forecast API for the exercises.
We started using exploratory tests in order to identify the variables, test hypothesis from the documentation and heuristic applicability.
After that, we performed component tests where assertions were added based on the return code, contract tests to guarantee that the API’s input/output match the specification and that will not change over time, non functionals and end-to-end tests.
The workshop material can be found at:
Organize anything, together. Trello is a collaboration tool that organizes your projects into boards. In one glance…trello.com
They want it WHEN? UX in an Agile Environment
At 10:45am was Diana Getman’s time (@diana-getman-98425999), team leader with over 25 years of experience, to talk about how to integrate the UX design to an agile process. In this 30-minute talk, she presented techniques that can help in a software development process. The presented UX approach integrated to development team was the most interesting one and raised many questions in the audience.
Testing as Part of the Team Culture
The highly regarded Claudia Badell (@claubs_uy), elected one of the most influential latin American women in software engineering, started her presentation, Testing as Part of the Team Culture, showing a panorama of changes in development process. Badell defended how to build team culture based on lessons learned.
“The importance of nourishing your team culture to truly adopt testing as team responsibility.” (Claudia Badell, 2018).
Automation for the People
With a fascinating title, Christin Wiedemann (@c_wiedemann) managed to captivate an audience eager to find out what she had to offer in a short 30 minute talk. And she did not disappoint: she took to the event the importance of automation for people who test, and differently from what people expected, she did not defend automation for everything. On the contrary, she made it clear the importance of manual tests and how they can add value when integrated with automated tests. Wiedemann advocated respect for diversities within a project, showing that not every tester needs to be a coder.
Owning Our Narrative
After lunch, it was Angie Jones’s turn (@ techgirl1908) to show her scenic, yes, scenic ability — an eloquent and effusive way of holding everyone’s attention — inside the main hall of the event, where all participants gathered before moving on to other lectures. A programming professor, automation consulting engineer and inventor of 22 patented creations, Jones licensed the software dynamics to enter the world of music, showing the whole history of the music technology industry, taking us to a context very different from our realities and with such common problems. With a very interesting comparison, she listed what the musicians taught us:
- Let automation work for you
- Our tools cannot always evolve with us
- Changing imposes limitations
- Adjust success measures
And from there, we could understand that the goal was to lead us to think “out of the box”, give us a view that just like music, our market has gone through many changes. And on this roller coaster of transformation, we can stay within the limited box where we are placed daily by countless factors or we can choose to innovate, empowering ourselves without fear of following what we believe.
Using your testing mindset to explore requirements
In this 1h30m workshop, Janet Gregory (@janetgregoryca) and Ardita Karaj (@ardita_k) talked about the importance of testers to be present since the beginning, being part of discussions around planning and scope. The use of mind maps were presented to be helpful when you have a large backlog and so it’ll help to understand the ecosystem and from where to begin. The context diagram is also a good alternative as it’s more structured then the mind maps.
We also used charters in order to explore and understand better the requirements, before they become a story. From those charters, we practiced the example mapping in order to explore alternatives. The idea is that from a charter we identify the rules and each rule has examples. All this process will also raise questions which helps on the understanding.
Another technique presented was the learning from experiments, especially when there are many business rules, based on a minimum codification, named learning release. They believe that the MVP term shall be replaced by this term because it simplifies expectations.
Defining Heuristics In An Agile Environment
30 minutes of lecture, little time to explore all the content that Ash Coleman (@ AshColeman30) brought us. At that point, she was able to show the vision that other team members often have about the work the quality team has been doing. For the quality teams, who feel in a process isolated from that touched by the development team, Coleman presented some important tips to make our work seen as an important part of all the gear.
Accelerating Enterprise QA to Meet the Modern Performance Imperative
Paul Bruce brought to us a bit of the DevOps and Quality integration, showing how the quality team can have the various platforms and tools to implement, execute and scale automatic tests.
Challenges to effective performance testing in CI
The Uruguayan community was well represented at the event. Among its representatives was Federico Toledo (@fltoledo), who besides being a listener for several lectures, was the speaker of Challenges to effective performance testing in CI. Toledo provided us with his experience in performance tests, showing us how companies have organized themselves to measure their systems, from the time of the survey of the scenarios to the execution of the scripts. The lecture gained more brilliance when the speaker showed points of continuous integration infrastructure, displaying ideas on how to run the scripts several times a day within this practice commonly called CI. The Agile Testing Days itself sparked a few discussions about performance testing — perhaps one more point to explore better in future editions — and it was in this talk that the fans of this test practice, still so laborious, little explored, and still with a small community, were able to explore and share their experiences on the proposed theme. There has been a feeling that we need more advocates for the inclusion of performance testing in the development process.
“Performance testing in CI is a must. If you are looking to invest effort and money in it, you’ll better take certain things into account from day one.” (Federico Toledo, 2018).
TestClash on Automation
Undoubtedly the funniest moment of the day. In the main room, Noah Sussman (@noahsussman) — automation engineer — joined Paul Holland (@PaulHolland_TWN) — senior director of engineering testing — for a duel of giants. With the help of Ash Coleman, Daniël Maslyn and José Diaz, Sussman and Holland used the staging of a boxing match to talk about “the automation of absolutely everything” or “no automation.” According to Sussman, it is difficult to say what specific value a tester could add to his team and that the unit tests implemented by the developers would already guarantee a quality needed for their products: “I can detect issues quickly, recover quickly, and the integrity of my data is guaranteed because of deleted soft deletions and non-destructive schema changes. Because I can detect, recover, and restore data so quickly, I see very little risk if the occasional bug hits my clients. I can just fix this fast!”; on the other hand Holland argues that test engineers are extremely important in the development process, that unit tests do not reach all possible types of bugs and that not everything is automated: “I realized that automation will not find ‘new’ code errors. Once an automated script is up and running, the script will find only newly introduced bugs that are placed throughout the development through code that is being automated. Thus, one can expect that once a script turns green, it will remain green unless developers make mistakes that interfere with running those streams. What about all the other alternative paths or the same paths with different values? Our customers do not follow scripts when they use our code!”.
In the end, after several rounds, Holland won the fight and convinced everyone that automation is important but that alone does not guarantee the quality of the products. You need to automate various levels of testing (unit, integration, system, acceptance) and even then you will still need manual testing to cover what automation cannot do.
Third day: lots of information and learning
Get off the hamster wheel and start adding value!
After some recovered from Agile Morning Yoga or Agile Morning Run, and soon after an enhanced breakfast, everyone headed to the main room to watch Alex Schladebeck (@alex_schl) and Huib Schoots ( @huibschoots) showing us the relationship between people and hamsters. The testers-speakers showed us how many of us still run in circles (like hamsters), trying to improve our processes all the same way. One of the signs to indicate that we are on the hamsters’ wheel: working out of time, small problems seem big, inability to think clearly, moodiness, fear of failure, fatigue. Through the lecture, we could see some examples of how not to be a hamster, coming out of the race in a circle and effectively starting to change the scenarios of our processes. Some pattern changes are: recognizing and accepting, saying no and acting, slowing down, setting realistic goals, reducing the number of things being done at the same time, managing attention and productivity, asking for help, and valuing yourself.
Demo Driven Development and the Agile UX Show
In this workshop, Oana Juncu (@ojuncu) presented alternatives to the demonstration performed at the end of Sprint, a.k.a Sprint Review, integrating the concept of story telling and creation of personas to move from the language abstraction level to something more concrete.
Digital Transformation Requires Continuous Testing & Continuous Testing for DevOps
Solution Architect Jonathan Morar has taken to the public a bit of what he has learned while working in Europe, South Africa and the United States. Showing how the process they use at Microsoft works, which, he says, serves as a reference for other organizations who want to be agile. He argued throughout the lecture that testers can not be a bottleneck in the rapid delivery of new features and fixes. His fierce defense of responding to changes more quickly made several listeners raise numerous questions about how this might affect the quality of what was being delivered. Morar’s speech was controversial and generated disagreements.
Larry Johnsen also presented these points in his talk. The most intriguing of them led the audience to the debate when it was said that “testing is the biggest barrier in interactive delivery because they rely heavily on manual efforts.” Agreeing or not, the audience left both talks debating the subject.
Innovate and Invigorate Your Agile Discovery Practices
Ellen Gottesdiener (@ellengott) is not just a nice classmate of Mob Programming, she is the founder of her own business. Having worked in various IT positions, she was able to contribute with her experience in a lecture that showed us how to go beyond the requirements already defined, not getting stuck with them during software development. Through creative forms, Gottesdiener argued that we should use good practice, but not get stuck with them. We can adapt or improve them within our projects.
The day had been long, with many workshops and keynotes, which would make it difficult for the last speaker to catch the attention of all the audience that was concentrated in the main hall. But Ray Arell (@elmoray) got that done! Founder of a company of consultants and coaches, he shared at the event what he has learned about failures during his 30-year career. Arell argued that during a development process we need to be open-minded about failures that inevitably happen and will happen, but stressed that we need to be prepared to mitigate them when they occur.
“Overconfidence in the level of certainty and predictability is perhaps the most expensive form of avoidable waste in product development.” (Ray Arell, 2018).
With more than 19 exhibitors, including the well-known PARASOFT, Sauce Labs, SmartBear — famous for their automation tools — , the event had space for these companies to showcase their products. Many of them were automation or support tools for Continuous Integration, which unbelievably caught the attention of even the listeners already working with automation, whether for the sake of simplicity or the ease of creating their scripts. mabl, that introduced a test automation service based on machine learning (ML), which automatically maintains tests and identifies regressions to the systems, allows enthusiasts to create new scripts with the guidance of their patrons.
Here’s how mabl works in the video below:
Every day, after the lectures, Social Events were made available, moments in which the participants met for various activities of social integration.
On the first day, Agile Social Night encouraged participants to seek out people with characteristics similar to theirs and to interact with these people in order to build good professional networks.
On the second night of the event, Agile Games Night brought, through gamification, day-to-day work issues to be solved together and didactically.
But the most anticipated moment was the Roller Derby Party After, which took place on the third night in Danvers, outside the venue. Everyone was able to enjoy a female match of the Roller Derby — a sport played by two teams skating counterclockwise.
All these events had good music, food and drinks, and lots of social interactions.
A great opportunity to learn, socialize and share
The Agile Testing Days was a great time to meet professionals from European countries and the United States, who showed us several examples of automation and continuous delivery tools, but also to meet professionals from Latin countries, such as Mexico and Uruguay, who enriched the event with good test practice. The participation of Latin Americans has shown how well represented our region is, as well as the opportunity to meet professionals from countries with other cultures, but with dynamics and dilemmas so similar to ours, as India.
Contact with representatives of renowned companies answered questions that are routine: Mob Programming as a response to the passage of knowledge about automation and better techniques to run performance tests in CI as a solution for projects that need to continually measure their systems. The tutorials were a great opportunity to be immersed in a specific context and to learn in a practical way with professional experts how to execute the proposed subject.
The organization was flawless, providing early morning events such as races and yoga to prepare participants mentally for the day loaded with knowledge, and all lectures and activities always conducted with good humor, not making the event monotonous. We missed lectures and workshops related to performance testing, which was little explored, despite the fantastic speech of Federico Toledo. The deepening in this aspect can be a suggestion of improvement for the next editions. Beyond the quality of the lectures and workshops, already detailed above, the event guaranteed that the nights were moments of more interactions with other people and discussion about what happened during the day.
In addition to all the learning, at different times, we were able to take answers to other companies’ questions, through the exemplification of processes and good practices that we already use, which proved that we are on the right track and that, yes, we are reference in various aspects of test engineering to the world.
This year we will still have the Agile Testing Days in Potsdam-Germany, celebrating its tenth anniversary, and with content as interesting as the one presented in Boston-USA. Therefore, it is extremely important to have new employees representing our organization and enjoying this festival, which is extraordinary.