Why We Prefer DAMP to DRY
Or, How We Found Balance Between The Immovable Object and The Unstoppable Force. Hahaha.
You’ve seen these phrases before: Always reuse code. Abstract. Synthesise. Don’t Repeat Yourself, aka DRY. Less lines is better.
And then there is the other side: Readability. Less is More.
Wat dis? These two aren’t mutually exclusive, but so often in practice, they seem to be. So, how sia?
A Sharing on How We Structure Our Tests and Found Balance Between The Immovable Object and The Unstoppable Force.
In the Beginning
Long time ago, we believed in DRY. Reuse where we can. Abstract Functions when used in various places. Abstract constants when used more than once. Blah blah blah. The whole she-bang.
But soon, we began to see that there was one place, one perhaps unlikely place, where it was a good idea to be repetitive.
We were facing a few problems.
One, lots of people were contributing to the code base (which is a good thing), but people were not finding functions that had been previously written by others. So there were many repetitive functions littered all over the code base.
Remark: It’s not as easy finding similar functions written by others as one might think. Someone might decide to name it “clickButtonAndGo”, whilst you might be thinking along the lines of “Next”. Someone might have put the function at an entirely different layer, or folder.
Two, it was often not immediately obvious to contributors, especially new contributors, what a particular test was doing.
Point Two might just be the single biggest gripe of all new contributors to any repository. A lot of times, things aren’t deliberate, or intentional, and though it makes perfect sense to the original author, because that’s the way they think — it can be a nightmare to others. This is especially intolerable on testing code, as you want to be able to know what is the reason behind a test failure immediately, rather than after some investigation.
If I had a dollar for every time a developer told me, “I tested this. It looks fine. But I’m not sure if I broke anything. Please help me check.”, I’d be pretty rich.
The solution to that, hard as it is to say, was standardisation. Standardisation of structure, format and a set of best practices on what choice to make when faced with a technical dilemma.
And yes, that also meant getting rid of/reformatting/reframing heritage code, that you may or may not have feelings for.
The Solution — Data, Page-Object, Script
We grouped our test files into three categories: data, page-object classes and test user scripts.
Why separate the data out? Because I wanted it to be easy to see what data was being fed in. That would make it easy to update or change the data, if business side necessitated.
Why have page-object classes? Because they were the most natural organising construct. The most true-to-form with what the user sees, and thus the most likely for a new contributor to grasp and sensible for them to search. For example, if I told a new contributor, hey we have page-object models around here, and follow it strictly, they would then know that if they wanted to perform an action on a particular model (be it, Modal, Section or Page), they would first look for all the respective (Modal, Section or Page) classes to see if it had already been implemented. This would facilitate easy reuse of written classes and their methods. You see, it’s not just about writing reusable functions; it’s about getting people to reuse them by making them easy/possible to find.
One format we use and try to follow is [PageObject].[verb][noun]. So page.clickButton would be permissible, or even button.doSomething. but samsRandomUtility.feature21 would not be.
Why have test user scripts? Here’s where DAMP comes in. The acronym DAMP comes from Google’s TotT: Descriptive and Meaningful Phrases. I think of it as a reminder to make things as readable and understandable to anyone who will read the script in the future. Once again, we take our cues from the application itself. Most people will say that things become obvious when you use an application (or they should be). So since we are doing Automated User Testing, we might as well write our test scripts from the perspective of a user. There’s no need to clutter the script with extraneous jQuery get statements. Those should be tucked away under the hood in our page-object classes.
In another way, our test scripts form documentation on appropriate user behaviour, or user behaviour that we test for. We don’t really use it as acceptance criteria, but we certainly know what a user can do on a particular build by reading the test scripts that passed. And if something does fail, we know at which user step it has failed. Coupled with our data files, we know exactly 1. what data we need to provide, plus 2. which user step/action triggers failure. This makes understanding and reproducing failures a breeze.
So let’s take a look at a few examples of this brave new world. We are using Cypress as our testing framework, on the JavaScript language.
Here is a sample data class:
We have a data class for every test script. Each data file feeds to its matching test script.
Here is a sample page-object class:
What this means is that in the event that the component name changes, it won’t, and shouldn’t affect my test script. It would only affect this function in the page-object class. Because that’s an implementation that changed, not a user flow that changed.
So yay, I get to hide all the nastiness of cy.get (or any kind of Inspector-based DOM-based jQuery statement) in a separate layer, away from the test script where I only want user action logic. And if ever it made sense to extend some of the page object classes, in order to reuse functions and/or structure, I’m free to do so. Say I wanted to extend modal.js with another Page Object file licenceConfirmationModal.js , no problem. I could just create another class file, extend the original class, add the functions I need, and start calling licenceConfirmationModal.[methodName] and other functions in my test script.
Here is a sample test script:
- If a test were to fail at a function called
fillInCreditCardDetails, we would know that it could possibly be a problem with a recently done frontend refactor, unless Product had decided that users could no longer pay via credit card on the platform. Previously, if you had put yourcy.get/DOM-selector commands in the test script file, and it fails, you wouldn’t know if it was a problem with yourcy.get/DOM-selector query and/or a change in your component, or a change in business logic.
Now take a look below. Notice that when I do a diff between two different test scripts, there’s virtually no difference between them, except the data being inputed.
You laugh. Sam! What’cha doing! What is this nonsense!
Flows, Not Features
It’s not nonsense at all. The reason we maintain two test script files that are virtually the same is because from our business owner’s perspective, these are two different types of applications, with two different results (one allows for instant approval, the other, not). Does it matter that they are almost exactly the same form during application? No. We decided that we want to test both these flows, as they are both critical flows in our application. The fact they are so similar informs us about their similarity in user flow. That’s what matters.
We could have written a kind of loop to loop over the one test file with different data. We could have done that. But we didn’t. Just to make it obvious when one of them has failed. We are not testing that the user can apply for some form. We are testing that the user can successfully run a through-flow for each of these critical flows that we have artfully earmarked. If any of these test files fail, we’ll know which flow was affected. If all of the test files fails, then all the flows were affected.
Some of you might suggest that even with a loop, it would still be possible to do logging and make it clear which application type was failing. That is true. Even the benefit of being able to run the tests concurrently as separate files is not unassailable. There probably is also a way to run items concurrently, without repeating code as we did. But would it be easier to understand for a newcomer than this? Would there be more boilerplate code or libraries you would have to import to support that additional unintentional functionality, resulting in a Rube Goldberg-like design?
We do use loops in some situations/circumstances. It’s in a file that we run in our standalone environment. Here, we are not concerned with user flow, but only ascertaining certain core functionality that we want repeated via a particular parameter.
The point is not about having rules like “use loops where possible” or “no loops”. Similarly, it’s not about “less lines of code” or “more lines of code”. Or “no repetition” versus “some repetition”. It’s about what makes the most sense for that particular purpose.
How does one choose the essential flows? Choosing flows is more an art than a science. Ours is a licensing portal, so we have a flow for every licence application. At any given time, we know that a user can login, open up a premise, and apply for a particular licence. Do we have flows that have users with existing licence applications and then applying for more? No. Why? How would we then maintain this complexity of tests? Remember, we’ll have to maintain that suite of tests. And no one wants to maintain more Automated User Testing tests unless necessary. Recall the testing pyramid?
Earnest testers, or those that get paid by lines of code generated per hour, may like to write automation code after every feature card they test. If so, get them to write it as low on the testing pyramid as possible, starting from Unit Tests, then to Integration/API/Service Tests, and if they can’t be written on those lower layers, then and only then, at the E2E/UI Test layer. A pyramid is thickest at its base, guys. And whilst coverage is good, it needs to be done at the right layer so that it A. makes sense for the amount of resources consumes, B. does what its supposed to do and is not flaky, C. is maintainable and will not be thrown away. Also, I don’t know anyone whose KPI is to write as many lines of code as possible; but I’ve certainly seen agents who behave this way.
Give me a good way how we can maintain flows with existing licences and I’ll be happy to listen to it. (It’s on the backlog.)
Finally, your testing should not be coupled with your implementation. It’s okay to have one modal class for all modals, but because it works, not because you know your frontend reuses modal components. Your user has no idea how you implemented something, and doesn’t care. Neither should you (when it comes to testing).
Are these methods foolproof? Of course not. A malicious actor who sees existing classes can still go ahead to implement the same thing as a different function. And people can still be repetitive and illogical. Or grammatically incorrect.
In the end, I’m not saying that you shouldn’t try to abstract and reuse functions. It’s where you do it that matters. We do that at the page-object level. We have clear steps (and sometimes a lot of repetition) at the scripting level. And our fixtures (data) also contain similarly repetitive structure. All in the name of readability and reuse.
There’s a design principle known as The Principle of Least Astonishment. It’s about making sure your code base follows certain consistent design rules and patterns; so that it becomes intuitive and easy to understand, maintain and add to. We started out with a similar aim of making our code base sensible and intuitive for any newcomer. I think we’ve maximised and reaped the benefits of both readability and reuse. Don’t you?
Mega thanks to Steven Koh, and a huge shout-out to my team-mates in GoBusiness Licensing: Saiful Saini Shahril, Dawa Law, Cheryl Wong, Cho Zhi Ying, Curtis Tan, Lai Kwan Cheng, Yew Kwok Chung and Chai Sheng Hau.
A presentation of similar content was given on 5th March 2020 to the Quality Engineering Chapter in GDS led by Koh Keng Hun.
Thank you Lim Xyng Fei, Wendy Soh, Thia Chang Chao, Ong Jin Jie, HATS Team and others for your warm support.
