Merge Request Template Example in Automated Testing
In this article, we will share about one of the best practices that QA Engineering team could develop to ensure the quality of the test scripts.
WHAT is a MR Template
Merge Request Template (MR Template) — it’s a checklist for us to ensure our test scripts/code is good to go and also as a ‘QA’ to the test scripts. It’s also a short summary for MR reviewers to know the story of this MR and why do we need the changes.
WHY do QA team need MR Template
Not only developers will make mistakes during development, QA could also have mistakes while coding the test scripts.
By submitting MR template:
- We can prevent contributors from simply merging code that are ‘not elegant’ into the master branch.
- Can get the idea that how teammates design the AT cases. It is similar to test case review process.
- MR template also acts as a reference to each QA’s contribution to the AT repo as the MR contains all the detailed code change information.
MR Template Example
This example will be based on coding language Go.
Story
- [Why do we need this MR? Let your code reviewers to know your story!]
Summarise your reason of code change in this story section. Basically includes: why, what, how.
Test Case Naming
✅ Test case name should tell what it is, i.e. Test_{server}_{module}_{function}_{behaviour}
✅ Should use all lowercase character except the first letter ‘T’
✅ Test case name should be unique and ‘the only one’ across the AT repo. If you are having test cases: Test_x_y_z [1] & Test_x_y_z_missing_shop_id [2], update the name of [1] to Test_x_y_z_p0 to make it ‘unique’
Test case naming convention helps to increase the consistency among thousands of cases.
Test Case Name are always a very important feature point to a test repo. By using snake_casewe can increase the readability, reader can have a rough idea of: ‘what is this test case trying to do’.
Why not camelCasebut snakeCasefor Test Case Naming? Below article explains subjects can identify the snakeCasestyle more quickly compared to camelCase.
Product Code Not Merged into Master Yet?
✅ Add TO_BE_UPDATED_AFTER_RELEASE_Test_xxx as prefix to your test cases to skip them for AT Daily Run
✅ Promise me you won’t forget to update your case name back to normal when your feature code has been merged into master
Usually we will develop new test scripts that test against new product code. We always want it to merge into AT repo master branch when it’s ready to be merge. This is to prevent the MR is left behind by many commits in master branch (which merge conflicts might occurs).
But, if:
- Product code has not merged into master yet and our test scripts already be merged into our AT repo master, AT daily run might face failures.
- Hence, adding prefix in test case name could skip them in daily run.
We should raise another MR to remove the prefix after the product code has goes live/merge into master. This is to ensure our test scripts are always on track with the product master branch.
Is Test Passed?
✅ Is Test Passed? Make sure your test runs are all passed in the last three times with the same code
- [Provide Your Test Result Here: ]
To assure the quality of test scripts, we should always make sure test cases in our MR are always ALL-Passed. Provide the evidence of the Jenkins report link. (Honesty are very important as a QA!)
Avoid using local report as local report might have some settings that only set in the machine, but not in others.(i.e. our golden machine in Jenkins that is used for nightly run)
Is Chaos-Proof? (in flavour of chaos Engineering)
✅ Kill your test case when it is still running, and then resume, it doesn’t generate any dirty data or fail your second run
Generally, test scripts will start with preparing the test data for our test execution. When we are having the setup of data, we will need a data clean-up at the end of test execution ( a.k.a setup and teardown).
At this point we could simulate the scenario where 1st run is fail to complete/pass the run (i.e. due to network issue), when we give it a retry, it should not fail the 2nd run. (at least not a failures cause by dirty data)
Avoid Magic Number
✅ Instead of using status := proto.Int32(1) , should use status := proto.Int32(yourProtobuf.ACC_STATUS_NORMAL)
Keep in mind, avoid using such magic number as nobody will knows what does 1 means here except the ‘magician’ who has written this. Probably the ‘magician’ will also doubt what is that magic after a few days. It’s always hard for others to find out the historical of the magic number. — “We DON’T need any magic in Engineering!!”
Assert Enough?
✅ My test cases have enough assert to verify the result (note: it is unacceptable that just simply to verify the response code. You need to verify the return data is as expected)
Test cases should verify the correctness of response data, logic of the requirement. It’s unacceptable to just verify the response code.
Method/Variable Naming
✅ Use camelCase to name a variable and make it verbs, say, eatAnApple is a good example while eatanapple and eataa are bad
- It is okay to use t, l, k a single alphabet to name a ‘very local variable’ to keep the code tidy
✅ Use CamelCase to name a method and make it verbs as possible. Just consider all methods are public
✅ For DB CRUD method, use CreateXXX, RetrieveXXX, UpdateXXX, or DeleteXXX to name them
✅ For Redis method, use SetXXX, GetXXX, HSetXXX or HGetXXX
- Easier to read eatAnApple vs. eatanapple
- For database CRUD/Redis method we are encourage to use
CreateXXX, so that the purpose of this method is self-explanatory.
Comment
✅ Add comment only when you are really sure your test code can’t tell all of the story (but why?)
To keep our test scripts tidy and professional, we should only add necessary comment in our test code. Please include the reason if you really need to add in some comment.
Code is like humor, When you have to explain it, it’s bad.” — Cory House
Panic Over Fatal
✅ Use log.Panic(YourYellMsg) instead of log.Fatal(YourFatalMsg). This will give you a chance to recover from it if applicable
Dependencies Check
✅ Run dep ensure -v before MR?
✅ Review your Gopkg.toml already?
It’s normal for us accidentally import some wrong/unused dependencies during development of test scripts. We should prevent from keeping those unwanted dependency in our code as it might have failures caused by unknown. Thus, it’s important for us to do the dependencies check.
Test Cases should be independent
✅ My test case result doesn’t have implicit or explicit dependencies with other test cases
Each test cases should be able to run, passed individually without the needs of running other test cases first.
Beautify Your Code
✅ I have run ‘option+command+L’ to format every files involve in this MR
We should always keep our test scripts tidy and formatted. Above shortcut is to format the code in GoLand. Other IDE should have the function to format the code too.
Rule: Your Mom Does NOT Work Here
✅ I will not break other people’s test cases.
Teammates are here to make contribution as a team, making the testing process better and not here to help us to fix the code. Always bear in mind when developing the test scripts, do not break the existing code.
When all else fails
✅ [List anything you want to mention here:]
In case there are some rules/practices that fails to follow, please note down here so that reviewers can evaluate case by case.
We have listed down all the best practices that a QA team could follow when submit a MR. Hope this can help test scripts written by QA to be more powerful! (of course, the most important: to make tester’s life easier!)
Do not forget and try to follow the rules of — KISS (Keep It Simply and Stupid)!
Stay tuned to our next article! Cheers! 🎉
Merge Request Template Example
## Story
[Why do we need this MR? Let your code reviewers to know your story!]## Test Case Naming
- [x] Test case name should tell what it is, ```Test_{server}_{module}_{function}_{behavior}```
- [] Should use all lowercase character except the first letter 'T'
- [] Test case name should be unique and 'the only one' across the AT repo. If you are having test cases: `Test_x_y_z` [1] & `Test_x_y_z_missing_shop_id` [2], update the name of [1] to `Test_x_y_z_p0` to make it 'unique'## Product Code Not Merged to Master Yet? [The Running of Three-Oh-Three](<http://allegedly.petebevin.com/303.html>)
- [ ] Add ```TO_BE_UPDATED_AFTER_RELEASE_Test_xxx``` as prefix to your test cases to skip them for AT Daily Run
- [ ] Promise me you won't forget to update your case name back to normal when your feature code has been merged into master## Is Test Passed
- [ ] Is Test Passed? Make sure your test runs are all passed in the last three times with the same code
- [Provide Your Test Result Here: ]## Is Chaos-Proof? (in flavour of chaos Engineering)
- [ ] Kill your test case when it is still running, and then resume, it doesn't generate any dirty data or fail your second run
## Avoid Magic Number
- [ ] Instead of using `status := proto.Int32(1)` , should use `status := proto.Int32(yourProtobuf.ACC_STATUS_NORMAL)`## Assert Enough?
- [ ] My test cases have enough assert to verify the result (note: it is unacceptable that just simply to verify the response code. You need to verify the return data is as expected)## Method/Variable Naming
- [] Use camelCase to name a variable and make it verbs, say, eatAnApple is a good example while eatanapple and eataa are bad
- it is okay to use t, l, k a single alphabet to name a very local variable to keep the code tidy
- [] Use CamelCase to name a method and make it verbs as possible. Just consider all methods are public
- [] For DB CRUD method, use ```CreateXXX```, ```RetrieveXXX```, ```UpdateXXX```, or ```DeleteXXX``` to name them
- [] For Redis method, use ```SetXXX```, ```GetXXX```, ```HSetXXX``` or ```HGetXXX```# Comment
- [] Add comment only when you are really sure your test code can't tell all of the story (but why?)## Panic Over Fatal
- [] Use ```log.Panic(YourYellMsg)``` instead of ```log.Fatal(YourFatalMsg)```. This will give you a chance to recover from it if applicable## Dependencies Check
- [] Run ```dep ensure -v``` before MR?
- [] Review your Gopkg.toml already?## Test Cases should be independent
- [ ] My test case result doesn't have implicit or explicit dependencies with other test cases## Beautify Your Code
- [] I have run "option+command+L" to format every files involve in this MR## Rule: [Your Mom Does NOT Work Here](<https://www.facebook.com/alvina.vang/videos/10218785238363679/?extid=VeNau2r6VLLv9hAd>)
- [] I will not break other people's test cases## When all else fails
- [List anything you want to mention here:]
Additional Info: To have a Merge Request Template (MR Template) in our git repo, can make use of the alternative that GitLab provided which is creating a markdown files as a MR template in our project.
Here are the documentation about MR Template in GitLab.
