In-Depth Look at the Programmer Problem Solving Checklist for Testers Writing Automation Scripts
A More In-Depth Look at Items on the Programmer’s Problem Solving Checklist
In my article about a Programmer’s Problem Solving Checklist, the items on the checklist just got a brief mention. Here we’ll go in-depth related to each item, providing an understanding of why we do those things. This understanding will get your problem solving juices flowing and help you excel at writing automated scripts. And, don’t forget to meditate, it’s great for programmers and programming.
By way of reminder, here is the checklist in short form:
Problem Solving — Quick and Easy Steps
- Have I added logging to see what the script is doing at key steps?
- Are my variables actually being set?
- Do my variables and parameters use the proper type?
- Are my mapped objects I want to take action on properly identified?
- Am I using the mapped object’s correct property?
- Is my automation script running too fast?
Problem Solving — More Advanced Steps
- What do the scripting language’s or tool’s docs say?
- Is there a community specific to my language or tool and have I looked up answers related to my problem in the community?
- Have I checked general tutorials and communities to see if helps are available there?
- Have I googled the answer?
- Are there other automation scripts in the project that do this same thing?
- Is this the only way to do this thing I’m doing?
If you’re wondering — what do these things really mean and why is googling in the advanced section, continue reading. We’ll dive into the whys below.
Detailed Look at the Quick and Easy Checklist Steps
Logging
Adding in log messages with details about the test will save you many hours of headache. This is easy to do and after a while you’ll start doing it up front while initially writing your automation script instead of waiting until trouble arises. Some tips on logging include:
Remember to include variables’ values in the log messages (using python logging as an example)
import logging
logging.info("expense rpt total = " + varExpRptTotal)Add log messages inside of Conditionally invoked code, e.g., if, else statements and loops
import logging
if varExpRptTotal > 1000:
logging.info("expense rpt total gt 1,000")NOTE: Most, if not all, of these trouble shooting log messages should be removed before checking in your code for prime time.
Variables, Variables, Variables
The amount of times I’ve run into problems b/c the variable wasn’t set is too many to count. Typically solve these problems using logging to find out when the problem occurs.
Variable and Parameter Types
Problems related to data types, e.g., sending an Integer value into a String-typed variable will be covered below.
Not Yet Solved Example
import logging
expectedExpenseRptTotal = 1000
actualExpenseRptTotal = mappedObject.ExpRptTotal
##this mapped is a String when retrieved from the AUT
if expectedExpenseRptTotal == actualExpenseRptTotal:
logging.info("SUCCESS")This check will fail because it will be comparing the following:
1000 == "1000"
Problem Solved Example
import logging
expectedExpenseRptTotal = 1000
actualExpenseRptTotalFromScreen = mappedObject.ExpRptTotal
actualExpenseRptTotal = int(actualExpenseRptTotalFromScreen)
if expectedExpenseRptTotal == actualExpenseRptTotal:
logging.info("SUCCESS")Mapped Objects can be Dastardly
Mapped Objects are a common source of problems in automation scripting. If the object you’d like your automation script to interact with is improperly mapped, your automation script will be doomed to failure.
The biggest example to consider is mapped objects with dynamic properties that are also used in identification.
There are many good resources online for web-based testing. Also utilize your team and other tests to determine how this has been solved before.
Mapped Object Properties
Even more dastardly than the mapped objects themselves are the mapped object’s properties. Consider an Image for example. It could have many properties with which an automation script may want to interact, for example the source (src), the alternate text (alt), or the style.
<img src="smiley.gif" alt="HTML tutorial" style="width:42px;height:42px;">Since properties can be set differently for different objects and can change as your script interacts with the system, this is one place to quickly look to see if your script is utilizing the correct property and the correct time.
Run too Fast and You’ve Got Problems
Automation tools typically wait for objects, but not in the way a human does. If there’s a slight delay the automation script may just assume a thing has happened when that thing has not yet happened.
Especially watch for differences between running in your local environment and in the pipeline environment. There can be failures in the nightly runs, that you don’t see locally due to the automation scripts running too fast.
Detailed Look at the More Advanced Checklist Steps
Check the Documentation — Language or Tooling
Why is reading an advanced skill you might ask? Well, it’s reading in a brand new language, which is always difficult. A language’s or scripting tool’s documentation is great for the author of that language or scripting tool. They know the tool, they know the language, everything is fine.
For you, the tourist in this new found land, things could be confusing and you may not understand the cues and signs and know which direction is best. How do you start using the tool’s or language’s documentation when you’re still new?
- Start small
- Give yourself time
- Take a somewhat known topic and read about that topic first, for example logging or adding waits, which are quite similar across tools
- When moving to a newer or more complex topic, check your understanding with a colleague
- Take notes about the documentation’s style, approach, and short cuts you use to quickly find things
- Learn what terms work well in Searches and which terms don’t
Use Forums provided by Language or Tool
Some tool providers also provide a forum, where users or license holders can ask questions. This forums can be similarly daunting to using tool or language documentation, but that’s okay. Follow steps similar to those listed above in the tool documentation section.
Note that since users are providing the answers to other users’ questions in most forums, this can be more helpful to a beginner or more frustrating. Stick with it and use when your able to grow your skill in using online forums to find answers to your automation scripting questions.
Use General Forums for your Language or Tool
If you’re using open source tools or languages forums on the interwebs can also be helpful. Same notes apply here.
Google Stuff
This sounds great, right? This sounds easy and quick, right?
Wrong! Or, it can be just the opposite. I have googled many a programming problem only to find myself needing to go back to meditation practices before continuing.
I think the main problem with googling is that it sounds easy. I’ve googled before and found the nearest restaurant, how hard could it be?
It can be very hard for software engineering questions. Here’s two things that helped me.
- Learn the right phrasing for your problem. This will take a bit of research on the general topic first. Followed by multiple attempts to adjust the phrasing so you find helpful information instead of the types of pythons.
- Give yourself time — the problems your solving are challenging and take work to solve, so it stands to reason that you’ll need more than a quick scan to find the help you need.
Copying from other Automation Scripts in the Project
Wait — why is this more advanced? Can’t anyone copy and paste. It should take seconds. Watch yourself there, that type of talk can easily lead you back into those negative self-talk patterns. Take a few deep breaths and then let’s continue true problem solving.
Why is this challenging? I can think of a few reasons:
- You have to know a lot about a lot of tests you probably didn’t write
- You have to be able to weigh the pros and cons of the way you’re planning to do things and they way other tests have done similar things
Are these things doable? Yes, but they take time. So, make it part of your journey to get familiar with other automation scripts and how they operate. It will benefit you in maintaining the scripts and give you good ideas when you write your own automation scripts.
I’d also recommend asking colleagues questions, but be as specific as possible in your questions.
I’m trying to do this certain type of checkpoint. Are there examples of this being done in other tests?
Is This Way the Only Way to Do Things
Usually — NO! There are multiple ways to do things, and you may only be focusing on one way. Why have this been your focus, because you’ve always done it that way, or because you are in fact new and don’t know another way, or because you weren’t aware of new options in the upgraded tool, etc.
Regardless of how you’re doing things, there are times when you need to throw out your preconceived notions and go back to the drawing board.
I do find discussions with colleagues helpful here, but that’s not the only method.
If colleagues, forums, or online documentation start to lead you down a new path requiring significant change, EMBRACE it, at least for a time, and see where it leads you. You’ll surprise yourself as new options and opportunities open up.
In Conclusion
Be a problem solving wizard and embrace the opportunity to grow this new skill. Do this and soon enough you’ll be the test automation programmer people look up to and talk about.
