Automation is a ladder: ModelBuilder
Welcome back to Automation Is A Ladder, your friendly guide to Esri’s automation and productivity tools. In part one, we saw an overview of the choices for automation in ArcGIS Pro, and a couple of quick tricks that are baked in to the user interface.
Now it’s time for a gently opinionated take on a common starting-point: the ModelBuilder (MB) framework.
MB is a mature and well-documented feature of Esri’s desktop tools. Rather than just repeating the docs, I’m going to share what I’ve learned about the limitations of MB, as well as its strengths, and suggest some criteria you can use to decide whether it’s the right tool for the task at hand.
The good
MB uses a code-free, schematic interface to represent data and processes in an intuitive way.
This makes it a welcoming first stop on your automation journey. It’s part and parcel of ArcGIS Pro, meaning that you can easily drag-and-drop data and tools from your map, your table of contents, or the Calalog pane. It’s also well-integrated with all of Pro’s other automation options, meaning that you shouldn’t ever have to re-create work that you’ve done in MB.
The less-good
The awesome time-saving power of MB quickly becomes obvious. However, having used it on countless projects large and small, I’ve found that it saves me less time as the complexity of the task increases.
Three contributing factors come to mind.
Custom logic
MB makes a carefully considered trade-off between power and accessibility. As a result, support for detailed custom logic is limited. You will discover the boundaries of this compromise when you begin to incorporate more complex logic into a model. For example, MB has its own iterators, but they come with two important health warnings,
- An iterator will repeat the entire model on each iteration. This is a bit counter-intuitive. I say this because the very sequential execution of all other MB behaviour would lead most people to expect that the iterator, and everything after it, is the part which repeats. Not so.
- Only one iterator is allowed per model. This makes sense in light of point 1, but it is quite a major limitation. You can sometimes work around this by nesting your iterators inside separate models — but by this point, you are better off brushing up on some Python.
The lesson: don’t use ModelBuilder to try and automate something that isn’t straightforward to achieve in Pro in the first place.
ArcGIS Pro application state
Another caveat is that Pro’s application state changes as you work. As you run through iterations of testing and troubleshooting, perhaps adding datasets to your map, different parameters are created and cached in memory. This raises the spectre of unexpected errors when you start up fresh after a long day of debugging. As the complexity of your model increases, you will encounter more idiosyncrasies which become burdensome to keep track of.
The lesson: always consider whether time spent troubleshooting a model could be better spent on applying an alternative approach.
Good engineering practice
Error handling, unit tests and version control are all essential to long-term stability of your apps and scripts. There is no facility for error handling in MB, which is understandable due to the nature of the tool. In theory, Pro’s own error handling takes care of any issues arising from MB.
Because MB models are binary files that can only be understood by Pro, there are no practical solutions for unit testing (I’m happy to be corrected on this point). Binary models mean that many useful features of version control tools, such as detailed change tracking, aren’t available.
It is still extremely important to use version control in order to revert changes when necessary, but you won’t be able to easily check what has changed between versions. This is not helpful when you return to a very complex model that doesn’t work properly, and you need to find out why.
The lesson: don’t neglect best practices in production tools, or the time you save at first will become a false economy.
So when is best to use ModelBuilder?
None of the above is intended to disparage MB. It’s just advice that I would have benefited from at day one (and which I still occasionally fail to stick to). These caveats can help to avoid some sunk-cost productivity traps, and would have saved me a few days, if not weeks over the course of my career so far.
Python is the long-term destination for most people who do a lot of automation work in Esri. However, the Esri product stack is so large and complex that there is always going to be room for MB to save a bit of time. Some of my ever-green use cases are,
- Simple tasks
- Collaborating with colleagues
- Designing analysis — maintaining the habit of planning and testing analytical processes before running them on an entire dataset
When shouldn’t I use it?
Whilst there are no black and white rules, I can think of some hallmarks of particularly painful projects.
Extract, transform and load process for large or complex data
It is worth remembering that the geoprocessing framework, which MB orchestrates, originated as a collection of analytical tools. It has now grown to encompass virtually all operations that are possible in Pro. However, just because you can do something in MB, doesn’t mean that you should.
Because of all the reasons that I’ve outlined above, I prefer to stick to analysis tasks rather than heavy data management in MB.
Nested models
There are times when you may be compelled to put models inside other models in order to achieve your desired behaviour.
I’ve encountered this when working with iterators, and with transport network analysis. My strong advice to you, is that if you are working at such a level of complexity, it’s time to do your future self a favour and sharpen up your Python skills.
Iterators and conditional logic
At the risk of beating a dead horse, all but the most basic iteration is really the domain of Python rather than MB.
Tips and tricks
Loyal reader, I’ll sign off with some handy hints for getting the most out of MB.
Parameterisation
Parameterisation is the bridge between the flow-chart model that you see on your screen, and a usable tool that you can share with colleagues. It’s a simple concept, but one that can get lost in the extensive MB docs.
Invoke your model in Python
You’ve mastered MB and Python. How do you make sure that your old MB projects don’t go to waste?
Use this snippet to invoke a MB model from a Python script.
Invoke a script in your model
Your colleague has built a totally ingenious script, and you need to invoke it from MB. You can achieve this by means of a Script Tool… which I’ll cover in part three.
Stay safe!
