To monitor or control?
Cast yourselves, for a moment, into the shoes of an environmental manager responsible for Ashmore Reef: you are very concerned about tropical fire ants, which have taken to eating the eggs of native turtles and seabirds — but it’s hard to know exactly how bad the problem is. You’re working with limited resources, which you could use to control the infestation, by dropping poisoned baits from a chartered helicopter, or send a team to monitor the ant population, a 120 hour round trip.
This is a tough problem, and one that crops up across invasive species management. How should we allocate our conservation resources to problems of unknown severity? How much of our effort should be given to monitoring the problem, to better inform future decisions? This is the subject of our recent paper, published in Methods in Ecology and Evolution.
The situation we analyse is this: a manager is faced by an invasive species, whose population is of uncertain size and is growing at an uncertain rate. In response, they can: act to control the invasion; monitor, to reduce uncertainty; or do nothing, saving resources to be used elsewhere.
If the manager can observe the state — that is, the size of the invasive population — perfectly, then what we have described is a Markov decision problem. The theory of stochastic dynamic programming (due to Bellman in the 1950s) can solve these problems, producing answers like “if the population is greater than X, $Y should be spent on control.”
Once we add the uncertainty back in, we have a partially-observed Markov decision problem (POMDP). In brief: a manager is tasked with making a sequence of decisions, and in response to each of their actions, the environment changes to some new state. The manager observes the state of the system, but this observation may be imperfect. For example, on Ashmore reef the manager is the Australian government and the system’s state is the fire-ant population. The manager makes their choice to monitor, control or do nothing, and in response the ant population changes — possibly grows by reproduction, or is reduced by the control efforts. The observation the manager receives is the outcome of any monitoring they decided to carry out.
Following Åstrom (1965), we can solve a POMDP by changing the state space: instead of analysing the process itself (the population of ants), we study the manager’s belief about the system. Then, instead of deciding what to do in cases of low, medium and high population, for instance, we have to examine every “belief state”, one of which might look like figure 2.
Put another way, the decision problem asks us to compute a function π:S→A (“pi” for “policy”), from the set of states S to the set of actions A={do nothing,control,monitor}. In the fully observed case, S contains the possible sizes of the invasive population — say S={1,2,…,100} or {low,medium,high} — meaning π is a function of one variable. In the belief state approach, S is the set of all possible beliefs about the size of the population. In this case, π has one variable for each population size! Each variable keeps track of the probability that the population is that size — viz. each row in figure 2.
Now it’s clear that if the invasive population varies continuously over a wide range, the belief state approach quickly gets computationally difficult. Stepping back, it’s not even clear that all these dimensions have added anything useful. For example, a reasonable belief distribution might look like the blue curve in figure 3: an estimate of the population (≈20), with some spread to higher and lower values. But how are we supposed to deal with the orange? This doesn’t correspond to any reasonable real-world situation! Including it in our model makes decision-making hard, by introducing a huge number of dimensions to the model, and makes interpretation hard, as the model contains beliefs that would never crop up in reality. Number crunchers file this under the “curse of dimensionality”.
What I’ve described is how out-of-the-box POMDP solvers approach problems like this. The user tells the program what the states are, and the solver computes (at great effort) the function π of 3–10-however-many variables that they need. Then what happens? Does an environmental manager have a good idea of the probability that there are 1, 2, 3, 4, … invasive animals? Probably not. More likely, they have a guess of the size of the invasive population, and an idea of how good this guess is — that is, of the uncertainty in their estimate. We could convert the manager’s values into the format the solver needs, but what would happen if we went the other way? This is our approach.
Let’s look at an example of what results. In figure 4 we visualise the belief space of our manager in two dimensions, split into regions based on the optimal management policy.
The vertical axis measures to the abundance of the pest: regions higher up the graph correspond to situations with a larger pest population. Moving up the left-hand side of the picture, the decision-colour changes from “ignore” at low abundances, to “control” and upward to more intense control methods.
The horizontal axis quantifies the uncertainty in the manager’s population estimate — as we move rightward, it becomes more likely that the manager is advised to monitor the size of the pest population before making further decisions. Note that monitoring is more useful for medium-sized pest populations, since then the decision is uncertain, not just our knowledge of the population.
Our technique is known in the POMDP literature as density projection, though to our knowledge we are the first to apply it to an ecological problem. We lift the curse of dimensionality by looking at a lower dimensional section. Importantly, the dimensions we pick have intuitive content, and as such we can produce meaningful qualitative recommendations, like those in figure 4.
Returning to our fire-ant case study, we ran simulations to validate the application of our method. On the left is a time-series, showing the manager’s belief compared to the true abundance of the invasive pest. Note that after the monitoring step, the difference between the two vanishes, and the range of uncertainty collapses. On the right is the same time-series, with the steps plotted on the belief space of the manager. The state moves up and down as the ant population grows and shrinks according to the manager’s control interventions. After the monitoring step, the state jumps to the left, where uncertainty is much lower.
We hope that the methods we have developed might help environmental managers make decisions. We’ve created a repository containing all the code you need to run our models with your own parameters, including making plots like those above. Beyond this, however, we hope that our overarching methodology inspires similar approaches to questions in ecology and elsewhere. Results that no-one can interpret are no good! If you start your model-building process from the needs of the model’s potential users, the resulting tool will be more useful, and will provide more qualitative insight than one developed in full mathematical generality.
— Tom Waring on behalf of researchers Vera Somers, Michael McCarthy and Christopher Baker.
