Building Data Science capability at the UKHO: our October 2020 Research Week
This blog was co-authored by Andrew Smith, Pascal Philipp, Rachel Keay, Cate Seale and Mike Hudgell.
Using research weeks to push the boundaries of data science at the UKHO
Welcome to our quarterly blog post in which we share our ‘research week’ findings, as we explore the interesting bits of data science and machine learning. This month, the team look at reinforcement learning, automate choosing spectral indices, pre-process satellite images, attend a conference and investigate a whole load of useful tools and techniques!
What is research week?
Members of the Data Science team at the UKHO come up with excellent ideas for how we can apply the latest in machine learning or big data research to the marine data domain. Sometimes these ideas are outside the scope of our current project deliverables, so to push the boundaries of what is possible, every quarter we conduct a ‘research week’ — an allocated time for data scientists to experiment with new ideas, techniques and technologies. In this blog we outline our work, how it might tie into our future projects and how the lessons learnt will help with future projects.
Initial investigations into reinforcement learning: Andrew Smith
Reinforcement learning is a branch of machine learning that involves learning how to take optimal actions through interaction with an environment. An agent observes the current state of an environment (often defined in the form of a Markov Decision Process or MDP), and uses this to determine what action to take. In doing so, a reward is returned that the agent uses to update the best course of action to take in the future (i.e. take more actions that result in a positive reward whilst avoiding actions that result in a negative reward).
Whilst we have embraced many other branches of machine learning in our work, reinforcement learning has had minimal investigation. Hence, this research week focused on learning more about reinforcement learning with an aim of establishing how it might help with some of our problems.
Consequently, to keep things as simple as possible (during this learning stage), I explored how reinforcement learning might be useful for tuning of hyper-parameters when training deep learning models. I decided to focus on if it helps when tuning the learning rate within an Adam optimiser. The learning rate seemed like a good candidate for reinforcement learning because the tuning process typically evolves with the number of epochs a deep learning model has trained on, and thus involves a sequence of decision-making that fits in well with the reinforcement learning framework.
The following images illustrate the set-up for the experiments: action is how to change the learning rate (increase, decrease or no change), state is based on how the loss value has changed over the last 10 epochs, reward is based on if the loss improved, and for the agent, Q-Learning was used with an epsilon-greedy policy.
To evaluate how successfully tuning the learning rate with reinforcement learning worked, a number of commonly used methods for learning rate selection were compared against (e.g. linearly decaying the learning rate). The results suggested that the reinforcement learning approach was as successful as the commonly used methods. Nonetheless, the reinforcement learning set-up is much more complicated than other approaches. Likewise, it has a number of hyper-parameters itself to tune, somewhat making this particular problem redundant. That said, the main aim for this research week was to learn more about reinforcement learning in practice and it has provided many insights into future research directions to which this tool could be applied.
Automating the choice of index bands for image segmentation tasks: Pascal Philipp
Training an image segmentation model often starts like this:
- We have an array of data of the form (num_images, height, width, num_bands)
- Depending on the task, we often add a suitable index band (e.g. the normalised difference water index or the normalised difference vegetation index). The index band is a combination of the bands in the original data block and is appended to it, giving data of the form (num_images, height, width, num_bands + 1).
- This is then fed into a machine learning algorithm
The choice in step 2 requires experience/research/experimentation and may not be the best solution — wouldn’t it be better if the ML model could learn which index band to use? The aim of this research week project is to explore this option.
Modelling
We generalise step 2 by letting the index band be of the form:
This takes place inside a custom layer called IndexFinder, which takes the data with num_bands bands and produces data with num_bands + 1 bands. This is then fed into the ML architecture that would otherwise have received the data block with the handpicked index band from step 2 above. The parameters ai, bi are initialised randomly and trained just like all the other weights. The pre-defined water and vegetation index bands mentioned above have
for some combinations i,j. Can our IndexFinder learn those configurations or find better ones?
Results
Training the IndexFinder layer at the same time as the rest of the model requires callbacks to control the training, as otherwise the small number of weights in the IndexFinder layer wouldn’t get enough attention among the thousands or millions of weights in the rest of the model. For example, we can freeze all but the IndexFinder layer for certain epochs. That worked, but despite the callbacks, I didn’t see much change in the way the new index band is computed. An interpretation of that could be that letting the model find a suitable index band at the same time as letting it learn to work with that index band is asking too much.
Another experiment to try to get more action in the IndexFinder layer was to work with a fixed model that has been trained to work for a handpicked index band. Instead of feeding that index band into the model, let IndexFinder find it. Does that work? Well, we’re not getting the clean parameters (2) above, but there’s definitely more going on now in the IndexFinder layer. Here are some results:
On the left, we have the original image; then the true labels (here: water and land); then two different types of normalised difference water indices which one may use as in step 2 above; then the index band that is added by IndexFinder: before training (random initialisation) and after training!
Conclusion
This was a fun project, a good opportunity to dig in a little deeper and implement custom layers and custom callbacks in Keras. Performance of different experiments wasn’t assessed much — the focus was more on technical capabilities — and I don’t have any results like “the IndexFinder layer improves performance”, but the option to find and use a custom index band is nevertheless something I’ll keep in mind.
Atmospheric and glint correction of Sentinel-2 imagery for marine and coastal machine learning: Rachel Keay
Rachel has written a separate blog post on her research week.
Notes from the IEEE International Geoscience And Remote Sensing Symposium 2020: Cate Seale
For my research week I attended IGARSS 2020 which had a packed technical program of remote sensing, satellites and machine learning. Whilst I was sad not to be learning about these topics from a beach in Hawaii (original location Waikoloa), I was excited to see the latest research.
From the start of the first UK lockdown in March, I have attended numerous webinars, workshops and working groups that were held remotely (I’m sure many others have had a similar experience). This was the first time I’d attempted a week-long conference with a full day’s agenda, and as expected, some things were good, and some not-so-good…
On the one hand:
- I went to a conference (for $10!) that would have been prohibitively expensive otherwise.
- It’s super easy to type a question in the chat and have it answered, in comparison to sticking your hand up in a full auditorium and/or standing up to speak into a microphone.
- Attending conferences in person with a physical body limits you to attending one track at a time and inevitably there were 10+ sessions running concurrently. Here, the poster presentations were prerecorded videos, with the authors available to ask questions after broadcast, and slides and video available for download. Very useful if you get FOMO.
- I avoided the environmental impact of travel, had no jet-lag or hotel costs — multiple wins here!
- I was able duck out of a session without doing a walk of shame. 😊
On the other hand:
- It’s so easy to get distracted by work and home life when you aren’t physically removed from it all, so conferencing ends up adding to your mental load.
- Getting the information is only achieved when the presenter is understandable, but not all microphones are equal! Presenters ranged from crisp and clear to seemingly recorded-underwater-next-to-pneumatic-drill. Not really anyone’s fault, but unfortunately degraded the experience.
- I personally didn’t get along with the use of text-to-speech software and found it hard to concentrate (my brain obviously switches off when it hears computer-voice), but appreciate this may be preferable for speakers that were confident in English.
- There was no networking, and I really missed the opportunity to talk to people.
Overall, it was a positive experience and my respect and admiration go out to the organisers for the smooth coordination of hundreds of speakers and thousands of attendees. The technology needed to run an event like this remotely seems intimidating at best, but (barring the odd terrible microphone) everything went well, and I appreciated being able to continue to attend conferences and training courses while COVID rages outside my door.
Investigation of tools and techniques: Mike Hudgell
Moving window technique
We have written a geomorphology function that classifies pixels of a bathymetric surface into 10 canonical forms, namely: flat, summit, ridge, shoulder, spur, slope, hollow, foot-slope, valley, depression. The function is smart in that you can have different focal ranges and thresholds to suit your use-case, and is proving useful in our current work with LIDAR imagery and seagrass detection. However, the function is slow as it needs to calculate the slope at eight compass points for every pixel. Therefore, the investigation was to see if we could improve the performance of the function by using a DASK moving window technique.
The “moving window” technique essentially splits the raster into many thousands of windows (with overlaps so you don’t get any unwanted edge artefacts) and processes the windows in parallel. We achieved the following results:
So, this was a successful experiment showing an 82% improvement in performance was possible and one that will be worth factoring into the development of our functions going forwards. However, it is worth observing:
- we saw a 35% performance improvement by running the baseline code on a higher spec machine, without adding complexity to the code.
- we saw a 44% and 72% performance improvement (business and experimental laptops respectively) with DASK, indicating that if we go to the trouble of parallelising our functions, we will gain most benefit on higher spec machines.
RAPIDS
We have started to use a new library called RAPIDS that enables us to run many different data science algorithms on GPU rather than CPU which has the potential not just to save us time but also make some algorithms viable — as sometimes the algorithm we really want to use can be just too slow.
Two experiments were carried out, timing k-means clustering and loading a data frame:
So, these experiments were successful in showing improvements in performance and an early indication that we are likely to want to exploit GPUs more than we do currently.
Additional techniques
Finally, I spent time reading around various techniques that we think are going to be useful, namely: Morphological Building Index; Morphological Shadow index; Bathymetric Position Index; Topographic Position Index.
Here is an example synthetic surface we have created from an aerial image using the Morphological Building Index algorithm:
Conclusions
Thanks to all of our contributors this month! Developing new techniques to solve complex problems is an important part of being a data scientist, as off-the-shelf algorithms and techniques only get you so far when it comes to solving new data problems.
Knowing where and how to spend time developing new approaches can be difficult, especially when deadlines are looming. But, by protecting time each quarter for our data scientists to explore new ideas and techniques without the pressure of immediate project deliverables, we’ve found that our understanding of complex problems has increased, our delivered solutions have improved, and the overall skill and knowledge level of our team has increased.
We’d recommend the approach is taken up by other data science teams — no matter your domain.
