Custom scripts: faster, cheaper, better!
15 tips for better scripts and more responsive requests
Do you use the Sentinel Hub API? Do you want to improve your evalscripts while saving processing time and cost?
With advanced custom scripts, it’s important to write clear and efficient code. When every second of processing counts, every line of code matters. This blog post will give you some tips on how to write efficient evalscripts (custom scripts). It will also provide insights into possible uses of evalscripts which are not necessarily obvious at first glance.
When it comes to optimization, while making a single OGC/API request, the performance differences are often negligible. In some cases, however, they are not, and with even a single request the difference in responsiveness can be easily visible. Of course, when making thousands of requests and/or very large requests, even small differences add up.
At Sentinel Hub, we are constantly striving to shorten processing time while increasing flexibility. We want to share the best tips from our expert team with you, which will hopefully help you write faster scripts.
Tip 1: Avoid map! Use FOR loops
This one is big because it can significantly speed up even a single request.
Use for loops instead of the javascript functions .map, .reduce, .filter, .find in evaluatePixel. Your app may be noticeably slower if using these functions. While they may be convenient, they are not fast. And for very simple map functions even a for loop may be redundant, as it may be possible to compute the values explicitly for each component.
Additionally, avoid foreach, as for loops are still faster.
The performance hit can be significant!
Avoid:
Do:
Or in this most simple case even:
Tip 2: Request what you need
Avoid requesting a larger area than you actually need. Similarly, inputting a longer time range than necessary for temporal requests will take more time to process and use more processing units. Temporal requests can easily get out of hand as there is a LOT of data available. If unsure and if possible, consider testing with minimal bands and minimal scenes on a minimal area before launching the juggernaut.
Tip 3: Delete code that isn’t used
No need to define variables or functions that are not used in the final computation. If you do not wish to delete this code because you are testing, for example, consider commenting it out instead.
Tip 4: Calculate as needed
If the output depends on a number of factors, calculate your indices/functions after you evaluate the conditions and not before. This makes the evalscript faster since computations are done only when needed. For example, instead of calculating both the NDVI and LAI for each pixel and then using one, compute these values only when you know which one is required.
Avoid:
Do:
Tip 5: Avoid excess input bands
It is bad practice to specify more input bands than requested in the output. The script will work, but slower, as the system needs to load more data than it needs. It’s also the bands defined in the setup, that determine the number of processing units used.
Tip 6: Avoid unnecessary output formats
In some cases, an output image is not actually necessary. If you find yourself in such a situation (for example if you need some kind of metadata only), set the number of output bands to zero so you don’t need to deal with an image.
Tip 7: Output smaller formats
If you don’t need a large output format, such as float32, consider using UINT16, or even smaller, UINT8, instead. More about output formats here.
Tip 8: Use digital numbers if possible
For Sentinel-2, digital numbers are divided by 10,000 to get reflectance. If you want to calculate normalized difference indices for optical bands, such as the NDVI, scaling is relevant. In such cases, it makes sense to use digital numbers (DN) as units. Doing so will be very slightly faster since reflectance conversion will be avoided. More on the setup function here.
Additionally, requesting UINT16 data of Sentinel-2 digital numbers carries the same information as FLOAT32 data, while costing fewer processing units and requiring less bandwidth (thus faster).
Tip 9: Use global variables
You may set global variables in the evalscript which you can access/modify in evaluatePixel (or any other function). This is useful for basic statistics computation, for example. There is no performance hit or benefit in doing so, we would simply like to mention that it’s possible.
Tip 10: Use visualizers
Visualizers are convenient classes you can use to control visualization within your scripts. There are multiple types available. As they are optimized, they should work faster than the custom function (how much we can’t say as it depends on the function).
You can use visualizers with two different functions:
- viz.process(value) is used when only working with one value/band
- viz.processList(valueArray) is used when working with multiple values/bands. (The availability of this depends on the visualizer)
For example:
Tip 11: Use predefined products
Many visualizations are already available as predefined products in the Sentinel Hub layer configurator. We make improvements to these as needed, and in some cases using a predefined product will be faster than its corresponding evalscript version. This is because we can further optimize processing for often used fixed products. Check the configurator, it is possible that the visualization you are interested in is already there.
Note, that you should select the “set product” option instead of “copy to script editor”, to get all the latest updates.
Tip 12: Use a single request for multiple visualizations
Let’s suppose you are interested in the false color and NDVI visualizations of an area. For the false color, the corresponding Sentinel-2 bands are B08, B04, and B03. For the NDVI, you need bands B08 and B04. If you request the visualizations separately, you use two requests, both of which get the B08 and B04 bands. Instead, you can return both at once by specifying the outputs for both in a single request.
To do so, first, in the output object in setup, specify both outputs by giving them IDs, the number of output bands and a sample type. Next, under evaluatePixel, return both outputs pointing to their corresponding IDs set above. A full request for the NDVI can be found here.
Tip 13: Use data masks
Evalscripts run for each pixel, no matter the underlying data. Valid or invalid, the pixel will be processed. If your evalscript is computationally expensive, consider adding dataMask and skipping processing with no data. Note that you can only use dataMask with Version 3 evalscripts.
Tip 14: Filter your scenes
Use filterScenes to additionally filter scenes that enter your evalscript. This is useful for example if wanting to compare two images, but not knowing when exactly they were acquired. A longer time range can be used and filterScenes can then remove all but two. Not only will such an evalscript load faster, but it will also use fewer processing units since extra unneeded scenes are no longer loaded.
Tip 15: Read the docs, see the examples
We understand that using custom scripts can present quite a steep learning curve. Evalscripts are incredibly powerful and it is simply not possible to explain everything in a single blog post.
We, therefore, invite you to visit our ever-improving evalscript documentation and the examples pages. These are written by us and represent various best-practices approaches to evalscripts. They should hopefully get you up to speed without causing overload and provide a starting point for scripts and requests, which can be modified to your specific needs.
Finally, if you haven’t yet seen, the second Sentinel Hub Custom Script Contest is currently ongoing! We hope you find these tips useful and don’t forget to send in your best scripts for a chance to win attractive prizes!
Happy scripting!
