Comparison metrics simulation challenge for wind resource assessment

Published in

The Wind Vane

4 min readSep 12, 2019

Background

In wind energy, the accuracy of the estimation of the wind resource has an enormous effect on the expected rate of return of a project. Due to the complex nature of the weather and of the wind flow over the earth’s surface, it can be very challenging to measure and model the wind resource correctly. For a given project, the modeller is faced with a difficult choice of a wide range of simulation tools with varying accuracies and costs. If this choice is made incorrectly, either many resources are wasted in needless high accuracy simulations, or the rate of return is incorrect and investors risk losing large amounts of money. As there are currently no guidelines or tools available to the modeller to help with this choice, it is usually left to gut feeling — and this can be catastrophic for investors or acquirers of wind parks.

This challenge is organised as part of the Swiss-funded project “Comparison metrics simulation challenge”, which aims to develop transfer functions between predicted and actual accuracy and costs for a range of tools for the accurate prediction of wind modelling tool suitability, helping modellers choose the best model for a given wind energy project. It is coordinated within the International Energy Agency IEA-Wind Task 31 “Wakebench”.

Goal of this challenge

The goal of this challenge is to collect hundreds of comparison metrics data regarding the “skill” and “costs” of simulation tools both before and after carrying out the simulations, enabling transfer functions to be developed for the accurate prediction of tool “skill” and “costs”. This will help modellers choose the best model for the job for a given wind energy project, without having to carry out hundreds of simulations first.

The participants of this challenge are asked to submit simulation results in a pre-defined template containing comparison metrics and simulation results, based on an initial study carried out at the University of Applied Sciences Rapperswil at the Bolund Hill site. The comparison metrics include parameters such as the quality of the underlying aerodynamic equations, the size of time step, the length of simulation period, the grid quality, the skill of the user, etc., and are currently under development.

Challenge details

This public challenge will be open to the international community and has two stages following a pre-study:

Pre-study: Evaluation of template — Oct 2019-Dec 2019

In this pre-study, the template for assessing the skill scores and costs will be publically released, and participants will be asked to evaluate the template based on their pre-existing simulation results for the Bolund Hill site.

Stage 1: Open data complex terrain (Perdigão site) — Apr 2020-Sep 2020

In this stage, the participants will submit their results in the template allowing us to calculate weighted parameters related to the skill scores and costs both before and after carrying out the simulations. The simulation case will be clearly defined to allow all the results to be compared with each other. The Perdigão site in Portugal, consisting of flow over a double ridge, has been chosen due to its relative complexity combined with the high number of field tests and open measurement data available for validation. All the results will then be plotted on one graph and may look something like Figure 1(a), where each point represents one tool. The clusters are expected due to different category of tool (e.g. linear model, RANS-CFD, LES-CFD). As can be seen, there will be a discrepancy between the metrics predicted beforehand and those deter-mined using the results of the simulations. Transfer functions to better predict the skill scores and costs will be developed based on these results.

Stage 2: Open test cases — Jul 2020-Mar 2021

In this stage, the participants will also submit their results in the template, allowing us to calculate weighted parameters related to the skill scores and costs both before and after carrying out the simulations. However, in this case, no particular test case will be pre-defined, allowing us to collect a much wider range of different sites and external conditions. The results will be clustered according to different categories of input conditions and may look something like Figure 1(b). In this figure, only the best-fit lines through the data are shown, and an example for only two different categories is shown for simplicity. Transfer functions to better predict the skill scores and costs will be developed based on these results.