Cloudrun’s Fall 2022 Update: Faster and More Reliable with Simpler Pricing
It’s been a while and I’ve been quietly working on several important updates to Cloudrun’s internals. No UI changes or new features this time — this update is all about making Cloudrun more reliable, efficient, and correct. Let’s see what’s new in store and how it affects you as a Cloudrun user:
- Newer CPUs, more parallel cores, and improved adaptive time-stepping lead to significant gains in speed and a reduction in WRF completion times
- Compute node provisioning and upstream data downloads have been improved for a faster and more reliable start-up of your WRF runs.
- Simplified pricing of $48 per forecast day, pro-rated to the hour, takes effect on November 1, 2022.
- Cloudrun now offers an on-premise, end-to-end numerical weather prediction system that’s easy to configure, fully automated, and comes with output post-processing and visualization built-in.
Continue reading below for more details.
New hardware, faster compute
Our compute core has been completely rebuilt to take advantage of the newer hardware we now have access to via our cloud provider. Further, your WRF simulations will now run on 50% more parallel CPU cores than before. As a result of both the newer CPUs and more cores, you can expect most of your forecast to run about 1.5 times faster, and for some configurations up to 2 times faster than before.
Updated WRF compute core to v4.4
If you’ve used Cloudrun before, you know that WRF — the Weather Research and Forecasting model — is its key compute component. At Cloudrun we’ve been running WRF 4.0 for a few years now, so an update has been long overdue. This WRF update brings many bug fixes and performance improvements relative to 4.0 (see the list of all changes on the WRF releases page). I follow WRF development and releases closely, so if there’s ever a critical update or bug fix in a future WRF release, rest assured that Cloudrun will incorporate it promptly.
Faster and more reliable compute node startup
When you create a new forecast using the Cloudrun Wizard, our server spins up a compute node dedicated to crunching through your forecast as soon as possible. This involves sending an API request and awaiting an affirmative response that the requested compute is available. Occasionally, the resources may not be available despite an affirmative answer from the cloud provider that they are. Worse, it may sometimes take an unusually long time for the compute node to become responsive, despite it appearing active in the cloud provider’s records. Despite our best efforts to handle such edge cases with defensive engineering, occasionally we would have a hiccup, which would appear as a “hung” forecast on the user’s end.
To address this issue, I’ve completely rebuilt from scratch how we handle the provisioning of new compute nodes and starting up our WRF compute core on them. So far it has proven quite successful, with zero failures in the past month. Of course, that doesn’t mean that it’s perfect and that it won’t have any glitches going forward, but it gives me confidence that it’s a significant improvement over what we had before.
Faster and more reliable downloads of upstream data
If you have experience with regional weather modeling, you know that regional models require external model data for initial and boundary conditions. At Cloudrun, we depend on publicly available government services such as NOAA’s GFS and ECMWF’s Open-ECMWF forecast datasets to initialize our WRF models. Rarely enough but too often, upstream model data is either not available on time, completely unavailable, or slow to download. Over the past few months, I’ve been experimenting with alternative sources and algorithms to produce what seems to be the most reliable and efficient approach. Cloudrun now gets the public model data from a nearby regional CDN in a matter of seconds — in contrast to a minute or more that the downloads took before.
New adaptive time-stepping algorithm
Placing a power tool such as a numerical weather prediction model like WRF into the hands of end-users makes it a challenge to correctly handle all edge cases that could cause the model to go unstable. To work around occasional numerical instabilities, we implemented our own “self-healing” system to allow a failed model to reduce its time-step and resume without the need for user intervention. Over the past year, however, we’ve been experimenting with and stress-testing WRF’s own internal adaptive time-stepping algorithm. After several months of successful operational use with a few customers, it’s time to release it system-wide. WRF’s adaptive time-stepping enables considerably longer time steps in fair weather conditions, thus leading to sooner completion of the run, and automatically reduces the time step when heavier compute is needed to crunch through intense weather patterns such as strong topographically driven convection.
Simpler pricing for on-demand forecasts
To date, Cloudrun has been using a step-wise linear pricing model wherein a 1-day forecast costs $49, a 3-day forecast costs $119, and a 5-day forecast costs $149. With the increasing inflation and cloud computing costs over the past year, Cloudrun’s pricing model needs to adapt as well. At the same time, I don’t want this change to adversely affect our loyal customers with smaller budgets, especially in light of the coming recession.
Starting November 1, 2022, all forecasts will cost $48 per forecast day, pro-rated to the hour. This means that your Cloudrun forecasts up to 24 hours long (nowcasts) will remain at the same price, while only more extended forecasts will increase in price. It also simplifies the pricing model, making it easy to calculate your Cloudrun bill in the long term.
If you simply want to try Cloudrun and see if it’s a good solution for you, please write to us at help@cloudrun.co with a short description of your application or use case and we’ll provide you with evaluation credits.
On premise Cloudrun NWP
Over the past year, I’ve developed a custom on-premise NWP solution for a customer that needed to run WRF operationally, multiple times per day, each forecast several days long. As you can imagine, running this amount of weather forecasts would be prohibitively expensive with a cloud-based and on-demand solution like the Cloudrun Forecast Wizard with scheduling. Instead, the customer wanted to run Cloudrun’s WRF system on their hardware, with a variety of settings to be customizable. The on-premise system’s features include:
- Your choice of WRF and WPS versions and easy upgrading when new releases become available
- Initial and boundary conditions based on GFS, ECMWF, NAM, or HRRR
- Easy configuration of regions and schedules via a TOML-based configuration file
- Set it and forget it — the system runs automatically year-round without your intervention or need for maintenance.
- WRF output post-processing for derived meteorological variables, as well as interpolation of 3-d fields to custom pressure or height levels
- WRF output visualization via a built-in web server
While the on-premise Cloudrun NWP solution continues to evolve and improve, it’s ready for wider adoption. Please reach out to help@cloudrun.co if you need a fully automated, easy-to-configure, on-premise numerical weather prediction system.
What are you waiting for? Get started with Cloudrun today at https://cloudrun.co.
