Climbing the stairway to HPC cloud
Hardly a day goes by without hearing about some success story of HPC in the cloud. This is reflected in the recent Hyperion Research May 2021 HPC Market Update, where the HPC Cloud Forest projects that “HPC in the cloud is expected to grow more than 2.5 times faster than the on-prem HPC server market”. However, ease of use of running HPC workloads in the cloud remains a challenge.
So what hurdles are there on the stairway to HPC cloud that need consideration? To begin, are you considering a hybrid HPC cloud or moving your HPC workloads entirely to the cloud? Are you looking to adopt a multi-cloud strategy, meaning that you’ll leverage multiple public cloud vendors? How will data for your jobs be managed? How can you make use of the cloud transparent to the HPC users? These are just some examples of considerations in addition to security, cost and, performance.
Containers provide the means to easily port workloads from one HPC environment to another, including cloud. But whether the workload is containerized or not, a workload scheduler is also a crucial component in building out an HPC environment. For almost 30 years, IBM Spectrum LSF (LSF) has been managing the workloads in some of the world’s largest HPC environments. In addition to providing a number of advanced scheduling policies from first-come first-serve and pre-emption to fairshare and backfill, LSF provides dynamic hybrid cloud capabilities which allow you to burst out to public cloud resources automatically, based on workload demand and scale back when no longer required. The LSF resource connector supports dynamic bursting of HPC workloads to IBM Cloud, AWS, Microsoft Azure, Google Compute Cloud, OpenStack, OpenShift and Kubernetes. And to round out the capabilities which are crucial for those cloud bound workloads, LSF supports intelligent movement of data and support for containerized workloads including Docker, Shifter, Singularity, Nvidia Docker, Enroot and Podman.
Making it easy on IBM Cloud
IBM Spectrum LSF is now available on the IBM Cloud. With the goal of providing as streamlined a path to HPC cluster readiness, IBM Cloud now provides a fully automated and configurable deployment of LSF on IBM Cloud VPC HPC clusters. IBM Cloud automatically configures LSF auto-scaling and multi-cluster to allow for forwarding of jobs from on-prem to the IBM Cloud at the flick of a switch. All of the underlying complexities of the deployment and configuration are taken care of for you, so that you can focus on your workloads.
In case you missed our webinar on August 19, 2021 covering new features and futures in IBM Spectrum LSF, you can access the recording here. This autumn we’ve planned a series of webinars on IBM Spectrum LSF covering a wide variety of technical topics, which you won’t want to miss:
- September 2, 2021: What’s new in LSF Service Pack 12
- September 16, 2021: Best Practices for Upgrading your LSF Clusters
- September 30, 2021: What if….? Using the LSF Simulator to answer those hard questions
- October 14, 2021: Simplifying HPC — Just push the button! Tips & Tricks
- October 25–28, 2021: IBM Tech U — Spectrum LSF sessions
- November 11, 2021: Expediting PMR’s with the LSF Support Tools
- November 25, 2021: High Performance Computing — Health Check Services from Lab Services
Learn more and register for these sessions here.
