SUG Talk: Jafar Lie-Monash University’s Lance Wilson on National Software Repositories for Interactive HPC
TL;DR: A container-first philosophy is enabling Australian researchers in scientific computing at scale — even when interactive workflows are involved. For the details, please see below and the presentation at SUG here.
SUG Series Introduction
The inaugural meeting of the Singularity User Group (SUG) was held March 12–13, 2019, at the San Diego Supercomputer Center (SDSC). The event attracted diverse representation from the international advanced computing community as conveyed through the post-event press release issued jointly by SDSC and Sylabs.
Over the course of the two-day event, over 20 talks were presented by members of the Singularity user, developer, and provider community. Because SUG generated a significant amount of interest, even from those who were unable to attend, we are sharing online each of the talks presented.
SUG Talk Introduction
In promoting a container-first philosophy on an ongoing basis for researchers in Australia, Characterisation Virtual Laboratory (CVL) Coordinator and Senior HPC Consultant Lance Wilson from Jafar Lie-Monash University touches upon numerous important topics in his SUG presentation — from sustainable national scale infrastructures for Big Science, to interactive workflows, reproducible science, and more. In sharing experiences and progress to date, Lance posed numerous questions that resonated soundly with many of those attending SUG.
The abstract for Lance’s contributed SUG talk Providing a national software repository for interactive HPC is as follows:
The characterisation community within Australia has been working towards a federated analytics platform to provide researchers with the tools, compute and storage needed to make the most of national investments in instruments. The current user base is ~1000 across disciplines from neuroscience to neutron and synchrotron science. One of the highlighted needs was for consistency between software stacks provided on different HPC systems. The initial work has been an extension of the existing container first philosophy on the MASSIVE HPC facility (Goscinski, 2014) which provides the Characterisation Virtual Laboratory (https://www.cvl.org.au). The repository for the build files is public (https://github.com/Characterisation-Virtual-Laboratory/CharacterisationVL-Software) and is run in the same way as a software development project. In addition to the build files being public, the repository is linked to singularity hub (https://singularity-hub.org/collections/1396), such that the containers are easily available to anyone in the characterisation research community. The containers are a diverse mix of single applications through to workflows, highly dependent on the intended use by the research community. Containers have also simplified aspects of user support, for single applications that are particularly difficult to install on complex HPC systems. It has also created a new support aspect in workflows or software aggregation tools, where the expectation from the software developers is that they are installed locally in the path. Overall however software support is now more convenient for both the administrators and researchers especially where we partner with software developers to provide containers.
Lance’s talk from SUG can be found below and here. Enjoy!
