SUG TALK: LENOVO’S J. J. FALKANGER ON DRIVING THE CONVERGENCE OF HPC & AI WITH LICO AND SINGULARITY
TL;DR: Lenovo Intelligent Computing Orchestration (LiCO) and Singularity are driving the ongoing convergence between HPC and AI; the video of J. J. Falkanger’s SUG talk is available here.
SUG Series Introduction
The inaugural meeting of the Singularity User Group (SUG) was held March 12–13, 2019, at the San Diego Supercomputer Center (SDSC). The event attracted diverse representation from the international advanced computing community as conveyed through the post-event press release issued jointly by SDSC and Sylabs.
Over the course of the two-day event, over 20 talks were presented by members of the Singularity user, developer, and provider community. Because SUG generated a significant amount of interest, even from those who were unable to attend, we are sharing online each of the talks presented.
SUG Talk Introduction
As an event sponsor, Lenovo’s commitment to SUG was made clear. By making use of it in Lenovo Intelligent Computing Orchestration (LiCO) — a software solution that simplifies the management and use of distributed clusters for High Performance Computing (HPC) workloads and Artificial Intelligence (AI) model development — their ongoing interest in Singularity is also evident. To quote from a solution brief published earlier this week in HPCwire:
Managing the environment needed to run both machine learning and deep learning applications can be a major challenge. Especially when the user attempts to do this in a multi-tenant environment, simply getting a job to run in the correctly configured environment proves problematic. To solve this problem LiCO leverages Singularity. The user can download any of a number of popular Singularity containers from Singularity Hub or using a single pull command can import Docker containers into Singularity from Docker Hub. This is a powerful tool that allows data scientists and AI engineers to update AI frameworks, add new frameworks quickly, and effectively manage a multi-tenant environment.
Of course, the Sylabs Cloud Container Library offers another option for acquiring images in the native Singularity Image Format (SIF); additionally, this portal from Sylabs provides services for signing and verifying container images, as well as building those images in the cloud.
Lenovo’s senior AI software and solutions manager J. J. Falkanger participated in person at SUG, and presented the talk LiCO and Singularity — Driving the Convergence of HPC & AI; the abstract for this talk is as follows:
Scientific researchers and commercial enterprises have been increasingly applying high performance computing (HPC) and artificial intelligence (AI) techniques for driving insights and business value, often in the same computing environment. This convergence demands flexible tools to effectively utilize the resources and workflows. Lenovo Intelligent Computing Orchestration (LiCO) provides an intuitive web-based interface for managing and deploying both AI and HPC workloads on the same cluster. Built upon an OpenHPC management stack, LiCO enables even inexperienced users to utilize cluster resources, thanks to Singularity container management and job template integration. This session covers the capabilities of LiCO, and the many benefits for users Singularity provides as part of the architecture.
As you’ll learn from J. J. Falkanger’s SUG talk, LiCO makes use of OpenHPC. Literally the day after SUG, OpenHPC v1.3.7 was announced. Because this release of OpenHPC includes version 3.1 of Singularity, it is anticipated that those making use of LiCO can expect this update to be propagated downstream in due course.
J. J. Falkanger’s talk from SUG can be found below and here. Enjoy!
Originally published at www.sylabs.io on March 29, 2019.
