CDS’ William Falcon Introduces Test Tube
CDS Ph.D. student William Falcon describes new library
CDS Ph.D. student William Falcon introduced his GitHub project, Test Tube, on Wednesday, October 24 during a Moore-Sloan Data Science Lunch Seminar at NYU’s Center for Data Science. Falcon, who began his career as a naval officer with Navy SEAL training, also worked as a software developer at Goldman Sachs and built iOS applications as a pastime. Falcon describes his latest contribution, Test Tube, as “a python library to track, optimize and parallelize Deep Learning experiments across GPUs and compute clusters. It’s framework agnostic and is built on top of the python Argparse API for ease of use.”
During his presentation, Falcon discussed the motivations behind the project, explained ways to use the library, and provided examples of simple code to implement the various functionalities Test Tube offers. Specific reasons to use Test Tube include the ability to parallelize hyperparameter optimization across multiple GPUs and CPUs, parallelize hyperparameter optimization across HPC cluster using SLURM, and automate the start of continuation jobs when walltime approaches. Test Tube allows the user to specify the number of GPUs and CPUs per neural network, as well as set the walltime.
Falcon discussed how Test Tube makes it easier to log experiments. He demonstrated this advantage and contrasted it with traditional, cumbersome methods of experiment logging via Excel. Addressing concerns over fitting models into memory, Falcon explained data parallel, which clones the network across GPUs, slices the batch, and processes the halves. He also outlined the distributed model, which arbitrarily distributes across GPUs and returns to the main GPU.
Test Tube offers a wide variety of new tools to optimize performance for programmers. The Center for Data Science congratulates William Falcon on this exciting new library!
For more details, check out the original presentation here.
By Sabrina de Silva