Custom Conda Environments for Data Science on HPC Clusters
Ryan Abernathey
12

Updates to my previous comment — now I can run notebooks on Pleiades as well. My first reference is a general guide of running jupyter on HPC:

http://ipyrad.readthedocs.io/HPC_Tunnel.html

However, the two-stage login on Pleiades seems kind of different, until I see this guide:

https://www.nas.nasa.gov/hecc/support/kb/running-applications-on-pleiades-with-your-local-license_511.html

It turns out that you just need to repeat the same forwarding option one more time in the two-step connection. Something like “ssh -L 8889:localhost:8889 sfe” followed by “ssh -L 8889:localhost:8889 -N pfe”. The notebook itself can run on pfe (not a compute node) which is pretty convenient.

Show your support

Clapping shows how much you appreciated Jiawei Zhuang’s story.