All entries for Friday 16 October 2020

October 16, 2020

Using PyTorch on the SC–RTP machines

Follow-up to Installing your own TensorFlow/Kears + FastAI on Ubuntu from Rudo's blog

We (the DisQS team) have now moved back from Cergy to the University of Warwick where the Scientific Computing RTP provides a centrally managed Linux desktop setup as well as bespoke HPC solutions, including some GPU-enabled machines. This means that we need to switch from our self-managed Ubuntu/Anaconda solution to the managed SC-RTP desktop. The following lines, executed in an SC-RTP terminal, allow us to use PyTorch, or preferred ML package at the moment, as follows

module load GCC/8.3.0  OpenMPI/3.1.4
module load IPython/7.9.0-Python-3.7.4
module load PyTorch
module load scikit-learn
pip install --user torchvision==0.4.0
module save ML (or some such name)

and later one can retrieve these settings via

module restore ML

Please not that we need to install torchvision locally via pip as it's not available on the SC-RTP system as a module. The pip mechanism is what replaces the "conda install ..." used under anaconda for missing libraries. Also, we found that the newer torchvision version 0.7.0 gave some error with our codes, so we are using an older version that seems to work. This may not be needed for your code. Obviously, the use of "module save/restore" is optional, but very convenient.

jupyter notebook &

then works as before/desired.


October 2020

Mo Tu We Th Fr Sa Su
Sep |  Today  | Nov
         1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31   

Search this blog

Tags

Galleries

Most recent comments

  • Please note that this link to download is now obsolete. Attendees for the 2021 MPAGS/CYU course shou… by Rudo Roemer on this entry
  • To load Tensorflow and PyTorch (and Keras) modules on ORAC may you may have to module load a few mor… by Daniel Paget on this entry
  • That's cool. Perhaps you want to show people during class next week how to run on Kaggle? by Rudo Roemer on this entry
  • Regarding possible alternatives to benefit form a GPU, I have personally used Kaggle, by Google, its… by Juan on this entry
  • Hello to you all, I was just trying to make the programs run in my VirtualBox with Ubuntu 18.04 inst… by Malaquias Correa on this entry

Blog archive

Loading…
Not signed in
Sign in

Powered by BlogBuilder
© MMXXI