All entries for October 2020

Keras, TensorFlow and more on the SC–RTP desktops

Follow-up to Using PyTorch on the SC–RTP machines from Rudo's blog

This set of commands allows to load a working Keras/TF/scikit ML environment which will run on a GPU if present

$ module load GCC/8.3.0  CUDA/10.1.243  OpenMPI/3.1.4
$ module load IPython/7.9.0-Python-3.7.4
$ module load TensorFlow/2.1.0-Python-3.7.4 
$ module load Keras
$ module load scikit-learn

$ jupyter notebook &

using the versions

tensorflow:  2.1.0 , keras:  2.3.1 sklearn:  0.21.3

Rudo Roemer : 23 Oct 2020 11:28 |  Tags: Machine-Learning Pytorch Sc-Rtrp |  Comments (1) |  Close comments |  Report a problem

Using PyTorch on the SC–RTP machines

Writing about web page /rroemer/entry/installing_your_own/

Writing about an entry you don't have permission to view

We (the DisQS team) have now moved back from Cergy to the University of Warwick where the Scientific Computing RTP provides a centrally managed Linux desktop setup as well as bespoke HPC solutions, including some GPU-enabled machines. This means that we need to switch from our self-managed Ubuntu/Anaconda solution to the managed SC-RTP desktop. The following lines, executed in an SC-RTP terminal, allow us to use PyTorch, or preferred ML package at the moment, as follows

module load GCC/8.3.0  OpenMPI/3.1.4
module load IPython/7.9.0-Python-3.7.4
module load PyTorch
module load scikit-learn
pip install --user torchvision==0.4.0 
module save ML (or some such name)

and later one can retrieve these settings via

module restore ML

Please not that we need to install torchvision locally via pip as it's not available on the SC-RTP system as a module. The pip mechanism is what replaces the "conda install ..." used under anaconda for missing libraries. Also, we found that the newer torchvision version 0.7.0 gave some error with our codes, so we are using an older version that seems to work. This may not be needed for your code. Obviously, the use of "module save/restore" is optional, but very convenient.

jupyter notebook &

then works as before/desired.

Rudo Roemer : 16 Oct 2020 11:12 |  Tags: Machine-Learning Pytorch Sc-Rtrp |  Comments (0) |  Close comments |  Report a problem