All 3 entries tagged Sc-Rtrp

No other Warwick Blogs use the tag Sc-Rtrp on entries | View entries tagged Sc-Rtrp at Technorati | There are no images tagged Sc-Rtrp on this blog

February 02, 2021

Updating to FastAI v2 on ubuntu and anaconda

Follow-up to Keras, TensorFlow and more on the SC–RTP desktops from Rudo's blog

I wanted to upgrade my FastAI installation to V2, using my existing ubuntu installation of anaconda. After a few wrong tipps on google, I converged to the following.

First, I made a new conda environment, i.e.

$ conda create -n FastAI2

Then I installed all that was needed into the new environment:

$ conda update -n base -c defaults conda
$ conda install -c fastai -c pytorch fastai
$ conda install jupyter
$ conda install -c fastai nbdev

OK, that seems to work, i.e. the example jupyter notebooks from the FastAI V2 course now run.

October 23, 2020

Keras, TensorFlow and more on the SC–RTP desktops

Follow-up to Using PyTorch on the SC–RTP machines from Rudo's blog

This set of commands allows to load a working Keras/TF/scikit ML environment which will run on a GPU if present

$ module load GCC/8.3.0  CUDA/10.1.243  OpenMPI/3.1.4
$ module load IPython/7.9.0-Python-3.7.4
$ module load TensorFlow/2.1.0-Python-3.7.4 
$ module load Keras
$ module load scikit-learn
$ jupyter notebook &

using the versions

tensorflow:  2.1.0 , keras:  2.3.1 sklearn:  0.21.3

October 16, 2020

Using PyTorch on the SC–RTP machines

Follow-up to Installing your own TensorFlow/Kears + FastAI on Ubuntu from Rudo's blog

We (the DisQS team) have now moved back from Cergy to the University of Warwick where the Scientific Computing RTP provides a centrally managed Linux desktop setup as well as bespoke HPC solutions, including some GPU-enabled machines. This means that we need to switch from our self-managed Ubuntu/Anaconda solution to the managed SC-RTP desktop. The following lines, executed in an SC-RTP terminal, allow us to use PyTorch, or preferred ML package at the moment, as follows

module load GCC/8.3.0  OpenMPI/3.1.4
module load IPython/7.9.0-Python-3.7.4
module load PyTorch
module load scikit-learn
pip install --user torchvision==0.4.0
module save ML (or some such name)

and later one can retrieve these settings via

module restore ML

Please not that we need to install torchvision locally via pip as it's not available on the SC-RTP system as a module. The pip mechanism is what replaces the "conda install ..." used under anaconda for missing libraries. Also, we found that the newer torchvision version 0.7.0 gave some error with our codes, so we are using an older version that seems to work. This may not be needed for your code. Obviously, the use of "module save/restore" is optional, but very convenient.

jupyter notebook &

then works as before/desired.

October 2021

Mo Tu We Th Fr Sa Su
Sep |  Today  |
            1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

Search this blog



Most recent comments

  • Please note that this link to download is now obsolete. Attendees for the 2021 MPAGS/CYU course shou… by Rudo Roemer on this entry
  • To load Tensorflow and PyTorch (and Keras) modules on ORAC may you may have to module load a few mor… by Daniel Paget on this entry
  • That's cool. Perhaps you want to show people during class next week how to run on Kaggle? by Rudo Roemer on this entry
  • Regarding possible alternatives to benefit form a GPU, I have personally used Kaggle, by Google, its… by Juan on this entry
  • Hello to you all, I was just trying to make the programs run in my VirtualBox with Ubuntu 18.04 inst… by Malaquias Correa on this entry

Blog archive

RSS2.0 Atom
Not signed in
Sign in

Powered by BlogBuilder