All 10 entries tagged Machine-Learning

Updating to FastAI v2 on ubuntu and anaconda

Follow-up to Keras, TensorFlow and more on the SC–RTP desktops from Rudo's blog

I wanted to upgrade my FastAI installation to V2, using my existing ubuntu installation of anaconda. After a few wrong tipps on google, I converged to the following.

First, I made a new conda environment, i.e.

$ conda create -n FastAI2

Then I installed all that was needed into the new environment:

$ conda update -n base -c defaults conda
$ conda install -c fastai -c pytorch fastai
$ conda install jupyter
$ conda install -c fastai nbdev

OK, that seems to work, i.e. the example jupyter notebooks from the FastAI V2 course now run.

Rudo Roemer : 02 Feb 2021 17:59 |  Tags: Machine-Learning Pytorch Sc-Rtrp |  Comments (0) |  Close comments |  Report a problem

Keras, TensorFlow and more on the SC–RTP desktops

Follow-up to Using PyTorch on the SC–RTP machines from Rudo's blog

This set of commands allows to load a working Keras/TF/scikit ML environment which will run on a GPU if present

$ module load GCC/8.3.0  CUDA/10.1.243  OpenMPI/3.1.4
$ module load IPython/7.9.0-Python-3.7.4
$ module load TensorFlow/2.1.0-Python-3.7.4 
$ module load Keras
$ module load scikit-learn

$ jupyter notebook &

using the versions

tensorflow:  2.1.0 , keras:  2.3.1 sklearn:  0.21.3

Rudo Roemer : 23 Oct 2020 11:28 |  Tags: Machine-Learning Pytorch Sc-Rtrp |  Comments (1) |  Close comments |  Report a problem

Using PyTorch on the SC–RTP machines

Writing about web page /rroemer/entry/installing_your_own/

Writing about an entry you don't have permission to view

We (the DisQS team) have now moved back from Cergy to the University of Warwick where the Scientific Computing RTP provides a centrally managed Linux desktop setup as well as bespoke HPC solutions, including some GPU-enabled machines. This means that we need to switch from our self-managed Ubuntu/Anaconda solution to the managed SC-RTP desktop. The following lines, executed in an SC-RTP terminal, allow us to use PyTorch, or preferred ML package at the moment, as follows

module load GCC/8.3.0  OpenMPI/3.1.4
module load IPython/7.9.0-Python-3.7.4
module load PyTorch
module load scikit-learn
pip install --user torchvision==0.4.0 
module save ML (or some such name)

and later one can retrieve these settings via

module restore ML

Please not that we need to install torchvision locally via pip as it's not available on the SC-RTP system as a module. The pip mechanism is what replaces the "conda install ..." used under anaconda for missing libraries. Also, we found that the newer torchvision version 0.7.0 gave some error with our codes, so we are using an older version that seems to work. This may not be needed for your code. Obviously, the use of "module save/restore" is optional, but very convenient.

jupyter notebook &

then works as before/desired.

Rudo Roemer : 16 Oct 2020 11:12 |  Tags: Machine-Learning Pytorch Sc-Rtrp |  Comments (0) |  Close comments |  Report a problem

list of submitted projects and information about late submission penalties

Follow-up to Submission date and instructions for your project work from Rudo's blog

Dear all,

thus far I have received submissions from (10/02/30 10:00)

1. Jettawat Khantaveramongkol
2. Juan Roman Roche
3. Hendrik Schlueter
4. Alexios Malaquias
5. DanielPepin Fowan
6. Burak Civitcioglu
7. Roman Perrier (one day late)
8. Jeremy Cadran (two days late)
   

Please note that I intend to apply late submission penalties, following roughly similar Warwick penalties which penalize a late submission with a 5% reduction in final points for every day late. A late day starts in the first minute after the dealine and the time of submission follows from the time of the last upload.

Regards,

-Rudo

Rudo Roemer : 05 Mar 2020 13:16 |  Tags: Fastai Keras Machine-Learning Tensorflow |  Comments (0) |  Close comments |  Report a problem

PS: don't worry too much about maximizing epochs

Follow-up to Submission date and instructions for your project work from Rudo's blog

Just to add: do not worry too much about running your code to the maximum of epochs. It is clear to us that most people do not have access to state of the art graphics cards or hundreds of processors. What is more important is to see that you came up with a good idea where machine learning can be used, executed the right commands to get some training and analysis done, and found the right data to train on. In fact, we migth even try to run some of your code for more epochs to see how results can get better!

Rudo Roemer : 02 Mar 2020 10:26 |  Tags: Fastai Keras Machine-Learning Tensorflow |  Comments (0) |  Close comments |  Report a problem

Submission date and instructions for your project work

Follow-up to teaching schedule, now ready from Rudo's blog

Dear all,

you should now all be close to finishing your project work for the machine learning course. I hope it has all gone well and that you had some fun exploring machine learning. The official date for submission is

Friday, March 6 at 17:00.

For submission, please remember that we want you to submit a running machine learning code using a jupyter notebook with comments that explain step by step what you did and why. Please also supply the data so that Djena and I can run the code ourselves. The easiest way to send us the submission is via

https://files.warwick.ac.uk/rroemer/sendto

where you can upload both jupyter notebook and data (preferably as a single .zip file). If your data files are too big for upload there (I have 4.5GB in total on that upload drive), please send a link at which we can then download the data. Please remember that it is your responsibility to make sure data and notebook work "out of the box", i.e. without us having to change anything before they work, including directories.

We aim to have feedback for you by the end of March and will communicate this by email to each of you (so please also let me have your preferred email address in your jupyter notebook).

Cheers!

Rudo Roemer : 02 Mar 2020 09:08 |  Tags: Fastai Keras Machine-Learning Tensorflow |  Comments (0) |  Close comments |  Report a problem

teaching schedule, now ready

Follow-up to Machine Learning – a hands–on course from Rudo's blog

Dear all,

the schedule of the course has now become clearer: we continue to teach every Monday 0930-1230 in room E4.28 (except for one day in January and one day in February) for 10 weeks overall. For the 1st 5 weeks (with 4 more weeks to come), I will actually "teach", i.e. with prepared lectures for 2 hours 0930-1130, while Djenabou will be available for 1 hr more 1130-1230 to help you with the exercises. Then in the last 5 weeks, Djenabou will be available 0930-1230 to help with your project work (she won't do it for you, but might be able to help with technical problems).

A full schedule should be available soon.

See you next Monday!

-Rudo

Rudo Roemer : 12 Dec 2019 15:53 |  Tags: Fastai Keras Machine-Learning Tensorflow |  Comments (0) |  Close comments |  Report a problem

Thoughts on the 1st lecture session

Writing about web page /rroemer/entry/installing_your_own/

Writing about an entry you don't have permission to view

Dear all,

[1] today in the lecture, we saw that it was not entirely simple to install your own ML machine with Tensorflow/Keras and FastAI. I have now made a blog entry on how to do this. You can find it here.

[2] In addition, when running the FastAI code, we saw that it might take a long time even for just the pets categorization. I just ran the pets notebook (please download a new version) and have made the pets-stage-1 and pets-stage-1-50 trained models available on the download area. You need to move these models into where FastAI keeps them which in my set-up is

/home/phsht/.fastai/data/oxford-iiit-pet/images/models

Just copy the twp .pth files there and you should be able to run the pets code after the

learn.load('pets-stage-1')

fastai commands in the notebook.

[3] When you "play" with the notebook called "FastAI-lesson2-download.ipynb", note that the file data.zip contains the cats/teddys/wolves that I managed to download and use for the exercise.

[4] Some of you argued that your own machines may not be able to do meaningful ML as you do not have a GPU. Have a look at what the FastAI guys have collected in terms of possible free GPU instances, i.e. Using a GPU. I have not yet used any of these services, but would be keen to learn from your experiences!

OK, keep checking this blog for up-to-date information before we meet again next week.

Rudo Roemer : 09 Dec 2019 16:29 |  Tags: Fastai Keras Machine-Learning Tensorflow |  Comments (2) |  Close comments |  Report a problem

Things to do before the 1st lectures

Follow-up to Machine Learning – a hands–on course from Rudo's blog

Dear all,

[1] some of you will have registered for the short Machine Learning (ML) course that we are planning to give, starting next week. ML is best "learned" by doing it, so we are planning to use the computer room E4.28 for exercises during all the lectures. But we think you might also find it more useful if you were to bring your laptops along and install+run the ML codes there.

[2] We are expecting to run all the ML on a Ubuntu installation (Francois Chollet, one of the key people in the field, says "it’s possible to use Windows, too, but I don’t recommend it"). Other Linux installations should also work. If you only have Windows, Rudo recommends installing

VirtualBox, https://www.virtualbox.org/

with a Ubuntu 19.10 installation (http://releases.ubuntu.com/19.10/).

[3] During the course, we intend to offer you downloadable ML source codes via Rudo's blog at

https://blogs.warwick.ac.uk/rroemer/entry/machine_learning_a/

where you can also see a copy of this email. The direct download link is in the first entry of the blog and given here as well, i.e.

https://cloud.csc.warwick.ac.uk/index.php/s/JomfpoMj87X3Jbk

[4] We want to share literature on ML with you using Mendeley, i.e. https://www.mendeley.com/. Although the Mendeley Desktop might be convenient to install (we have it on our machines), the online interface could be ok for the course. But in order to "share" PDFs with you, you need to have a (free) Mendeley account. Please create one.

[5] Last, we, Djena and Rudo, are both new to ML, so this will be a course where we will all explore ML together! We hope you'll enjoy it.

- Rudo and Djena

Rudo Roemer : 06 Dec 2019 09:13 |  Tags: Fastai Keras Machine-Learning Tensorflow |  Comments (0) |  Close comments |  Report a problem

Machine Learning – a hands–on course

Dear all,

for this course, you should be able to find all course material for download at

https://cloud.csc.warwick.ac.uk/index.php/s/JomfpoMj87X3Jbk

Please let me know in case this does not work!

-Rudo

Rudo Roemer : 05 Dec 2019 16:29 |  Tags: Fastai Keras Machine-Learning Tensorflow |  Comments (4) |  Close comments |  Report a problem