All entries for Thursday 24 May 2018

Multi–site Electronic Lab Notebook in Wordpress – how we did it!

This is the first post on the CAMDU blog! Our aim is to give everyone a small glimpse into our day-to-day work, explain how we did some of the things we did and (hopefully) help people out there who have similar issues.

As this is the first post, it might be worth explaining who we are: CAMDU (Computing and Advanced Microscopy Development Unit) is a small team of dedicated researchers at Warwick Medical School who support microscopy-based research, from acquisition to image analysis and storage. We're home to multiple commercial light microscopes and custom-built systems alongside Wellcome-funded lattice light sheet microscopy and visitor programme (coming soon); computational workstations, software development and petabyte data storage array are also in place.

For our first entry, we have decided to talk about our solution for Electronic Lab Notebooks for multiple labs. It is Wordpress-based: the reasons for picking Wordpress have been detailed by Steve Royle on this blog post. In summary: it's easy to use, free, ubiquitous and takes care of issues like backups and versioning in a nice, transparent way. Also, Steve had already been using it as a solution for his own lab for more than 6 months when we started implementing our solution, so we already knew it worked!

- Our infrastructure

We happened to have a pretty decent server already running VMware ESXi in the building - it's inside our local network, which would make sense from a lab notebook point of view (you're not supposed to take them home anyway). It was super easy to spin up an Ubuntu 16.04.3 virtual machine and start playing with it. Nick, our resident expert in all things everything, handed me that VM and gave me a hand on setting up a local IP and local domain for that machine.

Having the whole install encased on a virtual machine was a great idea for ease of transfer and backup; the physical server running the ELNs is (as I understand) quite old and might just give up the ghost at any time. Our IT Services Linux hosting team also works based on VMs, so our contingency plan has always been telling them "here's the virtual machine backup, can you get us some resources to run this?". Our server is still holding up, though!

So why not go straight to an IT Services-hosted virtual machine from the start? Well, we like having control over our machines, it turns out. Also, having the server in our local network means we have control over who can see what, and what kind of firewall exceptions make or don't make sense, without having to deal with a third party that, as good as it is (and the Linux hosting team at Warwick is fantastic!), would always introduce a bit of delay and extra complications to the process.

- Installing Wordpress

This is the easy part! There are plenty of tutorials out there (I basically followed the one at Wordpress Codex). If you have some familiarity with terminal commands you can probably do it without any issues. I am not particularly competent when it comes to anything web, and I had a server running in about 10 minutes.

If you don't feel particularly confident just going for it, I strongly recommend running a local install before you try it on your server. I followed this tutorial for a local install to make sure I actually knew what I was doing before putting my hands on the actual server!

- Making Wordpress Multisite

A basic wordpress install supports a single site. That's fine if you are establishing an ELN for a single lab, but if you want multiple labs with independent feeds, and you want to keep each individual lab's information contained, then you will need to set up Wordpress to work as a multisite. In this mode, each lab can have its own site. Each site can, then, have its own admin structure, plugins can be activated granularly, you get a lot of flexibility to operate multiple streams of information in parallel.

Again, the Wordpress Codex has an excellent guide to migrate your Wordpress install to a multisite. Dave Mason also has an excellent guide on this process on his blog - it helped me a lot when setting this up! My experience is that making a fresh WP install into multisite is very straightforward, and that I only ran into issues when trying to convert an install that was already being used into a multisite install.

- Site structure and Permissions

So we have a multisite Wordpress installed and ready to go. We have multiple labs who want their own ELN. The obvious choice is the right choice: we just added one site per lab. Each lab member can see everyone's posts on their lab site, but cannot see anything else. Site admins (i.e. PIs or the "tech person" in the lab) have some degree of authonomy over their own site (activating plugins, changing themes, etc).

The biggest debate we've had was regarding super admins. They have permission to do anything on the whole network, and importantly they are the only ones who can add new users to the multisite. The big question was: should PIs be super admins? If the answer is yes, that gives us a lot of flexibility, with different PIs being able to just add new users/researchers to their groups, install new plugins they might require and so on. Of course, the downside is that every PI can do everything, which means that it takes a single super admin to download a malicious plugin and the whole network is infected. We have decided to trust people to do the right thing and gave our PIs super admin permissions.

- Customising things

Luckily, a lot of the customisation work had already been done by Steve on his group's Wordpress install. We are reusing lots of his choices there.

• Theme: we're using Steve's fork of the gitsta theme. It looks super nice and clean!

• Plugins: My Private Site (by David Gewirtz) is absolutely essential if you want to make sites non-public. We are using TinyMCE Advanced (by Andrew Ozz) and WP-Markdown (by Stephen Harris) for extra features when editing posts. To make sure all kinds of data look good, we have Code Prettify (by Kaspars Dambis), Mathjax-LaTeX (by Phillip Lord, Simon Cockell, Paul Schreiber) and TablePress (by Tobias Bäthge). Finally, we have added PDF Embedder (by Dan Lester) Mammoth .docx converter (by Michael Williamson) for importing data from elsewhere.
• Contingency planning: for the moment, we have an automated weekly backup that is encrypted (just in case) and pushed to our IT-services managed storage server, where it's further backed up according to their policies. This is a "manual" backup: we're not using any plugins for that. Both the Wordpress folder and a dump of the MySQL database are included. We've tested restoring an install from these backups and it's a very straightforward, 5-minute process.

- Issues we still want to deal with

• I still don't like having all PIs as super admins. Not that I don't trust them (I do!), but people make mistakes, and limiting permissions to what's necessary and nothing beyond that is always a good idea.
• The virtual machine image is not being backed up. It's not a huge deal since we can restore our install from the backups we currently take, but I'd like to have the extra redundancy there!
• Adoption: this is the hardest challenge we face. Currently, there are only 3 or 4 groups using the ELN solution heavily, while everyone else still relies on their paper notebooks. Even for the groups where adoption is widespread, there's still a lot of resistence to what's seem as "duplicated effort".

Erick Martins Ratamero : 24 May 2018 15:41 |  Comments (0) |  Close comments |  Report a problem