Departmental impact evaluation of e–learning initiatives

Here’s the initial plan for the departmental impact analysis of e-learning initiatives, part of the elearning Pathfinder evaluation workpackage. See attached document for appendices. List of names is tentative and will be subject to invitation.

Comments/suggestions on the plan, questions or people to interview?

1. Scope

This work is being undertaken between February and September 2007 by Terry Wareham, Fourstones evaluation & consultancy for HE, in response to a request from Dr Jay Dempster, Deputy Director of the Centre for Academic and Professional Development (CAPD) who is leading the evaluation component of the University’s HEA-funded e-learning ‘Pathfinder’ project. The data gathering and analysis will be conducted over 15 working days as agreed, which will cover initial discussion, planning and reading, preparation of interview schedule and gaining appointments, data analysis, report drafting and presentation.

Concurrently a university-wide web-based survey (see appendix 4) aimed at academic staff has been developed that aims to capture the spread of e-learning activity across the institution. This data will be reported separately.

2. Aims

a) identify the impact of current and new e-learning projects and initiatives on the student learning experience using an approach which emphasises appreciative enquiry in order to draw out the positive benefits of specific approaches in particular departmental contexts
b) make draft recommendations to inform the development of institutional strategic goals for e-learning
c) provide a model for evaluation of e-learning developments

3. Methods

The methods used for the evaluation will be a) gathering of existing data on departmental impact from the WELA programme, uptake and outcomes of initiative funding at departmental level, participation and outcomes of the Elab faculty activities and b) interviews with staff at a range of levels in 6 departments across 4 faculties and further interviews with a sample of staff who provide or support development activities for e-learning (see appendix 3). The sample of staff will be selected to ensure a range of engagement with e-learning initiatives and will thus include those who have made changes at programme and module levels and also those who have yet to embed e-learning approaches in their teaching.

The data will be analysed thematically using a process of constant comparison. Where comparable data are available from other institutions these will be used in the analysis as appropriate to draw out indications for future recommendations.

4. Deliverables

a) draft report to be submitted to CAPD by 17th August 2007

b) Final report to be submitted 14th September 2007

5. Phases of the evaluation

Phase 1 (February to May 2007)

Initial scoping of the work, establishment of list of interviewees background reading of institutional documentation and existing data.

Phase 2 (May to June 2007)
Interviews with identified sample of participants.

Phase 3 (June to August 2007)

Data analysis and report drafting.

Phase 4 (September 2007)

Completion and submission of report.

Note that phases overlap.

6. Interview protocol

Interview subjects will be contacted with a request for an interview, lasting 30 -45 minutes. Interviews will be confidential in order for subjects to feel comfortable in speaking about their experiences and views. All efforts will be made to ensure that particular views cannot be ascribed to particular individuals in the final report. Interview data will remain confidential to the consultant conducting the research. The interviews will be recorded, with the subjects’ permission, in order to facilitate data analysis.

The interviews will be dialogic and thus semi-structured. They will be iterative in the sense that themes that emerge in earlier interviews will be tested with later subjects where appropriate. It is important that interview subjects have the opportunity to develop ideas and views that were not necessarily envisaged in the initial interview schedule.

A list of questions are given in appendix 3. These will be used as a semi-structure to the interviews.

: 08 May 2007 13:04 |  Comments (0) |  Close comments |  Report a problem

WMG E–learning Lunch report

On Thursday 19th April we held the first e-learning lunchtime event in the Science Faculty. The event was held in the International Manufacturing Centre for staff in WMG and Engineering and was attended by over 40 academic staff. Representatives from WMG, the E-learning Advisor Team, the Library and E-Lab were on hand to offer advice and hands-on demonstrations of e-learning. We also displayed nearly 30 posters showcasing examples of e-learning around the University.

The event was a great success; many people stayed for over two hours to try out e-learning tools including podcasting, personal response systems, Quizbuilder and ELAT development projects. Our video and audio communication and streaming software projects proved very popular and we now have a number of new contacts who wish to be involved. Some of the participants had started the WELA course the previous day, so this was a great opportunity for them to see the types of activity already taking place. The ELAT would like to thank everyone involved with making the event a success.

Steven Carpenter : 20 Apr 2007 12:31 |  Tags: E-Learning Events Faculty Science |  Comments (0) |  Close comments |  Report a problem

Second Arts Faculty E–learning Exhibition Lunch

The exhibition was well attended. The poster display showcased more real examples of work from within the faculty, addressing the theme of “Supporting Students at a Distance”, with a consideration of these four issues:

1. keeping students and tutors focussed;
2. keeping people connected (community and communications);
3. developing roles, responsibilities and identities, making them appropriate and well understood;
4. supporting research, creativity and enterprise.

Technical demonstrations from Elab and the Library again proved popular, and Matt Jones from the Sitebuilder support team gave advice and guidance to a range of people.

The next event will be:

Arts Faculty E-coffee drop-in
Wednesday 28th February, 2:30pm – 4:30pm, Graduate Space

Robert O'Toole : 20 Feb 2007 10:47 |  Comments (0) |  Close comments |  Report a problem

Sitebuilder versus Blackboard debate

Writing about web page /stevencarpenter/entry/blackboard_and_firefox/

Writing about an entry you don't have permission to view

Looking beyond the expensive sales pitch into the reality, here is a very telling blog entry by a Blackboard user.

I comment with an important phrase:

decentralised customer led content creation and delivery workflows

...which is precisely what we have here with Sitebuilder, on a really huge scale, but also what commercial VLE users desire with increasing futility. Sitebuilder is deeply embedded across the institution, much more so than any commercial VLE installation that I know of. Why is this the case? I can give three good answers:

1. Sitebuilder (and its associated skillset) is used for research, corporate and personal communications, and teaching, a VLE only does one job. Could we expect people to learn Sitebuilder and a VLE? Not likely.
2. Sitebuilder’s functionality and features have evolved with the institution, guided closely by its users.
3. Sitebuilder is a very high quality product, designed by people who really understand and care for useability and the customer. Blackboard, as you can see from Steve’s blog entry, is not.

Robert O'Toole : 24 Jan 2007 09:40 |  Tags: E-Learning Elearning |  Comments (2) |  Close comments |  Report a problem

Second Arts Faculty E–learning Exhibition Lunch

Follow-up to Report on the first Arts Faculty e–learning exhibition from Pathfinder blog, University of Warwick

Here is the plan for the next Arts Faculty E-learning Exhibition Lunch.

This second exhibition in the series will showcase a wide range of successful applications of technology in teaching, along with demonstrations of new hardware and software. Special exhibits will focus in particular upon the topic of:

Supporting students at a distance (more information below).

The exhibition is open to all staff and teaching postgraduate students within the Arts, as well as anyone else from across the University with an interest or expertise in this topic. It is an informal session, so please drop by at any time during the exhibition. A buffet lunch and drinks will be provided.

Supporting students at a distance

Students at Warwick are frequently expected to undertake work in a location away from the University campus and in less frequent contact with teaching staff and peers. This is not only true of fully distance based courses, but also of the research oriented student undertaking a project out in the field or writing a thesis. Some common examples are:

• full distance learning courses;
• hybrid distance and on-site courses;
• transnational courses;
• year abroad students;
• placement students;
• students in low-contact phases of a course (e.g. writing-up).

Many problems are posed by these arrangements, both to the student and to the tutor, but also to the administrator. Difficulties commonly occur in areas such as:

• establishing and relating to a peer group;
• connecting with the department and faculty community;
• connecting with the wider academic community;
• access to resources;
• understanding (and confirming) module or activity purpose;
• individual study guidance;
• discussing work;
• presenting work;
• submitting work.

Of course these problems may occur in any mode of study, including on-site high-contact courses. The techniques and technologies that have evolved to address these problems are therefore useful in many situations.

Warwick has developed a range of sophisticated but easy to use technologies that allow us to effectively address these problems. The E-learning Exhibition will demonstrate these technologies and techniques, with showcase examples of real solutions by people at Warwick.

We will show how tools including Sitebuilder, Warwick Forums, Warwick Blogs and Perception can be applied to create a learning environment customised to your particular mode of teaching and course delivery, whether at a distance or in close proximity.

Robert O'Toole : 23 Jan 2007 11:31 |  Tags: Arts Faculty Arts Faculty Exhibitions E-Learning Elearning |  Comments (0) |  Close comments |  Report a problem

Report on the first Arts Faculty e–learning exhibition

Follow-up to Arts Faculty e–learning exhibition from Pathfinder blog, University of Warwick

On Friday 24th November we held the first in a series of Arts Faculty E-lunch events, sponsored by the Pathfinder Project. Many of the participants (estimated to be around 30) commented on how useful, enjoyable and informative it was. One of our key objectives was to meet a much wider range of lecturers from the faculty, rather than the usual “early adopters”. This certainly did happen, with all kinds of people visiting to see the exhibits.

The next event will be held between 12 and 2pm on Friday 2nd of February. It will follow a similar format, but with a focus on “distance learning” to include supporting students on a year abroad. There will also be an e-learning coffee and cakes drop in session on the 28th of February.

The success of the e-lunch should be attributed to the long list of exhibitors. Posters and presentations were used to showcase e-learning work. Eight A2 posters were on display, and proved the focus of discussion and consultation. This was extended with web sites explored in more detail using a PC, projector and screen (well managed by Lisa Lavender of History and Nicoleta Cinpoes of Renaissance Studies). The exhibitors were:

1. Comparative American Studies, CAS Image Database
2. Maria Luddy, History, Women in Modern Irish Culture database;
3. Annunziata Videtta, Italian, IT112 module web site;
4. Molly Rogers, History of Medicine, Coventry and Warwickshire Hospital Project
5. Beat Kumin, HI203 module web site
6. Elizabeth Robbery, Language Centre, German with Perception;
7. Nicoleta Cinpoes, Renaissance Studies, Thomas Kyd web site;
8. Piotr Kuhiwczak, Centre for Translation and Comparative Cultural Studies, Avoding Plagiarism online activity.

The Elab Sitebuilder team were also present (Karen Mortimer, Julie Moreton, John Dale and Matt Jones). At a desk with a PC they advised people on the use of Sitebuilder, and on the training opportunities available. Natasha Nakariakova provided advice on online assessment with Perception.

A third table was used to demonstrate hardware from the ELAT TOOLS evaluation service, including MP3 recorders (very popular), projectors, electronic whiteboards, and video cameras. There was also a chance to see and give feedback on new online video, audio and conferencing tools currently being developed by ELAT. Steven Carpenter (Sciences Advisor) and Chris Coe (Social Studies Advisor) facillitated at this table.

We also provided leaflets on:

• Warwick Skills Programme
• Sitebuilder Training
• E-learning Training by the E-learning Advisor Team
• PDP
• Graduate School Training
• The Learning Grid
• Graduate Skills Planning and Recording

And of course there was an excellent buffet organised by Kerry Drakely of the Arts Faculty Office.

I should also thank Sarah Richardson of History for helping out with the event.

Thanks to everyone who helped.

Robert O'Toole : 04 Dec 2006 11:47 |  Tags: Arts Faculty E-Learning Elearning |  Comments (0) |  Close comments |  Report a problem

Arts Faculty e–learning exhibition

News of an event sponsored by the Pathfinder project.

On Friday 24th (12:30-14:30 in the Graduate Space) we will be holding an ‘e-learning exhibition’ for the Arts Faculty. This will showcase successful work within the faculty. As requested by the Arts Faculty IT Committee, it will be an informal ‘drop-in’ session, consisting of:

• two presentation stands with presenters able to demonstrate showcases and tools when requested;
• poster display of showcases;
• a practical support table, hopefully with a representative of the Sitebuilder team;
• a giant (hopefully A0) concept map of e-learning activities within the faculty;
• an ideas and suggestions sheet, with coloured pens
• a buffet with food, drink and guidance booklets and leaflets.

So far I have offers of help with the presentations from Lisa Lavender (History IT), Nicoleta Cinpoes (Renaissance Studies), Chris Coe (Social Studies E-learning).

This is the first time we have run an event in this format, so it is a bit of an experiment. Publicity at such short notice and at this present time is also a challenge. But hopefully we will put on a good show.

Here’s an image of the poster that I am using to publicise the event:

And an example of a showcase poster (this is intended for printing at A2, so the detail will be blurry in this image):

Robert O'Toole : 16 Nov 2006 14:29 |  Tags: Arts Faculty E-Learning Elearning |  Comments (0) |  Close comments |  Report a problem

Leicester e–learning events

Writing about web page http://www.le.ac.uk/beyonddistance/events.html

I’ve just received an email from Gilly Salmon’s PA at Leicester advertising the Learning Futures event in January ‘07. Looks as though it could be an interesting event – Warwick ought to send someone/some people, wouldn’t you think? I can forward the email to you if you’re interested.

: 26 Oct 2006 16:03 |  Tags: E-Learning Pathfinder |  Comments (2) |  Close comments |  Report a problem

what did i mean?

Follow-up to HEA York launch event from Pathfinder blog, University of Warwick

When I said we didn’t have the expertise, I meant particularly in the break-out group of four people who were discussing e-assessment. There were no doubt other people even at this event who know more than the four of us in some areas, so what eventually appears on the HE Academy’s pathfinder blog on the outcomes of that particular session should have the large caveat that it was the product of four brains, ably captured by Liz, but I don’t think we’d claim to be the national experts on the subject.

An issue that has come up a number of times at this event is the need for some sort of a nationally coordinated collecting point for existing and emerging research in e-pedagogy variously. The HEA are talking about setting up an observatory function to do this – which sounds like a grand plan to me.

: 04 Oct 2006 12:20 |  Tags: E-Learning Pathfinder |  Comments (0) |  Close comments |  Report a problem

HEA York launch event

Congratulations to the HE Academy for arranging the launch event for the Pathfinder programme in a hotel which provides wireless broadband – it means I can blog as we go along! I know Jay has been taking notes on her PDA, so she will probably be uploading them in due course. But this for the final session this afternoon we were in different groups – Jay went to the one on CPD and Staff Development, while I went to the one on e-assessment.

The idea of these break-out discussions was to help the Academy identify gaps in their provision or opportunities to provde targeted support for Pahtfinding institutions, as well as a chance for us to talk amongst ourselves about our challenges and the like. As you’d expect, we talked through a number of issues – how e-assessment links with assessment more broadly, how and why we assess, how electronic methods can help students to learn better and staff teach better and so on. Questions of whether there is enough support “out there” (research into e-assessment, resources and advice from the Academy including the subject centres, other sources of good practice and so on) were more difficult for our small group to answer – we just don’t have the expertise which others in our Pathfinder teams do. The joys of specialisation.

You might like to look at Trevor Hawkes’s blog – he’s leading a project on CAA (computer-aided assessment) across Warwick’s Faculty of Science

It’s been useful to talk to people from other Pathfinder institutions and I think some of us will certainly keep in touch on particular issues, as well as scanning each other’s blogs from time to time.

What else from today? I gave a brief overview of our project this morning and I got a giggle when I described our e-learning as “managed diversity” – a giggle of recognition, I think/hope. Jay wonders whether we might more usefully aim towards “nurtured diversity”.

And the wikipedia entry on mind maps is apparently rather good, with some pretty pictures too:

I’ll check it out in due course.

: 03 Oct 2006 18:37 |  Tags: E-Learning Pathfinder |  Comments (2) |  Close comments |  Report a problem