All 129 entries tagged Teaching

Learnings & suggestions based on implementing diverse assessments – Lucy Ryland

Learnings and suggestions based on implementing diverse assessments in the foundation year at Warwick

By Lucy Ryland

Diverse assessments, sometimes known as ‘alternative assessments’, is a growing area of interest in higher education, and Warwick is no exception. As a member of the WIHEA Diverse Assessment Learning Circle, I delivered a presentation on my own practice in relation to diverse assessments. This triggered a discussion around how my collection of assessments could be expressed in terms of an ‘assessment strategy’. Here I highlight my rationale for diversifying assessment and reflect on my experiences with implementing a diverse assessment diet with international foundation students.

Diverse assessment can be defined as an assessment method “that is less familiar (to you and/or the students) and increases the range of assessments in your discipline” (O’Neill and Padden 2021:3). Interested in this concept and the potential benefits it could bring, in the 2021-22 academic year, as convener of three modules on the Warwick International Foundation Programme in Social Science, I used 11 different assessment methods with my group of students, and embedded choice into almost all of them.

Research suggests there are many potential benefits to adopting a diverse assessment strategy. I identified six advantages that I hoped to bring to my teaching and my students’ learning through this diversity.

1. In some instances, diverse assessment was required in order for the intended learning outcomes of my modules to be assessed, to ensure the constructive alignment of my programme. One such ILO was that students should be able to “justify informed solutions to social science problems to a varied audience using a variety of media formats”. Diverse assessment methods of blog post writing, infographic design and film production were therefore appropriate methods to assess students’ attainment of this learning outcome.
2. Using a variety of assessment methods, and embedding choice into students’ assessments, is argued by Bloxham and Boyd (2007:190) to enhance the inclusive nature of assessment. It enables students to pursue their own interests, preferences, and strengths. If students are always assessed using written, essay-style tasks, this will advantage students who are strong in the relevant skills, and disadvantage those whose skills and strengths lie in different areas. Therefore, including written assessments alongside oral presentations and more visual or creative methods of assessment is likely to make the students’ assessment diet more inclusive.
3. Part of the purpose of higher education has to be to prepare students for life beyond studies, in the ‘real world’. There is therefore value in encouraging students to develop skills and knowledge that is going to help them pursue their future goals (Knight and Yorke 2003). A number of the different assessment methods I use, including blog writing, annotated bibliographies, case studies and research projects, are designed to be authentic, to use real world contexts and facilitate the growth of skills that the students may well need in their future studies and careers. They have a function beyond simply receiving a grade at the end of the process.
4. In a similar vein, evidence suggests that the ‘traditional’ examination type assessment has limited value in terms of developing students’ skills rather than focusing on simple knowledge recall (Dunn et al. 2004; Bloxham and Boyd 2007:195). It is also apparent that more traditional timed examinations induce high levels of stress in students, and this can limit how effectively they can really show their full capability. Conversely, it may be the case that diverse assessments, that are more inclusive and authentic, are less stressful for students and enable them to perform to the best of their ability (Hong-Meng Tai et al. 2022:6).
5. Preparing and submitting the same type of assessment on multiple occasions can be monotonous for students, especially in an intense year of study and assessment. Introducing variety and authenticity into the assessments can enhance students’ motivations to complete them: they can do something different, something new and relevant for them (Bloxham and Boyd 2007:191; Struyven et al. 2005). There is also motivation for the teacher in reviewing and marking a variety of assessment types rather than reading multiple 2000-word essays throughout the year.
6. Finally, diverse assessments open many opportunities for more peer to peer and group learning experiences, which have value in themselves. Compared to a simple essay or exam, adding comments to each other’s blog posts, acting as the audience for each other’s presentations, watching each other’s film productions, and collaborating in case study assessments, provide numerous opportunities for students to learn from each other and work together, enhancing their learning experience (Hong-Meng Tai et al. 2022:9).

As an outcome of the above, across my three 30 CAT modules of the students’ 120 CAT programme, I implemented the following assessment diet:

On reflection, after implementing this for one academic year, I found that the diversity and variety did indeed facilitate the nurturing of new skills, both for me and the students, that are likely to be useful for their future careers. In a feedback survey conducted at the end of the academic year, all of the respondents (6 out of the 8 registered on the course) agreed or strongly agreed that the assessments helped them develop skills that will be useful to them in the future. I found the variety motivating and helped to keep things exciting, both in terms of the types of assessments and the students’ choice within them. Within Understanding Society for example, for the infographic, the blog post and the film project, students had the option to choose to focus on any aspect of the given topic they are interested in, leading to a broad range of pieces of work which were a joy to mark. In collecting student feedback, all of the students who completed the survey either agreed or strongly agreed that the variety of assessment methods was exciting and motivated them to work hard.

I had been concerned about how student performance might be affected by the variety of assessment and the cognitive load that necessarily comes with it, but students actually performed very well, with the mean score for FP009 11 points higher than the previous academic year and three points higher in FP053, whilst FP036 did see a three-point fall in the mean overall outcome. All students responded to my survey that they agreed or strongly agreed that the variety of assessments enabled them to demonstrate the full range of their knowledge and skills and meant that at least some of the assessment types had catered to their personal strengths.

Nonetheless, there were challenges along the way. Introducing lots of new skills to the students alongside new knowledge was a heavy load both for me as the teacher and the students. Students reported at the end of the year feeling sufficiently prepared to complete all of the assessments, but a lot of time and energy had gone into this preparation. Further challenges from the teacher perspective included preparing appropriate assessment criteria for all the different forms of assessment, and considering parity across different assessment types. For example, I had to consider whether a 1500-word essay in Politics is of comparable workload and complexity to a 5-minute film production in Understanding Society, since they are both worth 40% of the 30 CAT modules.

The only one of my reasons for introducing such assessment diversity that was not evident in the experience was the level of stress students felt around their assessments. Two students disagreed that not doing only traditional exams reduced how stressful they found assessments and one response was neutral. Three did agree, but none strongly agreed. Interestingly, the assessments that the students reported enjoying the most were quite different to those they reported having learned the most from, so it may not always be the case that we learn the most when we are enjoying the experience. This appears to go against much research in the area of enjoyment and learning (Lucardie 2014).

There are thus many aspects for me to continue to reflect on and develop, including how my practice can be expressed in terms of an assessment ‘strategy’ (see the earlier post from the WIHEA Learning Circle), and the WIHEA Learning Circle is an excellent space for me and anyone else interested in diverse assessment to discuss good practice and share experiences as we look to improve our students’ experiences of assessment. If you would like to join this learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk).

Reference list:

Bloxham, S and Boyd, P (2007) Developing Effective Assessment in Higher Education: A Practical Guide, London: McGraw-Hill Education

Dunn, L et al. (2004) The Student Assessment Handbook: New Directions in Traditional and On-line Assessment, London: RoutledgeFalmer

Hong-Meng Tai, J. et al. (2022) ‘Designing assessment for inclusion: an exploration of diverse students’ assessment experiences’ in Assessment & Evaluation in Higher Education DOI: https://doi.org/10.1080/02602938.2022.2082373

Knight, P. T and Yorke, M (2003) Assessment, Learning and Employability, Maidenhead: OUP

Lucardie, D (2014) ‘The impact of fun and enjoyment on adults’ learning’ in Procedia – Social and Behavioural Sciences 142, pp.439-446

O’Neill, G and Padden, L (2021)’Diversifying assessment methods: Barriers, benefits and enablers’ in Innovations in Education and Teaching International DOI: https://doi.org/10.1080/14703297.2021.1880462

Struyven, K et al. (2005) ‘Students’ perceptions about evaluation and assessment in higher education: a review’ in Assessment and Evaluation in Higher Education 30(4) pp.325-341

This is the 5th blog in our diverse assessment series, the previous blogs can be found here:

Blog 1: Launch of the learning circle: https://blogs.warwick.ac.uk/wjett/entry/interested_in_diverse/

Blog 2: Creative projects and the ‘state of play’ in diverse assessments: https://blogs.warwick.ac.uk/wjett/entry/creative_projects_and/

Blog 3: Student experience of assessments: https://blogs.warwick.ac.uk/wjett/entry/a_student_perspective/

Blog 4: Assessment Strategy – one year after starting the learning circle https://blogs.warwick.ac.uk/wjett/entry/one_year_on/

If you would like to join this learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk).

Abigail Ball : 05 Dec 2022 11:56 |  Tags: Assessment Lucy Ryland Teaching |  Comments (3) |  Close comments |  Report a problem

Well–being pedagogies: activities and practices to improve the student experience online

In this short THE Campus piece, Warwick’s own Elena Riva shares some helpful practices that can boost student well-being in the online teaching and learning environment:

https://www.timeshighereducation.com/campus/wellbeing-pedagogies-activities-and-practices-improve-student-experience-online

Abigail Ball : 28 Nov 2022 09:27 |  Tags: Elena Riva Social Student Teaching |  Comments (0) |  Close comments |  Report a problem

Making connections – embedding values in the student journey

Maha Bali is an educator who writes and speaks frequently about social justice, critical pedagogy, and open and online education. You can find her blog on this webpage. She is co-facilitator of Equity Unbound which, in collaboration with OneHE, has curated a range open educational resources focused on community building activities that educators can use with their students. You can also watch Maha’s keynote address at our recent TEALfest 2022 event by following this link: Building community online with equity and care.

Abigail Ball : 21 Nov 2022 09:53 |  Tags: Equity Unbound Maha Bali Teaching |  Comments (0) |  Close comments |  Report a problem

One year on: Progress update on our Diverse Assessments Learning Circle

Isabel Fischer (WBS) and Leda Mirbahai (WMS)

One year ago we created an open WIHEA Learning Circle on Diverse Assessments. Since we seem to be building a reputation as the ‘godparents of assessments’. In addition to contributing to diversifying assessment strategies across Warwick we aim to work towards providing equity in our assessment practices and to improve student experience. Assessments, if used effectively, are key to promoting learning for our students.

To encourage reflection and to drive change in how we use and view assessments in our programmes, we hosted a series of keynote speeches to start our regular meetings. Here one example from Kerry Dobbins, Academic Development Centre, on How to create an effective assessment strategy (drilling down – or up – from institutional, via course, to module level)

Assessment Strategy

My aim for this presentation was to highlight the conflation that often occurs between assessment ‘strategies’ and assessment ‘methods’. The term ‘strategy’ is often used when we are actually referring to the mode of assessment, e.g. ‘our assessment strategy is coursework or an online exam’. It is important to disentangle these terms so that we can take an explicitly strategic approach to designing assessments that supports inclusion at all levels, i.e., module, course/programme and institution. An assessment strategy develops a shared and holistic view of the course/programme between students and academics. At a macro level, there needs to be constructive alignment between module learning outcomes (LOs), course/programme LOs and graduate attributes. In this way, a programme level view is taken to what LOs are being assessed across modules and how. For diverse assessments this is extremely important because it ensures that a holistic view is taken in relation to how comfort with, skills for and literacy of different types of assessment methods are developed and scaffolded for students as the programme progresses. This strategic and holistic view also recognises the various transition points of the students’ journey; so first year assessments may start to introduce elements of doing things differently, that are built on in the second year, etc.

In essence then, a strategic approach is vital for inclusive assessment practices as it provides an explicit framework for developing assessment literacy skills and for assignment feedback to be clearly directed towards feeding forward into future assessment activities. Taking a strategic approach also provides greater opportunities for teams to develop a coherent view about the purposes and values of assessment; and how those shared values are threaded through the course or programme. Assessment is not value-free as we are always conveying value messages to students about what we assess and how. A programme strategy allows us to really consider our values and what we are trying to achieve with our assessment practices and processes overall.

Assessment strategy also occurs at the module level. Again, at this level the strategy is not the mode of assessment but how support to achieve within the assessment is structured into the module. For example, how is assessment and feedback literacy designed into the module curriculum? What does the pre and post-assessment support look like? What is the rationale for the mode of assessment being used? How is assessment (formative and summative) being used within the module to support learning, not just quantify it?

You might find the attached presentation and some of the texts below useful to review:

Boud and Associates (2010) Assessment 2020: Seven propositions for assessment reform in higher education

Brunton et al (2016) Designing and developing a programme-focused assessment strategy: a case study

Scholtz (2016) (PDF) The assessment strategy: An elusive curriculum structure (researchgate.net)

If you are interested in this area, I would welcome you to get in touch: Kerry.Dobbins@warwick.ac.uk

For our learning circle we have also managed to secure funding to undertake a research project to capture both student and staff views of diverse assessments. Although the project is still ongoing, our student project officers, Molly Fowler and Pula Prakash, have managed to gather valuable data with an aim to feed into institutional considerations around assessment strategies.

Finally, if you want to find out more about our Learning Circle you can visit our webpage and you can read our previous blogs here:

Blog 1: Launch of the learning circle: https://blogs.warwick.ac.uk/wjett/entry/interested_in_diverse/

Blog 2: Creative projects and the ‘state of play’ in diverse assessments: https://blogs.warwick.ac.uk/wjett/entry/creative_projects_and/

Blog 3: Student experience of assessments: https://blogs.warwick.ac.uk/wjett/entry/a_student_perspective/

If you would like to join this learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk).

Abigail Ball : 14 Nov 2022 08:38 |  Tags: Assessment Isabel Fischer Leda Mirbaha Reflections Teaching |  Comments (0) |  Close comments |  Report a problem

The self and syllabus – Teaching in Higher Ed podcast

The discussion in this podcast focuses on how we can embed and bring ourselves into all aspects of our teaching practices, including student-facing artefacts like course documents and handbooks. The presenters offer some practical tips on making initial connections with students through their first encounters with written module documentation. All resources discussed in the podcast can be accessed from the episode webpage.

The self and syllabus – Teaching in Higher Ed podcast

Abigail Ball : 07 Nov 2022 09:39 |  Tags: Pedagogy Teaching |  Comments (0) |  Close comments |  Report a problem

A Student Perspective on Assessment Diversity and Strategy – Molly Fowler

Over the last two years I have completed 21 assessments of approximately 10 different types - I am a seasoned consumer of assessments. Truly diverse assessment should use a range of valid assessment types that stimulate learning and provide opportunities for learners with varied needs and capabilities.

Assessments where I have synthesised information mined without time pressure, have resulted in the deepest learning. I overcame an irrational fear of epigenetics by writing an essay on the topic. I have been wondering whether real time information acquisition and synthesis could form an equally valuable assessment modality. My recent work experience showed me the readiness of healthcare professionals at all levels to access and deploy ambulatory online information in real time on the ward and in the clinic. As far as I can see, assessment methodologies have yet to catch up with this new reality.

It is perhaps a truism to say that assessments should be a learning methodology. Assessment can be a stimulus to learn content but can also model useful skills and behaviours. I have found that assessments with low ecological validity are less enriching, and regrettably invite purely strategic learning. The best example I have is my French GCSE in which I achieved an A grade without being able to speak a word of the language. The nature of the assessment meant that I did not need to engage with the content or the objectives of the course. I was able to prevail by memorising and regurgitating chunks of text which I promptly forgot after the exam.

Of my recent exams, the methodology that strikes me as most at fault in this respect is the MCQ. This is a highly artificial assessment construct, relying crucially on recall of memorised information, in a specific and stressful environment. I accept that there may be other skills involved such as prioritising options, but this is barely possible without the memorising and recall.

Group assessments might seem to reflect real world behaviour, but I have not always found this to be the case. With no agreed hierarchy it is very difficult to distribute responsibility or manage group members who are either overly assertive or who fail to contribute. This is particularly tricky when all those involved are required to be equal contributors with equal status. Surely this is an unusual circumstance in any working environment. The contrived dynamics of group assessments can introduce an uncontrolled variable that impacts the results of even very good students.

I feel it is of critical importance that students have the opportunity to thoroughly rehearse assessment methodologies that they will face in the high-stakes final year. For example, having done two poster presentations, previously unfamiliar to me, I feel less daunted by the prospect of doing it again for my dissertation module. I now appreciate that this is a common way of disseminating information at scientific conferences so it may be an important skill for me to have in my career.

Personal lives are complex and we are all subject to sudden or dramatic change in our circumstances. Assessments by essays can more easily accommodate and mitigate unforeseen events through flexibility, but fixed assessments such as exams or in-person presentations are much more difficult to rearrange. An advantage of continuous assessment over end-of-year assessments is the feasibility of allowances for such happenings as fewer assessments would be affected. It is also true that students with a reported disability tend to do less well than students with no reported disability (Office for Students, 2022). This raises important questions around equity, and whether the available reasonable adjustments are good enough.

Properly diverse assessment could serve many purposes. Within the mix of assessments should be opportunities for different students to demonstrate their individual proficiency at their preferred method. Diverse assessment should be inclusive of students with reasonable adjustments, but they should also allow for unpredictable adverse events. Finally, where possible, assessments should engender learning and embody activities that will be useful in later life.

Office for Students (2022) Access and Participation Data Dashboard [online] Available from: https://www.officeforstudents.org.uk/data-and-analysis/access-and-participation-data-dashboard/ (Accessed 8 October 2022).

Molly is member of the Diverse Assessment Learning Circle. If you would like to join the LC please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk)

Link to blog one : Introduction to diverse assessment learning circle

Link to blog two: Creative Projects and the ‘state of play’ in diverse assessments

Abigail Ball : 31 Oct 2022 09:28 |  Tags: Assessment Molly Fowler Student Teaching |  Comments (0) |  Close comments |  Report a problem

Tips for new lecturers on the human elements that make students feel included

This short THE Campus piece offers advice about simple things lecturers can do in their lectures to enhance students’ feelings of belonging and inclusion from the very start:

https://www.timeshighereducation.com/campus/tips-new-lecturers-human-elements-make-students-feel-included

Abigail Ball : 24 Oct 2022 10:21 |  Tags: Teaching |  Comments (0) |  Close comments |  Report a problem

Community–sourced introductory and setting the tone activities

Community-sourced introductory and setting the tone activities

Equity Unbound and OneHE have developed a values-led initiative that curates community building activities for educators to use with their students. Their Introductory activities section includes creative ways to undertake student introductions in your first week of classes. The Setting the tone activities section outlines approaches that you can use throughout the term to let students know that you care.

Abigail Ball : 17 Oct 2022 09:27 |  Tags: Teaching |  Comments (0) |  Close comments |  Report a problem

Introducing Warwick’s “AI Essay–Analyst”

By Isabel Fischer, Zhewei Zhang, Lichuan Xiang, Aiqi Jiang, Yiran Xu and Joe Nandhakumar

Three years since its first conceptualisation, we are pleased to introduce the “AI Essay-Analyst”, an academic-writing-tool in support of the mission of Warwick Business School (WBS) “to enable our stakeholders to realise their full potential” and the University of Warwick’s 2030 strategy “to ensure that, irrespective of background, disability, faith, gender, race and sexual orientation, all students have access to equal opportunities to thrive and progress at Warwick”.

A recent WBS survey showed that the majority of students perceive poor academic writing as their main barrier to success. In response, a group of WBS faculty and students developed and piloted a machine-generated automated formative essay feedback tool in-house which is now being made available to an increasing number of students on an optional basis.

Academic writing feedback tools have the potential of providing students with “personalised feedback that is currently only available to a privileged minority”^[1] and can enhance students’ self-determined learning: Formative assessments are seen as “one of the most important mechanisms for improving student learning. Self and peer-assessment are particularly effective in formative learning as they require students to engage more fully with the assessment process”^[2]. Currently, students of select modules are being offered the opportunity to trial the software by submitting their draft essays or dissertations prior to their submission deadlines. Participating students receive a personalised AI-generated feedback report of approximately 15 pages. The report includes images, charts and graphs which students are encouraged to review prior to the formal submission of their assignment.

External providers, such as Grammarly, Turnitin Draft Coach, Bartleby, Writefull, and Hemingway Editor also offer feedback to students, however, most of these tools focus on grammar and spelling. In addition, in most cases, students have to agree that external providers can use their data. The “AI Essay-Analyst” does not use student data and is substantially more comprehensive. For example, by also including the CABS ABS ranking we can check the quality of the journal articles that are cited and referenced. In addition, we offer visualisations such as knowledge graphsand argumentative zoning^[3], which are expressed as PIE charts and knowledge graphs^[4]. These visualisations are very much appreciated by students.

Students who opted to take part in the project so far were very satisfied, commenting: “The overall feedback is very useful for the general understanding of your academic writing skills”, “It is quite cool and it is a new approach I never tried before”, “I have enjoyed the visualisations most since they are interactive and easy to understand” and “Grammar suggestions are useful since they show some spelling and small mistakes that I ignored before.”

Detailed student feedback on specific features included:

1. The most useful are grammar suggestions, because it helps me revise the essay most directly.
2. The spider graph is useful to help me understand where the essay is lacking.
3. The Word Cloud is useful to help me check if the essay is on topic.
4. Systematic stages of negation is helpful as it let me know if my critical thinking has been fully applied.
5. For readability, this is an aspect that I usually find difficult to notice, because everything is readable in my own mind. So that is very helpful.
6. The knowledge graph allowed me to see the bigger picture at a time when I was too focused on the detail. It helped me to break down my essay and also showed the correct as well as incorrect relationships between key concepts.

For comments or questions please contact the project lead Isabel.fischer@wbs.ac.uk

[1] https://oro.open.ac.uk/46517/1/LAK16%20Writing%20Analytics%20Wkshp%20-%20FINAL.pdf

[2] https://www.tandfonline.com/doi/full/10.11120/plan.2010.00230040

[3] https://www.cl.cam.ac.uk/~sht25/az.html

[4] https://medium.com/nlplanet/building-a-knowledge-base-from-texts-a-full-practical-example-8dbbffb912fa

Abigail Ball : 10 Oct 2022 09:08 |  Tags: Isabel Fischer Teaching |  Comments (0) |  Close comments |  Report a problem

A values–laden approach to teaching and learning in Higher Education and why it matters

In this short (20mins) video, Julie Taylor from the Centre for Teaching Education (CTE) shares the work that she and her colleagues have done in CTE to embed a values-laden approach to teaching and learning. The key messages shared are relevant to any discipline or department. The resources she refers to in the video can be accessed below:

• Character education in universities: a framework for flourishing
• The Jubilee Centre framework for character education in schools
• Statement on teacher education and character education
• PowerPoint slides used in the video

Abigail Ball : 03 Oct 2022 10:05 |  Tags: Character Julie Taylor Teaching Values |  Comments (0) |  Close comments |  Report a problem