All 24 entries tagged Assessment

Ungrading: more possibilities than some might think

Assessment is often the biggest cause of student anxiety and distress. Some have begun to explore ‘ungrading’ as a way to enhance the developmental rather than judgemental aspects of assessment. Ungrading can be implemented in various ways and is a process of decentring summative grades or marks. In this 17-minute video, Martin Compton from UCL explores the potential of ‘ungrading’ and the various ways that he has implemented elements of it.

https://www.youtube.com/watch?v=xdBYm8K_pVI&list=PLAbF8wnSF-e9H54nDtvsCXehw4xDS9xQb&index=2

Abigail Ball : 30 May 2023 09:00 |  Tags: Assessment Martin Compton Ucl |  Comments (0) |  Close comments |  Report a problem

Democratising the feedback process

Blog 7 of our Diverse Assessment Learning Series, based on a recent keynote address as part of the Diverse Assessments WIHEA Learning Circle

By Dr Linda Enow, from Newman University, Birmingham UK

We need to rethink feedback in Higher Education (HE). To examine feedback in HE, this contribution engages with democratic principles, and through Audio-Visual feedback (AVF) interrogates current feedback practices. Some current gaps in feedback research are on understanding the nature of student engagement and interaction with feedback, relational constituents of feedback processes, and the role of technology in supporting feedback processes. This paper posits that power imbalance with feedback is a barrier to effective feedback engagement and interaction. Embedded in the, now challenged, conceptualisation of feedback giver and feedback recipient dynamic is implicit power imbalance. A further challenge for feedback is the over-reliance on written feedback within which is arguably entrenched the ‘product’ conceptualisation of feedback. Through the exploration of democratic and cognitive requirements of feedback, this contribution traces an outlook which values equitable relationships and emphasises the positioning of feedback as a process. Insights on feedback processes in this piece have emerged from empirical work on audio-visual feedback.

Where we are in HE with assessment design?

Significant strides are being taken to design diverse assessment tasks in HE. These tasks are designed with due consideration of the diverse student population and established knowledge of andragogy. Typically, assessment design aims to showcase inclusion. With graduate outcome requirements for HEIs in the UK, assessments subsequently engage with the more utilitarian constituent of seeking applicability in workplace settings. Perhaps assessments should be more dynamic, for instance generated from cohort identity and depth of knowledge of the practice requirements of students, rather than current static assessment practices. In any case, progress made with assessment design and the thinking behind assessment practices demonstrates the ongoing effort of Higher Education Institutes (HEIs) to be diverse in their practices, and this is welcome effort in the right direction.

As we diversify assessments, is feedback left behind?

In the context of assessment, the corresponding element of feedback is left behind. Written feedback remains the dominant feedback mode. A small proportion engages with audio-feedback, typically as a one-off. Audio-Visual Feedback (AVF) despite demonstrating its multimodal composition (see West & Turner, 2016) is minimally used compared to written feedback and audio feedback (Lowenthal, 2021; Nicol, 2012). Audio-visual feedback is inclusive, multimodal, precise, and retains its quality over time hence guaranteeing portability. The nature of AVF embeds verbal feedback with access to the written assignment, and the use of technology to enable precision. Verbalising naturally offers more depth in explanation with the added advantage of capturing relational constituents of feedback. AVF is a suitable format for the thinking or the cognitive composition of the feedback to become explicit (see Nicol, 2021). For all involved in the feedback process engagement and interaction are both enhanced. A summation of these strengths is reflected in Payne et al. (2022) positioning audio and video feedback modes as humanising.

We no longer ‘give’ feedback

Contemporary research on feedback processes tracks the evolution of feedback from the now defunct perception of information which was given to the students, to the contemporary understanding that; first of all feedback is a process, and secondly feedback is dialogic, ongoing and embedded with andragogical expectations, as well as aspirations of transferability and portability (e.g. de Kleijn, 2023; Winstone et al., 2022a). Equipped with the understanding that saying you are giving feedback is no longer acceptable (Winstone, et al., 2022b) from the andragogical perspective, this paper makes the case to re-think feedback processes from the bases of democratic principles. Educating, especially in HEIs in the UK, is based on foundational democratic principles imbued with a variety of duties, rights and responsibilities. Power imbalance is therefore in conflict with these democratic principles. From the position of evaluating HEIs in the UK, Winstone et al., (2022b: 1524) raises the concern that National Student Survey (NSS) questions “promote an outdated view of feedback as information transmitted from teacher to student in a timely and specific manner, largely ignoring the role of the student in learning through feedback processes”.

How do we democratise feedback?

Democratic principles advocate freedoms, rights and responsibilities. If we think about feedback following democratic principles, we interrogate primarily the power imbalance in the lecturer-student relationship. The lecturer is in the powerful position of ownership of the feedback which they ‘give’ to the student (see Matthews, et al., 2021). The sense of ownership of the feedback content is taken away from the student and the student becomes a recipient. This power imbalance means the student has a skewed relationship with this feedback which is being imposed on them. What is the student to do with this feedback which has been given to them? What if the student does not identify with the feedback which has been given? Carless (2015:28) declares; impact on learning is limited unless students are actively engaging with feedback processes, and ultimately acting on feedback. Consensus on what these processes constitute is yet to be arrived at as De Kleijn (2023) requests clarity of the activities and strategies which must be applied in these feedback processes. When democratic principles are not followed, distance is created between the student and the feedback process.

In contrast to the lecturer-student feedback element, there is strong uptake of peer-feedback. Peer feedback (see Deneen & Hoo, 2023) supports feedback dialogue, evidences students’ feedback literacy and supports development of self-regulation skills. The strong interest in peer feedback is a result of the significant benefits. Nicol and McCallum (2022) assert that powerful insights are generated from student feedback, at times more powerful than teacher feedback. From the research which this contribution on democratising feedback draws on, the strength of peer feedback is being explored and understood from the position of power. There is the opportunity to redesign assessments to possibly capture and support these peer feedback partnerships linking them to assessment design. Moving away from the conceptualisation of feedback as product, to feedback as process serves to further democratisation of feedback. Careful consideration in incorporating peer feedback within this process contributes to improving feedback dialogue, and to establishing tangible steps. Continuing this dialogue using the audio-visual mode opens up relational spaces and further enhances feedback processes.

Audio-Visual Feedback (AVF)

The discursive nature of AVF assists in personalising feedback and modelling the expectation for engaging with feedback. The multimodal nature of audio-visual feedback (Lacković, and Popova, 2021) necessitates active designation of time to not only engage with feedback, but to interact with feedback. AVF supports the transition of feedback into a process. There is a challenge embedded in this; assessment design minimally factors in feedback avenues and suitable allocation of time for AVF. This is understandable as written feedback is dominant in HEIs in the UK. In order for AVF to attain its full potential, HEIs need to re-work their assessment and feedback policies. Re-working policies is definitely not to give more time to turnaround times; rather this is more time to work on effectively embedding AVF as a viable feedback avenue. AVF is not without its potential challenges. One example is large class sizes and staffing limitations in some HEIs. A potential solution is considering AVF for group assessments and group feedback. Teaching in HEIs is inherently multimodal; combining written, audio and audio-visual content. Why is feedback disproportionately in one format?

Some points to facilitate change

1. Undertake an audit of feedback formats; recording written format, audio format, and audio-visual format.
2. Interrogate power imbalance relating to feedback and explore institution-focused and relevant ways to improve the power imbalance.
3. Co-design feedback processes in line with contemporary co-designing of assessments. Ensure feedback is not an add-on or an after-thought to assessments (see Ajjawi and Boud, 2018).
4. Review assessment and feedback policies.
5. Equity: establish feedback processes reflective of advances in learning, teaching and scholarship.

What next?

Feedback is a process involving students and lecturers in a shared space. Cognitive, sociocultural, social constructivist positions, amongst others are at play and space needs to be created for democratic principles. Alongside making sense of feedback, cognitive drivers pass judgement on the utility of feedback, and decision making guides portability of feedback. This contribution advocates the externalisation of these cognitive constituents as a precursor to democratising feedback. The argument is; the student has a right to know. Knowing in this way is empowering. Through democratisation, the student evolves from the position of a recipient to that of a partner in the feedback process. Partnership conjures images of ownership, responsibility and duties. Democratising feedback, through the medium of audio-visual feedback (AVF) removes the power imbalance, showcases the dialogic nature of feedback, and enhances subsequent portability of knowledge and skills. Whilst there is empirical work on feedback as a process, implementation in the structures of HEIs is yet to follow.

This is the 7th blog in our diverse assessment series. Previous blogs can be found here:

Blog 1: Launch of the learning circle (Isabel Fischer & Leda Mirbahai): https://blogs.warwick.ac.uk/wjett/entry/interested_in_diverse/

Blog 2: Creative projects and the ‘state of play’ in diverse assessments (Lewis Beer): https://blogs.warwick.ac.uk/wjett/entry/creative_projects_and/

Blog 3: Student experience of assessments (Molly Fowler): https://blogs.warwick.ac.uk/wjett/entry/a_student_perspective/

Blog 4: Assessment Strategy – one year after starting the learning circle (Isabel Fischer & Leda Mirbahai): https://blogs.warwick.ac.uk/wjett/entry/one_year_on/

Blog 5: Learnings and suggestions based on implementing diverse assessments in the foundation year at Warwick (Lucy Ryland): https://blogs.warwick.ac.uk/wjett/entry/learnings_suggestions_based/

Blog 6: How inclusive is your assessment strategy? (Leda Mirbahai): https://blogs.warwick.ac.uk/wjett/entry/blog_6_how/

Join the Diverse Assessment Learning Circle: If you would like to join the learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk). This LC is open to non-WIHEA members.

References

Ajjawi, R. & Boud, D. (2018) 'Examining the nature and effects of feedback dialogue', Assessment & Evaluation in Higher Education, 43:7, 1106-1119, DOI: 10.1080/02602938.2018.1434128 https://doi.org/10.1080/02602938.2018.1434128

Carless, D. (2015) Excellence in university assessment: learning from award-winning practice. London: Routledge.

Deneen, C. C. & Hoo, H-T. (2023) 'Connecting teacher and student assessment literacy with self-evaluation and peer feedback', Assessment & Evaluation in Higher Education, 48:2, 214-226, DOI: 10.1080/02602938.2021.1967284 https://doi.org/10.1080/02602938.2021.1967284

de Kleijn, R. A. M. (2023) 'Supporting student and teacher feedback literacy: an instructional model for student feedback processes', Assessment & Evaluation in Higher Education, 48:2, 186-200, DOI: 10.1080/02602938.2021.1967283

Lacković, N. & Popova, B. (2021) 'Multimodality and socio-materiality of lectures in global universities’ media: accounting for bodies and things', Learning, Media and Technology, 46:4, 531-549, DOI: 10.1080/17439884.2021.19286

Abigail Ball : 03 Apr 2023 08:00 |  Tags: Assessment Leda Mirbaha Linda Enow Teaching |  Comments (0) |  Close comments |  Report a problem

Blog 6: How inclusive is your assessment strategy?

By Leda Mirbahai (Warwick Medical School)

Assessments are a fundamental part of student experience - with students learning by doing, i.e. by engaging with assessment tasks and then, after submission through the feedback they receive on their performance and progress Internal and External Examiner often ensure that assessment strategies are reliable, effective, and accurate, however, whether they are inclusive is often overlooked.

Considering the diversity of our student population, it is clear that ‘one size fits all’ approach to assessment design and delivery is not an inclusive assessment strategy which links with diversification of our assessment approaches (a point that we will come back to shortly). The Quality Assurance Agency (QAA), UK’s quality code for higher education, lists 10 guiding principles on expectations and practices for assessment which are:

1. Assessment methods and criteria are aligned to learning outcomes and teaching activities.
2. Assessment is reliable, consistent, fair and valid.
3. Assessment design is approached holistically.
4. Assessment is inclusive and equitable.
5. Assessment is explicit and transparent.
6. Assessment and feedback are purposeful and supports the learning process.
7. Assessment is timely.
8. Assessment is efficient and manageable.
9. Students are supported and prepared for assessment.
10. Assessment encourages academic integrity.

As mentioned, a criteria of good assessment ‘strategy’ is to be inclusive and equitable. If one really reflects on the 10 principles, they are all interlinked. For example, it is very difficult to demonstrate an assessment strategy that is inclusive where assessment loads are not manageable for our students or students don’t receive equitable level of support. With Toolkits such as ‘Embedding inclusive assessment reflective toolkit’, a project funded by QAA, becoming available we can reflect on the assessment strategy of our courses and programmes by considering how well we align to attributes of an inclusive assessment. The toolkit starts to encourage a triangulation critic of our assessment approaches by involving major stakeholders from students, academics and leaders in the process and asking some key reflective questions. Reflecting on the 9 attributes of inclusive assessment as mentioned by this toolkit, most questions are basically reflecting on the principles of good assessment, which in my view indicates that if your assessment aligns to the principles, it should promote inclusivity.

As an academic staff leading the assessment strategy for a new UG course at WMS, I have been involved in the planning and delivery of a course level assessment strategy. As the course is new, it has removed some of the challenges of trying to alter and adjust existing assessments for individual modules. This raises an important concept; assessment strategies should be seen at course/programme level. Going back to principles of good assessments, we need to demonstrate how our assessments map to course level, year level and module level learning outcomes as well as ensuring students are supported and prepared for assessments. This highlights the gradual building of our learner’s skills and knowledge in a spiral curriculum and enabling them to receive continuous feedback on their progress in a meaningful way. Learners need to be able to visualise and reflect on their progress across the programme and to achieve this, our assessments in one module (skills, attribute and knowledge) need to meaningfully build on the previous modules; hence programme level approach being more desirable.

Using diverse assessments is a great way of acknowledging that ‘one size fits all’ approach to assessment design and delivery is not an inclusive assessment strategy. However, this also doesn’t mean that we should sprinkle our assessment programmes with as many different modes of assessments possible. Introducing too many different modes of assessments that are not revisited or are not utilising or building on skills and knowledge of a learner would just add to the level of stress encountered by our learner as it means our students need to learn a new and unfamiliar assessment approach just to use it once! Therefore, diversification should be achieved in a meaningful way rather than just for the sack of introducing new assessment modes.

Finally, in my view one of the most important aspects of a good assessment strategy is continuous reflection and improvement. There is no fixed perfect assessment strategy as our learners and their requirements are constantly changing. An assessment strategy that is developed and never revised will soon become unfit for the purpose it was developed. I know this as I am already making a list of changes that we need to embed into our assessment strategy and the course is only 3 years old! So I leave you with one question. How inclusive is your assessment strategy?

References

The Quality Assurance Agency (QAA) for Higher Education (2018) UK Quality Code for Higher Education advice and guidance: Assessments.

Embedding inclusive assessment-Reflective toolkit (2022), a QAA funded project. Developing a Set of Inclusive Assessment Design Attributes for use Across the Higher Education Sector (qaa.ac.uk)

This is the 6th blog in our diverse assessment series. Previous blogs can be found here:

Blog 1: Launch of the learning circle (Isabel Fischer & Leda Mirbahai): https://blogs.warwick.ac.uk/wjett/entry/interested_in_diverse/

Blog 2: Creative projects and the ‘state of play’ in diverse assessments (Lewis Beer): https://blogs.warwick.ac.uk/wjett/entry/creative_projects_and/

Blog 3: Student experience of assessments (Molly Fowler): https://blogs.warwick.ac.uk/wjett/entry/a_student_perspective/

Blog 4: Assessment Strategy – one year after starting the learning circle (Isabel Fischer & Leda Mirbahai): https://blogs.warwick.ac.uk/wjett/entry/one_year_on/

Blog 5: Learnings and suggestions based on implementing diverse assessments in the foundation year at Warwick (Lucy Ryland): https://blogs.warwick.ac.uk/wjett/entry/learnings_suggestions_based/

Join the Diverse Assessment Learning Circle: If you would like to join the learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk). This LC is open to non-WIHEA members.

Abigail Ball : 20 Feb 2023 09:38 |  Tags: Assessment Inclusion Leda Mirbaha Teaching |  Comments (0) |  Close comments |  Report a problem

Learnings & suggestions based on implementing diverse assessments – Lucy Ryland

Learnings and suggestions based on implementing diverse assessments in the foundation year at Warwick

By Lucy Ryland

Diverse assessments, sometimes known as ‘alternative assessments’, is a growing area of interest in higher education, and Warwick is no exception. As a member of the WIHEA Diverse Assessment Learning Circle, I delivered a presentation on my own practice in relation to diverse assessments. This triggered a discussion around how my collection of assessments could be expressed in terms of an ‘assessment strategy’. Here I highlight my rationale for diversifying assessment and reflect on my experiences with implementing a diverse assessment diet with international foundation students.

Diverse assessment can be defined as an assessment method “that is less familiar (to you and/or the students) and increases the range of assessments in your discipline” (O’Neill and Padden 2021:3). Interested in this concept and the potential benefits it could bring, in the 2021-22 academic year, as convener of three modules on the Warwick International Foundation Programme in Social Science, I used 11 different assessment methods with my group of students, and embedded choice into almost all of them.

Research suggests there are many potential benefits to adopting a diverse assessment strategy. I identified six advantages that I hoped to bring to my teaching and my students’ learning through this diversity.

1. In some instances, diverse assessment was required in order for the intended learning outcomes of my modules to be assessed, to ensure the constructive alignment of my programme. One such ILO was that students should be able to “justify informed solutions to social science problems to a varied audience using a variety of media formats”. Diverse assessment methods of blog post writing, infographic design and film production were therefore appropriate methods to assess students’ attainment of this learning outcome.
2. Using a variety of assessment methods, and embedding choice into students’ assessments, is argued by Bloxham and Boyd (2007:190) to enhance the inclusive nature of assessment. It enables students to pursue their own interests, preferences, and strengths. If students are always assessed using written, essay-style tasks, this will advantage students who are strong in the relevant skills, and disadvantage those whose skills and strengths lie in different areas. Therefore, including written assessments alongside oral presentations and more visual or creative methods of assessment is likely to make the students’ assessment diet more inclusive.
3. Part of the purpose of higher education has to be to prepare students for life beyond studies, in the ‘real world’. There is therefore value in encouraging students to develop skills and knowledge that is going to help them pursue their future goals (Knight and Yorke 2003). A number of the different assessment methods I use, including blog writing, annotated bibliographies, case studies and research projects, are designed to be authentic, to use real world contexts and facilitate the growth of skills that the students may well need in their future studies and careers. They have a function beyond simply receiving a grade at the end of the process.
4. In a similar vein, evidence suggests that the ‘traditional’ examination type assessment has limited value in terms of developing students’ skills rather than focusing on simple knowledge recall (Dunn et al. 2004; Bloxham and Boyd 2007:195). It is also apparent that more traditional timed examinations induce high levels of stress in students, and this can limit how effectively they can really show their full capability. Conversely, it may be the case that diverse assessments, that are more inclusive and authentic, are less stressful for students and enable them to perform to the best of their ability (Hong-Meng Tai et al. 2022:6).
5. Preparing and submitting the same type of assessment on multiple occasions can be monotonous for students, especially in an intense year of study and assessment. Introducing variety and authenticity into the assessments can enhance students’ motivations to complete them: they can do something different, something new and relevant for them (Bloxham and Boyd 2007:191; Struyven et al. 2005). There is also motivation for the teacher in reviewing and marking a variety of assessment types rather than reading multiple 2000-word essays throughout the year.
6. Finally, diverse assessments open many opportunities for more peer to peer and group learning experiences, which have value in themselves. Compared to a simple essay or exam, adding comments to each other’s blog posts, acting as the audience for each other’s presentations, watching each other’s film productions, and collaborating in case study assessments, provide numerous opportunities for students to learn from each other and work together, enhancing their learning experience (Hong-Meng Tai et al. 2022:9).

As an outcome of the above, across my three 30 CAT modules of the students’ 120 CAT programme, I implemented the following assessment diet:

On reflection, after implementing this for one academic year, I found that the diversity and variety did indeed facilitate the nurturing of new skills, both for me and the students, that are likely to be useful for their future careers. In a feedback survey conducted at the end of the academic year, all of the respondents (6 out of the 8 registered on the course) agreed or strongly agreed that the assessments helped them develop skills that will be useful to them in the future. I found the variety motivating and helped to keep things exciting, both in terms of the types of assessments and the students’ choice within them. Within Understanding Society for example, for the infographic, the blog post and the film project, students had the option to choose to focus on any aspect of the given topic they are interested in, leading to a broad range of pieces of work which were a joy to mark. In collecting student feedback, all of the students who completed the survey either agreed or strongly agreed that the variety of assessment methods was exciting and motivated them to work hard.

I had been concerned about how student performance might be affected by the variety of assessment and the cognitive load that necessarily comes with it, but students actually performed very well, with the mean score for FP009 11 points higher than the previous academic year and three points higher in FP053, whilst FP036 did see a three-point fall in the mean overall outcome. All students responded to my survey that they agreed or strongly agreed that the variety of assessments enabled them to demonstrate the full range of their knowledge and skills and meant that at least some of the assessment types had catered to their personal strengths.

Nonetheless, there were challenges along the way. Introducing lots of new skills to the students alongside new knowledge was a heavy load both for me as the teacher and the students. Students reported at the end of the year feeling sufficiently prepared to complete all of the assessments, but a lot of time and energy had gone into this preparation. Further challenges from the teacher perspective included preparing appropriate assessment criteria for all the different forms of assessment, and considering parity across different assessment types. For example, I had to consider whether a 1500-word essay in Politics is of comparable workload and complexity to a 5-minute film production in Understanding Society, since they are both worth 40% of the 30 CAT modules.

The only one of my reasons for introducing such assessment diversity that was not evident in the experience was the level of stress students felt around their assessments. Two students disagreed that not doing only traditional exams reduced how stressful they found assessments and one response was neutral. Three did agree, but none strongly agreed. Interestingly, the assessments that the students reported enjoying the most were quite different to those they reported having learned the most from, so it may not always be the case that we learn the most when we are enjoying the experience. This appears to go against much research in the area of enjoyment and learning (Lucardie 2014).

There are thus many aspects for me to continue to reflect on and develop, including how my practice can be expressed in terms of an assessment ‘strategy’ (see the earlier post from the WIHEA Learning Circle), and the WIHEA Learning Circle is an excellent space for me and anyone else interested in diverse assessment to discuss good practice and share experiences as we look to improve our students’ experiences of assessment. If you would like to join this learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk).

Reference list:

Bloxham, S and Boyd, P (2007) Developing Effective Assessment in Higher Education: A Practical Guide, London: McGraw-Hill Education

Dunn, L et al. (2004) The Student Assessment Handbook: New Directions in Traditional and On-line Assessment, London: RoutledgeFalmer

Hong-Meng Tai, J. et al. (2022) ‘Designing assessment for inclusion: an exploration of diverse students’ assessment experiences’ in Assessment & Evaluation in Higher Education DOI: https://doi.org/10.1080/02602938.2022.2082373

Knight, P. T and Yorke, M (2003) Assessment, Learning and Employability, Maidenhead: OUP

Lucardie, D (2014) ‘The impact of fun and enjoyment on adults’ learning’ in Procedia – Social and Behavioural Sciences 142, pp.439-446

O’Neill, G and Padden, L (2021)’Diversifying assessment methods: Barriers, benefits and enablers’ in Innovations in Education and Teaching International DOI: https://doi.org/10.1080/14703297.2021.1880462

Struyven, K et al. (2005) ‘Students’ perceptions about evaluation and assessment in higher education: a review’ in Assessment and Evaluation in Higher Education 30(4) pp.325-341

This is the 5th blog in our diverse assessment series, the previous blogs can be found here:

Blog 1: Launch of the learning circle: https://blogs.warwick.ac.uk/wjett/entry/interested_in_diverse/

Blog 2: Creative projects and the ‘state of play’ in diverse assessments: https://blogs.warwick.ac.uk/wjett/entry/creative_projects_and/

Blog 3: Student experience of assessments: https://blogs.warwick.ac.uk/wjett/entry/a_student_perspective/

Blog 4: Assessment Strategy – one year after starting the learning circle https://blogs.warwick.ac.uk/wjett/entry/one_year_on/

If you would like to join this learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk).

Abigail Ball : 05 Dec 2022 11:56 |  Tags: Assessment Lucy Ryland Teaching |  Comments (3) |  Close comments |  Report a problem

One year on: Progress update on our Diverse Assessments Learning Circle

Isabel Fischer (WBS) and Leda Mirbahai (WMS)

One year ago we created an open WIHEA Learning Circle on Diverse Assessments. Since we seem to be building a reputation as the ‘godparents of assessments’. In addition to contributing to diversifying assessment strategies across Warwick we aim to work towards providing equity in our assessment practices and to improve student experience. Assessments, if used effectively, are key to promoting learning for our students.

To encourage reflection and to drive change in how we use and view assessments in our programmes, we hosted a series of keynote speeches to start our regular meetings. Here one example from Kerry Dobbins, Academic Development Centre, on How to create an effective assessment strategy (drilling down – or up – from institutional, via course, to module level)

Assessment Strategy

My aim for this presentation was to highlight the conflation that often occurs between assessment ‘strategies’ and assessment ‘methods’. The term ‘strategy’ is often used when we are actually referring to the mode of assessment, e.g. ‘our assessment strategy is coursework or an online exam’. It is important to disentangle these terms so that we can take an explicitly strategic approach to designing assessments that supports inclusion at all levels, i.e., module, course/programme and institution. An assessment strategy develops a shared and holistic view of the course/programme between students and academics. At a macro level, there needs to be constructive alignment between module learning outcomes (LOs), course/programme LOs and graduate attributes. In this way, a programme level view is taken to what LOs are being assessed across modules and how. For diverse assessments this is extremely important because it ensures that a holistic view is taken in relation to how comfort with, skills for and literacy of different types of assessment methods are developed and scaffolded for students as the programme progresses. This strategic and holistic view also recognises the various transition points of the students’ journey; so first year assessments may start to introduce elements of doing things differently, that are built on in the second year, etc.

In essence then, a strategic approach is vital for inclusive assessment practices as it provides an explicit framework for developing assessment literacy skills and for assignment feedback to be clearly directed towards feeding forward into future assessment activities. Taking a strategic approach also provides greater opportunities for teams to develop a coherent view about the purposes and values of assessment; and how those shared values are threaded through the course or programme. Assessment is not value-free as we are always conveying value messages to students about what we assess and how. A programme strategy allows us to really consider our values and what we are trying to achieve with our assessment practices and processes overall.

Assessment strategy also occurs at the module level. Again, at this level the strategy is not the mode of assessment but how support to achieve within the assessment is structured into the module. For example, how is assessment and feedback literacy designed into the module curriculum? What does the pre and post-assessment support look like? What is the rationale for the mode of assessment being used? How is assessment (formative and summative) being used within the module to support learning, not just quantify it?

You might find the attached presentation and some of the texts below useful to review:

Boud and Associates (2010) Assessment 2020: Seven propositions for assessment reform in higher education

Brunton et al (2016) Designing and developing a programme-focused assessment strategy: a case study

Scholtz (2016) (PDF) The assessment strategy: An elusive curriculum structure (researchgate.net)

If you are interested in this area, I would welcome you to get in touch: Kerry.Dobbins@warwick.ac.uk

For our learning circle we have also managed to secure funding to undertake a research project to capture both student and staff views of diverse assessments. Although the project is still ongoing, our student project officers, Molly Fowler and Pula Prakash, have managed to gather valuable data with an aim to feed into institutional considerations around assessment strategies.

Finally, if you want to find out more about our Learning Circle you can visit our webpage and you can read our previous blogs here:

Blog 1: Launch of the learning circle: https://blogs.warwick.ac.uk/wjett/entry/interested_in_diverse/

Blog 2: Creative projects and the ‘state of play’ in diverse assessments: https://blogs.warwick.ac.uk/wjett/entry/creative_projects_and/

Blog 3: Student experience of assessments: https://blogs.warwick.ac.uk/wjett/entry/a_student_perspective/

If you would like to join this learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk).

Abigail Ball : 14 Nov 2022 08:38 |  Tags: Assessment Isabel Fischer Leda Mirbaha Reflections Teaching |  Comments (0) |  Close comments |  Report a problem

A Student Perspective on Assessment Diversity and Strategy – Molly Fowler

Over the last two years I have completed 21 assessments of approximately 10 different types - I am a seasoned consumer of assessments. Truly diverse assessment should use a range of valid assessment types that stimulate learning and provide opportunities for learners with varied needs and capabilities.

Assessments where I have synthesised information mined without time pressure, have resulted in the deepest learning. I overcame an irrational fear of epigenetics by writing an essay on the topic. I have been wondering whether real time information acquisition and synthesis could form an equally valuable assessment modality. My recent work experience showed me the readiness of healthcare professionals at all levels to access and deploy ambulatory online information in real time on the ward and in the clinic. As far as I can see, assessment methodologies have yet to catch up with this new reality.

It is perhaps a truism to say that assessments should be a learning methodology. Assessment can be a stimulus to learn content but can also model useful skills and behaviours. I have found that assessments with low ecological validity are less enriching, and regrettably invite purely strategic learning. The best example I have is my French GCSE in which I achieved an A grade without being able to speak a word of the language. The nature of the assessment meant that I did not need to engage with the content or the objectives of the course. I was able to prevail by memorising and regurgitating chunks of text which I promptly forgot after the exam.

Of my recent exams, the methodology that strikes me as most at fault in this respect is the MCQ. This is a highly artificial assessment construct, relying crucially on recall of memorised information, in a specific and stressful environment. I accept that there may be other skills involved such as prioritising options, but this is barely possible without the memorising and recall.

Group assessments might seem to reflect real world behaviour, but I have not always found this to be the case. With no agreed hierarchy it is very difficult to distribute responsibility or manage group members who are either overly assertive or who fail to contribute. This is particularly tricky when all those involved are required to be equal contributors with equal status. Surely this is an unusual circumstance in any working environment. The contrived dynamics of group assessments can introduce an uncontrolled variable that impacts the results of even very good students.

I feel it is of critical importance that students have the opportunity to thoroughly rehearse assessment methodologies that they will face in the high-stakes final year. For example, having done two poster presentations, previously unfamiliar to me, I feel less daunted by the prospect of doing it again for my dissertation module. I now appreciate that this is a common way of disseminating information at scientific conferences so it may be an important skill for me to have in my career.

Personal lives are complex and we are all subject to sudden or dramatic change in our circumstances. Assessments by essays can more easily accommodate and mitigate unforeseen events through flexibility, but fixed assessments such as exams or in-person presentations are much more difficult to rearrange. An advantage of continuous assessment over end-of-year assessments is the feasibility of allowances for such happenings as fewer assessments would be affected. It is also true that students with a reported disability tend to do less well than students with no reported disability (Office for Students, 2022). This raises important questions around equity, and whether the available reasonable adjustments are good enough.

Properly diverse assessment could serve many purposes. Within the mix of assessments should be opportunities for different students to demonstrate their individual proficiency at their preferred method. Diverse assessment should be inclusive of students with reasonable adjustments, but they should also allow for unpredictable adverse events. Finally, where possible, assessments should engender learning and embody activities that will be useful in later life.

Office for Students (2022) Access and Participation Data Dashboard [online] Available from: https://www.officeforstudents.org.uk/data-and-analysis/access-and-participation-data-dashboard/ (Accessed 8 October 2022).

Molly is member of the Diverse Assessment Learning Circle. If you would like to join the LC please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk)

Link to blog one : Introduction to diverse assessment learning circle

Link to blog two: Creative Projects and the ‘state of play’ in diverse assessments

Abigail Ball : 31 Oct 2022 09:28 |  Tags: Assessment Molly Fowler Student Teaching |  Comments (0) |  Close comments |  Report a problem

Promoting Metacognition Through Student Self–Assessment – Emily Atkinson

Please see this blog post by CTE alumni Emily Atkinson on Promoting Metacognition Through Student Self-Assessment:

https://researchschool.org.uk/billesley/news/promoting-metacognition-through-student-self-assessment

There is also an EEF report called Moving forwards, making a difference: A planning guide for schools 2022–23 that Emily features in (page nine).

Abigail Ball : 27 Jun 2022 08:54 |  Tags: Assessment Emily Atkkinson Research |  Comments (0) |  Close comments |  Report a problem

Creative Projects and the ‘state of play’ in diverse assessments – Lewis Beer

As a member of the newly formed ‘Diverse Assessments’ Learning Circle in WIHEA (led by Leda Mirbahai and Isabel Fischer), how have you engaged with this topic during your time in Higher Education, and are there any lessons for the future?

Back in 2015, while teaching on a Shakespeare module in Warwick’s English Department, I worked with several students who chose not to write a standard academic essay, but instead to submit a Creative Project – for instance a painting, an animated film, a screenplay – accompanied by a reflective essay. Students were advised to centre their project on a clear research question, and were also told that the project would be assessed not for its artistic quality but for its achievement ‘as a piece of intellectual exploration’.

None of this was my idea, and frankly it made me nervous. But when I saw the students’ creative outputs and accompanying reflections, and read the feedback my colleagues gave them, I could see the unique benefits of this assessment method. One student, Amy Brandis, produced a 55-minute documentary called Lady Macbeth: From Stratford to Stage. She sourced A/V equipment, enlisted the help of students, staff, and people outside the university, and developed filming and editing skills to a high standard. She also reflected incisively on the intellectual exploration entailed by this process. Amy has since pursued a career as a writer, film-maker, and script consultant. For students like her, the Creative Project opened up types of intellectual discourse, and levels of insight, that fall outside the scope of a ‘standard’ essay. It also enabled students to explore topics in a way that fed into their broader development goals: not just academic development, but also personal and professional.

Five years later, compiling the Student Innovation at Warwick report during my time in Warwick Enterprise, I spoke to over 120 colleagues across 33 departments about the myriad ways in which Warwick students can ‘engage with innovation’. One way is through innovative pedagogy, and especially through non-standard modes of assessment that encourage students to be creative and take risks. Not only in the English Department, but also in Engineering, Psychology, History, and many others, academics model the risk-positive mindset they want to foster: it’s no mean feat to overcome the administrative and pedagogical challenges of creating new assessment schemes, getting them approved, and introducing them to curious but anxious students. In my current role, supporting Warwick’s involvement in the EUTOPIA Alliance, I see this spirit of pedagogic experimentation playing out on an even larger scale. It’s wonderful that there is so much interesting work being done in this area, but more than a little overwhelming.

The WIHEA-funded project we will carry out in the ‘Diverse Assessments’ Learning Circle, between now and the end of 2022, is an opportunity to explore these issues more deeply, and with a sense of focus that makes the exploration feasible. The Learning Circle members are now pooling their knowledge and experiences to understand the current ‘state of play’ regarding diverse assessments at Warwick. We look forward to consulting a wider range of staff and students to understand the benefits and challenges of diversifying assessment schemes, and to help shape Warwick’s future vision regarding diverse assessments.

There’s still room for more people to join our circle! For further information, please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk).

Link to blog one: Interested in diverse assessments?

Link to blog three: A Student Perspective on Assessment Diversity and Strategy

Abigail Ball : 11 Apr 2022 09:42 |  Tags: Amy Brandis Assessment Lewis Beer |  Comments (0) |  Close comments |  Report a problem

Interested in diverse assessments? – Isabel Fischer et al.

Interested in diverse assessments? Join our learning circle for an exciting grant-funded project on the future of assessments.

Authors: Isabel Fischer; Leda Mirbaha; Lewis Beer; Dawn Collins; Peter Fossey; Celine Martin; Natasha Nakariakov; Pula Prakash; Farrah Vogel-Javeri

We have recently created an interdisciplinary learning circle which aims to optimise the learning opportunities for Warwick students. We want to ensure that the teaching and learning opportunities are inclusive and cater for our diverse student community. Assessment and feedback are critical stages in the learning process. Using diverse assessments will ensure that students are not unfairly disadvantaged or advantaged by a specific form of assessment. Although it is worth noting that what may count as diverse assessment in one faculty may not necessary be seen as a diverse assessment approach in another. Therefore, using diverse assessments comes with its own challenges and barriers such as:

• Diversification without sufficient opportunity for students to practice and get familiar with the new and different forms of assessment, disadvantaging group of students that may not be as familiar with certain style of assessments
• The resource and time component needed for familiarisation then reduces the uptake and engagement by faculty with more innovative assessment approaches

Therefore, the aim of this learning circle is to capture both staff and student experience of diverse assessments and to involve students, staff, and other stakeholders in shaping the future of assessments. Furthermore, the learning circle aims to develop practical recommendations on overcoming some of the challenges associated with use of diverse assessments which will significantly benefit the community.

To achieve this we need your support!

If we have not yet convinced you to join our learning circle, then read on:

Vision: Our vision is to foster an inclusive environment where assessments are designed and developed in partnership with students, staff, and external stakeholders, to effectively promote learning, valuing students’ uniqueness and considering their future employment(s) and wellbeing, as well as the social and environmental responsibility and sustainability of the wider community.

Mission. Our mission is to:

• Gather existing data on practices around use of diverse and inclusive assessments, including Warwick staff and student experiences
• Develop shared understanding of principles and practice of diverse assessment
• Develop an evaluative framework for measuring the success of diversified assessment strategies at module, year, and course level
• Capture student and staff views on diverse assessments

To help us achieve our mission we have successfully applied and been awarded a WIHEA funding which will enable us to capture staff and student experiences of some of the diverse and innovative assessment approaches used in different disciplines to address some of the key questions around: 1) perception of diverse assessments from a staff and student perspective, 2) practical tips for successful application of the assessment method and marking, 3) communicating assessed skills and requirements, 4) overcoming challenges. To achieve this, we will conduct interviews with staff and students and will share resources, included but not limited to examples of assessments, marking and feedback rubrics and assessment briefs.

For further information on joining the learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk)

Stay tuned for further updates and blogs on our initiative, such as: Creative Projects and the ‘state of play’ in diverse assessments – Lewis Beer

Abigail Ball : 14 Mar 2022 09:16 |  Tags: Assessment Celine Martin Dawn Collins Farrah Vogel-Javeri Isabel Fischer Leda Mirbaha Lewis Beer Natasha Nakariakov Peter Fossey Pula Prakash |  Comments (1) |  Close comments |  Report a problem

AI Ethics for Assessments in Higher Education

AI Ethics for Assessments in Higher Education: A project example of an interdisciplinary social sciences undergraduate summer research scheme

By Isabel Fischer (Warwick Business School) and Thomas Martin (Economics)

Warwick’s Social Sciences offer students and faculty from economics, education and Warwick Business School (WBS) the opportunity to take part in an interdisciplinary summer research project to improve awareness and understanding of collaborative research work on a topic of the students’ choice. One of this summer’s group focused on AI ethics for assessments where students applied the EU Ethics Guidelines for Trustworthy AI to Higher Education Assessment. Students concluded that despite the limitations of AI, AI has the potential to make assessment processes in higher education more effective and fairer. Students suggested that AI should be embraced, but only with human oversight and agency, and with clear stakeholder communication in place.

Linden Davison, a student from the Department of Economics, commented: “Getting involved in the UG research scheme broadened my awareness beyond my single subject discipline - working in a field I wasn't aware existed when we started! It was a pleasure to work alongside students from different departments and be guided by such engaging, motivated staff alike.” Toby Pia, also from Economics, added: “Throughout my URSS experience I improved my ability to explain complex ideas in a more understandable way. I also got better in conveying a balanced argument as previously I tended to get more entrenched into one side of a debate rather than looking at it from both sides.”

At the end of the project each student produced a research poster, depicting one of the seven overarching themes of the EU Ethics Guidelines for Trustworthy AI. Below an example by Shubhangi Bhatt, a student from the Department of Education Studies, on Transparency.

poster_example.pdf

AI poster (text only version)

Finally, here a link to five articles that explain how AI could be made ethical and trustworthy:

https://www.wbs.ac.uk/news/five-reads-you-need-to-make-ai-ethical-and-trustworthy/

And you also might want to read here how AI can (positively) influence education generally:

https://www.wbs.ac.uk/news/five-areas-ai-can-have-the-greatest-impact/

Abigail Ball : 25 Oct 2021 13:30 |  Tags: AI Assessment Isabel Fischer Research Teaching Thomas Martin |  Comments (0) |  Close comments |  Report a problem