All 3 entries tagged Kerry Dobbins

Past reflections, future outlook

Past reflections, future outlook: Update from the Warwick Learning Circle on diverse and inclusive assessment practices

by Kerry Dobbins, Isabel Fischer, Sam Grierson and Leda Mirbahai

Warwick’s International Higher Education Academy (WIHEA) has many features - one of them is that members only remain active members for three years before becoming alumni. The imminent handover of the co-leadership of the open learning circle on diverse and inclusive assessment practices made us, outgoing and new co-leaders, reflect on past initiatives and future directions.

Let’s start with an exciting future - what is new?

As we continue to move forwards with this work, we will become the ‘Inclusive Assessment Learning Circle’. This name change recognises our intention to embed the work on diverse assessments into a broader outlook which sees assessment in HE as a vehicle to promote equity and social justice. The wellbeing of students and staff will feature strongly in our ongoing discussions.

In line with our focus, the learning circle will be inclusive of:

1. Assessment strategies and methods that are diverse, authentic, applied and decolonised - this broad understanding and acknowledgement of relevant issues in teaching, learning and the designing of assessments reflects the current landscape in HE and indicates the interest of our members. We hope to support the university reimagining the assessment design narrative, taking a view from programme level through to single assessment and placing assessment at the heart of curriculum design.
2. Our members - we want members to be and feel actively engaged and involved in the learning circle.
3. Students’ experiences – the work of the learning circle will be firmly focused on working with students so that collectively we can work to understand and support students’ needs and their personal and professional ambitions through assessment.

As for ‘past reflections’: What are the existing features of the learning circle worth keeping? There are in particular four features that worked really well in the past:

1. Since its inception membership has grown steadily for three reasons: The membership of this particular LC is open to WIHEA as well as non-WIHEA members from across Warwick and other institutions, also internationally. The open membership has enriched the discussions and enabled establishment of networks nationally and internationally. Secondly, creating sub-groups not just raised interest, it also offered leadership opportunities to more members. The most popular sub-group was AI in education which in turn was split into six further strands / interest groups: Artificial Intelligence in Education (warwick.ac.uk). Lastly, the topic of the learning circles (assessment in its broadest term) has significant implications not only for educators but also for our learners.
2. Many of our invited keynote speakers at the start of the bi-monthly meetings and at our mini-conference on assessments, captured the essence of their talks in blogs. Most of the blogs were published within WJETT (see the end of this blog for some examples, others were in other outlets, such as SEDA and SCiLAB. Even if not captured in blogs, did the keynote speeches result in interesting discussions and network opportunities.
3. Extensive student corporations, with students supported by two WIHEA grants. One of the student participants even drafted an academic article based on her learning about assessments during her membership.
4. Hosting workshops enabled dissemination of our findings and new resources, including the outputs of our funded projects. It also provided a platform to encourage co-learning and sharing amongst participants and facilitators (including student facilitators). Overall we hosted three workshops with the details of the workshops available from our webpage.

Finally, this is blog 15 in our diverse assessment series, some of the more recent blogs can be found here (with further links to previous blogs shown within some of the blogs below):

• Using AI for Formative Feedback: https://blogs.warwick.ac.uk/wjett/entry/using_ai_for/
• Assessments: Capturing Lived Experience and Shaping the Future
• Blog 7: Democratising the feedback process (Linda Enow): https://blogs.warwick.ac.uk/wjett/entry/democratising_the_feedback/

Abigail Ball : 16 Oct 2023 09:00 |  Tags: Assessment Isabel Fischer Kerry Dobbins Leda Mirbaha Reflections Sam Grierson Teaching |  Comments (0) |  Close comments |  Report a problem

Scholarship of Teaching and Learning (SoTL): Considering and measuring impact

Scholarship of Teaching and Learning (SoTL): Considering and measuring impact

By Dr Kerry Dobbins (Assistant Professor, Academic Development Centre) and Dr Isabel Fischer (Reader in Information Systems at Warwick Business School)

Questions about how to evaluate and measure impact of Scholarship of Teaching and Learning activities (SoTL) have always generated much lively discussion and debate within the SoTL community. For example, a recent LTHE Tweetchat focused on the topic of ‘Breaking boundaries: career progression and education-focused roles’. At the heart of the discussion was the nature and purpose of SoTL in relation to career advancement. Two specific questions discussed were ‘what forms and outputs can SoTL take?’ and ‘how should quality and impact be measured?’ In recent times, these questions have become more important than ever for all colleagues on education or teaching-focused contracts due to their connection to institutional promotions criteria. Routes to promotion for these colleagues will often involve criteria including:

• Making a significant contribution to pedagogy.
• Gaining recognition for quality and impact of scholarship.
• Demonstrating educational impact and influence within and beyond the University.
• Producing scholarly outcomes that advance learning, teaching and assessment.

Outputs and demonstrable impact of SoTL underpin these criteria and highlight why the recent Tweetchat was such a topical discussion. Considering the various forms or outputs that SoTL can take connects to how impact from those outputs may be measured or viewed to have made a significant contribution. The Twitter community identified a wide range of forms that SoTL outputs can take. These included:

• Conference/webinar presentations
• Publications
• Podcasts, blogs, vlogs
• Infographics
• Content for repositories, e.g. the National Teaching Repository
• Books
• Invited talks
• Staff/student development training
• Well-designed courses/modules

There was much leaning towards open access forms and critical scrutiny occurring through the teaching community, rather than tying outputs largely to journal publications and scrutiny through the standard academic peer review process. This leaning is still in line with the SoTL principles of sharing findings for communal use and review, but importantly the Twitter community placed considerable value on practitioner-focused scrutiny.

There was also recognition that different outputs serve different purposes and that any output can have an impact. This takes us into the realms of questioning what ‘impact’ means in SoTL and how it might be measured. Again, the Twitter community identified various ways that ‘impact’ of SoTL could be considered and/or measured:

• Policy changes – national, institutional, departmental, etc.
• Creation of staff development provision.
• Explicit recognition and reward of teaching through, for example, promotion and recruitment criteria.
• Improved module/course evaluations and student grades.
• Enhanced student engagement.
• Paper citations, usage metrics.
• Engagement in networks or communities of practice.
• Improved resourcing for teaching/development.
• Changes in colleagues’ practices.
• Engagement in collaborative activities.

It is interesting to see from the responses the different levels at which impact may play out (e.g. individual, institutional, etc). Of course, one of the ultimate aims of SoTL is to understand how students learn effectively so that learning experiences can be enhanced. However, SoTL is also recognised as an essential component of academic professional development and a mechanism through which HE teaching as a profession is advanced. As the responses above show, this wide remit of SoTL means that impacts from its activities can take many forms.

It is also apparent that some of the impact examples offered above may take longer to realise than others and be harder to measure in a quantifiable sense. A key question emerging in the Twitter discussion was whether SoTL impact needs to be measurable. This again takes us back to the question of what ‘impact’ means in SoTL and more specifically, what type of impact is being sought.

Different stakeholders (e.g., individuals, community groups, institutions) may of course be seeking different types of impact from SoTL activities but in reality, the needs of the different stakeholders are not so easily separated. There continues to be a challenging balancing act being played out in SoTL between it being an activity to develop individual and community practices, and it becoming a significant feature of aspects like promotions criteria. Whilst inclusion in promotions criteria demonstrates institutional recognition and reward of SoTL, it adds a layer of trying to measure or capture a narrative of identifiable and evidence-based impact that resonates with a panel who may or may not be familiar with the various lenses and dynamics of SoTL.

What all of this leads to is the need to keep engaging in institutional conversations about SoTL so that purposes, outputs and impacts of this activity are not viewed in too reductive terms and the various lenses and levels identified by the Twitter community are not overlooked. This is the vast flavour of SoTL and if we are too reductive, we may only taste elements of the impacts that it could have.

Two further questions should also be asked in the context of these institutional discussions:

1. How to align expectations about output and impact with time allocated for SoTL, e.g. what should be the output for somebody who receives a workload allocation of, say, two to three hours a week versus somebody who receives one to two full days per week?
2. How might expectations about output and impact be aligned on an inter-disciplinary instructional-level to encourage transferability?

It is important to ensure that discussions to define impact more clearly (and broadly) at an institutional level do not deter us from seeing the sharing of SoTL work and findings as an activity worthy in and of itself. At Warwick, we aim to create and encourage opportunities for the sharing of SoTL activities as part of our internal communities of practice. This way inspirational SoTL findings can be spread and impact upon colleagues’ practices, independent of views about measurability.

Abigail Ball : 10 Jan 2022 09:43 |  Tags: Isabel Fischer Kerry Dobbins Learning Scholarship Teaching |  Comments (0) |  Close comments |  Report a problem

Scholarship of teaching and learning (SoTL)

Scholarship of teaching and learning (SoTL): How to get started with a journey in mind

By Dr Isabel Fischer (Reader in Information Systems at Warwick Business School) and Dr Kerry Dobbins (Assistant Professor, Academic Development Centre)

Scholarship of teaching and learning (SoTL) represents the opportunity for critical reflection both on generic pedagogic practices as well as on subject specialisms. It leads to knowledge‐in‐practice, knowledge‐of‐practice and productive disciplinary engagement.1 Based on a panel discussion at Warwick’s Education forum here an overview of how we got started, as well as lessons that we have learned along the way. We hope that the following ten suggestions might help colleagues either to get started, or to reflect on and engage with discussions on their SoTL journey. For colleagues considering applying for promotion, we have loosely aligned our suggestions to our interpretation of the current promotion research bands.

Approx. Band 2:

1.Decide on your interest and intellectual position, this might be pedagogic and/or disciplinary. For Isabel, this was (and still is) the intersection of Ethics & ESG (Environmental, Social, and Governance), Technologies, and Education. For Kerry, it was initially the interplay between particular HE policies and individual teaching practices and has now evolved to communities of practice, particularly in relation to SoTL.

2. Start reading articles in the area of your interest, taking note of what was researched and how it was researched, i.e. methodology. Google Scholar can be a good place to start finding relevant articles and you might also try an online education database like British Education Index (BEI). One learning point here is that it is actually good to align yourself with relatively recent and frequently cited papers rather than trying to develop something completely new. Another learning point is that you might want to consider a systematic literature review to perhaps be able to convert this later into a ‘systematic literature review and research agenda paper’.

3. Develop your research proposal including ethics approval. Both of us have conducted several research projects on our own modules (with students explicitly agreeing to these as per ethics requirements) and this is our favourite part of SoTL. For all of our projects, we have learned so much from students. Action research for your own teaching is also quite popular for SoTL and if the activity is purely for professional development purposes, it would not require ethical approval (though it would still have to be conducted in an ethical manner of course). For those new to SoTL and education research, you might start first with a question that you have about your teaching in relation to your area of interest. From there, you could identify various sources of evidence that already exist that will allow you to gain insight into your question, e.g., published studies, module evaluation data, student feedback, performance data, etc. You might then begin to make changes to your teaching practices using insights gained from this evidence-informed approach whilst building confidence to develop a more formal research proposal. It is important to note that it is possible to engage in SoTL without conducting primary research but through engagement with literature and/or communities of practice. Up to hear, this represents roughly band 2 of the research-related academic promotion criteria (there is no band 1).

Approx. Band 3:

4. Having gained these insights, it’s time to talk to colleagues at conferences and/or events. There are always a range of internal and external conferences as well as sharing practice events. These are good ways to connect to a wider network of colleagues and build confidence in presenting your education-focused work to broader and interdisciplinary audiences. My (Isabel’s) first conference presentation was at Advance HE. I have since expanded to international conferences and also organised my own specific events. While applying for conferences and organising events seems time consuming, these are extremely useful both for disseminating own findings and for getting feedback on these, as well as for hearing about and getting inspired by other research. Presenting at conferences and being a reviewer for journals translate roughly to band 3 of the promotion criteria. And don’t forget, presenting at conferences might also help with demonstrating impact in FHEA or SFHEA applications.

5. Similar to conferences, a good way to engage with other academics and to understand more about research as well as teaching, is to become a reviewer for journals and/or an external examiner. One way to do this is to apply for these roles as they are often advertised. Another is to talk to colleagues about their external engagements as they might be looking for reviewers for the journals that they are editing. Do not hesitate to contact external editors too as most are always eager for reviewers and usually very happy to be contacted directly. This is particularly relevant if there is a journal which focuses on your area of interest and many articles on topics that you find interesting are published in that particular journal. Once you have been a reviewer for some time as a next step you could consider guest editing or assistant editing of journals.

Approx. Band 4+:

6. The more you engage with academic literature you start to understand some of the complexities of publication, especially the length it takes from writing a first draft to publication. For high impact journal the review and resubmit process takes several years. You might therefore want to consider developing a portfolio of output and consider writing a blog, producing some videos and/or writing article(s) for journals with lower impact factors. Many journals publish conceptual or case-study papers as well as research articles. All output, independent of the journal’s impact factor, might be helpful for future FHEA or SFHEA applications as by disseminating to colleagues and/or the public - rather than ‘just’ your own students - these demonstrate an increased sphere of influence. For promotion criteria, depending on the journal’s impact and ranking this might be classified as band 4 and if sustained even higher.

7. You might consider inviting co-author(s) for your research projects. Of course, you can also consider doing so from the start of your research process or alternatively colleagues might ask you if you want to join some of their research projects. Both of us really enjoy working with co-authors, our second favourite part of SoTL after the primary research, however, the downside is that your co-author(s) might work at a different pace to yours or have different aspirations of journal impact factors.

8. Instead of peers as co-authors you might want to consider doing a joint student-project where students are the co-authors or co-creators. This is also very enjoyable, however, when initially starting with SoTL, you might feel that you want to exchange ideas with colleagues who are more or as experienced rather than guiding students.

Outlook:

9. Possible barriers? You might think there is not enough time for SoTL (and you should check your departments policy on workload allocation for SoTL), or you might encounter ‘representativeness heuristics’ where you are compared to an existing prototype of colleagues who have already published many articles, however, this might only be you thinking it (no need to feel like an amateur or imposter, or to hide your educational research!)

10. Finally, and possibly most importantly, SoTL allows you to be slightly less theory-driven and more action oriented than traditional research. Hence, SoTL provides you with the opportunity to make change happen. Enjoy the journey!

References

Gregory J. Kelly, The social bases of disciplinary knowledge and practice in productive disciplinary engagement, International Journal of Educational Research, Volume 64, 2014, Pages 211-214, ISSN 0883-0355

https://warwick.ac.uk/services/humanresources/internal/academicprocesses/academicpromotion/academic_promotions_criteria.pdf

Abigail Ball : 09 Nov 2021 16:40 |  Tags: Isabel Fischer Kerry Dobbins Learning Scholarship Teaching |  Comments (0) |  Close comments |  Report a problem