All 3 entries tagged Learning

Scholarship of Teaching and Learning (SoTL): Considering and measuring impact

Scholarship of Teaching and Learning (SoTL): Considering and measuring impact

By Dr Kerry Dobbins (Assistant Professor, Academic Development Centre) and Dr Isabel Fischer (Reader in Information Systems at Warwick Business School)

Questions about how to evaluate and measure impact of Scholarship of Teaching and Learning activities (SoTL) have always generated much lively discussion and debate within the SoTL community. For example, a recent LTHE Tweetchat focused on the topic of ‘Breaking boundaries: career progression and education-focused roles’. At the heart of the discussion was the nature and purpose of SoTL in relation to career advancement. Two specific questions discussed were ‘what forms and outputs can SoTL take?’ and ‘how should quality and impact be measured?’ In recent times, these questions have become more important than ever for all colleagues on education or teaching-focused contracts due to their connection to institutional promotions criteria. Routes to promotion for these colleagues will often involve criteria including:

• Making a significant contribution to pedagogy.
• Gaining recognition for quality and impact of scholarship.
• Demonstrating educational impact and influence within and beyond the University.
• Producing scholarly outcomes that advance learning, teaching and assessment.

Outputs and demonstrable impact of SoTL underpin these criteria and highlight why the recent Tweetchat was such a topical discussion. Considering the various forms or outputs that SoTL can take connects to how impact from those outputs may be measured or viewed to have made a significant contribution. The Twitter community identified a wide range of forms that SoTL outputs can take. These included:

• Conference/webinar presentations
• Publications
• Podcasts, blogs, vlogs
• Infographics
• Content for repositories, e.g. the National Teaching Repository
• Books
• Invited talks
• Staff/student development training
• Well-designed courses/modules

There was much leaning towards open access forms and critical scrutiny occurring through the teaching community, rather than tying outputs largely to journal publications and scrutiny through the standard academic peer review process. This leaning is still in line with the SoTL principles of sharing findings for communal use and review, but importantly the Twitter community placed considerable value on practitioner-focused scrutiny.

There was also recognition that different outputs serve different purposes and that any output can have an impact. This takes us into the realms of questioning what ‘impact’ means in SoTL and how it might be measured. Again, the Twitter community identified various ways that ‘impact’ of SoTL could be considered and/or measured:

• Policy changes – national, institutional, departmental, etc.
• Creation of staff development provision.
• Explicit recognition and reward of teaching through, for example, promotion and recruitment criteria.
• Improved module/course evaluations and student grades.
• Enhanced student engagement.
• Paper citations, usage metrics.
• Engagement in networks or communities of practice.
• Improved resourcing for teaching/development.
• Changes in colleagues’ practices.
• Engagement in collaborative activities.

It is interesting to see from the responses the different levels at which impact may play out (e.g. individual, institutional, etc). Of course, one of the ultimate aims of SoTL is to understand how students learn effectively so that learning experiences can be enhanced. However, SoTL is also recognised as an essential component of academic professional development and a mechanism through which HE teaching as a profession is advanced. As the responses above show, this wide remit of SoTL means that impacts from its activities can take many forms.

It is also apparent that some of the impact examples offered above may take longer to realise than others and be harder to measure in a quantifiable sense. A key question emerging in the Twitter discussion was whether SoTL impact needs to be measurable. This again takes us back to the question of what ‘impact’ means in SoTL and more specifically, what type of impact is being sought.

Different stakeholders (e.g., individuals, community groups, institutions) may of course be seeking different types of impact from SoTL activities but in reality, the needs of the different stakeholders are not so easily separated. There continues to be a challenging balancing act being played out in SoTL between it being an activity to develop individual and community practices, and it becoming a significant feature of aspects like promotions criteria. Whilst inclusion in promotions criteria demonstrates institutional recognition and reward of SoTL, it adds a layer of trying to measure or capture a narrative of identifiable and evidence-based impact that resonates with a panel who may or may not be familiar with the various lenses and dynamics of SoTL.

What all of this leads to is the need to keep engaging in institutional conversations about SoTL so that purposes, outputs and impacts of this activity are not viewed in too reductive terms and the various lenses and levels identified by the Twitter community are not overlooked. This is the vast flavour of SoTL and if we are too reductive, we may only taste elements of the impacts that it could have.

Two further questions should also be asked in the context of these institutional discussions:

1. How to align expectations about output and impact with time allocated for SoTL, e.g. what should be the output for somebody who receives a workload allocation of, say, two to three hours a week versus somebody who receives one to two full days per week?
2. How might expectations about output and impact be aligned on an inter-disciplinary instructional-level to encourage transferability?

It is important to ensure that discussions to define impact more clearly (and broadly) at an institutional level do not deter us from seeing the sharing of SoTL work and findings as an activity worthy in and of itself. At Warwick, we aim to create and encourage opportunities for the sharing of SoTL activities as part of our internal communities of practice. This way inspirational SoTL findings can be spread and impact upon colleagues’ practices, independent of views about measurability.

Abigail Ball : 10 Jan 2022 09:43 |  Tags: Isabel Fischer Kerry Dobbins Learning Scholarship Teaching |  Comments (0) |  Close comments |  Report a problem

Scholarship of teaching and learning (SoTL)

Scholarship of teaching and learning (SoTL): How to get started with a journey in mind

By Dr Isabel Fischer (Reader in Information Systems at Warwick Business School) and Dr Kerry Dobbins (Assistant Professor, Academic Development Centre)

Scholarship of teaching and learning (SoTL) represents the opportunity for critical reflection both on generic pedagogic practices as well as on subject specialisms. It leads to knowledge‐in‐practice, knowledge‐of‐practice and productive disciplinary engagement.1 Based on a panel discussion at Warwick’s Education forum here an overview of how we got started, as well as lessons that we have learned along the way. We hope that the following ten suggestions might help colleagues either to get started, or to reflect on and engage with discussions on their SoTL journey. For colleagues considering applying for promotion, we have loosely aligned our suggestions to our interpretation of the current promotion research bands.

Approx. Band 2:

1.Decide on your interest and intellectual position, this might be pedagogic and/or disciplinary. For Isabel, this was (and still is) the intersection of Ethics & ESG (Environmental, Social, and Governance), Technologies, and Education. For Kerry, it was initially the interplay between particular HE policies and individual teaching practices and has now evolved to communities of practice, particularly in relation to SoTL.

2. Start reading articles in the area of your interest, taking note of what was researched and how it was researched, i.e. methodology. Google Scholar can be a good place to start finding relevant articles and you might also try an online education database like British Education Index (BEI). One learning point here is that it is actually good to align yourself with relatively recent and frequently cited papers rather than trying to develop something completely new. Another learning point is that you might want to consider a systematic literature review to perhaps be able to convert this later into a ‘systematic literature review and research agenda paper’.

3. Develop your research proposal including ethics approval. Both of us have conducted several research projects on our own modules (with students explicitly agreeing to these as per ethics requirements) and this is our favourite part of SoTL. For all of our projects, we have learned so much from students. Action research for your own teaching is also quite popular for SoTL and if the activity is purely for professional development purposes, it would not require ethical approval (though it would still have to be conducted in an ethical manner of course). For those new to SoTL and education research, you might start first with a question that you have about your teaching in relation to your area of interest. From there, you could identify various sources of evidence that already exist that will allow you to gain insight into your question, e.g., published studies, module evaluation data, student feedback, performance data, etc. You might then begin to make changes to your teaching practices using insights gained from this evidence-informed approach whilst building confidence to develop a more formal research proposal. It is important to note that it is possible to engage in SoTL without conducting primary research but through engagement with literature and/or communities of practice. Up to hear, this represents roughly band 2 of the research-related academic promotion criteria (there is no band 1).

Approx. Band 3:

4. Having gained these insights, it’s time to talk to colleagues at conferences and/or events. There are always a range of internal and external conferences as well as sharing practice events. These are good ways to connect to a wider network of colleagues and build confidence in presenting your education-focused work to broader and interdisciplinary audiences. My (Isabel’s) first conference presentation was at Advance HE. I have since expanded to international conferences and also organised my own specific events. While applying for conferences and organising events seems time consuming, these are extremely useful both for disseminating own findings and for getting feedback on these, as well as for hearing about and getting inspired by other research. Presenting at conferences and being a reviewer for journals translate roughly to band 3 of the promotion criteria. And don’t forget, presenting at conferences might also help with demonstrating impact in FHEA or SFHEA applications.

5. Similar to conferences, a good way to engage with other academics and to understand more about research as well as teaching, is to become a reviewer for journals and/or an external examiner. One way to do this is to apply for these roles as they are often advertised. Another is to talk to colleagues about their external engagements as they might be looking for reviewers for the journals that they are editing. Do not hesitate to contact external editors too as most are always eager for reviewers and usually very happy to be contacted directly. This is particularly relevant if there is a journal which focuses on your area of interest and many articles on topics that you find interesting are published in that particular journal. Once you have been a reviewer for some time as a next step you could consider guest editing or assistant editing of journals.

Approx. Band 4+:

6. The more you engage with academic literature you start to understand some of the complexities of publication, especially the length it takes from writing a first draft to publication. For high impact journal the review and resubmit process takes several years. You might therefore want to consider developing a portfolio of output and consider writing a blog, producing some videos and/or writing article(s) for journals with lower impact factors. Many journals publish conceptual or case-study papers as well as research articles. All output, independent of the journal’s impact factor, might be helpful for future FHEA or SFHEA applications as by disseminating to colleagues and/or the public - rather than ‘just’ your own students - these demonstrate an increased sphere of influence. For promotion criteria, depending on the journal’s impact and ranking this might be classified as band 4 and if sustained even higher.

7. You might consider inviting co-author(s) for your research projects. Of course, you can also consider doing so from the start of your research process or alternatively colleagues might ask you if you want to join some of their research projects. Both of us really enjoy working with co-authors, our second favourite part of SoTL after the primary research, however, the downside is that your co-author(s) might work at a different pace to yours or have different aspirations of journal impact factors.

8. Instead of peers as co-authors you might want to consider doing a joint student-project where students are the co-authors or co-creators. This is also very enjoyable, however, when initially starting with SoTL, you might feel that you want to exchange ideas with colleagues who are more or as experienced rather than guiding students.

Outlook:

9. Possible barriers? You might think there is not enough time for SoTL (and you should check your departments policy on workload allocation for SoTL), or you might encounter ‘representativeness heuristics’ where you are compared to an existing prototype of colleagues who have already published many articles, however, this might only be you thinking it (no need to feel like an amateur or imposter, or to hide your educational research!)

10. Finally, and possibly most importantly, SoTL allows you to be slightly less theory-driven and more action oriented than traditional research. Hence, SoTL provides you with the opportunity to make change happen. Enjoy the journey!

References

Gregory J. Kelly, The social bases of disciplinary knowledge and practice in productive disciplinary engagement, International Journal of Educational Research, Volume 64, 2014, Pages 211-214, ISSN 0883-0355

https://warwick.ac.uk/services/humanresources/internal/academicprocesses/academicpromotion/academic_promotions_criteria.pdf

Abigail Ball : 09 Nov 2021 16:40 |  Tags: Isabel Fischer Kerry Dobbins Learning Scholarship Teaching |  Comments (0) |  Close comments |  Report a problem

Enhancing student learning through the use of ICT – Alexandra

During my complementary placement, I planned and delivered the first unit of the new Design and Technology GCSE which focuses on New and Emerging Technologies. This provided me with an opportunity to develop a new Scheme of Learning (SOL) with an entirely paperless agenda and digital delivery.

The teacher resources were produced in PowerPoint in which a variety of additional tools were imbedded such as YouTube videos and links to external applications Kahoot, Padlet, Google Classroom Slides and Surveyhero.

A flipped classroom was created by releasing resources to students in advance of each lesson from which they developed their own digital workbook in PowerPoint which was submitted periodically by email or via the cloud for marking and feedback. From these, I was able to develop a digital archive of student work and monitor progress. Individual feedback was given by email in response to each submission and group feedback, to highlight common areas of success and misconception, was given at the start of the following lesson. The SAMR model (Puentedura, 2014) was used to structure the transformation and enhancement of this SOL through ICT.

Kahoot quiz results provided insight into areas of weakness that needed addressing and this informed my short-term planning with the start of the following lesson allocated to this due to 45-minute lessons.

Padlet and Google Classroom Slides enabled collaborative working however the novelty of these provided an opportunity for misbehaving. The software has the option for teachers to verify all comments before they go live but this can disrupt the flow of student contributions. Kirkman (2017) refers to dialogic practice as ‘that in which students are active, engaged and empowered participants in a conversation from which learning emerges’ which was evident through the immediate feedback from the class enabled by not using this facility. This outweighed the behaviour issues but with other classes I would consider putting this control in place to monitor the content and pace of the lesson and to encourage students to consider their contributions more carefully.

Student engagement was high throughout this unit and even with the issues identified, the use of ICT assisted behaviour management and improved engagement with students who had been previously identified as reluctant contributors to lessons. This supports my findings from my base school where I have previously used these applications to support SOLs and found them to be useful tools in behaviour management and engaging disruptive students. However, student demographic and data must be considered to ensure that all students have access to ICT and to be aware of any Pupil Premium students who may require additional support in this area.

The feedback from the wider department was very positive however, as I reflect on the success of this approach I shall alter elements of the scheme and allow more time for student reflection. At my base school I developed the use of Surveyhero to encourage students to reflect on their skills, progress and outcomes. I shall also incorporate more collaborative, student led tasks and develop my pace of delivery and the flow and fluency of the plan in line with the use of technology. I would also like to develop the use of ICT and digital resources to further differentiate the SOL to support a wider range of learners.

References

Kirkman, P. (2017). Digital technologies in the classroom. [ebook] Cambridge Assessment International Education. Available at: http://www.cambridgeinternational.org/images/271191-digital-technologies-in-the-classroom.pdf [Accessed 12 April 2018].

Puentedura, R. (2014). Ruben R. Puentedura's Weblog. [online] Hippasus.com. Available at: http://www.hippasus.com/rrpweblog/ [Accessed 2 April 2018].

Abigail Ball : 06 Aug 2018 14:23 |  Tags: Ict Learning Student |  Comments (0) |  Close comments |  Report a problem