All 17 entries tagged Isabel Fischer

Jagged frontiers of AI in academic practice

Dominik Lukeš (University of Oxford) and Isabel Fischer (Warwick Business School) explored with a group of Developing Consulting Expertise students the integration of AI into academic practice.

We looked at the concept of the Jagged Frontier of AI capabilities that illustrates the mismatch between what people expect AI would be good at and what AI is actually good at. We asked the question of how to explore the jagged frontier, what is the best way to do so and what to pay attention to.

Dominik introduced his framework for evaluating AI tasks along five dimensions. For each AI task, we asked five questions based on exploring various AI tools.

1. AI capability mismatch: What is the level of mismatch against known AI capabilities and limitations?
2. Hallucination / unpredictability problem: How manageable is hallucination here? Where is it likely to show up?
3. Prompt sensitivity: Is the output sensitive to how the prompt is formulated?
4. Context window dependence: Can the AI tool see everything you present it?
5. Variability across tools and models: Does the tool and/or the model the tool uses matter for the task?

The students worked in groups and came up with the following comparison of various common academic tasks. The table represents their work.

While the table is not the final word on any of these tasks, it was the subject of a much wider conversation. Perceptions along the five dimensions are very much dependent on how the specifics of the task are conceived. A different group, at a different time, may come up with a table that is quite different. It is the conversation and joint exploration that matters. You can read more about the workshop on AI integration and explore some of the presentation materials here.

Abigail Ball : 03 Jun 2024 09:00 |  Tags: AI Dominik Lukes Isabel Fischer Teaching |  Comments (0) |  Close comments |  Report a problem

Want to create a Digital Community? Some insights from our WBS module

Building digital communities is seen as a way to create connectedness and a sense of belonging and at the same time is seen by digital marketers as a tool to create brand loyalty and drive business. This year - for the first time - we have created a module at WBS that provides students with insights on how to create digital communities. Based on three weeks of teaching, here are some key things to keep in mind when creating a digital community (for any educator reading this blog who teaches year 1 undergraduate students, what about creating a digital community for ‘commuting students?’).

So, what is important to consider when creating digital communities?

1. Define your purpose and target audience (we used Design Thinking to help us with this step)

One example of a digital community that defines its purpose and target audience well is Barry’s Bootcamp. The purpose is defined as providing high-intensity, boutique physical and digital fitness experiences that cater to individuals seeking challenging workouts and a dynamic community environment. Its target audience primarily includes fitness enthusiasts who value being part of a motivated, like-minded community that holds each other accountable.

Equinox Fitness sells a sense of belonging to achievement-oriented metropolitan influencers who want to define their life by both vitality and success. They strive to promote an ethos of achieving equilibrium between physical discipline, career ambitions, creative pursuits, and relationships by offering programming spanning mediums like dance, art, podcasts and more alongside Founder training.

2. Choose the right platform (for our own digital community we created we used Discord)

It is important to identify the right platform to reach your audience while creating digital communities. The current social media landscape hosts a variety of channels including Facebook, X, Instagram, Blog posts, Newsletters and TikTok among others. Different platforms appeal to different people based on age, interest, location etc. The focus should begin with first identifying your audience, then deciding on the most appropriate platform and communication channels to build a community.

We thought that the Kenyan company Healthy U chose their platform (Facebook) well. Questions that informed their decision-making for creating a digital community included: How can we best reach a community of individuals in Kenya and Uganda that is keen on living a happier, healthier life by consuming alternative health and organic food products? What product or service are we providing? Who is our audience? What platform do they commonly use to interact socially? What is the best way to engage with them and is this achievable on that platform?

3. Foster a welcoming and inclusive environment

We suggest creating a sense of belonging from the start: When you create a community, it's essential to provide a sense of belonging to ensure that participants feel welcomed. For instance, consider Asian Beauty's approach: when you join their community, they ask several questions about your preferences, such as 'What do you believe this community is about?', 'What is your skin type?' and also instead of ‘what is your name’: 'How would you prefer to be called?’

4. Create engaging content and activities

Another important thing to consider when creating digital communities is creating interesting content and fun activities to get each member engaged and interested in participating. To illustrate, our chosen industry was the racecourse industry. Here, the community owners offer members first-hand access to events and shows through displaying them on the membership apps and newsletters. Members are also given intel on renovations ongoing at the racecourse to ensure that they get first-hand access to the exclusive renovated spots by seeing them through an interactive map.

5. Build relationships and connections

Building relationships, connections, and networks among the wealthy seems to be one trend that we saw across many communities, and it is one of the primary aims of racecourses, such as the Warwick Racecourse. It can certainly be linked to capitalism and reflects how economic and social capital often intertwine. The influence of wealth in this community, and in many others, influences the access to opportunities.

6. Measure success and iterate

To know the performance of one of the digital communities that we reviewed, we measured the engagement rate of Asian Beauty on discord and found that the community has a relatively high engagement rate with 14,3% of members online at the exact time when we checked out the community (1 Feb 2024 12.30pm).

7. Promote your community outside your community

While Asian Beauty has very engaged members, we felt they could do better in promoting their community. Some suggestions:

Market your community through various channels to increase membership, e.g., for google search unlocking first page visibility.

• Advertising: A/B test with Google ads optimization
• Organic approach: write more blogs / articles that contain popular keywords that are trending. e.g., for beauty communities, the keyword 'skin barrier' on Google Trends increased by 1.5x in 2023; tap into these conversations!

Level Up Online Presence: Dive into the Digital Dialogue

• Create a social media that could attract more members. Conduct survey and apply design thinking (building personas) to find the suitable social media for Instagram/TikTok/Facebook
• Be where your audience is, e.g., for beauty communities, #KoreanSkincare on TikTok has 9.9m engagement, joining this conversation might increase visibility.
• Engage with influencers beyond the community wall!

8. Belonging and responsiveness

The one aspect successful communities have in common is that they do create a sense of belonging - with members feeling safe to communicate. This leads to our final tip: Be responsive: Address member concerns, be transparent and keep the community informed. This might require a good team of moderators across various time zones. It might also require AI, something we are going to focus on in the coming weeks.

For the next four weeks, we will drill down on specifics of important concepts and technical aspects in digital communities. For example, understanding user behaviours is essential to manage digital communities. Theoretically, we will discuss users’ innate psychological needs that motivate them to come to digital communities and how users engage with different affordances (action possibilities) provided by digital communities to satisfy those needs based on academic research. Practically, we will develop hands-on skills to analyse and understand user behaviours in digital communities, such as regression analysis and sentiment analysis with user data. We will explore other important agendas in a similar manner, such as user social networks, AI considerations (e.g., chatbots), and current challenges to digital communities (e.g., fake news).

Disclaimer: We wrote this blog collectively in one workshop. We used GenAI to provide us with the structure and found illustrative examples through internet research.

For further questions on the Creating Digital Communities module, please contact Isabel Fischer: Isabel.fischer@wbs.ac.uk

Abigail Ball : 19 Feb 2024 09:00 |  Tags: Amira Badri Angie Nasirumbi Anindya Dena Boluwatife Buraimoh Bowen Zhang Community Digital Isabel Fischer Jingya Wen Milind Bage Qinyao Li Samantha Reindorf Seyoung Seol Swara Kalghatgi Tiang-Bahumi Kgothule Yichen Zhao Yimeng Zhu Zhiping He Zhuochen Wu |  Comments (0) |  Close comments |  Report a problem

Past reflections, future outlook

Past reflections, future outlook: Update from the Warwick Learning Circle on diverse and inclusive assessment practices

by Kerry Dobbins, Isabel Fischer, Sam Grierson and Leda Mirbahai

Warwick’s International Higher Education Academy (WIHEA) has many features - one of them is that members only remain active members for three years before becoming alumni. The imminent handover of the co-leadership of the open learning circle on diverse and inclusive assessment practices made us, outgoing and new co-leaders, reflect on past initiatives and future directions.

Let’s start with an exciting future - what is new?

As we continue to move forwards with this work, we will become the ‘Inclusive Assessment Learning Circle’. This name change recognises our intention to embed the work on diverse assessments into a broader outlook which sees assessment in HE as a vehicle to promote equity and social justice. The wellbeing of students and staff will feature strongly in our ongoing discussions.

In line with our focus, the learning circle will be inclusive of:

1. Assessment strategies and methods that are diverse, authentic, applied and decolonised - this broad understanding and acknowledgement of relevant issues in teaching, learning and the designing of assessments reflects the current landscape in HE and indicates the interest of our members. We hope to support the university reimagining the assessment design narrative, taking a view from programme level through to single assessment and placing assessment at the heart of curriculum design.
2. Our members - we want members to be and feel actively engaged and involved in the learning circle.
3. Students’ experiences – the work of the learning circle will be firmly focused on working with students so that collectively we can work to understand and support students’ needs and their personal and professional ambitions through assessment.

As for ‘past reflections’: What are the existing features of the learning circle worth keeping? There are in particular four features that worked really well in the past:

1. Since its inception membership has grown steadily for three reasons: The membership of this particular LC is open to WIHEA as well as non-WIHEA members from across Warwick and other institutions, also internationally. The open membership has enriched the discussions and enabled establishment of networks nationally and internationally. Secondly, creating sub-groups not just raised interest, it also offered leadership opportunities to more members. The most popular sub-group was AI in education which in turn was split into six further strands / interest groups: Artificial Intelligence in Education (warwick.ac.uk). Lastly, the topic of the learning circles (assessment in its broadest term) has significant implications not only for educators but also for our learners.
2. Many of our invited keynote speakers at the start of the bi-monthly meetings and at our mini-conference on assessments, captured the essence of their talks in blogs. Most of the blogs were published within WJETT (see the end of this blog for some examples, others were in other outlets, such as SEDA and SCiLAB. Even if not captured in blogs, did the keynote speeches result in interesting discussions and network opportunities.
3. Extensive student corporations, with students supported by two WIHEA grants. One of the student participants even drafted an academic article based on her learning about assessments during her membership.
4. Hosting workshops enabled dissemination of our findings and new resources, including the outputs of our funded projects. It also provided a platform to encourage co-learning and sharing amongst participants and facilitators (including student facilitators). Overall we hosted three workshops with the details of the workshops available from our webpage.

Finally, this is blog 15 in our diverse assessment series, some of the more recent blogs can be found here (with further links to previous blogs shown within some of the blogs below):

• Using AI for Formative Feedback: https://blogs.warwick.ac.uk/wjett/entry/using_ai_for/
• Assessments: Capturing Lived Experience and Shaping the Future
• Blog 7: Democratising the feedback process (Linda Enow): https://blogs.warwick.ac.uk/wjett/entry/democratising_the_feedback/

Abigail Ball : 16 Oct 2023 09:00 |  Tags: Assessment Isabel Fischer Kerry Dobbins Leda Mirbaha Reflections Sam Grierson Teaching |  Comments (0) |  Close comments |  Report a problem

Building knowledge on the pedagogy of using generative AI in the classroom and in assessments

By Matt Lucas and Isabel Fischer (WBS)

Matt Lucas is a Senior Product Manager at IBM, and Isabel Fischer is an Associate Professor (Reader) of Information Systems at WBS (Warwick Business School). Isabel also co-convenes an IATL (Institute for Advanced Teaching and Learning) module. This blog represents their own opinions and not those of their employers.

After two terms of including generative AI (GenAI) in my teaching and for assessments I am still building my knowledge and understanding around the pedagogy of using GenAI. Students seem to like the entertainment of playing around with music and art (e.g. DALL.E 2 and midjourney), creating images and also memes, with all of these being user-friendly for big screens and also for huddling around one laptop as part of teamwork. Text-outputs seems less intuitive for ‘collective use’: There does not seem to be yet an app available that allows for hands-on collaborative refinement of prompts (e.g. similar to students working on the same Google doc). And displaying a string of words on a shared screen clearly does not have the same entertainment value for students as ‘customers and consumers’.

In addition to a lack of entertainment value I also found that students seem to actually appreciate word-based GenAI (e.g. ChatGPT and Bard) as ‘their secret tool’ at their disposal and for them to use. They appear to appreciate it, if lecturers show them the exact prompts that they can copy that allows them to make the most of ‘their secret tool’. They seem less keen about having to be transparent about using the tool themselves and having to justify and critically reflect on usage. It not only means additional work, more importantly, they dislike the thought of the tool’s hidden power being exposed. They appear even less keen for lecturers to use GenAI for the lesson preparation and to be transparent about it because otherwise, what is the ‘perceived added value’ of attending the lecture if they could have just reviewed GenAI?

With this in mind, what are the skills that students can learn from using GenAI in the classroom and in assessments?

In the attached blog Matt Lucas and I suggest that by including innovative aspects into assessments, students can learn and practise four skills that are relevant for their future careers in a world disrupted by AI:

1. Cognitive flexibility, abstraction and simplification

2. Curiosity, including prompt engineering

3. Personalisation, reflection and empathising to adapt to different audiences

4. Critical evaluation of AI

For each of the four skills we explain in the attached blog the relevance for student learning with some illustrative examples, before outlining how we have incorporated these four skills into students’ assessments in the recent term.

Abigail Ball : 12 Jun 2023 09:00 |  Tags: Assessment Isabel Fischer Matt Lucas Teaching |  Comments (0) |  Close comments |  Report a problem

Is interdisciplinarity learnable in 10 weeks?

A Student - Staff co-analysis from IATL’s Creating Digital Futures module

by Anil Dhariwal, Isabel Fischer, Jamal Lawal, Matthew Leslie, Claire Rocks, Shwetha Roy, Bryon Wandera, and Devon Williams

We created this blog collectively in 1h as part of our last Creating Digital Futures workshop. In the interest of time, instead of using Harvard Referencing we entered Weblinks.

In this blog we look back at a 10-week module which brought together students from different fields for the collaborative development of a digital product, which we pitched to the Warwick #Creative Futures Incubator (https://warwickinnovationdistrict.com/creative-futures-incubator-application/). The purpose of the blog is to reflect on whether students can effectively learn and apply the fundamentals of interdisciplinarity, and if 10 weeks is a suitable amount of time for the benefits of interdisciplinarity to take effect.

3 min read

Source: DALLE.E 2 by OpenAI - retrieved 15/3/23 using these prompts: ‘Interdisciplinarity Creating Digital Futures’

From an educator-side our research has shown that a key pedagogical challenge is to encourage students to move beyond scanning of information towards critical engagement and action (Preuss, Fischer, Luiz, 2023, see: https://onlinelibrary.wiley.com/doi/10.1111/hequ.12429). Payne (1999, p.173) emphasises the potential of interdisciplinarity to enhance ‘democratisation of knowledge construction processes in society’ with Klein (2000, p.18) explaining that ‘Interdisciplinary knowledge strengthens connections between disciplines and in that process it weakens the division of labour in disciplines, exposes gaps, stimulates cross fertilisation and creates new fields of focus for knowledge inquiry’.

Problem based learning is often thought of as a pedagogical approach that can facilitate interdisciplinary learning. Adopting this approach, Creating Digital Futures asked students to create a product or service at the intersection of Digital Technology, Creativity and the Global Sustainable Development Goals. Focussing first on identifying a problem using Design Thinking was intended to encourage students to come out of their disciplinary boundaries to search for solutions. As research has shown that we cannot take for granted that interdisciplinary learning will take place, and that it is necessary to address issues of interdisciplinary learning specifically (Stentoft et al, 2017, https://doi.org/10.1177/1469787417693510), in Creating Digital Futures we scaffolded the interdisciplinary learning by providing 10 weekly taught sessions covering topics such as working in interdisciplinary teams, design thinking, ethical frameworks, storytelling with data and presentation skills.

A ChatGPT (2023) prompt tells us: ‘Interdisciplinarity has become a buzzword in academia, with many universities and institutions promoting collaborative learning and research. But can students truly learn the skills and mindset needed for interdisciplinary work in just 10 weeks? In this blog, we explore [if] it is possible to truly break down the boundaries between disciplines and instil a collaborative mindset in a matter of weeks.’ (ChatGPT, 2023a)

We - the students - found that our interdisciplinary module has a number of benefits, including:

• We were able to develop real world skills such as: critical thinking; communication; collaboration and the ability to synthesise information (Anil, Econ)
• Interdisciplinarity gives the opportunity to see areas that you may lack in due to your subject background and allows you to learn and improve them with the help of others with different perspectives (Byron, Biomed)
• Providing a better understanding of the utility of one’s own skills from their field in the context of the real world (Devon, Comp Sci)
• Really highlights the inherent interdisciplinarity of traditional subjects (Jamal, PPE)
• The helpfulness of application of theory in a practical matter, experiencing, growing and learning from and with others (Matthew, WBS)
• Allows you to produce an impactful and cohesive piece of work that brings together the ideas from different disciplines (Shwetha, Econ).

In conclusion, students have found many aspects of the module rewarding, and have also reflected that it was a good preparation in anticipation of embracing interdisciplinarity in the workplace. A problem based approach seems to be good to foster interdisciplinary learning in just 10 weeks. More importantly though, students appreciated having been taught by members from five different departments. The module convenors - Claire Rocks (Computer Science) and Isabel Fischer (WBS) - would therefore like to thank Elke Thonnes (Stats) and Robert O’Toole (Arts) as well as Rachel Davis, Clare Green and Holly Smeeton (Warwick’s Creative Futures Incubator / Warwick Entreprise) for making time to take this cohort of IATL students on a journey of interdisciplinary discovery.

ChatGPT prompt to help with the initial thoughts on the blog:

ChatGPT 2023 prompt used on 15/3/23: ‘Is interdisciplinarity learnable in 10 weeks, write a short intro to hook readers in for a blog with this title with an interesting question’.

Abigail Ball : 27 Mar 2023 08:00 |  Tags: Anil Dhariwal Bryon Wandera Claire Rocks Creating Digital Futures Devon Williams Iatl Isabel Fischer Jamal Lawal Matthew Leslie Shwetha Roy |  Comments (0) |  Close comments |  Report a problem

On ‘Opportunities of AI in Higher Education’ by DALL.E and ChatGPT

Prologue and epilogue by Isabel Fischer (Warwick Business School)

Since the creation of our task and finish groups on evaluating and shaping the opportunities of AI in Higher Education (for further information see here: https://blogs.warwick.ac.uk/wjett/entry/ai_for_good/) we have had many very insightful ‘human to human’ discussions. The aim of this blog is, however, to show what ‘machines’ draw and write on the topic, to illustrate both the strengths and the weaknesses of generative AI.

The images at the start of the blog represent DALL.E’s take on the topic. Having always been untalented in drawing, I use DALL.E in my business teaching to show live how AI translates the topic we discuss by imitating distinct artists. Students enjoy how DALL.E (OpenAI) shows other – very creative - images while we wait for AI ‘to think & draw’ our images – there are always four images coming up for each prompt. Of course, we also discuss the risks, threats, and ethics behind AI (here an open-access case study which we wrote on AI ethics in the field of ESG investing: https://journals.sagepub.com/doi/full/10.1177/2043886920961782).

While DALL.E generates images that I could not have drawn, ChatGPT generates text that I could have written better and more concisely, with different logical priorities. I could - and actually did - create visualisations to illustrate my points. Nevertheless, ChatGPT did beat me by far on ‘speed’. For anybody who has not yet asked ChatGPT any questions, here are the ChatGPT responses based on five prompts:

1. Opportunities for AI in HE
2. Strategy for implementing and integrating AI in HE
3. Developing research on the opportunities of AI in HE
4. Developing an AI based marking and feedback tool
5. Common areas of weakness of students’ academic writing

Opportunities for AI in HE

Artificial intelligence (AI) is transforming the world around us, and higher education (HE) is no exception. The potential opportunities of AI in HE are vast, ranging from improving learning outcomes and enhancing student engagement to streamlining administrative tasks and reducing costs. In this article, we explore some of the key opportunities of AI in HE and how they can be realized.

1. Personalized learning: AI can help personalize the learning experience by adapting the content, pace, and style of instruction to the individual needs and preferences of each student. This can be achieved through intelligent tutoring systems, which use machine learning algorithms to analyze student performance data and provide customized feedback and support. Personalized learning can improve student engagement, increase retention rates, and enhance learning outcomes.
2. Predictive analytics: AI can be used to analyze large datasets of student information, such as grades, attendance records, and behavioral data, to predict future performance and identify at-risk students who may need additional support. This can help educators intervene early and provide targeted interventions to improve student success.
3. Virtual assistants: AI-powered virtual assistants can help students with routine administrative tasks such as scheduling, registration, and accessing resources. This can free up staff time and reduce wait times for students, improving their overall experience.
4. Automated grading: AI can be used to automatically grade assignments and provide feedback to students, reducing the workload on instructors and improving the consistency and objectivity of grading. This can also provide students with immediate feedback, enabling them to improve their performance more quickly.
5. Research and data analysis: AI can be used to analyze large datasets of research publications and help identify trends, patterns, and gaps in the literature. This can help researchers stay up-to-date with the latest developments in their field and generate new insights.
6. Adaptive assessments: AI can be used to create adaptive assessments that adjust the difficulty and content of questions based on the student's level of knowledge and ability. This can help educators more accurately assess student performance and provide targeted feedback and support.
7. Smart campus: AI can be used to optimize campus operations, such as energy management, facility maintenance, and security. This can reduce costs and improve the overall efficiency of campus operations.

While the opportunities of AI in HE are significant, there are also challenges that need to be addressed, such as ethical considerations, data privacy concerns, and ensuring that AI is used to enhance, rather than replace, human interaction and support. To fully realize the potential of AI in HE, it is essential that educators, researchers, and policymakers work together to develop responsible and effective strategies for implementing and integrating AI technologies.

Strategy for implementing and integrating AI in HE

Implementing and integrating AI technologies in higher education (HE) requires careful planning and consideration to ensure that these technologies are used in a responsible and effective manner. Here are some responsible and effective strategies that HE institutions can use to implement and integrate AI technologies:

1. Develop a clear vision and strategy: HE institutions should develop a clear vision and strategy for how AI technologies will be used to enhance the student experience, improve learning outcomes, and support research. This should involve identifying specific use cases and goals for AI, as well as assessing the potential benefits and risks.
2. Foster collaboration and communication: Implementing and integrating AI technologies requires collaboration and communication across departments and stakeholders. HE institutions should establish cross-functional teams that include educators, administrators, IT professionals, and students to ensure that all perspectives are considered and to foster buy-in and support for AI initiatives.
3. Ensure transparency and ethical considerations: HE institutions should ensure that the use of AI technologies is transparent and ethical. This includes being clear about how AI technologies are being used, ensuring data privacy and security, and ensuring that AI is used to enhance, rather than replace, human interaction and support.
4. Invest in staff training and development: AI technologies require specialized skills and knowledge to implement and use effectively. HE institutions should invest in staff training and development programs to ensure that educators and administrators have the necessary skills and knowledge to use AI technologies in their work.
5. Pilot and evaluate AI initiatives: Before scaling up the use of AI technologies, HE institutions should pilot and evaluate their effectiveness. This involves testing AI technologies in small-scale pilots, collecting feedback from stakeholders, and using data to evaluate their impact on learning outcomes, student engagement, and other key metrics.
6. Engage with the wider AI community: HE institutions should engage with the wider AI community to stay up-to-date with the latest developments and trends in AI. This includes attending conferences and workshops, collaborating with other institutions, and partnering with industry experts and researchers.
7. Align AI initiatives with institutional goals and values: Finally, HE institutions should ensure that their AI initiatives are aligned with their institutional goals and values. This involves considering the unique needs and priorities of the institution, as well as the needs and preferences of students, educators, and other stakeholders. By aligning AI initiatives with institutional goals and values, HE institutions can ensure that they are using these technologies in a responsible and effective manner.

Developing research on the opportunities of AI in HE

There are several ways to develop ideas for future opportunities of AI in higher education and organize it as a research topic:

1. Literature review: Conducting a literature review of existing research on AI in higher education can help identify the gaps and potential areas for future research. This can involve searching for relevant articles, books, and reports in databases such as Google Scholar, ACM Digital Library, and IEEE Xplore.
2. Expert interviews: Talking to experts in the field, such as educators, researchers, and AI professionals, can help identify emerging trends and potential opportunities for AI in higher education.
3. Case studies: Conducting case studies of AI implementation in higher education institutions can provide insights into the benefits and challenges of adopting AI technologies.
4. Surveys: Conducting surveys of stakeholders such as students, faculty, and administrators can help identify their needs and preferences for AI in higher education.
5. Workshops and conferences: Attending workshops and conferences on AI in higher education can provide opportunities to learn from experts and exchange ideas with peers.

Once you have developed ideas for future opportunities of AI in higher education, organizing it as a research topic can involve the following steps:

1. Identifying research questions: Based on the ideas developed, identify research questions that are relevant, feasible, and significant.
2. Defining the scope of the research: Clarify the boundaries of the research topic and define the key concepts and variables.
3. Developing a research methodology: Decide on the appropriate research methods, such as qualitative, quantitative, or mixed methods, and develop a research design.
4. Conducting data collection: Collect data using appropriate methods such as surveys, interviews, and case studies.
5. Analyzing data: Analyze the data using appropriate statistical or qualitative analysis techniques.
6. Drawing conclusions and making recommendations: Based on the findings, draw conclusions and make recommendations for future research and practice.
7. Writing up the research: Write up the research in a clear and concise manner, following the conventions of academic writing.

When reviewing emerging trends of AI in higher education after conducting qualitative research, a theory that could be useful is grounded theory.

Grounded theory is a research methodology that involves systematically collecting and analyzing qualitative data to develop a theory or explanation of a phenomenon. The approach emphasizes the need to generate theory from the data itself, rather than imposing preconceived notions or theories on the data.

When conducting a qualitative study of emerging trends of AI in higher education, the grounded theory approach can help identify key themes and patterns in the data that may point towards emerging trends. These trends can then be used to develop a theory that explains how AI is being used in higher education, what the benefits and challenges are, and what the future implications may be.

Grounded theory can also be helpful in identifying gaps in the literature and areas for future research. By systematically analyzing the data, researchers can identify areas where more research is needed, and use the emerging trends to develop research questions that are relevant and meaningful.

Developing an AI based marking and feedback tool

Designing an AI system that can be used to automatically grade assignments and provide personalized feedback to students can be a complex task, but it can be accomplished by following these steps:

1. Define the grading criteria: The first step is to define the grading criteria for the assignment. This involves breaking down the assignment into its key components and determining how each component will be evaluated. The grading criteria should be clearly defined and objective, so that the AI system can easily understand and apply them.
2. Create a training dataset: To train the AI system, you need to create a training dataset that includes a set of annotated assignments. These annotated assignments should be graded by human experts according to the defined grading criteria. The dataset should be large enough to ensure that the AI system can learn to recognize a variety of different assignment types and styles.
3. Choose the appropriate machine learning algorithm: There are several machine learning algorithms that can be used for automatic grading, including decision trees, support vector machines, and neural networks. The choice of algorithm will depend on the complexity of the grading criteria and the size of the training dataset.
4. Train the AI system: Once you have chosen the appropriate machine learning algorithm, you can train the AI system using the annotated training dataset. The system should be trained to recognize patterns and relationships in the data, and to identify which components of the assignments are important for grading.
5. Test the AI system: After training the AI system, you should test it using a set of unseen assignments that have not been included in the training dataset. This will help you evaluate the system's accuracy and identify any areas where it needs further improvement.
6. Provide personalized feedback: To provide personalized feedback to students, you can use the AI system to identify areas where a student may need additional support or guidance. For example, if a student consistently struggles with a particular component of the assignment, the system can provide targeted feedback and resources to help the student improve.
7. Continuously improve the AI system: Finally, you should continuously monitor and improve the AI system. This involves regularly updating the training dataset, evaluating the system's performance, and incorporating feedback from students and educators to refine the grading criteria and feedback mechanisms.

In summary, designing an AI system for automatic grading and personalized feedback involves defining the grading criteria, creating a training dataset, choosing the appropriate machine learning algorithm, training the system, testing it, providing personalized feedback, and continuously improving the system. By following these steps, you can develop an effective and efficient AI system that enhances the learning experience for students and educators alike.

Common areas of weakness of students’ academic writing

Academic writing is an essential skill for students in higher education. However, there are common areas of weakness that students often struggle with. Here are some of the most common areas of weaknesses for students' academic writing in higher education:

1. Poor organization and structure: Students often struggle with organizing their ideas and structuring their writing in a way that is clear and logical. This can make their writing difficult to follow and reduce the overall effectiveness of their argument.
2. Weak thesis statements: A strong thesis statement is essential for guiding the direction of an academic paper. However, many students struggle to develop a clear and concise thesis statement that effectively summarizes the main argument of their paper.
3. Inadequate research and citation skills: Effective academic writing requires the use of credible sources to support arguments. However, many students struggle with finding appropriate sources, evaluating their credibility, and correctly citing them in their writing.
4. Poor grammar and syntax: Grammatical errors and poor syntax can make writing difficult to read and understand. This can detract from the clarity and effectiveness of students' arguments.
5. Lack of clarity and concision: Academic writing should be clear and concise, but many students struggle with using language that is overly complex or wordy. This can make their writing difficult to read and reduce the impact of their argument.
6. Inappropriate tone or style: Students may struggle to adopt an appropriate tone or style for academic writing, which can lead to writing that is overly informal or lacking in scholarly rigor.
7. Failure to address the assignment prompt: Finally, students may struggle with addressing the assignment prompt in a way that fully meets the expectations of the instructor. This can result in writing that is off-topic or fails to fully address the main requirements of the assignment.

Overall, addressing these common areas of weakness in academic writing can help students improve their writing skills and achieve greater success in their academic pursuits.

Epilogue: For comparison to the above, here are some ChatGPT responses to prompts by Chaoran Yang, a student who is part of my consulting module. I suggested to all students of that cohort to try ChatGPT for idea generation at the start of a consulting exercise for the Warwick Esports centre. Similar to my findings from above, the students also felt that ChatGPT did not provide any major insights. Possibly because we did not ask the right questions, and, more importantly, because ChatGPT is still in its infancy.

In our team we have spent over 2,000 hours developing the Warwick AI Essay Analyst. We used a mixture of non-AI rule-based statistical features and deep-learning algorithms and databases, e.g., Pytorch, Hugging face framework, and Transformer (for further information on our AI-based tool see here: https://nationalcentreforai.jiscinvolve.org/wp/2022/11/16/interested-in-receiving-formative-feedback-on-your-draft-essays-and-dissertations-on-demand-introducing-warwicks-ai-essay-analyst/).

With the current progress in the field of generative AI, developments of future tools will be faster – let’s work together to ensure that all tools, whether developed in-house or bought / endorsed by the university have robust ethical underpinnings. My final suggestions for readers is to review here the Ethical guidelines on the use of artificial intelligence (AI) and data in teaching and learning for educators, produced by the Office of the European Union: https://data.europa.eu/doi/10.2766/153756

Abigail Ball : 06 Mar 2023 12:01 |  Tags: AI Chaoran Yang Isabel Fischer Teaching |  Comments (0) |  Close comments |  Report a problem

AI for Good: Evaluating and Shaping Opportunities of AI in Education

By Isabel Fischer, Leda Mirbahai, and David Buxton

Following the rise of awareness of the opportunities (and threats) of artificial intelligence (AI) in education, we have created a task and finish group which aims to review and ‘imagine’ the opportunities and challenges of AI in education, incl. assessments. Our vision is to deploy AI as a tool to support all students, independent of background and socio-demographic characteristics, to be successful in their studies and in their future work, while ensuring academic integrity, as well as to support educators feel confident in using AI effectively in promoting learning. We are working in five (sub)groups:

1. General AI in Education (AIEd) Opportunities & information sharing
2. Novel and Diverse Assessment Designs
3. Feedback, Marking, Authorship Detection
4. Designing Teaching Content - ‘what is out there being developed?’
5. 'Red Team': AI Ethics and Academic Integrity

As we are still interested in colleagues from within Warwick as well as other institutions and the wider community of stakeholders to join us, here some further information per (sub)group:

1) General AI in Education (AIEd) Opportunities & information sharing: We review how to capture, shape, and disseminate the opportunities for both learner-facing and educator-facing AI, mainly in HE but also considering how HE can support the secondary and even primary school sector (e.g. how to help teachers to experiment with different forms of AI in a low-stake environment). We also consider the benefits, such as reducing inequality, fairness and democratisation that AI offers, evaluating how we can support SDG 4 (equitable and quality education) and SDG 10 (reducing inequalities). We want to help educators to know how to potentially embrace recent AI developments for their professional practice. Combined with sub-group / Strand 5, the ‘red team’ we also want to inform colleagues on research (similar to mini literature reviews) on topics such as Algorithmic Fairness.

Target Output: A WIHEA page that is informative for colleagues new to AIEd (explanations, links to other resources, links to discussions / brainstorming exercises / blogs, suggestions for their practice)

2) Designing Assessments: We review the opportunities for designing and setting diverse assessments (Learner-facing), including embedding our work within our different Learning Circle’s work. It is in this strand that most of the student co-creation will take place.

Target Output: WIHEA page, blogs, and talks

3) Feedback and Marking: We review the opportunities of using AI for formative feedback (Learner-facing), summative feedback (Educator-facing), ‘AES – automated essay scoring’ (educator-facing), and stylometry (authorship authentication) as well as ChatGPT detection. One aspect of this strand (but not constrained to this strand) is also ‘Move fast, Break fast, Learn fast’ – doing small scale experiments and testing them (e.g., Consulting Students will experiment with mind maps this term and then can, but don’t have to, submit their work to the Warwick AI Essay Analyst for formative feedback and we can analyse their work).

Target Output: A WIHEA page that disseminates information and possibly diffusion of the actual Warwick AI Essay Analyst tool at Warwick, potentially producing research output

4) Designing Module and Lesson Content & Recommendations for institutional Developments / Purchases: Educator-facing, we review tools and initiatives that might help educators in planning and organising their modules and lessons, as well monitoring their email and forum entries. This group looks at all educator-facing areas besides designing assessments (group 2) and providing feedback on assessments (group 3). This group might also make recommendations to the institution on what software to build or to purchase etc.

Target Output: A WIHEA page that disseminates information, possibly making recommendations for in-house developments / purchase of external software packages

5) A ‘red team’ acknowledges that AI is here to stay and ensures we follow AI Ethics guidelines and that everybody is clear about the risks. This team also reviews and mitigates the challenges to Academic Integrity more broadly. Moreover, it reviews the risk of bought-in products from EdTech and Tech companies, ensuring that AI Ethics is applicable both for in-house and off the shelf, bought-in products.

Target Output: A WIHEA page that provides information for colleagues worried about AIEd (explanations, links to other resources, links to discussions) especially on the topic of AI Ethics and Academic Integrity (what is OK to do, what isn’t – where should students / educators draw the line). Collaborating with stand 1, this group might want to explain (do a high-level literature review / providing links to important research) aspects of AI Ethics / Academic Integrity, such as explaining concepts such as ‘Algorithmic Fairness’. Building on work by other groups, e.g., last year’s ‘Online Assessment Data Ethics Group’, this group might want to develop a proposal for SLEEC (https://warwick.ac.uk/services/gov/committees/sleec/) and/or to provide guidance and advice to EPQ on suitable policy and guidance where appropriate.

Proposed Overall Target for entire Task and Finish Group, i.e. across the five groups / strands: Have some tangible outputs (WIHEA page, blogs, talks) that support colleagues when they embrace change in an ethically sound way that respects all stakeholders, especially learners and educators. Ideally collaborating with other universities, other education providers, and industry. Possibly develop a proposal for SLEEC and/or provide guidance and advice to EPQ on suitable policy and guidance where appropriate.

Please email Isabel.fischer@wbs.ac.uk if you are interested in joining any of the groups.

Please email Leda.Mirbahai@warwick.ac.uk if you are interested in joining our open WIHEA Diverse Assessment Learning Circle with interesting talks, such as our talks this month on Synoptic Assessments and on Democratising Assessments.

Abigail Ball : 13 Feb 2023 09:04 |  Tags: AI David Buxton Isabel Fischer Leda Mirbaha Teaching |  Comments (0) |  Close comments |  Report a problem

One year on: Progress update on our Diverse Assessments Learning Circle

Isabel Fischer (WBS) and Leda Mirbahai (WMS)

One year ago we created an open WIHEA Learning Circle on Diverse Assessments. Since we seem to be building a reputation as the ‘godparents of assessments’. In addition to contributing to diversifying assessment strategies across Warwick we aim to work towards providing equity in our assessment practices and to improve student experience. Assessments, if used effectively, are key to promoting learning for our students.

To encourage reflection and to drive change in how we use and view assessments in our programmes, we hosted a series of keynote speeches to start our regular meetings. Here one example from Kerry Dobbins, Academic Development Centre, on How to create an effective assessment strategy (drilling down – or up – from institutional, via course, to module level)

Assessment Strategy

My aim for this presentation was to highlight the conflation that often occurs between assessment ‘strategies’ and assessment ‘methods’. The term ‘strategy’ is often used when we are actually referring to the mode of assessment, e.g. ‘our assessment strategy is coursework or an online exam’. It is important to disentangle these terms so that we can take an explicitly strategic approach to designing assessments that supports inclusion at all levels, i.e., module, course/programme and institution. An assessment strategy develops a shared and holistic view of the course/programme between students and academics. At a macro level, there needs to be constructive alignment between module learning outcomes (LOs), course/programme LOs and graduate attributes. In this way, a programme level view is taken to what LOs are being assessed across modules and how. For diverse assessments this is extremely important because it ensures that a holistic view is taken in relation to how comfort with, skills for and literacy of different types of assessment methods are developed and scaffolded for students as the programme progresses. This strategic and holistic view also recognises the various transition points of the students’ journey; so first year assessments may start to introduce elements of doing things differently, that are built on in the second year, etc.

In essence then, a strategic approach is vital for inclusive assessment practices as it provides an explicit framework for developing assessment literacy skills and for assignment feedback to be clearly directed towards feeding forward into future assessment activities. Taking a strategic approach also provides greater opportunities for teams to develop a coherent view about the purposes and values of assessment; and how those shared values are threaded through the course or programme. Assessment is not value-free as we are always conveying value messages to students about what we assess and how. A programme strategy allows us to really consider our values and what we are trying to achieve with our assessment practices and processes overall.

Assessment strategy also occurs at the module level. Again, at this level the strategy is not the mode of assessment but how support to achieve within the assessment is structured into the module. For example, how is assessment and feedback literacy designed into the module curriculum? What does the pre and post-assessment support look like? What is the rationale for the mode of assessment being used? How is assessment (formative and summative) being used within the module to support learning, not just quantify it?

You might find the attached presentation and some of the texts below useful to review:

Boud and Associates (2010) Assessment 2020: Seven propositions for assessment reform in higher education

Brunton et al (2016) Designing and developing a programme-focused assessment strategy: a case study

Scholtz (2016) (PDF) The assessment strategy: An elusive curriculum structure (researchgate.net)

If you are interested in this area, I would welcome you to get in touch: Kerry.Dobbins@warwick.ac.uk

For our learning circle we have also managed to secure funding to undertake a research project to capture both student and staff views of diverse assessments. Although the project is still ongoing, our student project officers, Molly Fowler and Pula Prakash, have managed to gather valuable data with an aim to feed into institutional considerations around assessment strategies.

Finally, if you want to find out more about our Learning Circle you can visit our webpage and you can read our previous blogs here:

Blog 1: Launch of the learning circle: https://blogs.warwick.ac.uk/wjett/entry/interested_in_diverse/

Blog 2: Creative projects and the ‘state of play’ in diverse assessments: https://blogs.warwick.ac.uk/wjett/entry/creative_projects_and/

Blog 3: Student experience of assessments: https://blogs.warwick.ac.uk/wjett/entry/a_student_perspective/

If you would like to join this learning circle please contact the co-leads: Leda Mirbahai, Warwick Medical School (WMS) (Leda.Mirbahai@warwick.ac.uk) and Isabel Fischer, Warwick Business School (WBS) (Isabel.Fischer@wbs.ac.uk).

Abigail Ball : 14 Nov 2022 08:38 |  Tags: Assessment Isabel Fischer Leda Mirbaha Reflections Teaching |  Comments (0) |  Close comments |  Report a problem

Introducing Warwick’s “AI Essay–Analyst”

By Isabel Fischer, Zhewei Zhang, Lichuan Xiang, Aiqi Jiang, Yiran Xu and Joe Nandhakumar

Three years since its first conceptualisation, we are pleased to introduce the “AI Essay-Analyst”, an academic-writing-tool in support of the mission of Warwick Business School (WBS) “to enable our stakeholders to realise their full potential” and the University of Warwick’s 2030 strategy “to ensure that, irrespective of background, disability, faith, gender, race and sexual orientation, all students have access to equal opportunities to thrive and progress at Warwick”.

A recent WBS survey showed that the majority of students perceive poor academic writing as their main barrier to success. In response, a group of WBS faculty and students developed and piloted a machine-generated automated formative essay feedback tool in-house which is now being made available to an increasing number of students on an optional basis.

Academic writing feedback tools have the potential of providing students with “personalised feedback that is currently only available to a privileged minority”^[1] and can enhance students’ self-determined learning: Formative assessments are seen as “one of the most important mechanisms for improving student learning. Self and peer-assessment are particularly effective in formative learning as they require students to engage more fully with the assessment process”^[2]. Currently, students of select modules are being offered the opportunity to trial the software by submitting their draft essays or dissertations prior to their submission deadlines. Participating students receive a personalised AI-generated feedback report of approximately 15 pages. The report includes images, charts and graphs which students are encouraged to review prior to the formal submission of their assignment.

External providers, such as Grammarly, Turnitin Draft Coach, Bartleby, Writefull, and Hemingway Editor also offer feedback to students, however, most of these tools focus on grammar and spelling. In addition, in most cases, students have to agree that external providers can use their data. The “AI Essay-Analyst” does not use student data and is substantially more comprehensive. For example, by also including the CABS ABS ranking we can check the quality of the journal articles that are cited and referenced. In addition, we offer visualisations such as knowledge graphsand argumentative zoning^[3], which are expressed as PIE charts and knowledge graphs^[4]. These visualisations are very much appreciated by students.

Students who opted to take part in the project so far were very satisfied, commenting: “The overall feedback is very useful for the general understanding of your academic writing skills”, “It is quite cool and it is a new approach I never tried before”, “I have enjoyed the visualisations most since they are interactive and easy to understand” and “Grammar suggestions are useful since they show some spelling and small mistakes that I ignored before.”

Detailed student feedback on specific features included:

1. The most useful are grammar suggestions, because it helps me revise the essay most directly.
2. The spider graph is useful to help me understand where the essay is lacking.
3. The Word Cloud is useful to help me check if the essay is on topic.
4. Systematic stages of negation is helpful as it let me know if my critical thinking has been fully applied.
5. For readability, this is an aspect that I usually find difficult to notice, because everything is readable in my own mind. So that is very helpful.
6. The knowledge graph allowed me to see the bigger picture at a time when I was too focused on the detail. It helped me to break down my essay and also showed the correct as well as incorrect relationships between key concepts.

For comments or questions please contact the project lead Isabel.fischer@wbs.ac.uk

[1] https://oro.open.ac.uk/46517/1/LAK16%20Writing%20Analytics%20Wkshp%20-%20FINAL.pdf

[2] https://www.tandfonline.com/doi/full/10.11120/plan.2010.00230040

[3] https://www.cl.cam.ac.uk/~sht25/az.html

[4] https://medium.com/nlplanet/building-a-knowledge-base-from-texts-a-full-practical-example-8dbbffb912fa

Abigail Ball : 10 Oct 2022 09:08 |  Tags: Isabel Fischer Teaching |  Comments (0) |  Close comments |  Report a problem

Instructional Innovation

Including Esports in the curriculum to prepare for the Metaverse(s)

By Dr Isabel Fischer, Reader in Information Systems, WBS

Following the launch of our Warwick Esports centre, in collaboration with the Esports team Jack Fenton and Elenore Jiawen Li, we included Esports in four WBS modules (Digital Transformation, Design Thinking for Digital Innovation, Digital Marketing Technology and Management, and Developing Consulting Expertise). This allowed management students to experiment with and to reflect on digital innovations and to find solutions to problems. Specifically, we wanted to encourage students to develop tangible ideas for the future metaverse ecosystem.

While learning about technologies and business models related to Esports and the Metaverse, the topic also allowed students to reflect on the convergence of technology, ethics, science, psychology and digital wellbeing, as well as on the impact on environmental and social sustainability. Teaching delivery was accompanied by authentic assessments, with students able to choose their topic. These novel assessments (vlogs, blogs, board papers) were introduced in the previous year which allowed for comparisons.

We found on the one hand that the quality of submitted assignments improved, with students seemingly much more creative and also technological ‘savvy’, both for their choice of topics and content as well as for the delivery formats. On the other hand, module evaluation showed that the inclusion of the Metaverse and Esports early on in the module ‘hyped’ students and wet their appetite to ‘fully’ understand the potential applications of the Metaverse despite the Metaverse(s) still being conceptualised. While previous students were happy with carton-based headsets using their own mobile phones, some of this year’s students would have appreciated working with sophisticated VR headsets, possibly because our teaching delivery was further hyped up as it coincided with Microsoft’s $68.7bn acquisition of Activision Blizzard, clearly showing the current potential of the gaming industry.

Finally, here are some testimonials from students new to Esports, provided after the initial seminar on Esports:

1. I am doing the Design Thinking module where we had a chance to join the esports world. This enables us to think creatively! I love our Warwick Esports Centre.
2. Thank you for letting us use the Esports Centre - it was really fun and easy to learn.
3. First time and it was a very enjoyable experience. Very well organized and easy for beginners.
4. Really enjoyable experience, great equipment.
5. The game is really fun to play. It’s quite unexpecting and fun which the seminar works. I enjoyed it a lot. Thank you for the experience.
6. Thank you for giving students such a good opportunity on campus. It really is a good way to bring people from different backgrounds together.

For further information on this initiative you might want to listen to this podcast: https://anchor.fm/ldcuwarwick/episodes/Blogging--AI-Marking--Online-Learning--Communication--Metaverse--Esports-e1d8efu

Abigail Ball : 13 Jun 2022 10:14 |  Tags: Curriculum Esports Isabel Fischer |  Comments (0) |  Close comments |  Report a problem