All entries for Monday 10 October 2022

Introducing Warwick’s “AI Essay–Analyst”

By Isabel Fischer, Zhewei Zhang, Lichuan Xiang, Aiqi Jiang, Yiran Xu and Joe Nandhakumar

Three years since its first conceptualisation, we are pleased to introduce the “AI Essay-Analyst”, an academic-writing-tool in support of the mission of Warwick Business School (WBS) “to enable our stakeholders to realise their full potential” and the University of Warwick’s 2030 strategy “to ensure that, irrespective of background, disability, faith, gender, race and sexual orientation, all students have access to equal opportunities to thrive and progress at Warwick”.

A recent WBS survey showed that the majority of students perceive poor academic writing as their main barrier to success. In response, a group of WBS faculty and students developed and piloted a machine-generated automated formative essay feedback tool in-house which is now being made available to an increasing number of students on an optional basis.

Academic writing feedback tools have the potential of providing students with “personalised feedback that is currently only available to a privileged minority”^[1] and can enhance students’ self-determined learning: Formative assessments are seen as “one of the most important mechanisms for improving student learning. Self and peer-assessment are particularly effective in formative learning as they require students to engage more fully with the assessment process”^[2]. Currently, students of select modules are being offered the opportunity to trial the software by submitting their draft essays or dissertations prior to their submission deadlines. Participating students receive a personalised AI-generated feedback report of approximately 15 pages. The report includes images, charts and graphs which students are encouraged to review prior to the formal submission of their assignment.

External providers, such as Grammarly, Turnitin Draft Coach, Bartleby, Writefull, and Hemingway Editor also offer feedback to students, however, most of these tools focus on grammar and spelling. In addition, in most cases, students have to agree that external providers can use their data. The “AI Essay-Analyst” does not use student data and is substantially more comprehensive. For example, by also including the CABS ABS ranking we can check the quality of the journal articles that are cited and referenced. In addition, we offer visualisations such as knowledge graphsand argumentative zoning^[3], which are expressed as PIE charts and knowledge graphs^[4]. These visualisations are very much appreciated by students.

Students who opted to take part in the project so far were very satisfied, commenting: “The overall feedback is very useful for the general understanding of your academic writing skills”, “It is quite cool and it is a new approach I never tried before”, “I have enjoyed the visualisations most since they are interactive and easy to understand” and “Grammar suggestions are useful since they show some spelling and small mistakes that I ignored before.”

Detailed student feedback on specific features included:

1. The most useful are grammar suggestions, because it helps me revise the essay most directly.
2. The spider graph is useful to help me understand where the essay is lacking.
3. The Word Cloud is useful to help me check if the essay is on topic.
4. Systematic stages of negation is helpful as it let me know if my critical thinking has been fully applied.
5. For readability, this is an aspect that I usually find difficult to notice, because everything is readable in my own mind. So that is very helpful.
6. The knowledge graph allowed me to see the bigger picture at a time when I was too focused on the detail. It helped me to break down my essay and also showed the correct as well as incorrect relationships between key concepts.

For comments or questions please contact the project lead Isabel.fischer@wbs.ac.uk

[1] https://oro.open.ac.uk/46517/1/LAK16%20Writing%20Analytics%20Wkshp%20-%20FINAL.pdf

[2] https://www.tandfonline.com/doi/full/10.11120/plan.2010.00230040

[3] https://www.cl.cam.ac.uk/~sht25/az.html

[4] https://medium.com/nlplanet/building-a-knowledge-base-from-texts-a-full-practical-example-8dbbffb912fa

Abigail Ball : 10 Oct 2022 09:08 |  Tags: Isabel Fischer Teaching |  Comments (0) |  Close comments |  Report a problem