All entries for Monday 29 January 2007

CAA Fitness for Purpose: User Experience

Follow-up to Fit for Purpose? from Computer-aided assessment for sciences

Another heading in my check-list of criteria for judging whether CAA software, particularly that with mathematical capabilities, is up to the job. As usual, I welcome your comments and further ideas. Today I look at the heading

User Experience

• Logging in and Submitting: Within a given intranet, so-called “single sign-on” avoids having to distribute special user IDs and passwords to students, who then have to remember them to access an assignment. With single sign-on, it is easier to call on institutional databases to display personal information (name, number, mug-shot) onscreen as an identity check for student and invigilator alike. Once signed on, students should be able to click quickly to the test they want to take. It should be clear how their answers to questions should be submitted for marking (grading), singly and/or in one final submission, and whether multiple attempts are permitted. Answers should be regularly saved to a local drive in case of network or software failure.

• Navigation and Layout: It should be easy to navigate quickly through the questions (in any order), and to choose to display them all together on a scrolling page or one at a time. From any test page it should be clear which questions (i) have already been attempted and (ii) have been finally submitted. Each page layout should be visually easy to interpret (e.g. displayed equations, clear separation of question statement from answer boxes with hypertext for actions close by, adjacent questions with different background colours). Anchors to keep the right part of a long page in view, drop-down menus, and prompts to open help windows can all improve the user experience and sense of being in control.

• Entering Answers: Entering text with standard keyboard characters is usually unproblematic – answer-boxes should accommodate the longest imaginable answer, they should display an easily-readable font, and should have the focus with a flashing cursor when appropriate. Entering non-standard symbols, in particular mathematical expressions, presents a challenge. There are some well-tried ways of dealing with this: informal entry using pocket-calculator conventions, LaTeX markup, a palette of standard symbols that can be dragged into the answer box. CAA software is unforgiving when trying to make sense of the kind of informal entry easily understood by humans, and so rigorous adherence to correct mathematical syntax (brackets, arithmetical operations, functional notation) is usually required. (For instance, WeBWorK is fairly tolerant of informal entry and includes a summary of user syntax in a pane on the right-hand side of its pages.)

• Recording Progress: There are usually several stages in answering a question online: (1) entering the answer in the appropriate box(es); (2) validating the answer to check that the program correctly interprets it (especially if symbolic expressions are involved); (3) saving the answer (often combined with validation); (4) reviewing the answer and editing it; (5) submitting the answer for marking/grading; (6) making further attempts if allowed; (7) submitting the final attempt. It is important for this progress data to be displayed in a table on every page of the assessment, with direct navigation to uncompleted questions. It is also helpful to record individual question and total scores in this table and to display ”time remaining” in say minutes and an analogue clock.

• Training and In-Test Help: It is desirable to give students a practice assignment in conjunction with an online tutorial to familiarise them with the assessment format and the syntax for entering symbolic notation. This can be delivered in advance of the test or as an initial part of it. A summary of this user guidance should be easily accessible at any stage of the test, perhaps through a help-box or in a separate pane of the test window.

• Accessibility: Here is a short checklist of desirable feature for optimising access to web pages: (1) user control of font styles and sizes (especially important for the display of mathematics, which may be embedded as graphics); (2) text equivalents for graphics and multimedia; (3) simple and logical navigation; (4) control over text and background colour. (5) Compatibility with a screen-reader that handles mathematics and other symbolic notation (programs now exist to read mathematics that is coded in MathML – e.g. Design Science’s MathPlayer: see http://www.dessci.com/en/products/mathplayer/tech/accessibility.htm). Entering mathematical answers is particularly difficult for visually-impaired users and so an intelligent screen-reader for validation of answers would provide helpful reassurance.

: 29 Jan 2007 12:32 |  Tags: CAA Computer Aided Assessment Software |  Comments (0) |  Close comments |  Report a problem