All entries for April 2007

Quizbuilder's here

Warwick’s Elab has published the first draft of Quizbuilder, its elegant tool for writing simple online tests with the minimum of fuss. You might like to try your hand at this short test of 11 multiple-choice questions on elementary number theory, which took me less than an hour to write. The LaTeX equations are a little wobbly in their baselines, but perfectly fit for purpose.

: 27 Apr 2007 17:34 |  Tags: CAA |  Comments (3) |  Close comments |  Report a problem

OU = Open Utopia?

As part of its nearly-£5m investment in Moodle, the free open-source course management system (CMS, aka VLE), the Open University (OU) is currently adding its in-house assessment software OpenMark to Moodle’s assessment capabilities. It will also incorporate some of the authoring strengths of OpenMark into the Moodle Quiz. The full integration may take some time to complete but will mean that OpenMark becomes open source too.

OpenMark’s strengths include the ability to diplay complicated mathematical and symbolic expressions and to provide graduated targeted feedback in response to multiple attempts at variations of the same question. Given the OU’s high production standards and long term funding, this development can only bode well for the future of online assessment, in particular, the assessment of mathematics-based subjects.

: 27 Apr 2007 16:42 |  Tags: CAA |  Comments (0) |  Close comments |  Report a problem

CAA Fitness for Purpose: Data Security and Robustness

Follow-up to Fit for Purpose? from Computer-aided assessment for sciences

Data Security

Three kinds of data need to be kept safe: (i) the questions stored for a test; (ii) student answers entered during a test; (ii) submitted answers and results.

• Keeping tests safe: It is clearly important to keep tests, questions, solutions and feedback safe from prying eyes, especially if they are to be used in summative mode. So the question database should be encrypted or otherwise made hacker-proof. It should also be regularly backed up in case of hardware failure (having lost questions on a hosted service, I would strongly advise authors to back up their work locally too). If a degree of randomness is introduced to reduce the risk of cheating (via multiple question banks or question templates with parameters, say), then thought should be given to the ease with which determined students could circumvent the protection thus provided (see this blog entry, for instance).

• In-test Security: Some assessment software allows submission of answers to be postponed until the end of the test. This is dangerous. A user who has entered 9 out of 10 answers when the system crashes without saving them would have every reason to be angry. My preferred option is to require an entered answer to be validated (and simultaneously saved) before the user is allowed to proceed to the next question (or at least to be warned that they may lose their answer if they do not validate before moving on). Validation allows the software to interpret the answer and return its interpretation to the user in a standard form; it is an important stage in dealing with answers to questions with symbolic content, where the machine may not be able to cope with the informal context-dependent representation humans used to. Another kind of security involves limiting cheating during a test: impersonation, or copying from a neighbour, for example. Invigilation is still the safest answer to this.

• Securing Results: The most important thing about the results database, apart from the obvious needs to be backed up and proof against hacking, it that it should store every bit of activity engaged in by a student during a test. If a student challenges their test outcome, the examiner needs to be able to trace every step the student took, including false validations, inappropriate mouse clicks (some assessment software swoons at the click of a browser back button). and the relaunching of the test. Although it is a good idea to insist that students jot their work down on paper during a test, this is not much help if a system fault requires a new test to be generated and it comes with different values of th random parameters; When parameters are used, the system should also be able to deliver the same test to a student who, through no fault of their own, is forced to make a fresh start. As I have said elsewhere, it is a great help if the database fields are chosen to optimise efficiency and flexibility in searching the results.

: 24 Apr 2007 13:25 |  Tags: CAA Computer Aided Assessment Software |  Comments (1) |  Close comments |  Report a problem