All entries for April 2007

April 27, 2007

Quizbuilder's here

Warwick’s Elab has published the first draft of Quizbuilder, its elegant tool for writing simple online tests with the minimum of fuss. You might like to try your hand at this short test of 11 multiple-choice questions on elementary number theory, which took me less than an hour to write. The LaTeX equations are a little wobbly in their baselines, but perfectly fit for purpose.

OU = Open Utopia?

As part of its nearly-£5m investment in Moodle, the free open-source course management system (CMS, aka VLE), the Open University (OU) is currently adding its in-house assessment software OpenMark to Moodle’s assessment capabilities. It will also incorporate some of the authoring strengths of OpenMark into the Moodle Quiz. The full integration may take some time to complete but will mean that OpenMark becomes open source too.

OpenMark’s strengths include the ability to diplay complicated mathematical and symbolic expressions and to provide graduated targeted feedback in response to multiple attempts at variations of the same question. Given the OU’s high production standards and long term funding, this development can only bode well for the future of online assessment, in particular, the assessment of mathematics-based subjects.

April 24, 2007

CAA Fitness for Purpose: Data Security and Robustness

Follow-up to Fit for Purpose? from Computer-aided assessment for sciences

Data Security

Three kinds of data need to be kept safe: (i) the questions stored for a test; (ii) student answers entered during a test; (ii) submitted answers and results.

  • Keeping tests safe: It is clearly important to keep tests, questions, solutions and feedback safe from prying eyes, especially if they are to be used in summative mode. So the question database should be encrypted or otherwise made hacker-proof. It should also be regularly backed up in case of hardware failure (having lost questions on a hosted service, I would strongly advise authors to back up their work locally too). If a degree of randomness is introduced to reduce the risk of cheating (via multiple question banks or question templates with parameters, say), then thought should be given to the ease with which determined students could circumvent the protection thus provided (see this blog entry, for instance).
  • In-test Security: Some assessment software allows submission of answers to be postponed until the end of the test. This is dangerous. A user who has entered 9 out of 10 answers when the system crashes without saving them would have every reason to be angry. My preferred option is to require an entered answer to be validated (and simultaneously saved) before the user is allowed to proceed to the next question (or at least to be warned that they may lose their answer if they do not validate before moving on). Validation allows the software to interpret the answer and return its interpretation to the user in a standard form; it is an important stage in dealing with answers to questions with symbolic content, where the machine may not be able to cope with the informal context-dependent representation humans used to. Another kind of security involves limiting cheating during a test: impersonation, or copying from a neighbour, for example. Invigilation is still the safest answer to this.
  • Securing Results: The most important thing about the results database, apart from the obvious needs to be backed up and proof against hacking, it that it should store every bit of activity engaged in by a student during a test. If a student challenges their test outcome, the examiner needs to be able to trace every step the student took, including false validations, inappropriate mouse clicks (some assessment software swoons at the click of a browser back button). and the relaunching of the test. Although it is a good idea to insist that students jot their work down on paper during a test, this is not much help if a system fault requires a new test to be generated and it comes with different values of th random parameters; When parameters are used, the system should also be able to deliver the same test to a student who, through no fault of their own, is forced to make a fresh start. As I have said elsewhere, it is a great help if the database fields are chosen to optimise efficiency and flexibility in searching the results.

April 2007

Mo Tu We Th Fr Sa Su
Mar |  Today  | May
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29

Search this blog



Most recent comments

  • The LaTeX equations are a little wobbly in their baselines That's because they use mimeTeX. I am try… by Steve Mayer on this entry
  • Aha! You are right, John. I am familiar with a different terminology: Multiple Choice for what Quizb… by on this entry
  • Haven't you used the wrong question type, Trevor? Your answers look, to my unskilled eye, to be mutu… by John Dale on this entry
  • The usual factors in Information Assurance are: Confidentiality Integrity Availability And for syste… by Max Hammond on this entry
  • Is the workshop next Monday,26 March 07, open to anybody interested in computer aided assessment? If… by Kuldeep Singh on this entry

Blog archive

Not signed in
Sign in

Powered by BlogBuilder