April 24, 2007

CAA Fitness for Purpose: Data Security and Robustness

Follow-up to Fit for Purpose? from Computer-aided assessment for sciences

Data Security

Three kinds of data need to be kept safe: (i) the questions stored for a test; (ii) student answers entered during a test; (ii) submitted answers and results.

  • Keeping tests safe: It is clearly important to keep tests, questions, solutions and feedback safe from prying eyes, especially if they are to be used in summative mode. So the question database should be encrypted or otherwise made hacker-proof. It should also be regularly backed up in case of hardware failure (having lost questions on a hosted service, I would strongly advise authors to back up their work locally too). If a degree of randomness is introduced to reduce the risk of cheating (via multiple question banks or question templates with parameters, say), then thought should be given to the ease with which determined students could circumvent the protection thus provided (see this blog entry, for instance).
  • In-test Security: Some assessment software allows submission of answers to be postponed until the end of the test. This is dangerous. A user who has entered 9 out of 10 answers when the system crashes without saving them would have every reason to be angry. My preferred option is to require an entered answer to be validated (and simultaneously saved) before the user is allowed to proceed to the next question (or at least to be warned that they may lose their answer if they do not validate before moving on). Validation allows the software to interpret the answer and return its interpretation to the user in a standard form; it is an important stage in dealing with answers to questions with symbolic content, where the machine may not be able to cope with the informal context-dependent representation humans used to. Another kind of security involves limiting cheating during a test: impersonation, or copying from a neighbour, for example. Invigilation is still the safest answer to this.
  • Securing Results: The most important thing about the results database, apart from the obvious needs to be backed up and proof against hacking, it that it should store every bit of activity engaged in by a student during a test. If a student challenges their test outcome, the examiner needs to be able to trace every step the student took, including false validations, inappropriate mouse clicks (some assessment software swoons at the click of a browser back button). and the relaunching of the test. Although it is a good idea to insist that students jot their work down on paper during a test, this is not much help if a system fault requires a new test to be generated and it comes with different values of th random parameters; When parameters are used, the system should also be able to deliver the same test to a student who, through no fault of their own, is forced to make a fresh start. As I have said elsewhere, it is a great help if the database fields are chosen to optimise efficiency and flexibility in searching the results.

- One comment Not publicly viewable

  1. Max Hammond

    The usual factors in Information Assurance are:

    • Confidentiality
    • Integrity
    • Availability

    And for systems such as this, one might sensibly include:

    • Authentication
    • Non-repudiation

    Each of those aspects should be considered for the system as a whole, and for each class of data within it. Consider the threat actors, consider the impact of any given type of compromise, and consider the types of countermeasures which would help mitigate the risk.

    Next, consider how you can decide if your system has an adequate risk associated with it. What is your risk appetite? What’s the threat?

    So the question database should be encrypted or otherwise made hacker-proof.

    How hacker-proof? Which hackers? What access will they have to the system? Over what period? How motivated are they likely to be to attack? How skilled are they? Is it a bored student, or is it organized crime?

    How do you know it’s “hacker-proof”? Who’s going to do the audit/accreditation?

    24 Apr 2007, 23:25


Add a comment

You are not allowed to comment on this entry as it has restricted commenting permissions.

Trackbacks

April 2007

Mo Tu We Th Fr Sa Su
Mar |  Today  | May
                  1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30                  

Search this blog

Tags

Galleries

Most recent comments

  • The LaTeX equations are a little wobbly in their baselines That's because they use mimeTeX. I am try… by Steve Mayer on this entry
  • Aha! You are right, John. I am familiar with a different terminology: Multiple Choice for what Quizb… by on this entry
  • Haven't you used the wrong question type, Trevor? Your answers look, to my unskilled eye, to be mutu… by John Dale on this entry
  • The usual factors in Information Assurance are: Confidentiality Integrity Availability And for syste… by Max Hammond on this entry
  • Is the workshop next Monday,26 March 07, open to anybody interested in computer aided assessment? If… by Kuldeep Singh on this entry

Blog archive

Loading…
Not signed in
Sign in

Powered by BlogBuilder
© MMXIV