CAA Fitness for Purpose: Data Security and Robustness
Follow-up to Fit for Purpose? from Computer-aided assessment for sciences
Data Security
Three kinds of data need to be kept safe: (i) the questions stored for a test; (ii) student answers entered during a test; (ii) submitted answers and results.
- Keeping tests safe: It is clearly important to keep tests, questions, solutions and feedback safe from prying eyes, especially if they are to be used in summative mode. So the question database should be encrypted or otherwise made hacker-proof. It should also be regularly backed up in case of hardware failure (having lost questions on a hosted service, I would strongly advise authors to back up their work locally too). If a degree of randomness is introduced to reduce the risk of cheating (via multiple question banks or question templates with parameters, say), then thought should be given to the ease with which determined students could circumvent the protection thus provided (see this blog entry, for instance).
- In-test Security: Some assessment software allows submission of answers to be postponed until the end of the test. This is dangerous. A user who has entered 9 out of 10 answers when the system crashes without saving them would have every reason to be angry. My preferred option is to require an entered answer to be validated (and simultaneously saved) before the user is allowed to proceed to the next question (or at least to be warned that they may lose their answer if they do not validate before moving on). Validation allows the software to interpret the answer and return its interpretation to the user in a standard form; it is an important stage in dealing with answers to questions with symbolic content, where the machine may not be able to cope with the informal context-dependent representation humans used to. Another kind of security involves limiting cheating during a test: impersonation, or copying from a neighbour, for example. Invigilation is still the safest answer to this.
- Securing Results: The most important thing about the results database, apart from the obvious needs to be backed up and proof against hacking, it that it should store every bit of activity engaged in by a student during a test. If a student challenges their test outcome, the examiner needs to be able to trace every step the student took, including false validations, inappropriate mouse clicks (some assessment software swoons at the click of a browser back button). and the relaunching of the test. Although it is a good idea to insist that students jot their work down on paper during a test, this is not much help if a system fault requires a new test to be generated and it comes with different values of th random parameters; When parameters are used, the system should also be able to deliver the same test to a student who, through no fault of their own, is forced to make a fresh start. As I have said elsewhere, it is a great help if the database fields are chosen to optimise efficiency and flexibility in searching the results.
Max Hammond
The usual factors in Information Assurance are:
And for systems such as this, one might sensibly include:
Each of those aspects should be considered for the system as a whole, and for each class of data within it. Consider the threat actors, consider the impact of any given type of compromise, and consider the types of countermeasures which would help mitigate the risk.
Next, consider how you can decide if your system has an adequate risk associated with it. What is your risk appetite? What’s the threat?
How hacker-proof? Which hackers? What access will they have to the system? Over what period? How motivated are they likely to be to attack? How skilled are they? Is it a bored student, or is it organized crime?
How do you know it’s “hacker-proof”? Who’s going to do the audit/accreditation?
24 Apr 2007, 23:25
Add a comment
You are not allowed to comment on this entry as it has restricted commenting permissions.