All entries for Monday 30 January 2006
January 30, 2006
Follow-up to Fruit Flies Like An Apple … from Computer-aided assessment for sciencesMichael McCabe at the University of Portsmouth is giving some rigorous objective thought to marking schemes for objective tests in his SEXPOT Project (Scoring EXemplars and Principles of Objective Testing). Have a look at his very preliminary findings.
Fruit Flies Like An Apple …
… time flies like an arrow, which brings me neatly on to the theory of social choice functions (better known to most of us as 'voting systems'). Arrow's Theorem states, in a nutshell, that the only voting system that satisfies three very plausible 'fairness criteria' (for example, if more voters prefer X to Y, then X should appear above Y in the final list) is a dictatorship, where one person gets to decide for everyone else.
Could there be an analogue of Arrow's Theorem for marking objective tests? Is there a marking system that is fair to all? To answer this, one first needs a list of fairness criteria. Any suggestions for these? I'll start the ball rolling with
Criterion 1: If student X 'knows more' than student Y, then X should score more than Y. Of course, the examiner has to specify what is meant by 'knows more'.