Fruit Flies Like An Apple …
… time flies like an arrow, which brings me neatly on to the theory of social choice functions (better known to most of us as 'voting systems'). Arrow's Theorem states, in a nutshell, that the only voting system that satisfies three very plausible 'fairness criteria' (for example, if more voters prefer X to Y, then X should appear above Y in the final list) is a dictatorship, where one person gets to decide for everyone else.
Could there be an analogue of Arrow's Theorem for marking objective tests? Is there a marking system that is fair to all? To answer this, one first needs a list of fairness criteria. Any suggestions for these? I'll start the ball rolling with
Criterion 1: If student X 'knows more' than student Y, then X should score more than Y. Of course, the examiner has to specify what is meant by 'knows more'.
30 Jan 2006 12:00
| 
Tags: CAA Marking Objective Tests Pedagogy
| 
Comments (4)
| 
Report a problem
Loading…
Arrow assumes that everybody has voting preferences that can be expressed on a one-dimensional scale, so for every pair of candidates x and y, we have either xRy or yRx. Even having noted your preemptive defence of inverted commas, surely the same cannot be said for any reasonable understanding of the relation 'knows more'?
31 Jan 2006, 23:44
Yes, you have hit the nail bang on the head, Matthew. Knowledge has no bounds. Knowledge is in the eye of the beholder. Even if you could reasonably define the relation 'knows more', you cannot represent it by an ordered list.
Neverthless, in scoring a test that's exactly what we try to do. So is there a way of identifying and strictly limiting components of knowledge/understanding, and giving them a weighting through the marking system, to ensure that the scores on the test have a sensible meaning, or at least provide a useful comparison?
Theorists do you worst (if you haven't already done so)!
01 Feb 2006, 09:44
Vinayak Dalmia
Your criterion 1 sounds somewhat like the weak pareto condition in arrows theorem.
Another suggestion – the condition of independence of irrelevant alternatives implies objectivity in the marking scheme.
30 Aug 2006, 18:14
vinayak dalmia
By objectivity I mean that if one compares say knowledge of two students – that ranking should not be affected by the knowledge of another third student
30 Aug 2006, 18:16
Add a comment
You are not allowed to comment on this entry as it has restricted commenting permissions.