January 30, 2006

Fruit Flies Like An Apple …

… time flies like an arrow, which brings me neatly on to the theory of social choice functions (better known to most of us as 'voting systems'). Arrow's Theorem states, in a nutshell, that the only voting system that satisfies three very plausible 'fairness criteria' (for example, if more voters prefer X to Y, then X should appear above Y in the final list) is a dictatorship, where one person gets to decide for everyone else.

Could there be an analogue of Arrow's Theorem for marking objective tests? Is there a marking system that is fair to all? To answer this, one first needs a list of fairness criteria. Any suggestions for these? I'll start the ball rolling with

Criterion 1: If student X 'knows more' than student Y, then X should score more than Y. Of course, the examiner has to specify what is meant by 'knows more'.

- 4 comments by 1 or more people Not publicly viewable

  1. Arrow assumes that everybody has voting preferences that can be expressed on a one-dimensional scale, so for every pair of candidates x and y, we have either xRy or yRx. Even having noted your preemptive defence of inverted commas, surely the same cannot be said for any reasonable understanding of the relation 'knows more'?

    31 Jan 2006, 23:44

  2. Yes, you have hit the nail bang on the head, Matthew. Knowledge has no bounds. Knowledge is in the eye of the beholder. Even if you could reasonably define the relation 'knows more', you cannot represent it by an ordered list.

    Neverthless, in scoring a test that's exactly what we try to do. So is there a way of identifying and strictly limiting components of knowledge/understanding, and giving them a weighting through the marking system, to ensure that the scores on the test have a sensible meaning, or at least provide a useful comparison?

    Theorists do you worst (if you haven't already done so)!

    01 Feb 2006, 09:44

  3. Vinayak Dalmia

    Your criterion 1 sounds somewhat like the weak pareto condition in arrows theorem.

    Another suggestion – the condition of independence of irrelevant alternatives implies objectivity in the marking scheme.

    30 Aug 2006, 18:14

  4. vinayak dalmia

    By objectivity I mean that if one compares say knowledge of two students – that ranking should not be affected by the knowledge of another third student

    30 Aug 2006, 18:16

Add a comment

You are not allowed to comment on this entry as it has restricted commenting permissions.

January 2006

Mo Tu We Th Fr Sa Su
Dec |  Today  | Feb
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31               

Search this blog



Most recent comments

  • The LaTeX equations are a little wobbly in their baselines That's because they use mimeTeX. I am try… by Steve Mayer on this entry
  • Aha! You are right, John. I am familiar with a different terminology: Multiple Choice for what Quizb… by on this entry
  • Haven't you used the wrong question type, Trevor? Your answers look, to my unskilled eye, to be mutu… by John Dale on this entry
  • The usual factors in Information Assurance are: Confidentiality Integrity Availability And for syste… by Max Hammond on this entry
  • Is the workshop next Monday,26 March 07, open to anybody interested in computer aided assessment? If… by Kuldeep Singh on this entry

Blog archive

Not signed in
Sign in

Powered by BlogBuilder