All 1 entries tagged Mrqs

No other Warwick Blogs use the tag Mrqs on entries | View entries tagged Mrqs at Technorati | There are no images tagged Mrqs on this blog

February 06, 2006

I got it wrong!

Christopher Munro, who is working with Michael McCabe on the SEXPOT Project, has kindly pointed out that my interpretation of the following "partial grading" formula (for multiple-response questions in software package B)

was wrong. The reality is even more bizarre.

To explore the implications of this formula, let's lay down some ground rules for multiple-response questions (MRQs) and look at a concrete example.

Rule 1: The statements that make up the choices in are either true or false (although Martin Greenhow told me recently that he is exploring a question type for Mathletics that also allows the answer 'undecidable').

Rule 2: For each part of the MRQ, it is equally difficult to decide whether the statement is true or false (unlike competitions in popular magazines, which often make it humorously obvious which statements are false). I usually aim to satisfy this rule in my MRQs, although I freely acknowledge it's not an exact science and anyway, what is 'hard' varies from student to student.

Rule 3: It is equally likely that a statement is true or false (so that, on average for a four-part question, one question in sixteen has all parts false).

In the above formula the "# of correct answers" actually means the number of true statements in the question.

For our concrete example, take a four-part MRQ and a model student who gets all four choices correct. If, say, just one of the four statements is true, the student scores (4–0)/1 = 4 marks. On the other hand, if 3 statements are true and one is false, the student only gets (4–0)/3 = 4/3 marks for being just as clever as before. Maybe I am missing something, but that strikes me as a daft scoring system.

And for the one question in sixteen where all four answers are false, the student scores

$\frac{4-0}{0}=\infty$

Luckily here, the software under discussion violates Rule 3 by insisting that at least one statement is chosen to be true!

April 2023

Mo Tu We Th Fr Sa Su
Mar |  Today  |
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30

Galleries

• The LaTeX equations are a little wobbly in their baselines That's because they use mimeTeX. I am try… by Steve Mayer on this entry
• Aha! You are right, John. I am familiar with a different terminology: Multiple Choice for what Quizb… by on this entry
• Haven't you used the wrong question type, Trevor? Your answers look, to my unskilled eye, to be mutu… by John Dale on this entry
• The usual factors in Information Assurance are: Confidentiality Integrity Availability And for syste… by Max Hammond on this entry
• Is the workshop next Monday,26 March 07, open to anybody interested in computer aided assessment? If… by Kuldeep Singh on this entry