All 1 entries tagged <em>Mrqs</em>I expect to be leading a 12 month project promoting computer-aided assessment in the Warwick Science Faculty in 2005-06. Would like to keep a public diary on Project activitieshttps://blogs.warwick.ac.uk/caa4s/tag/mrqs/?atom=atomWarwick Blogs, University of Warwick(C) 20212021-01-25T04:34:09ZI got it wrong! byhttps://blogs.warwick.ac.uk/caa4s/entry/i_got_it_1/2006-02-08T14:02:59Z2006-02-06T09:59:28Z<p class="answer">Follow-up to <a href="https://blogs.warwick.ac.uk/caa4s/entry/unresponsive_multiple-response/" title="Related blog entry">Unresponsive Multiple–Response</a> from <a href="https://blogs.warwick.ac.uk/caa4s"> Computer–aided assessment for sciences</a></p>
<p>Christopher Munro, who is working with Michael McCabe on the <a href="http://blogs.warwick.ac.uk/caa4s/entry/sexpot/" title="">SEXPOT Project</a>, has kindly pointed out that my interpretation of the following "partial grading" formula (for multiple-response questions in software package B)<br />
<img style="padding-right:5px;padding-left:5px;" src="http://blogs.warwick.ac.uk/images/caa4s/2006/02/03/mta_grade_formula.png?maxWidth=500" alt='Grade Formula' title='Grade Formula' /><br />
was wrong. The reality is even more bizarre.</p>
<p>To explore the implications of this formula, let's lay down some ground rules for multiple-response questions (MRQs) and look at a concrete example.</p>
<p><strong>Rule 1:</strong> The statements that make up the choices in are either true or false (although Martin Greenhow told me recently that he is exploring a question type for <em>Mathletics</em> that also allows the answer 'undecidable').</p>
<p><strong>Rule 2:</strong> For each part of the <span class="caps">MRQ</span>, it is equally difficult to decide whether the statement is true or false (unlike competitions in popular magazines, which often make it humorously obvious which statements are false). I usually aim to satisfy this rule in my MRQs, although I freely acknowledge it's not an exact science and anyway, what is 'hard' varies from student to student.</p>
<p><strong>Rule 3:</strong> It is equally likely that a statement is true or false (so that, on average for a four-part question, one question in sixteen has all parts false).</p>
<p>In the above formula the "# of correct answers" actually means <em>the number of true statements in the question.</em></p>
<p>For our concrete example, take a four-part <span class="caps">MRQ</span> and a model student who gets all four choices correct. If, say, just one of the four statements is true, the student scores (4–0)/1 = 4 marks. On the other hand, if 3 statements are true and one is false, the student only gets (4–0)/3 = 4/3 marks for being just as clever as before. Maybe I am missing something, but that strikes me as a daft scoring system.</p>
<p>And for the one question in sixteen where all four answers are false, the student scores</p>
<blockquote></blockquote><img class="latex" src="http://blogs.warwick.ac.uk/cgi-bin/mimetex.cgi?\frac{4-0}{0}=\infty" alt="\frac{4-0}{0}=\infty"><p> </p>
<p>Luckily here, the software under discussion violates Rule 3 by insisting that at least one statement is chosen to be true!</p><p class="answer">Follow-up to <a href="https://blogs.warwick.ac.uk/caa4s/entry/unresponsive_multiple-response/" title="Related blog entry">Unresponsive Multiple–Response</a> from <a href="https://blogs.warwick.ac.uk/caa4s"> Computer–aided assessment for sciences</a></p>
<p>Christopher Munro, who is working with Michael McCabe on the <a href="http://blogs.warwick.ac.uk/caa4s/entry/sexpot/" title="">SEXPOT Project</a>, has kindly pointed out that my interpretation of the following "partial grading" formula (for multiple-response questions in software package B)<br />
<img style="padding-right:5px;padding-left:5px;" src="http://blogs.warwick.ac.uk/images/caa4s/2006/02/03/mta_grade_formula.png?maxWidth=500" alt='Grade Formula' title='Grade Formula' /><br />
was wrong. The reality is even more bizarre.</p>
<p>To explore the implications of this formula, let's lay down some ground rules for multiple-response questions (MRQs) and look at a concrete example.</p>
<p><strong>Rule 1:</strong> The statements that make up the choices in are either true or false (although Martin Greenhow told me recently that he is exploring a question type for <em>Mathletics</em> that also allows the answer 'undecidable').</p>
<p><strong>Rule 2:</strong> For each part of the <span class="caps">MRQ</span>, it is equally difficult to decide whether the statement is true or false (unlike competitions in popular magazines, which often make it humorously obvious which statements are false). I usually aim to satisfy this rule in my MRQs, although I freely acknowledge it's not an exact science and anyway, what is 'hard' varies from student to student.</p>
<p><strong>Rule 3:</strong> It is equally likely that a statement is true or false (so that, on average for a four-part question, one question in sixteen has all parts false).</p>
<p>In the above formula the "# of correct answers" actually means <em>the number of true statements in the question.</em></p>
<p>For our concrete example, take a four-part <span class="caps">MRQ</span> and a model student who gets all four choices correct. If, say, just one of the four statements is true, the student scores (4–0)/1 = 4 marks. On the other hand, if 3 statements are true and one is false, the student only gets (4–0)/3 = 4/3 marks for being just as clever as before. Maybe I am missing something, but that strikes me as a daft scoring system.</p>
<p>And for the one question in sixteen where all four answers are false, the student scores</p>
<blockquote></blockquote><img class="latex" src="http://blogs.warwick.ac.uk/cgi-bin/mimetex.cgi?\frac{4-0}{0}=\infty" alt="\frac{4-0}{0}=\infty"><p> </p>
<p>Luckily here, the software under discussion violates Rule 3 by insisting that at least one statement is chosen to be true!</p>