All entries for Wednesday 14 December 2005

December 14, 2005

Teaching Techniques: confidence based assessment as a solution to prediction failure

Writing about an entry you don't have permission to view

In a recent blog entry, John Dale discussed the poor performance of so called "experts" in making predictions.

There's an important point in this for university teaching, especially in subjects in which accurate prediction is paramount. Students must learn to assess the certainty of their own judgements. As Chris May points out, the problem is that "experts aren't very good at communicating the uncertainty of their predictions" – which probably means that they aren't very good at objectifying the degree of certainty to themselves.

I recently saw a technique demonstrated that helps with this. It is called "confidence based assessment", and is used in some medical schools. The student is given a series of questions. Obviously they must try to get the right answer, but more importantly they are asked to assess the confidence with which they give each answer. The answer/confidence combination works along these lines

  • If a student gives the wrong answer confidently, that recieves the worst possible grade;
  • If they give the wrong answer but are not confident, then they get a better grade;
  • Giving the right answer but without confidence is OK, but not ideal, as in reality it could end up with them waisting time having to consult others;
  • The best answer is that which is correct and made with confidence.

The outcome of this should be that the student knows when to consult others (or text books etc), they know when not to act independently. They also know when they can make a quick and precise decision, thus acting correctly and efficiently.

This is all very well, but exactly how do you equip someone with the skills to be confident when justified or doubtful when necessary? This is of course the predominant interest of a branch of philosophy called epistemology (or philosophy of science in a more restricted form).

Conclusion (not entirely justified): medical students must do philosophy.


: 14 Dec 2005 15:46 |  Tags:   |  Comments (0) |  Report a problem