All entries for Tuesday 13 December 2005

December 13, 2005

Prediction failure

Follow-up to Probability neglect from Autology: John Dale's blog

I was thinking yesterday about how people aren't very good at considering probability when they think about future events, and coincidentally today I've been reading about some pleasing research showing that people – especially "experts" – are in fact generally pretty rubbish at prediction generally.

This review in the New Yorker of Philip Tetlock’s new book, Expert Political Judgment: How Good Is It? How Can We Know? talks about the surprisingly large number of areas in which experts on a subject do no better in the accuracy of their predictions than people who know nothing about the subject, or even blind chance.

(As an aside, I love the New Yorker. Pretty much everyone who writes in it manages to be entertaining and witty, and for reasons I don't fully understand, they also seem to consistently write about things I find interesting. Malcolm Gladwell's articles are invariably great (must write about him in a separate entry at some point), and Anthony Lane is an insightful and funny film critic: talking about the new Narnia movie, he says "The one thing delaying any attempt to film the Narnia novels was the lack of technology; until recently, for example, there was no computer-imaging program powerful enough to re-create a wholly convincing wardrobe". I wonder how much international subscriptions cost?)

Where was I? Oh yes; experts are rubbish at predictions. Philip Tetlock's book is full of striking examples: data from a test used to diagnose brain damage were given to a group of clinical psychologists and their secretaries. The psychologists’ diagnoses were no better than the secretaries’. Political forecasters asked to choose between three predictive alternatives did worse than chance. And students were bested by lab rats because they over-thought the problem:-

A rat was put in a T-shaped maze. Food was placed in either the right or the left transept of the T in a random sequence such that, over the long run, the food was on the left sixty per cent of the time and on the right forty per cent. Neither the students nor (needless to say) the rat was told these frequencies. The students were asked to predict on which side of the T the food would appear each time. The rat eventually figured out that the food was on the left side more often than the right, and it therefore nearly always went to the left, scoring roughly sixty per cent — D, but a passing grade. The students looked for patterns of left-right placement, and ended up scoring only fifty-two per cent, an F. The rat, having no reputation to begin with, was not embarrassed about being wrong two out of every five tries. But Yale students, who do have reputations, searched for a hidden order in the sequence. They couldn’t deal with forty-per-cent error, so they ended up with almost fifty-per-cent error.

And if you read the article, it's instructive to consider whether you're a hedgehog or a fox.

Copulating deaf couple unaware of own volume

Writing about web page

Monday night, a record number of noise complaints were received by Residential Security Officers in Roger Revelle College. Officers responding to the calls found the sexual activity of a deaf couple to be the source of the noises, which were described as "cacophonous" by witnesses.

Is it just me or is this hilarious?

Alexa for sale

Writing about web page

John Battelle writes about an intriguing new idea from Amazon; they're making their Alexa index available for anyone to do anything they want with it. 5 billion documents or so, 100 terabytes of data, and you pay not a subscription or a licence fee, but a consumption fee – $1 per CPU hour consumed. $1 per gig of storage used, $1 per 50 gigs of data processed. Your code runs on their servers and the data you generate or retrieve also sits on their servers. It's like 1975 and mainframes all over again, except you've got much more interesting stuff to play with!

What could you do with unfettered access to the actual index, rather than an API such as Google or Yahoo offer? We'll find out soon enough, I guess.

Compulsory online learning

Writing about web page

According to this article in the Chronicle, the Michigan State Board of Education will approve a new graduation requirement today requiring every high-school student in the state to take at least one online course before receiving their diploma. Not because online learning necessarily offers a richer or better way to study, though:-

Mike Flanagan, the Michigan state superintendent of public instruction, said he proposed the online-course requirement, along with other general requirements, to make sure students were prepared for college and for jobs, which are becoming more technology-focused. "We don't want our kids left in the global dust," Mr. Flanagan said. "It's an experience we need to have."

Interesting idea. Online learning not because it's a better way to learn geography or chemistry or whatever, but because you need to know how to do things online.

Search this blog


Blog archive


Most recent comments

  • I'm looking for two authors/books from the 1970s or early 80s. The first was set in England. All I r… by Leo on this entry
  • I'm looking for two authors/books from the 1970s or early 80s. The first was set in England. All I r… by Leo on this entry
  • I am trying to find a book about a grandfather who tells his grandson that if he imagines hard enoug… by hilary woolf on this entry
  • Hi Looking for a series of books in which the main character was a knight. The knight was either a f… by Ely McKenna on this entry
  • I'm trying to track down my favourite children's book from the early 1970s (pub. 1970 – 1973?) about… by Ally Holloway on this entry
Not signed in
Sign in

Powered by BlogBuilder