Occam-pational Hazards

Yes, it's time for Occam's Razor again. Let's start with a question for all of us evil AGW "Deniers": do you really think there's a massive conspiracy to falsify data and that the East Anglia CRU is some sort of criminal cabal, conspiring with other climate scientists?

I can't speak for anyone else, but my answer is: no. It's not likely that such a conspiracy could really work. However, there are other possible explanations. Megan McArdle at the Atlantic is on the right track:

I can imagine a sort of selection bias in the grant process. I cannot imagine hundreds of scientists thinking, well, I put ten years into getting my PhD--time to spend the rest of my life faking data in order to get some grant money! One, yes. All of them, no.

To me, the worry is the subtler kind of bias that we indisputably know has led to scientific errors in the past.

McArdle then turns to an example from the great physicist and popular science writer Richard Feynman:

We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It's a little bit off, because he had the incorrect value for the viscosity of air. It's interesting to look at the history of measurements of the charge of the electron, after Millikan. If you plot them as a function of time, you find that one is a little bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher.

Why didn't they discover that the new number was higher right away? It's a thing that scientists are ashamed of--this history--because it's apparent that people did things like this: When they got a number that was too high above Millikan's, they thought something must be wrong--and they would look for and find a reason why something might be wrong. When they got a number closer to Millikan's value they didn't look so hard. And so they eliminated the numbers that were too far off, and did other things like that.

Is that what happened in this case? It's a far more likely explanation than a massive conspiracy. McArdle:

That is the actual worrying question about CRU, and GISS, and the other scientists working on paleoclimate reconstruction: that they may all be calibrating their findings to each other. That when you get a number that looks like CRU, you don't look so hard to figure out whether it's incorrect as you do when you get a number that doesn't look like CRU--and maybe you adjust the numbers you have to look more like the other "known" datasets. There is always a way to find what you're expecting to find if you look hard enough.

Indeed. Read the whole thing.