Is the Climate Changing on Climate Change?

By now you might have heard about the e-mails and other information from the East Anglia University Climate Research Unit that was hacked and put out on the internet. The MSM has been trying to avoid discussing the matter, with one notable and honorable exception: Declan McCullagh of CBS News. He moved the discussion forward in a big way yesterday:

Last week's leaked e-mails range from innocuous to embarrassing and, critics believe, scandalous. They show that some of the field's most prominent scientists were so wedded to theories of man-made global warming that they ridiculed dissenters who asked for copies of their data ("have to respond to more crap criticisms from the idiots"), cheered the deaths of skeptical journalists, and plotted how to keep researchers who reached different conclusions from publishing in peer-reviewed journals.

Cheering the death of skeptical journalists is bad form. Trying to keep contrary research out of peer-reviewed journals is something else entirely. One of the keys to the case of those who support the AGW theory is that there isn't any dissent and that the science is settled. Al Gore has made that argument, as have many others.

So why would people who believe in scientific inquiry try to stop the work of others from seeing the light of day? As McCullagh explains, some of the key research is cobbled together in a way that cannot be duplicated, a key to demonstrating the soundness of any scientific theory. He picks up the narrative of an East Anglia computer programmer named "Harry," who sets out his complicated tale of woe:

I am seriously worried that our flagship gridded data product is produced by Delaunay triangulation - apparently linear as well. As far as I can see, this renders the station counts totally meaningless. It also means that we cannot say exactly how the gridded data is arrived at from a statistical perspective - since we're using an off-the-shelf product that isn't documented sufficiently to say that. Why this wasn't coded up in Fortran I don't know - time pressures perhaps? Was too much effort expended on homogenisation, that there wasn't enough time to write a gridding procedure? Of course, it's too late for me to fix it too. Meh.

I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that's the case? Aarrggghhh! There truly is no end in sight... So, we can have a proper result, but only by including a load of garbage!

One thing that's unsettling is that many of the assigned WMo codes for Canadian stations do not return any hits with a web search. Usually the country's met office, or at least the Weather Underground, show up – but for these stations, nothing at all. Makes me wonder if these are long-discontinued, or were even invented somewhere other than Canada!

Knowing how long it takes to debug this suite - the experiment endeth here. The option (like all the anomdtb options) is totally undocumented so we'll never know what we lost. 22. Right, time to stop pussyfooting around the niceties of Tim's labyrinthine software suites - let's have a go at producing CRU TS 3.0! since failing to do that will be the definitive failure of the entire project.

Ulp! I am seriously close to giving up, again. The history of this is so complex that I can't get far enough into it before by head hurts and I have to stop. Each parameter has a tortuous history of manual and semi-automated interventions that I simply cannot just go back to early versions and run the update prog. I could be throwing away all kinds of corrections - to lat/lons, to WMOs (yes!), and more. So what the hell can I do about all these duplicate stations?...

As anyone who has ever worked with statistical analysis knows, one of the keys is getting past the noise in the numbers. It is exceedingly rare that you can get an entirely pure data set, but this description points to a number of highly red flags, especially the business about "manual and semi-automated interventions" and the lack of corroboration of the Canadian data and the suspicion of duplicate data in the set.

It's important to understand -- this data set is the basis for much of what we've been told is definitive about AGW. If there's this much noise in the numbers, we certainly shouldn't be relying on this data set. And we definitely shouldn't be basing an entire regulatory scheme (Cap and Trade) on it.

This is going to get very interesting.