Climategate (GW email leak)

[Pangloss]

What do you all think of this? The story "went wide" yesterday with a Washington Post article that actually likened it to the Pentagon Papers, and I counted almost 500 articles in Google News about it today, and there's also a Wikipedia article on it now. I've linked some articles below.

 

The focus seems to be on two areas:

 

1) Some emails seem to show examples of scientists willing to subvert their scientific principles for a political agenda. Does this mean that the consensus on global warming is artificial and political in nature, rather than wholly scientific? Or is it just an example of individuals getting carried away? (But then what IS a social agenda, if it's not individuals leading a trend?)

 

2) Some emails also seem to suggest that important evidence was covered up or downplayed. Has science leaped to the wrong conclusion about global warming, or is this just a blip on the radar?

 

Some articles on the subject:

http://en.wikipedia.org/wiki/Climategate

http://www.washingtonpost.com/wp-dyn/content/article/2009/11/21/AR2009112102186.html?hpid=moreheadlines

http://www.latimes.com/news/nation-and-world/la-fg-climate-hacker22-2009nov22,0,913036.story

http://www.google.com/hostednews/ap/article/ALeqM5j9MrjlmXzORMlHNvYfE9yAlgtiBwD9C4OSH03

 

(I'm still ridiculously swamped with work and study, but I just couldn't resist taking a short break to post something about this. I spend a huge amount of time these days reading peer-reviewed journals, and "dwelling" in that world as much as I can (spending some time with the editor of an award-winning journal the last few weeks), and it's an interesting community, to say the least. For all its power and usefulness, it certainly has its flaws.)

[padren]

I think it's really hard to sort through this without really reading all the leaked information, and with an idea of how much information was there in the first place that was not leaked, ie if it was leaked selectively at all.

My first thought is that most "anti-global warming activists" have very little appreciation for how much debate goes on within the scientific community on all research. Attacking a research paper as flawed that is later released as sound could look like a conspiracy to cover up those flaws - but the question is whether that attack was rebutted and debated until it was considered solid enough to release.

 

 

Lets examine the idea of a paper that contradicts AGW and someone trying to suppress it's publication - which would naturally appear to be a smoking gun to the "Antis" out there:

 

1) Paper contradicts much established evidence

2) The people who wrote it want to publish it

3) People in climatology are

- A) skeptical of the research due to bias (not evidence) and

- B) aware of the political damage that could be done if it is published prematurely

 

Would attempting to keep it from being published then be considered conspiracy, or due diligence in such a case? I would argue if publication was suppressed despite all vetting of the evidence and material pointing to it being sound, then it would be. If there are open questions they want answered first to ensure it's not a political hack job with bad numbers, selective sampling, and all the trimmings of a paper that would be DOA just so some nay-sayers without any regard for the scientific process to cite out of context later then there is nothing wrong with fighting that publication.

 

 

The real sad part is people in the scientific community that want to focus on research, are being forced to play politics. They have to deal with disingenuous individuals trying to "play the system" that if they ignore can damage their solid work and waste their time and energy. When they discuss how to deal with those who are playing the system, they are accused of playing the system themselves.

 

It would take some serious evidence to truly damage their reputations in my mind, perhaps because I am already sympathetic to their situation. That, and I think if you released all the climate change emails from all the "Antis" you'd get a much bigger plume of smoke.

[iNow]

It's pretty silly. If this is all they've got to attack the science, then they've got nothing, really...

 

 

http://www.realclimate.org/index.php/archives/2009/11/the-cru-hack/comment-page-3/

As many of you will be aware, a large number of emails from the University of East Anglia webmail server were hacked recently (

Despite some confusion generated by Anthony Watts, this has absolutely nothing to do with the Hadley Centre which is a completely separate institution

).

 

<...>

 

More interesting is what is not contained in the emails. There is no evidence of any worldwide conspiracy, no mention of George Soros nefariously funding climate research, no grand plan to ‘get rid of the MWP’, no admission that global warming is a hoax, no evidence of the falsifying of data, and no ‘marching orders’ from our socialist/communist/vegetarian overlords. The truly paranoid will put this down to the hackers also being in on the plot though.

 

Instead, there is a peek into how scientists actually interact and the conflicts show that the community is a far cry from the monolith that is sometimes imagined. People working constructively to improve joint publications; scientists who are friendly and agree on many of the big picture issues, disagreeing at times about details and engaging in ‘robust’ discussions; Scientists expressing frustration at the misrepresentation of their work in politicized arenas and complaining when media reports get it wrong; Scientists resenting the time they have to take out of their research to deal with over-hyped nonsense. None of this should be shocking.

 

<...>

 

No doubt, instances of cherry-picked and poorly-worded “gotcha” phrases will be pulled out of context. One example is worth mentioning quickly. Phil Jones in discussing the presentation of temperature reconstructions stated that “I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.” The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all. As for the ‘decline’, it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the “divergence problem”–see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend not using the post 1960 part of their reconstruction, and so while ‘hiding’ is probably a poor choice of words (since it is ‘hidden’ in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.

 

The timing of this particular episode is probably not coincidental. But if cherry-picked out-of-context phrases from stolen personal emails is the only response to the weight of the scientific evidence for the human influence on climate change, then there probably isn’t much to it.

 

 

 

But, wait... There's more:

 

 

http://www.skepticalscience.com/What-do-the-hacked-CRU-emails-tell-us.html

In the skeptic blogosphere, there is a disproportionate preoccupation with one small aspect of climate science - proxy record reconstructions of past climate (or even worse, ad hominem attacks on the scientists who perform these proxy reconstructions). This serves to distract from the physical realities currently being observed. Humans are raising CO2 levels. We're observing an enhanced greenhouse effect. The planet is still accumulating heat. What are the consequences of our climate's energy imbalance? Sea levels rise is accelerating. Greenland ice loss is accelerating. Arctic ice loss is accelerating. Globally, glacier ice loss is accelerating. Antarctic ice loss is accelerating.

 

When you read through the many global warming skeptic arguments, a pattern emerges. Each skeptic argument misleads by focusing on one small piece of the puzzle while ignoring the broader picture.

 

Further...

 

 

http://scienceblogs.com/deltoid/2009/11/on_those_stolen_cru_emails.php

The global warming denialists have predictably gotten very excited about the emails that were stolen from CRU, declaring that they prove that there's a big climate scientist conspiracy (presumably to install a COMMUNIST WORLD GOVERNMENT). We don't know whether or not the thief altered the emails, but since there isn't really anything incriminating it's likely that they are all genuine.

 

Most of the fuss has been generated by taking emails out of context and bad faith interpretations of what was written.

 

 

 

And if you want the real take on all of this which really offers perspective, go here:

 

 

http://carbonfixated.com/newtongate-the-final-nail-in-the-coffin-of-renaissance-and-enlightenment-thinking/

If you own any shares in companies that produce reflecting telescopes, use differential and integral calculus, or rely on the laws of motion, I should start dumping them NOW. The conspiracy behind the calculus myth has been suddenly, brutally and quite deliciously exposed after volumes of Newton’s private correspondence were compiled and published.

 

For those who are only paying minimal attention, that last one is satire... a parody of this ridiculous nonsense applied to the works of Isaac Newton... but it's damn funny.

 

 

 

 

 

 

 

 

---

Merged post follows:

Consecutive posts merged

For those of you who wish to explore some more of the actual content of the emails, this link does a nice job of bringing much needed context:

 

http://greenfyre.wordpress.com/2009/11/21/climate-deniers-hoax-themselves-again/

Edited November 23, 2009 by iNow
Consecutive posts merged.

[bascule]

I have no doubts that there are many scientists within the climate science community who have agendas. Generally, when you do have a general consensus in a scientific community, such as the one we see in the climate science community, it becomes much more difficult for outsiders of that community to present dissenting opinions.

 

All that said, the overwhelming majority of the anti-GW movement reeks of paranoid conspiracy theorists. An easy challenge to anyone who tries to draw too many conclusions from this: does this happen in other scientific communities? In my mind (having read books like The Trouble with Physics) the answer is an overwhelming yes.

 

Does this mean there's a conspiracy to convince the world that global mean surface temperatures are generally on the rise when really they're not. No.

 

I was traveling this weekend and caught a bit of Fox News. They were talking about how Obama didn't choose to go to Copenhagen for the climate conference there. They went so far as to say (I believe this is a direct quote) "proponents of global warming are in disarray" because Obama wasn't attending the conference. Disarray? They're holding a conference. That's what I call a fairly organized event.

 

Overall, I see this entire affair as a giant ad hominem. They can't attack the science, so they attack the scientists. OMG they're biased, therefore... they're wrong? No.

[CaptainPanic]

I think that the climate scientists should be objective about it.

 

But in the scientific world, there are plenty of people who have both authority (perhaps not on climate change though), and a strong opinion about climate change.

 

For a journalist, or for any layman it is very hard to distinguish between a real climate scientist, and somebody who is a scientist that works in a field related to the climate (like environmental research, sustainable energy - or oil and petrochemical industry). Look for example at this forum. How many of us are really involved in the climate research? And how many of us have joined in a discussion about climate change? Most of us are scientists, and most of us can claim to have some authority in some field - quite possibly related to the climate research.

 

And finally, many people might have two functions: one related to climate, one related to another topic. These two jobs might conflict.

[Pangloss]

If political influence did corrupt the integrity of global climate change science, what exactly would that look like, and how would we be able to identify that this was taking place?

[iNow]

If political influence did corrupt the integrity of global climate change science, what exactly would that look like, and how would we be able to identify that this was taking place?

 

The data itself would be inconsistent and show different things depending on who ran the test. We'd be able to identify it because two people running the same test would obtain different results, and it would direct further attention on to figuring out the source of the discrepancy and which conclusions and set of data are actually correct.

[bascule]

If political influence did corrupt the integrity of global climate change science, what exactly would that look like, and how would we be able to identify that this was taking place?

 

To basically reiterate iNow, it wouldn't look terribly distinct from anything else corrupting the scientific process. Corruption in science can take any number of forms: unreproducible data, falsified data, failure to make data available, failure to publish papers even though they pass the peer review process, "fast-track" peer review that overlooks errors in either the data or the conclusion, etc.

 

I'm sure you can find case instances of these sorts of things all over the place as well. So what would make it a "real" problem as opposed to the failings of individual scientists? If we saw systemic cases of these things happening in some of the larger groups whose analysis underlies the current consensus, like the IPCC, NOAA, or NASA GISS.

[Pangloss]

Well therein lies the rub, because lack of science education means that people not only can't fathom what climate science is reporting, but they can't even evaluate the people who are doing the telling because they don't understand the process.

 

Which I suppose puts an even higher burden on those who do understand to properly relate what's happening, and why.

[bascule]

The closest I've seen to someone calling out one of the major scientific organizations doing the relevant analysis is when Steve McIntyre located erroneous USHCN data in the NASA GISS analysis.

 

GISS's response was to correct their mistakes and update their analysis with the corrected data along with a post-mortem of what the problem was, how they corrected it, and what changed in the new analysis (see the Data Error section).

 

While yes, it's bad the data used by NASA GISS for their GISTEMP assessment was in error, GISS's response was very much in line with how a scientific organization should behave when confronted with an error in their analysis.

 

If we weren't seeing this sort of thing, then yes, we should be doubtful of the science.

[JohnB]

Corruption in science can take any number of forms: unreproducible data, falsified data, failure to make data available, failure to publish papers even though they pass the peer review process, "fast-track" peer review that overlooks errors in either the data or the conclusion, etc.

 

You are aware that "failure to make data available" is commonplace in certain areas? That is why FOI campaigns have been launched. Since the Hadley Centre is one of the major temp centres, isn't it reasonable that they reveal what stations are used and the methodology used to produce their results?

 

What is worrying to me is the mindset shown. People are being asked to delete data and emails in case of an FOI request. I don't know about other nations but that is a Federal offence in Australia. It is classified as corrupt activity.

 

It is also worrying that papers would be kept out of the IPCC process even if it meant "redefining" the meaning of the peer review process. This cannot be viewed as in any way "good" for science to have major players talking this way.

 

I note that other emails from lead authors are requested to be deleted, which is in contravention of IPCC rules. This is just not good.

 

BTW, nice misdirection on realclimates part. The issue is not with the Hadley Centre or the University of East Anglia, the issue is with a certain Dr. Jones who is associated with both.

 

Dr. Jones has consistently and constantly refused to provide data to back his results. Excuses ranging from "Why should I?" to "National Defence" to "Intellectual Property" to "Secrecy agreements with other nations, no, I can't tell you who because we have lost the list of who we have agreements with" to simply, "We've moved and lost the data".

 

Leaving aside for a moment the climate debate, the question is "Is this acceptable behaviour for any scientist?" I can't help but think no, and that this is harming climate science in particular and science in general.

[iNow]

Failure to share data is also common because you're not going to share information which is not credible. When errors get made, of course you're not going to publish it. Why is this so hard for the conspiracy-theorists to grasp?

[bascule]

You are aware that "failure to make data available" is commonplace in certain areas?

 

Yes, I have read about some pretty bad cases of this in climate science.

 

To a degree I can understand the technical difficulty of making this data available, as it used to be my job. I used to work for a climate center and was the support monkey who had to wander into a cramped, dust filled room and dig through tapes and tapes (or in some cases WORM discs) of archived data in order to locate the requested set.

 

After I found the tape/disc with the requested data, I also had to find a tape drive capable of reading it, along with all the necessary adapters to hook it up to a modern SCSI card.

 

And even after doing all that, in some cases the tapes were unreadable.

 

There are, in certain cases, legitimate technical reasons for which data cannot be made available upon request.

 

There are also cases where data is maliciously withheld.

 

But hey, never ascribe to malice that which is adequately explained by incompetence.

[The Bear's Key]

Well therein lies the rub, because lack of science education means that people not only can't fathom what climate science is reporting, but they can't even evaluate the people who are doing the telling because they don't understand the process.

 

Which I suppose puts an even higher burden on those who do understand to properly relate what's happening, and why.

Absolutely, the most critical point is to help remove the disconnect between the knowledgeable and the ignorant. It's not just education access, but making the science ever more digestable to the layman: simple, elegant, brief, accurate. (i.e. a real departure from the easy road in teaching)

 

You are aware that "failure to make data available" is commonplace in certain areas? That is why FOI campaigns have been launched.

.....

What is worrying to me is the mindset shown. People are being asked to delete data and emails in case of an FOI request.

.....

I note that other emails from lead authors are requested to be deleted, which is in contravention of IPCC rules. This is just not good.

.....

Dr. Jones has consistently and constantly refused to provide data to back his results. Excuses ranging from "Why should I?" to "National Defence" to "Intellectual Property" to "Secrecy agreements with other nations, no, I can't tell you who because we have lost the list of who we have agreements with" to simply, "We've moved and lost the data".

Please cite what you listed, for better reference.

 

Leaving aside for a moment the climate debate, the question is "Is this acceptable behaviour for any scientist?"

Definitely not. But without the context of the unreleased emails, we're unable to make solid conclusions. Did the hackers release all the information? I sense they were a bit dishonest. I thought hackers usually would reveal all the goods, not pick-and-choose.

 

Yet it doesn't look good for Jones in the OP's link of Washington Post, I admit. But out-of-context info's designed for just that purpose often enough.

 

However, it wouldn't surprise me if the guy did manipulate the situation. Question is, how widespread are such occurences -- especially compared to instances of proper science when its analysis treads on hot-button or politicized issues?

 

I don't know about other nations but that is a Federal offence in Australia. It is classified as corrupt activity.

Unfortunately here in the U.S. it's given way too much leeway.

 

...accounts were routinely destroyed, despite a law requiring preservation of presidential records

 

White House: 'We screwed up' on deleted e-mails

 

Senator wants to know why NFL destroyed Patriots spy tapes

 

Agency Admits It Destroyed More Tapes Than Was Previously Believed

 

Failure to share data is also common because you're not going to share information which is not credible. When errors get made, of course you're not going to publish it.

But they can release it stamped with "faulty data" across the page, just to maintain credibility and/or integrity. My opinion? Usually nothing good results from secrecy into things which have a strong potential for affecting much of the world.

[bascule]

This was an interesting read:

 

http://www.cbsnews.com/blogs/2009/11/24/taking_liberties/entry5761180.shtml

 

As someone who maintained something similar, I can certainly attest that the code quality of a lot of these data analysis models and programs is... not stellar.

 

I maintained a hacked together mess of Perl and C programs which did automated analysis of a network of weather stations.

 

I also worked on a model written in Fortran.

[Pangloss]

As bascule suggests above, this dog seems to have legs, and is not limited to the domain of conspiracy theorists.

 

Megan McArdle at The Atlantic talks about "the real problem" with the emails being the less-discussed issue of what this suggests about the validity of the climate models.

 

http://business.theatlantic.com/2009/11/the_real_problem_with_the_climate_science_emails.php

 

The emails seem to describe a model which frequently breaks, and being constantly "tweaked" with manual interventions of dubious quality in order to make them fit the historical data. These stories suggest that the model, and the past manual interventions, are so poorly documented that CRU cannot now replicate its own past findings.

 

That's not reason to abandon efforts to control our carbon emissions--as I say, they're still very likely to be problematic. But if the model turns out to be as bad as initial reports seem to imply, we should probably hold off on policy recommendations until we have a slightly better handle on the likely outcomes.

 

--------

 

On a side note, when all is said and done on this issue, even when we settle down and universally recognize the human contribution to this problem, I think scientific skepticism is going to take a well-deserved respect hit because of this. Skeptics should never be denounced as "deniers", and for skeptics themselves to rally religiously around a scientific problem that is not yet fully understood and declare any questioners to be somehow in violation is anathema to the very idea of skepticism.

 

The Michael Shermers out there are going to get a comeuppance, and it's not going to be pretty.

Edited November 25, 2009 by Pangloss

[iNow]

Megan McArdle at The Atlantic talks about "the real problem" with the emails being the less-discussed issue of what this suggests about the validity of the climate models.

 

That's just it, though... The models (in general) are frakkin stellar. Hackers... as a general rule... are going to share everything indiscriminately. What we see here is little more than a cherry-picked morass which attempts to tell a very specific story.

 

Further, the timing of the release of this information was (IMO) clearly strategic. They're merely trying to deflect everyone's attention from things like the below as we go to Copenhagen.

 

 

 

I don't know what your standards are, but in my world, that is an amazingly accurate projection and set of modeling data, especially when you consider it pertains to something as painfully complex as climate.

 

Pangloss... as your own post illuminates... apparently the distraction is working.

[padren]

How of curiosity, how many models are there for climate change, and how many variables do they each isolate? How much progress has been made on the "Global Dimming" factor due to particulate pollutants? I haven't heard about that aspect in a while, but when it came out it appeared to effect the models pretty significantly.

 

Thing is, I see a lot of talk about "the model" like there is one and a general consensus but is that really true?

[iNow]

Here's a decent overview, padren:

 

http://en.wikipedia.org/wiki/Global_climate_model

 

 

The short answer to your question is, "There are a metric-assload of them."

[JohnB]

Further, the timing of the release of this information was (IMO) clearly strategic. They're merely trying to deflect everyone's attention from things like the below as we go to Copenhagen.

 

I think that there are strategic moves being made on all sides. We had a "Report" released in the last week or so showing the destruction that will be caused by the 1.1m increase in sea level. Odd that an "alarmist" report should come out just before Copenhagen, don't you think?

 

If anyone is interested, and they are worth the read, the emails can be found here, complete with wordsearch facility.

 

From a cursory read, (I have little time ATM as I'm starting a business and preparing for throat surgery) there is a lot of interesting stuff in there. The extreme "antis" will certainly try to make out a conspiracy where there is none. There is certainly collusion, eg Mann asking Tamino to spearhead an attack on a paper, but so what? People on the same side co-ordinating their efforts, big deal. I would wager that a check of emails from the anti side would reveal similar activities.

 

Some, like the one I linked to, show that the IPCC process (at least in the past) has not been what it should be.

What made the last millennium graph famous (notorious!) was that Cxxxx Fxxxxxx must have seen it and reproduced it in the 1995 IPCC chapter he was editing. I don't think he gave a citation and it thus appeared to have the imprimatur of the IPCC. Having submitted a great deal of text for that chapter, I remember being really pissed off that Cxxxx essentially ignored all the input, and wrote his own version of the paleoclimate record in that volume.

 

So at least one IPCC report had it's paleo chapter written by one person and not a team of authors.

 

Another shows, I think, that "management" may have a larger part than many "pros" think.

I think that this story could possible catch on and make headlines, so I agree that we should be careful. But it's important that we bring the

*true* picture out, and it is best that this is done by RealClimate rather

than a sceptic site. The general scientific side of the IPCC report (i.e.

all the peer-reviewed papers ad the scientific theories) is still sound,

but to explain how *one* figure was shoe-horned into the report is harder

to defend. The sceptics may argue that the IPCC reports are political

after all, and this is also what it sounds like if governments 'hoisted

the national flag' by having it's own figures inserted last minute.

However, by providing an account of the 'evolution of the IPCC report

writing', we could possibly give the story a softer landing. E.g. how many

times of review the first report underwent as compared to the present

report. We should also put this in perspective - the report is large and

covers a wide range of topics, and most (all but our case?) is true to the

science. There are sometimes a few rotten apples in a good batch,

unfortunately. But the important part is that we don't accept rotten

apples and that we sort it out! Forthcoming and up-front. Another

important side is that this can provide a lesson for the scientific

communities.

 

This email from Rxxxxx, an author at RC leads me to ask;

1: What does he mean by *true* picture? Why the emphasis?

2: What does he mean by "all but our case" is true to the science? Is his case not true to the science?

 

I think it will take quite a while for all this to be digested, but with luck some more openness in climate science cannot be a bad thing.

[iNow]

Here's a copy of the most relevant email:

http://www.monbiot.com/archives/2009/11/23/the-knights-carbonic/

[JohnB]

"Knights Carbonic" I love it.

[Pangloss]

So next week we're going to get the Copenhagen interpretation of climate science. Do we call this story, then, the "many models" interpretation?

 

 

 

 

(sorry)

[iNow]

It's not as catchy, but I'd prefer we call it the reality-based interpretation.

[swansont]

I think this sums it up rather nicely. Rated R for language.

 

http://www.rifters.com/crawl/?p=886

 

 

A snippet:

 

There’s this myth in wide circulation: rational, emotionless Vulcans in white coats, plumbing the secrets of the universe, their Scientific Methods unsullied by bias or emotionalism. Most people know it’s a myth, of course; they subscribe to a more nuanced view in which scientists are as petty and vain and human as anyone (and as egotistical as any therapist or financier), people who use scientific methodology to tamp down their human imperfections and manage some approximation of objectivity.

 

But that’s a myth too. The fact is, we are all humans; and humans come with dogma as standard equipment. We can no more shake off our biases than Liz Cheney could pay a compliment to Barack Obama. The best we can do— the best science can do— is make sure that at least, we get to choose among competing biases.