I observed recently that Hansen’s GISS series contains an apparent error in which Hansen switched the source of GISS raw from USHCN adjusted to USHCN raw for all values January 2000 and later. For Detroit Lakes MN, this introduced an error of 0.8 deg C. I’ve collated GISS raw minus USHCN adjusted for all USHCN sites (using the data scraped from the GISS site, for which I was most criticized in Rabett-world). Figure 1 below shows a histogram of the January 2000 step for the 1221 stations (calculated here as the difference between the average of the difference after Jan 2000 and for the 1990-1999 period.) The largest step occurred in Douglas AZ where the Hansen error is 1.75 deg C! There is obviously a bimodal distribution.

hansen40.gif

Next here is a graph showing the difference between GISS raw and USHCN adjusted by month (with a smooth) for unlit stations (Which are said to define the trends). The step in January 2000 is clearly visible and results in an erroneous upward step of about 0.18-0.19 deg C. in the average of all unlit stations. I presume that a corresponding error would be carried forward into the final GISS estimate of US lower 48 temperature and that this widely used estimate would be incorrect by a corresponding amount. The 2000s are warm in this record with or without this erroneous step, but this is a non-negligible error relative to (say) the amounts contested in the satellite record disputes.

hansen41.gif

UPDATE:
On the weekend, I notified Hansen and Ruedy of their Y2K error as follows:

Dear Sirs,
In your calculation of the GISS “raw” version of USHCN series, it appears to me that, for series after January 2000, you use the USHCN raw version whereas in the immediately prior period you used USHCN time-of-observation or adjusted version. In some cases, this introduces a seemingly unjustified step in January 2000.

I am unaware of any mention of this change in procedure in any published methodological descriptions and am puzzled as to its
rationale. Can you clarify this for me?

In addition, could you provide me with any documentation (additional to already published material) providing information on the
calculation of GISS raw and adjusted series from USHCN versions, including relevant source code.

Thank you for your attention,
Stephen McIntyre

Today I received the following response:

Dear Sir,

As to the question about documentation, the basic “GISS Surface Temperature Analysis” page starts with a “Background” section whose first paragraph contains the sentence: “Input data for the analysis ,…, is the unadjusted data of GHCN, except that the USHCN station records were replaced by a later corrected version”. A similar statement appears in the “Abstract” and the “Introduction” section of our 2001 paper (JGR Vol 106, pg 23,947-23,948). The Introduction explains the above statement in more detail.

In 2000, USHCN provided us with a file with corrections not contained in the GHCN data. Unlike the GHCN data, that product is not kept current on a regular basis. Hence we used (as you noticed) the GHCN data to extend those data in our further updates (2000-present).

I agree with you that this simple procedure creates an artificial step if some new corrections were applied to the newest data, rather than bringing the older data in sync with the latest measurements - as I naively assumed. Comparing the 1999 data in both data sets showed that in about half the cases where the 1999 data were changed, the GHCN data were higher than the USHCN data and in the other half it was the other way round with the plus-corrections slightly outweighing the minus-corrections.

Although trying to eliminate those steps should have little impact on the US temperature trend (much less the global trend), it seems a good idea to do so and I’d like to thank you for bringing this oversight to our attention.

When we did our monthly update this morning, an offset based on the last 10 years of overlap in the two data sets was applied and our on-line documentation was changed correspondingly with an acknowledgment of your contribution. This change and its effect will be noted in our next paper on temperature analysis and in our end-of-year temperature summary.

The effect on global means and all our tables was less than 0.01 C. In the display most sensitive to that change - the US-graph of annual means - the anomalies decreased by about 0.15 C in the years 2000-2006.

Respectfully,

Reto A Ruedy

Well, my estimate of the impact on the US temperature series was about 0.18-0.19 deg C., a little bit more than Ruedy’s 0.15 deg C. My estimate added a small negative offset going into 2000 to the positive offset of about 0.15-0.16 after 2000 - I suspect that Ruedy is not counting both parts, thereby slightly minimizing the impact. However, I think that you’ll agree that my estimate of the impact of the impact was pretty good, given that I don’t have access to their particular black box.

Needless to say, they were totally unresponsive to my request for source code. They shouldn’t be surprised if they get an FOI request. I’ll post some more after I chance to cross-check their reply.

As to the impact on NH and global data, I’ve noted long before this exchange that the non-US data in GHCN looks more problematic to me than the US data and it would be really nice if surfacestations.org starting getting some international feedback. Ruedy’s reply was copied to Hansen and to Gavin Schmidt. I’m not sure what business it is of Gavin’s other than his “private capacity” involvement in a prominent blog.