[Andy Mayhew]

Quote

Blownaway - 16/8/2009 19:09 So the raw data on which the integrity of civilisation (or the bank balance of every tax payer) may depend has been lost.

No it hasn't.  But some people want you to think that.

[ldavidcooke]

Hey Blownaway,

A conclusion, if the experiment and maths can be replicated under the tested circumstances by anyone with the necessary techniques/processes/resources, it can be irrefutable. Most papers of hypothesis or theories are the out growth of observations. From these observations, if a pattern is observed you might be able to create a hypothesis. If the hypothesis is validated it can become theory and if there is no opposition and universal validation can be established for the theory, where a theory linked phenomena or pattern can be predicted under all circumstances it can become a principle

However, not all papers are about the creation of a hypothesis, sometimes they are validation tests of a theory. In these cases most times you can only state that under a given set of circumstances a population of observations demonstrates this property which either is in or not in alignment with the hypothesis/theory of others or earlier work. Hence, a conclusion generally has a very limited range, unless it is part of the definition of a hypothesis or theory.

Cheers!
Dave Cooke

[John Mason]

Dave, that's three out of hundreds of thousands! Get a sense of perspective, man!

I've published stuff in orefield mineralogy that has falsified previous papers, but never, ever have I seen anything in that field that I would say is anything more than making mistaken conclusions - and we're looking at papers that predated my work by decades.

In general mineralogy, there is one exception to add to your list: the work of A.W.G. Kingsbury. This is a very sad case. Kingsbury was a very well-respected mineral collector who published many new finds in the journal Mineralogical Magazine especially in the 50s and 60s. Many years later a lot of these were shown to be - as you say - frauds. To this day, nobody knows what motivated this deceit, and you would certainly not get away with it today. Sadder still, he did describe a lot of genuine finds that have been reconfirmed by subsequent workers - why he went the extra length to get more "notches on the stick" is something that has baffled leading mineralogists ever since. But the important thing is that his peers got him in the end!

If you want to read how the fraud was uncovered, see:

http://www.geocurator.org/arch/Curator/Vol6No9.pdf

Cheers - John 

[Uskys]

"the freedom of information act" + "redaction"
How on earth can temperature records be anything other than just temperature records? - Only if they show RAW data to be modified in favour. Temperature records are sensitive to stability? ...

come on , this is beyond a joke.

[John Mason]

Quote

Uskys - 16/8/2009 21:23 "the freedom of information act" + "redaction" How on earth can temperature records be anything other than just temperature records? - Only if they show RAW data to be modified in favour. Temperature records are sensitive to stability? ... come on , this is beyond a joke.

Exactly, Andres - I've never seen such a storm in a teacup!!

cheers - John 

[JohnG]

It is not unusual for organisations to "throw away" electronic data due to a lack of storage. Storage media which is capable of holdling terabytes of data has only recently become relatively inexpensive.

After reading the arcticle it sounds as if the Hadley Centre obtained what raw data it needed to do its study, adjusted the data so that is was easier to work with, and only kept the modified data as the raw data was already held by the governments/organisations that the Hadley Centre obtained the data set from. I do not see anything wrong with this, but IMO the Hadley Centre should release the information about where the data was sourced.

Many forecast organisations keep an archive of past forecasts for verification and contractual purposes, but how many acctually keep hold of the raw model output and observational data that was used to make the forecast? Not very many because they can't afford the cost of storage.

As far as I can tell the freedom of information requests are nothing more than attempts at getting the data for free, when we all know that the likes of the UK Met Office and many other European institutions charge for their data. That is no doubt why the Hadley Centre would not release the data even if they had retained it.

[ldavidcooke]

Hey John,

Exactly, the three off the top of my head, are not a great number at all and goes to point to the added value of the pre-published, peer-reviewed system.

However, we have had a number of papers authored over the last 10-15 years that would appear to lack the quality one would expect of a scientific paper. Either by the lack of isolating the degrees of freedom or the isolation of the potential variables in a given observation, causing conclusions to be drawn which are not supportable by replication of the work.

http://en.wikipedia.org/wiki/Degrees_of_freedom

To overcome this I was proposing that the documentation of the experiment and the data be archived similar to the effort we apply to judicial evidence. That we do not apply the same care with informational evidence is appalling, especially if the conclusions are being used to drive national or international policy.

Cheers!
Dave Cooke

[Simon Culling]

David, your last papragraph is the killer. This matter is so important because of the policy decisions that will follow on from it, that this data should be considered as valuable as the conclusions that have been derived from it. That is the point that some people either wont or cant understand. The request for the data and a thorough evaluation of it is seen as a slight on the scientists involved and not as an opportunity to improve our understanding.

John, if this is a storm in a teacup, you must have a very large teacup. [^]

It is also a nonsense to suggest that it is too expensive to store this data. No wonder some suspect a smokescreen.

[John Mason]

What David is suggesting is some form of central archive, which could indeed be a useful asset. It would clearly require a funding-stream and the manner of its administration would need to be addressed. The taxpayer would no doubt have to cough up for these things.

It would have to be global in nature, however, and in each country the same basic setup would be required. The main principle would be that the data would be available on an open-source basis. I'm all for that. Open source is an excellent thing (apart from anything else it's the exact opposite to Microsoft!)

You would doubtless get a few cranks trying to distort things (be it evolution, plate tectonics, AGW or whatever theorem). But most of the time the peer-review process would block these, and the only other point would be to take anything posted to the Blogosphere (where you can write any old crap) with a pinch of salt ;)

Cheers - John

 

[John Mason]

Quote

Village - 17/8/2009 10:46 It's very clear to most thinking people out there that there is a great deal of smoke-screening taking place with official data.

That's as good a description of the Denialosphere as I have heard! ;)

Cheers - John 

 

[PK2]

Quote

MPv - 16/8/2009 09:32 McIntyre says he doesn't expect any significant surprises after analysing the raw data

Anyone else noticed this bit? He certainly seems to work fast but if he confirms the above it would appear a disappointing result for some after all that's been said.

[JohnG]

Quote

Simon Culling - 17/8/2009 06:53

It is also a nonsense to suggest that it is too expensive to store this data. No wonder some suspect a smokescreen.

Simon, My point was that it most likely was prohibitively expensive for a research organisation in the 1980's.

[Simon Culling]

JohnG, the original temperature dataset (HadCRUT) was published in 1994 by Jones et al. It would have been simple and cheap to store this data. The Hadley Centre is not (and wasnt then) a Mickey Mouse organisation with a tiny budget.

We are talking about the data set that the IPCC uses to make its predictions. This is not a trifling matter.

[PK2]

Quote

Village - 17/8/2009 13:38 I think that the inept media can turn anything into the unexpected if it grabs attention and sells.

Assuming your post is in response to mine (quoted below), I wouldn't have thought MPv would quote something from a place he considers inept. Would you? It's certainly not the case that the rest of the article is biased in favour of the Hadley Centre, in fact it seems rather critical of it.

Quote

PK2 - 17/8/2009 11:49

Quote

MPv - 16/8/2009 09:32 McIntyre says he doesn't expect any significant surprises after analysing the raw data

Anyone else noticed this bit? He certainly seems to work fast but if he confirms the above it would appear a disappointing result for some after all that's been said.

[Andy Mayhew]

Quote

JohnG - 16/8/2009 23:55 

After reading the arcticle it sounds as if the Hadley Centre obtained what raw data it needed to do its study, adjusted the data so that is was easier to work with, and only kept the modified data as the raw data was already held by the governments/organisations that the Hadley Centre obtained the data set from.

That's what I understand happened

Quote

IMO the Hadley Centre should release the information about where the data was sourced.

I'm not sure that that question has been asked of them?

Quote

As far as I can tell the freedom of information requests are nothing more than attempts at getting the data for free, when we all know that the likes of the UK Met Office and many other European institutions charge for their data. That is no doubt why the Hadley Centre would not release the data even if they had retained it.

I suspect that some of the organisations that provided data do charge and that it is for commercial reasons they stipulated that their data should not be made freely available to the general public.   Though I may be wrong.

[Andy Mayhew]

Has anyone asked the Chinese to provide the data they provided to Philip Jones over 15 years ago?

I assume they'd be happy to provide it for free?

[MPv]

Quote

Andy Mayhew - 17/8/2009 20:34

Has anyone asked the Chinese to provide the data they provided to Philip Jones over 15 years ago?

I assume they'd be happy to provide it for free?

It doesn't exist.

[Andy Mayhew]

How do you know? Are you saying the Hadley global temp reconstructions included no data from China? Can you support that assertion?

But in any case, same question to Russia ..... ;)