We know from many other studies that local knowledge, even anecdotal knowledge can make a valuable contribution to knowledge production and decision making. Here is a related story on the Sunday Telegraph:
In its most recent report, it stated that observed reductions in mountain ice in the Andes, Alps and Africa was being caused by global warming, citing two papers as the source of the information.The IPCC report (AR4, WG 2) has a table that lists the two sources here.(Schwörer, 1997; Bowen, 2002 are the disputed sources)
However, it can be revealed that one of the sources quoted was a feature article published in a popular magazine for climbers which was based on anecdotal evidence from mountaineers about the changes they were witnessing on the mountainsides around them.
The other was a dissertation written by a geography student, studying for the equivalent of a master's degree, at the University of Berne in Switzerland that quoted interviews with mountain guides in the Alps.
The revelations, uncovered by The Sunday Telegraph, have raised fresh questions about the quality of the information contained in the report, which was published in 2007.
Is this bad practice just because it relies on non-scientific sources? How is 'propoer' data collected? What is the difference between them? Why is one better than the other?
Posts
29 comments:
Pachauri has been too busy to search for substantive scientific reports on the matter. He's been writing cheap racy novels. Anecdotes are quick and easy. Besides, the sceince is long settled and we're way past the point of having to provide real science. The debate is over.
Oh Dear!
The whole point is that there is a debate to be had.
Remember, the Inquisition told Galileo that the science was settled!
It's not about scientific or non-scientific sources, it is about the appropriate and systematic compilation of anecdotal (observational) knowledge.
Interviews my be sufficient to get impressions, but they are by no means primary information sources for longterm and global assessment.
Let's see, once again we have people pointing out two questionable references. Fair enough.
However, have these same people also checked the remainder of WG2? Of course not. If they would have, they would find several peer-reviewed references that essentially show the same as those two non-peer reviewed sources. See for example
http://www.ipcc.ch/publications_and_data/ar4/wg2/en/ch12s12-2.html#12-2-1 (check the table)
and
http://www.ipcc.ch/publications_and_data/ar4/wg2/en/ch13s13-2-4.html (in particular table 13.3)
@Marco
The recurrent theme in IPCC critcism is that the report makes a biased selection of existing the peerr reviewed literature and in some cases not even uses peer reviewed literature at all.
Hans, go ahead and list the papers that contradict that most glaciers all over the world are shrinking.
That the IPCC makes a biased selection is claimed by both sides. I've seen quite a few climate scientists note that the IPCC is rather conservative in its statements. In WG3 you can't expect many peer reviewed studies, there simply are too few, but that does not mean the references are bad. In WG2 it could have been better, especially (as I indicate) since there often are peer reviewed references available.
most glaciers all over the world are shrinking ... since 1850, at least.
I don't think this is the probem. The case is linking glacier shrinkage with CO2, and the aproximate amount of it.
"If they would have, they would find several peer-reviewed references that essentially show the same as those two non-peer reviewed sources. See for example" In other words if they come to the same conclusion they're right? Kinda two negatives gives a positive?
It's interesting to know Gerry North's description of the non-peer review process: that they “didn’t do any research”, that they got 12 “people around the table” and “just kind of winged it.” He said “that’s what you do in that kind of expert panel”
So this is what happened at the time they reviewed the melting Himalaya's:
"Okidoki... next we have here the melting of the Himalaya glaciers in 2035. Sounds interesting... What? Nature has published it? OK let's put it in. Anyone against it?
OK next...the Amazones...."
@Marco
Disappearing glaciers are usually good news, there used to be a glacier in the middle of The Netherlands. Few people also think that growing glaciers is good news.
Hydrologists who studied the subject have shown that the contribution of glacier melt to drinking water is negligable.
So all what's left is a hype on glaciers.
But that's beside the point that you did not respond to the grist of my comment: The IPCC is not the honest broker which it claims to be. Something that has been known for years.
Hans Erren (#9), what is "the subject" of your hydrologists?
Glacier melt water is certainly a negligible fraction of the flow in the lower part of large rivers, e.g. the Ganga (Ganges) or the Chang Jiang (Yangtze). But it comprises a substantial fraction in some small tributaries in the upper reach of them. It is important as water resource in the fine-grained regions.
(Also, drinking water is not large fraction of consumption of water by the human society. We need to think about water demand for food production.)
Glacier melt water is more important than its fraction in average river discharge. It can compensate for year-to-year fluctuation of water availability.
Incidentally, I was in Switzerland in the summer 1993. It was a hot summer with little rain there (contrary to the opposite situation in Japan then). Many rivers had low water levels, but some streams were vigorous. The latter had glaciers upstream, and the situation of that summer caused larger-than-normal melting there. Without glaciers, the situation would be drought. I did not scientifically study the situation but I read some scientific papers supporting the idea.
***
By the way, it is an unresolved issue how we can include regional assessments of climate change in world-wide efforts.
The wrong information appeared in the chapter of regional climate change in Asia of the Working Group 2 report. While Asia is a region with respect to the world, it is a very coarse-grained region. Glaciers matter in fine-grained regions. In theory they have some contribution to the aggregation in the coarse-grained region, it is difficult to quantify. In the chapter of Asia, matters of such small regions can only be either ignored or reported as examples.
@itisi69:
It isn't two negatives makes a positive. but positives reinforcing other positives. If both peer reviewed papers (with data to back) and anecdotal data point to the same thing, do you then think it is the opposite anyway?
@Hans Erren,
Kooiti already set you straight on the importance of glaciers. I'll add another little bit of news for you to consider:
http://www.expatica.com/nl/news/dutch-news/Melting-Alpine-glaciers-release-toxins-_57270.html
I won't go into your laughable example of glaciers in the Netherlands. They've been gone for millenia already. They just didn't disappear as fast.
The next issue is that there's a clear trend towards less snow cover (at least in the Alps), which is a thread to tourism.
The next issue is that there's a clear trend towards less snow cover (at least in the Alps)
Hardly unprecedented:
http://news.bbc.co.uk/2/hi/science/nature/7580294.stm
Update
http://www.telegraph.co.uk/earth/environment/climatechange/7111525/UN-climate-change-panel-based-claims-on-student-dissertation-and-magazine-article.html
Here's an interesting post by a hydrologist re Himalaya glaciers. Apparently there was no hydrologist included in the research, make sense... or does it?
http://ccgi.newbery1.plus.com/blog/?p=262
"Closer examination shows that the study only addresses mass loss from a single glacier in the Himalayas. It does not address the hydrology of the rivers at all. I don’t think there is even a single hydrologist on the crowded study team.
The scientists may think they have covered themselves with their speculation about the potential impact on water supplies by inserting the “if” and “could be”. But this is disingenuous. They have not even addressed the hydrology in a serious way. The last sentence in the abstract appears to have been inserted to make the research more relevant, a tactic that is quite common. I have served on funding committees for research organisations and the potential impact i.e. importance, of research is a component that is given high marks in funding decisions."
@plazamoyua:
did you read the article? As they note, those were the times we had a warmer climate due to orbital forcing. We SHOULD be colder now, but are going rapidly into the same warm (and likely warmer) period without that warming orbital forcing.
Thompson and his colleagues are hydrologists in a sense as glacier mass balance is their one of the important subjects of the International Association of Hydrological Sciences. But I agree that the remarks about water resources appeared in the particular paper of theirs are stories of possibilities.
Thinking about cases like this, I have arrived at the following understanding about scientific communiation. I am not so confident about the description. I would like to have help from cultural scientists who watch our behaviours.
I think that an article of scientific research has "hard" and "soft" parts. The hard part consists of the sections of "materials", "methods", "results" and "conclusion". The soft part includes "introduction" (except the part explaining the basic idea), "discussions" and "remarks". Peer reviewers are usually chosen to evaluate the hard part. They examine the hard part in detail, but they give criticism to the soft part only when the description seemed very strange. So the soft part does not usually have the quality of the "peer-reviewed literature". On the other hand, the existence of the soft part makes the professional scientific journals interesting. How boring they will be if they contain the hard part only! (And sometimes the soft part is really valuable.)
The problem in scientific communication is that the distinction between the hard part and the soft part is not always obvious to the non-experts of the subject. It is especially the case in short papers called "letters" in such journals as Nature, where such section headers describing the structure of the article are usually omitted.
This is a warning both to the people who want to use scientific papers as evidence supporting their ideas and those who want to criticize the authors. Please understand what are the hard part in the paper. If the paper is well structured, it does not require understanding of the disciplinary details. But unfortunately there are not so well structured papers, and you may need help informants of the disciplinary dialect languages. You may refer to the soft part, but you should make explicit that that is the soft part.
Kooiti - not sure of the distinction between hard and soft is very useful here. What I find common in all kinds of research (no matter if science or humanities) is the relation between 'data' and 'interpretation'. A typical question for a reviewer is this: is the interpretation given in the paper supported by the data?
This seems to be the worldview of many scientists, that there is a close relation between data and interpretiation. Of course, in many instances more than one interpretation is possible (and scientists sometimes are reluctant to acknolwedge this possibility). At the other extreme are some cultural theorists who tell us that there is nothing but interpretations.
Real science is done in between the two.
Reiner Grundmann - Yes, data and interpretation, and I wanted to mention that there are multiple levels of interpretation.
I try to paraphrase my thought again without using the words "hard" and "soft".
Usually a scientific paper must contain something proved to be of good quality within a certain narrow discipine of normal science -- isotope analysis of ice samples in the case of Kehrwald, Thompson et al.-- and this part is the primary target of peer review.
But the journal editors (or the peers, or the funding agencies) may also require the authors to state how relevant is the study to a wider community. (Sometimes to an interdisciplinary scientific community, and sometimes to the society at large.) Since this discourse is not normal science, peer review cannot assure the quality. If this part contains original thought of the authors, it can be valuable, but it can be nonsense. On the other hand, if the authors heavily rely on the standard literature, it does not carry new information, but it is unlikely to be wrong. Unfortunately for Kehrwald et al., what they thought a standard reference (the chapter on Asia of IPCC AR4 WG2) happened to contain wrong information.
An issue in this case is when does anecdotal evidence become scientifically relevant. If they're only interviewing current mountain climbers, they're using information from only a few decades. Worse, mountain climbing tends to be done more often by young people, so the time frame tends to be short. A study which examined descriptions over a 200 year period would be more interesting.
@ Marco
Yes, I did read it. But since I have never heard taht MWP or Roman Warmig Period where "caused by fluctuations in the orbital pattern of the Earth in relation to the Sun", I just separated the facts from the funny explanations. And the facts are: Alps glaciers don't seem to be in an unusual or unprecedented state.
Otherwise, they would have solved the "problem" of the hockey sticks, and this doesn't seem to be the case.
The issue is simple. Whenever the IPCC is criticised, the reply mentions the rigorous peer-review of the IPCC reports. Whenever someone puts forward an alternative claim, this is labelled "not peer reviewed" and therefore not valid.
The story in the Telegraph is about the claim in Chapter 1, WG2, AR4 that there has been a loss in ice climbs between 1900 and 2000 in the Andes, Alps and Africa.
This claim is based on two non-peer-reviewed papers. This blows the IPCC excuse "we're peer-reviewed and you're not".
The claim itself is astounding. The sources are unstructured interviews with mountain guides. The human mind is not wired for comparing situations then and now. Few mountain guides would have first hand knowledge of the ice situation in 1900. And the claimed loss in ice can be for all sorts of reasons.
So why did I put it in? I can only think of one explanation: They wanted as many observations as possible, affecting as many constituancies as possible.
It's a dumb error, and smacks of manipulation.
And that is before I read Schwoerer's summary:
http://climatequotes.com/2010/01/31/the-story-of-the-geography-major-dissertation/
@Richard Tol:
Please note that the continuing references to the IPCC using peer reviewed work (and being peer reviewed itself) has in the past been invariably aimed at WG1. For decades 'skeptics' have tried to attack the science behind climate science. When that didn't really work, the scientists became targets (see climategate). Now WG2 is the target.
And did you really read the WHOLE summary, Richard? It's nice the "climatequotes" site refers to the actual summary:
http://www.adventure.ch/news/climatch.htm
Take this section:
"If we compare the comments of mountain guides to the changes of the natural environment at hazardous spots, to changes of routes, about the reasons why some routes are used less and about seasonal changes concerning particular fields of activity, it becomes evident that the objective hazards and alpine-technical difficulties have been the central feature for the last few years.
The danger of rockfalls, followed by the danger of falls into crevasses has been increasing the most. Simultaneously mountain guides are confronted with these two hazards most frequently. The decrease of hazards in the region of the survey is, according to a general inquiry, insignificant compared to the increase.
(Permafrost and glacier shrinkage as main causes)
The principal causes of these changes are glacier shrinkage and the thawing of permafrost.
For about half of the mountain guides it was not clear, whether extreme wheather conditions which affect the demand for mountain guides negatively, have been occuring more often in the last few years."
And what does climatequotes do? It only cites the last sentence!
@Marco
I read the summary. It seems that the impact of climate change on mountain ice was only a small component of the study.
So the IPCC did not just cite a dubious source, it cited a dubious source that focussed on something else.
@Richard Tol:
The whole part I quoted focused on the change in mountain ice. That's about 1/4 of the summary, noting the observation repeated in the IPCC report: loss of mountain ice. It was NOT cited as investigating the impact of climate change, but as *reporting* an observation indicating climate change.
@Marco
It was not a master's thesis so, but a quarter of a master's thesis that supported the claim in the IPCC.
If a range of knowledge that the society needs fits a certain discipline of normal science, we can usually pick up good peer-reviewed scientific articles which satisfies the needs. But if not, maybe we need to resort to "grey literature", or to what I called "soft part" of peer-reviewed literature. Now I understand that the latter cases occurred few times in WG1 but many times in WG2 of IPCC.
Maybe we start a new journal, and sometimes it can achieve good quality and a new something-like-a-discipline emerges. Sometimes it is not so successful and the peer review there does not gurarantee quality. It does not mean that quality of all articles there is low, but that we need other ways to evaluate them.
The complaints about what made the Observed Effects section of WG2 are silly. The peer reviewed science is in WG1.
Are we to believe truck drivers can't notice when use of the Alaskan Highway is shortened by 3 months? That mountain climbers have no personal or organizational memory of lost climbs? That skiers can't tell when the ski season, snow depth and number of runs are all reduced?
Post a Comment (pop-up window,non-moderated)