On 16 November I have been invited to participate at a panel discussion about the past and future IPCC. The panelists have been asked for an initial statement. So far I am thinking of concluding my statement with this:
"• IPCC is an indispensable effort.
"• IPCC is an indispensable effort.
• IPCC
processes need to be improved.
- Mechanism to deal with claims about errors and conflicts of interest.
- Consideration that “payment” may take the form of societal or professional recognition, not only in terms of money
- Documentation and highlighting of contested areas and areas of significant lack of scientific analysis
- Mechanisms to prevent formation of networks of lead authors.
- Factoring into the assessment that scientifically constructed knowledge may be conditioned by culturally constructed (value-based) knowledge claims.
- Mechanism to deal with claims about errors and conflicts of interest.
- Consideration that “payment” may take the form of societal or professional recognition, not only in terms of money
- Documentation and highlighting of contested areas and areas of significant lack of scientific analysis
- Mechanisms to prevent formation of networks of lead authors.
- Factoring into the assessment that scientifically constructed knowledge may be conditioned by culturally constructed (value-based) knowledge claims.
• Decentralized
“regionalization” efforts (aka BACC etc.) where suitable."
I wonder if some Klimazwiebel-readers may have some constructive comments.
Posts
31 comments:
Hi Hans,
I would add a few words on the review process. At the moment, review editors are passive recipients of comments from people who may or may not be experts. I think that the review editors should identify the key parts of each chapter, and make sure that those parts are read by real experts.
And yes, this comment is borne from my frustration. Innocuous parts of "my" chapters have been extensively reviewed, while controversial parts were largely unread.
Hi Hans
Interesting, is it a national German discussion or is this meeting organized by the IPCC itself? Is it open to the press?
From my perspective as a science writer and expert reviewer of AR5 I have the following recommendations:
1) Open up the process; make scoping meetings accessible for the press and for the interested public.
2) pay more attention to alternative views; start e.g. by adding a special chapter about dissent or controversial issues that are not resolved. This suggestion was also done by Jeroen van der Sluijs in a 2012 peer reviewed paper
3) invite scientists with a minority view as lead author for a chapter. Only of course when they have relevant expertise. Think of Nic Lewis for the chapter in which climate sensitivity is discussed. This will make the danger of group think smaller.
4) stop producing the big reports and start making special reports only, that you publish online only and that can be update in case it is necessary.
5) Stop doing the big CMIP-runs to 2100 every 6 years as long as the models can't simulate the changes in the past 35 years.
These are some very concrete suggestions that imo have the potential to greatly improve future reports by the IPCC.
Good luck
Marcel
Dear Hans von Storch,
I read good points in the last two commentaries, so I make only one point, here.
My point would be to actively clarify and research specific uncertainties until the next report and specifically report on the results.
I try to make an example even though I have reported on it before.
ISCCP cloud data were rejected in AR4 based on a single paper by Evan et al. A GEWEX report actually rectified the ISCCP data and rejected the claim of Evan et al. at least partly (2/3).
I think it would be an improvement if such issues would be specifically adressed and clarified in the following report.
Therefore after the report a ToDo List should be reported on issues that need to be clarified in the next report.
A final question.
How would one measure the improvement?
Thanks for your comments, really useful.
It is not an offical event, but something internal in the Hamburg; thus, our debate will have not impact of what is happening with IPCC. Only on the thinking of the few peopple who will attend.
I am thinking of IPCC as an effort of describing the topology of knowledge about climate dynamcis, change and impacts - which means that one would try to determine which field is relatively less contested, which are contested, which are un-researched. Thus, asking for listing those issues which need further research and could possibly be resolved in the next report, makes sense. Indeed I wanted to ask for an assessment in the AR(n) report, which findings of accepted knowledge in AR(n-1) have turned out to be no longer generally accepted. (We did that in BACC 2 - with the result that revisions form BACC-1 were not necessary, but that BACC-2 found a broader field, with a few isssues no longer contested.)
What an improvement would constitute? - good question. My personal answer would be - a better description of the topology of knowledge and its change in time. And: less conditioned by the people involved in the assessment process. (We replaced in BACC all lead authors from #1 to #2, but found conclusions in #2 consistent with #1, even if the body of literature (and the issues) available for #2 had become considerably larger compared to #1.)
HvS,
I think Gavin Schmidt made some good points in his carbonbrief interview (http://www.carbonbrief.org/the-carbon-brief-interview-dr-gavin-schmidt)
Quote:
The problem with the IPCC process is the tasks that they set themselves, which every seven years is to write basically an encyclopaedia of climate change. Up to the minute. That’s an enormous burden on the whole community, not just the people writing for the IPCC. It sets up weird incentives to have certain publications come in before weird deadlines that really don’t make any sense. I don’t think that that is sustainable. The science doesn’t change so much on a five-to-seven year basis that you have to redo the encyclopaedia every single time. I would be supportive of an IPCC that set aside this idea of being totally comprehensive and instead focussed on directed questions that came from the member governments. So, instead of scientists getting together and coming up with a list of all the things that we think are important and that other people should pay attention to, I’d much rather that the IPCC was directed towards specific questions that the policymakers want answered. So if they want to know, “how do we combine controls on short-lived pollutants and long-lived pollutants in a coherent framework?”, then let’s have an IPCC special report just on that. And I think that would be very interesting. If policymakers wanted a special report, “what should we be planning in terms of sea level rise?”, then let’s have a special report just on that. I don’t think we need to go into every single piece of detail of climate science every single time, because things do get updated. There are things that are done better over time, but those are not things that are necessarily policy-relevant. The difference between the AR4 [Fourth Assessment Report] palaeo-history of carbon dioxide figure and the AR5 [Fifth Assessment Report] palaeo-history of carbon dioxide figure – there’s clear improvements in the science between one and the other – but none of it is policy-relevant, except in the very grossest sense. So, if IPCC kept to a more defined, policy-relevant set of questions, then I think the burden on the community would be lessened and the product would be more applicable and appropriate.
Andreas
A lot of good suggestions, nevertheless there are deeper issues that should be addressed.
The intergovernmental nature of the IPCC is problematic and has no parallel in other international policy advisory bodies. Governments meddling with the wording of the Summary for Policy Makers (SPM) is problematic. The reason was to bring them on board so that they could not cop out, and would be bound by the IPCC findings. This was an illusion. The IPCC should focus on the state of knowledge only.
The IPCC needs to re-think its communication strategy. At present, there are reports from three Working Groups, with SPMs, and a synthesis report. However, the SPMs are not summaries for policy makers, but for other scientists. The IPCC chair then takes the liberty to communicate (mainly via press conferences/releases) what he thinks the central findings are. This should change and the IPCC should sum up its most important findings in no more than 2 pages, in non-technical language. This should be done by the IPCC without government interference (see above). The "I" in IPCC would assume a new meaning: no longer intergovernmental, but international.
Any new report should re-evaluate central statements of the previous report saying what progress has been made (or not), in what way the new report comes to different conclusions (or not). At present there is a tacit script which assumes that the findings are basically the same.
@ReinerGrundmann
Hear hear, I totally agree.
Maybe such things become possible as the role of the IPCC is already diminishing. They have done their job, as Mike Hulme put it. Going from extremely likely to virtually certain in 2021 is not going to make any difference to the policy negotiations.
So it's time to depoliticise the IPCC.
Marcel
@Reiner
the full report itself is already a summary of all the publications that are out there.
I think the IPCC should not summarize its own report, but publish it without comment.
In order to facilitate a democratic discussion, I think a more pluralistic approach is needed compared to the SPM from the IPCC.
They have done their job, as Mike Hulme put it.
Nice to see Gavin Schmidt and Mike Hulme in good agreement ;-)
Andreas
It has all been done by the IAC, just implement a complete transparant organisation.
http://reviewipcc.interacademycouncil.net/
Why is the IPCC indespensible ?
One could easily argue that in fact the IPCC is holding back scientific advancement. How could a such a large and political body, whose entire existence depends on there being catastrophic global warming as a result of CO2 ever come to the conclusion that there was no such danger ?
Therefore It is indispensible.
Hi Hans,
Reading your text and scrolling down the comments I thought directly of the IAC review of the IPCC, and Hans Erren mentioned it rightfully. It’s a bit ironic, that a discussion on how to change the IPCC should be held on already well-based advices from the IAC review published August 30, 2010. Now, five years later, the discussion restarts? The IPCC is like an ocean steamer with a course that hardly can be influenced, despite all the nice words of its meanwhile former captain Pachauri, addressing the UNFCCC meeting (COP16) in November 2010 at Cancun, Mexico:
“In March this year, the UN Secretary General and I requested the Inter Academy Council (IAC) to carry out a review of IPCC procedures and processes and provide recommendations for strengthening the organization and its functioning. The IAC submitted its report in August 2010, and the last IPCC plenary held in Busan, Republic of Korea, in October 2010 deliberated on the IAC report, and initiated prompt action to consider and implement its findings. We are confident that the IPCC will emerge stronger as a result of this exercise and live up to the expectations of the global community and stand up to intense public scrutiny of its work.”
Marcel,
I think it makes sense to think about the set-up, the mission etc. of IPCC again. While the IAC advice was helpful, the IAC did its work in 2010, or so, five years ago! Quite a few things happened since then. In particular, a new report-series came out - without such errrors like those in the 2007-one. Also the perspective of doing regional assessments, by the regional communities has shown up.
It is certainly worth to re-visit this issue - and if it tunrs out that there is nothing to add to the IAC, then be it so. I personally think there is something to be added.
I will publish my comments after our little disccssion next Monday.
If you want to improve the quality of the IPCC's output, begin by taking several steps back and look at the most egregious failures.
1. How did thousands of propaganda advocacy pieces by activists get cited as if they were peer-reviewed work?
A: no quality control
2. How did a ridiculously incompetent study riddled with bad data, bad statistics, and improper methodology (Mann's hockey stick) become the iconic poster child of climate science? One that likely will be the IPCC's legacy forever.
A: no quality control
The quality control problem begins with the absurd notion held by scientists that every study which has been peer reviewed and published represents accurate, quality work. What's that? You say that you know that some studies are flawed? You don't act like it. If, by your actions, you treat all published studies as accurate, then you have no quality control. No one ever checks the data. No one ever checks the stats. No one even audits. NO QUALITY CONTROL.
If science as an institution continues to be so negligent and reckless about quality and the IPCC continues to embrace the negligence and recklessness, there is no reason for society to put much credence in the results.
Stan, most of what you wrote was simple ranting. How do you come to the conclusion that there would be a "notion held by scientists that every study which has been peer reviewed and published represents accurate, quality work.". But, what I found interesting were your claims 1 and 2:
"How did thousands of propaganda advocacy pieces by activists get cited as if they were peer-reviewed work?" and
"How did a ridiculously incompetent study riddled with bad data, bad statistics, and improper methodology (Mann's hockey stick) become the iconic poster child of climate science?".
We are talking about the IPCC process. Thus, a concrete question to you: to what extent and where were "advocacy pieces" be used in AR5? We had a problem in AR4, do we have it again in AR5? Has the hockstick been assigned a prominent role, aka truth, in AR5? Where?
I am really curious, because I had the impression that the quality of the IPCC process had improved. Since you are making these grand claims, you may be able to help me sorting this out. Please do so.
PS: Mann''s study was not incompetent; the problem was, in my view, that the results were too early accepted as accurate. Some problems emerged later, but that does not diqualify Mann, nor does it belittle the academic progress by developing this approach, which has certainly merits, even it was not perfect. His method received quite a bit of critique, and after an overly enthusiastic acceptance by the IPCC AR3 in 2001, ist was gradually replaced by more robust methodology. That Mann acted like a bully is something different, but compared to your ranting, he behaved allmost like a gentleman.
Lieber Hans,
I am looking forward to an interesting discussion on Monday, and I read with interest related contributions at Klimazwiebel.
I would like to draw people's attention to the "Principles and Procedures of the IPCC" provided at its website (http://www.ipcc.ch/organization/organization_procedures.shtml, http://www.ipcc.ch/organization/organization_review.shtml) as well as a consideration of the recent decisions of the Panel on its future work at the Plenary Session in Nairobi (http://www.ipcc.ch/scripts/_session_template.php?page=_41ipcc.htm). In particular, I would like to note that "the IPCC" is an intergovernmental body, i.e. its 195 member governments, and a scientific body, i.e. the various expert teams taking part in specific tasks. I wonder which of these two is referred to in some of the blog entries.
Beste Grüße
Christiane
I have now given my presentation - the powerpoint-transparencies are here:
http://www.hvonstorch.de/klima/PPT/ipcc/151116.IPCC.Clisap.pptx
The take-home messsage was essentially unchanged.
page 2:
"No known errors in AR5"
What about the error in AR5-WGIII? Remember Tol's flawed paper.
page 3:
"Advantage of lead authors – their own papers are elevated to significant contributions. Own scientific agenda tailored to IPCC usage."
Richard Tol as latest example?
Andreas
Andreas, I do not know what you refer to. Could you explain a bit?
Hans,
really? Never heard? I would prefer Richard Tol explaining it. Please read again his comment #1 and notice his "frustration".
Here is how The Guardian explains it: http://www.theguardian.com/environment/2014/oct/17/ipcc-corrects-claim-suggesting-climate-change-would-be-good-for-the-economy
I think, it's not a minor error. It's an eminently policy-relevant error.
And please not: It's an error in the opposite direction compared to AR4 errors. Made by a lead author who is associated to the GWPF...
Andreas
Andreas
The Guardian has not 'explained it'. The link you provide points to a comment by Bob Ward who had/has a spat with Richard Tol. Lots of axes to grind on Ward's side as well, don't you think? How can you mention him as a neutral source?
In the comment section beneath Ward's comment Tol accuses Ward of lying. This would not go down well with a news journalist (Guardian or otherwise) I guess. Has Ward made a legal case?
How can I dare to link to Bob Ward? Because I hoped for R. Tol to explain it here by himself and until then I provided my first google hit. Feel invited to provide better links.
Let's stick to the facts, Reiner: When AR5-WG3 report was presented in Berlin in April 2014 it contained conclusions based on a paper by R. Tol. This paper (which received also medial attention, message: global warming is beneficial up to 2°C) has got a correction, compare the figures for example here: https://andthentheresphysics.wordpress.com/2015/11/16/short-memory/
I would like to ask R. Tol other questions:
Was the error corrected in the WG3-report, when it was presented in Berlin? Or afterwards? Or are there still contradictions in the report? I'm unsure, because I read here (http://www.desmogblog.com/2015/08/03/richard-tol-s-gremlins-continue-undermine-his-work):
The IPCC sends First and Second Order Drafts of all of the chapters of its reports out to anyone who signs up to be an external reviewer. Any expert can review any part of the IPCC's work. That is a fundamental aspect of the IPCC process which gives it credibility. It's also an aspect that got ignored in this case.
Richard Tol was a Coordinating Lead Author for one of the chapters of the IPCC report. His chapter had an entire section added after the the last round of external reviews was completed. Another section received major revisions at the same time. None of these changes were seen by any of the experts who signed up to review the IPCC report.
Even worse, the material which was added included two tables and two figures derived entirely from Tol's own work. One table and figure directly cites his 2013 paper, reproducing errors from it. The text of the section referred to it, saying “the initial benefits of a modest increase in temperature are probably positive, followed by losses as temperatures increase further.”
Ward discovered this, and he contacted the IPCC to complain. The IPCC made some changes to the chapter. It modified that quote slightly, and it changed some of the data. Changed, not corrected. Some of the changes introduced new errors. Even more confusing, the IPCC continues to cite “Tol (2013)” despite the data it uses no longer matching the data used in “Tol (2013).”
Andreas
My reply to Reiner disappeared after being displayed. (???)
Short version:
Tol corrected his 2013-paper, which received a lot of media attention. "Global warming is beneficial up to 2°", do you remember? You can compare the old and new figure here:
https://andthentheresphysics.wordpress.com/2015/11/16/short-memory/
There has been some changes in WG3-report, but I'm unsure, if these changes have been made before or after the WG3 presentation in Berlin, April 2014. I'm unsure too, if the changes reflect all the corrections made in Tol's paper.
See for further information http://www.desmogblog.com/2015/08/03/richard-tol-s-gremlins-continue-undermine-his-work
Maybe R. Tol can help us here with his insights as lead author.
Andreas
It is truly a shame how statistical lightweights such as Andrew Gelmann get to demolish Richard Tol's groundbreaking work. After all, it is about Tol, the living legend, who can claim to be among "the first to show that [CO2 causes climate change] in a statistically sound way"
@ hvw
In 2014 Tol gave some advice about IPCC reform:
http://www.foxnews.com/opinion/2014/05/20/un-climate-change-expert-reveals-bias-in-global-warming-report.html
"The IPCC should therefore be reformed.
Here are some suggestions:
Away with the infrequent, massive set pieces. Away with alarmism – that has been tried for 25 years, with no discernible impact on emissions. Away with activists posing as scientists. Away with the freshman mistakes.
Just good, sober, solid science. Let the chips fall where they may."
Exactly, just good, sober, solid science without gremlins and polar bears in Paris.
Andreas
Andreas,
I did not know about the activities and claims about misconduct, you have mentioned. I would hope that Richard would comment on them, as he possibly is reading this. I would expect that he would desccribe the events very differently. When I made this statement of no known errors in the AR5 in the discussion, nobody opposed, even if heavyweights, also with respect to WG III, where part of the disucssion.
But, even if so - the issue is about the IPCC and its performance, not of the performance of individual actors. The question would have to be - how did the IPCC deal with the claim of alleged misconduct? Even if your claim of misconduct would be accurate (which you would have to convince me), then the question is - how could the text have been changed without involving oversight by the IPCC process? Or was that process part of the alleged manipulative regime?
Since the claim of misconduct was certainly made to IPCC - how did the machinery, which is supposedly now in place, react to this claim?
The IPCC is not supposed to accurate in all assertions about climate dynamics and impacts (etc)), but to be accurate in its assessment of the scientific insights and discussions about climate dynamics and impacts (etc.). As the process of generating science is a social process, it is not a process of detecting truth but but of constructing explanations given the knowledge o the time. This simple insight seems to be very difficult to digest by many.
Away with activists posing as scientists.
How can those be identified in an objective fashion? Maybe by association with activist organizations? Or rather by hairdo?
Andreas/#23. I checked the comments if something would have been moved to spam - there was your comment, and I moved it to legitimate comments. Hope all is ok now. [I am not checking this box often, in particular when travelling - if somebody has problems like this, please mail to me hvonstorch@web.de.]
Hans,
I would not use the word "misconduct". I see no reason to doubt that Richard Tol did an excellent job es IPCC lead author. He relied correctly on published literature, nobody could know that later one paper was shown to be flawed. It's a normal, unavoidable error.
I've just mentioned it because of two reasons:
1) You suggested the errors in AR4 had a common direction to alarmism. Here we have an example with opposite direction. I think the error occurred by chance.
2) You mentioned possible problems with IPCC authors writing about their own papers. True, this could impose a kind of bias, but a possible bias should not survive a good peer review process (see R. Tol in #1).
Again: I've made no allegations of misconduct. Maybe this is a good moment to thank R. Tol and all IPCC authors for their huge task and excellent work.
Andreas
Go with a hard limit on dates. Has to be published X (12?) months before the start of the synthetic work. This gives at least some chance for critical letters and comments to have been published also. And published means published (the journal issue is actually out...not accepted for publication, in review, minor changes or any of the other BS...archived literature version!) To the minute information is needed much less than careful synthesis and discussion of the literature (varying views etc.)
It is common sense (and was observed in the Climategate emails) that scientists are gaming the system to get their work in under various deadlines. This is unseemly enough. (In theory they should just do science as if IPCC did not exist and then IPCC would sort of look on and do a review. But there are career incentives or scientists to be featured in IPCC and also some of these scientists are not just chronicaling their guesses on how the climate system works but actively advocating for policy response.)
Post a Comment (pop-up window,non-moderated)