In 2009, Hans von Storch and myself published an article
titled ‘"Prediction" or "Projection"?: The Nomenclature of
Climate’ (Science Communication 2009; 30; 534) based on the results of a survey
of climate scientists. We concluded ‘Contrary
to established guidelines, approximately 29% of the respondents associated
probable developments with projections and approximately 20% of the respondents
associated possible developments with predictions.’ A facsimile of the publication was posted on http://academia.edu/ and seems to have drawn
considerable attention. For that reason,
the same questions used to draw these conclusions were repeated in a survey of
climate scientists in 2013. Not so much
has changed. In 2013 approximately 28%
(2008 – 29%) of respondents associated probable developments with projections
and approximately 16% (2008 – 20%)of respondents
associated possible developments with predictions. This is rather puzzling as the definitions of
projection and prediction as used by the IPCC have a considerable history of
debate (see http://scienceblogs.com/stoat/2007/08/23/projection-prediction
for example - and worth the effort of reading) and should have been a matter
that was resolved long ago.
Out of curiosity, I plotted the definitions selected against
the number of years of experience of the respondent. The data shows that the less number of years
of experience the more inclined the respondent to think they are making
predictions. With 0-5 years of experience,
approximately 35% of the respondents claimed that ‘From a scenario simulation
prepared with climate models, scientists are more inclined to make a
prediction; with 6 – 10 years of experience this was reduced to approximately 21%
of the respondents; 11-15 years of experience to approximately 17% of respondents;
and among those with more than 15 years of experience, the percentage dropped
slightly to 15%.
While the use of an operational definition (see Percy
Williams Bridgman) might lead to much less confusion, this does not seem to
have been a practice widely followed.
This brings me to the question ‘Is there something fundamentally lacking
in science education?’
And, without any intention of further comment, one might
wonder the same about education and journalism.
Posts
18 comments:
Very interesting observation, Dennis.
Dennis
Just a thought - I wonder if the climate scientists who think they are in the business of making predictions have a rather general definition of prediction. In positivist sciences, like physics or economics, there is the formula of a "hypothesis which would predict..." You then set up your experiment, or collect observation data, and see if the prediction is borne out by data. If yes, the hypothesis counts as true. If not, you have to discard the hypothesis.
Clearly the IPCC has a different definition and the deviation of (especially younger) colleagues may just indicate that they do not sing from the same hymn sheet. Rather than needing better education you could say they are in line with language use practiced outside the canonical (or sectarian?) climate community.
Hi Reiner
When talking hypothesis I am more familiar with the terminology 'fail to reject' or 'reject'. I would say the hypothesis seems more to suggest, than predict. I can look at the differences according to having participated in IPCC activities and not participated, rather than length of experience. I will post a comment concerning that in an hour or so.
A couple of weeks ago there was a discussion at climatedialogue.org about regional climate models. There was some disagreement between the three experts in the comments, to some extent caused by Roger Pielke's (senior!) opinion that the models give predictions rather than projections.
Very interesting discussion, you can find it here: http://www.climatedialogue.org/are-regional-models-ready-for-prime-time/
Andreas
To see if involvement with the IPCC has a bearing on the definitions of projection and prediction:
IPCC involvement/A description of the most probable outcome best defines
Projection = 25% Prediction = 68%
No IPCC Involvement/ A description of the most probable outcome best defines
Projection = 31% Prediction = 61%
It seems that the level of involvement with the IPCC does not play much, if any, role in how a scientist defines a most probable outcome.
IPCC involvement/A description of a possible outcome best defines
Projection = 67% Prediction = 16%
No IPCC Involvement/A description of a possible outcome best defines
Projection = 59% Prediction = 16%
It seems that the level of involvement with the IPCC does not play much, if any, role in how a scientist defines possible outcome
IPCC Involvement/From a scenario simulation prepared with climate models, scientists are more inclined
to make
Projection = 83% Prediction = 12%
No IPCC Involvement/From a scenario simulation prepared with climate models, scientists are more inclined to make
Projection = 69% Prediction = 23%
It seems that more scientists not involved with the IPCC are more inclined to think that from a scenario simulation prepared with climate models, scientists are more inclined to make a prediction as when compared to scientists who have been involved with the IPCC.
Which suggests that scientists with less experience, ie. not yet any involvement with the IPCC, are more inclined to claim that the output from using an IPCC scenario is a prediction. However, the claim that it is a prediction is limited to only 23% of the scientists who claim to have never had involvement with the IPCC.
This is a story about a man named Jack. Jack was born in 1985, at the very early stages of the global warming debate. Jack had his fist dream shattered at the age of 5 when he learned that there was no tooth fairy. The seventh year was bad too. Santa it seemed was as fickle as the tooth fairy. At age 10 Jack began to travel by bus each day to and from school. The talk on the bus was only about climate change, species extinction, weather induced catastrophes and a future of doom and gloom. All day long, between classes, on weekends, field trips, the same conversation was unrelenting. By 12 years old, this had become the norm. At 12 Jack also had another setback. He learned his parents likely had sex. The only constant in his life now was Global Warming and all it entailed. Setting his moral compass Jack assigned himself the mission of saving the planet. Like all naïve 12 year olds, what he chose to hear was a given fact that needed no further substantive evidence. At school he took all the environmental studies classes available, joined the Introductory Pupils’ Climate Class, and later on adopted every word of the early IPCC reports. Climate change in Jack’s mind was an uncontested phenomenon. Jack spent his teenage nights between nightmare of climate catastrophe and reading popular science. Jack was a dull boy. But it paid off when the board of admissions told Jack that he had a place in the environmental studies program. Jack worked his way through university with a vengeance, earning a PhD. Now in his early professional career how can we expect Jack to accept that his entire life had been based on speculations,projectiosn and not predictions? The tooth fairy, Santa, parental sex, ok – but not climate change!
Hi Andreas
You wrote
“A couple of weeks ago there was a discussion at climatedialogue.org about regional climate models. There was some disagreement between the three experts in the comments, to some extent caused by Roger Pielke's (senior!) opinion that the models give predictions rather than projections.”
The claim that “predictions” and “projections” are distinct from each other is false. Projections, as applied by the IPCC are just “what if” predictions, based, for instance on specified emission scenarios.
Most in the impacts and policy communities correctly interpret “projections” as predictions in this context.
I discussed this distinction several years ago in my paper
Pielke Sr., R.A., 2002: Overlooked issues in the U.S. National Climate and IPCC assessments. Climatic Change, 52, 1-11. Pielke Sr., R.A., 2002: Overlooked issues in the U.S. National Climate and IPCC assessments. Climatic Change, 52, 1-11.
Best Regards
Roger Sr.
@Roger
The difference between a prediction and projection is not only problematic in relation to climate change:
http://wiki.answers.com/Q/What_is_the_difference_between_projection_and_prediction
The answer for the climate change debate might lie in the acceptance of a single operational defintion that is used consistantly and stated clearly. For example, a climate scientist at a press release might well use the word prediction but in doing so without qualification the statement is transfered into the common use of the word by the time it reaches the public. Gone is any notion of the what if in the "“what if” predictions" as defined by the IPCC. Science is about precision, and this should be clear in scientific statements. Not everyone is aware of the IPCC terminology, where the prediction is only as good as the 'what if' that drives it. Furthermore, is the 'what if' that is driving the climate prediction, fiction, projection or prediction. The final prediction(s) of climate change are only as good as the drivers used to determine them. This is pretty much an absent discussion in the message reaching the public.
Roger Pielke sr.,
thanks for participating at climatedialogue. I enjoyed the discussion very much.
In my (layman) opinion I think it's clear, that a single run is a projection and not a prediction at all. But if we take a look at the whole picture (i.e. multi model projections) I would agree that there's some kind of prediction.
We predict that the real temperatures will probably be with the bands of uncertainty.
Best regards,
Andreas
Hi Andreas - A single run, in conventional mathematical definitions, is a prediction. Muliple model runs with slightly different initial conditions, forcings, parameterizations etc can be used to create an ensemble of model predictions. An individual model run in that context is generally called a "realization". The use of the term "projection" just introduces confusion.
None of the ensembles used in multi-decadal climate predictions accurately include all of the natural and anthropogenic climate forcings and feedbacks, and thus should not be used to claim robust envelopes of what could occur in the coming decades.
Best Regards
Roger
Hi Dennis
A solution to the misunderstanding of the uses of the terms "prediction" and "projection", would be to simply define a "projection" as a "what if prediction".
This can be easily accomplished in any communication either in the literature or to policymakers and the impacts communities. They would then know, that if a certain path of climate forcings occurs, this is the forecast.
With this framework, we can validate the multi-decadal models in hindcast since we can estimate, within an uncertainty, what the climate forcings have been.
In this testing framework, however, the multi-decadal climate model predictions have performed quite poorly even with respect to current climate, and essentially no regional and local skill with respect to changes in climate statistics, as we discuss in http://pielkeclimatesci.files.wordpress.com/2013/05/b-18preface.pdf
This lack of skill needs to be commuicated to the impacts and policy communities (or my conclusion refuted with robust testing).
Best Regards
Roger Sr.
Roger/7; Andreas/9
Roger, when you assert "The claim that “predictions” and “projections” are distinct from each other is false. Projections, as applied by the IPCC are just “what if” predictions, based, for instance on specified emission scenarios. " I do not agree. Terminology is not "false", but a matter of agreement. And the IPCC has formulated a terminology, which in the present context (see Bray and von Storch-paper, where this is made very explicit) clear and thus applicable.
Technically, most if not all projections are conditional/what-if forward simulations, as we run a GCM or RCM with assumed forcing and a random initial state consistent with climatology.
But the definition is more of the sort "description of a possible outcome, independent of the method". A prediction, also independent of the methodology, is an "effort for describing a probable outcome".
This distinction, possible vs. probable is an important difference for dealing with the objects, which may look rather similar, and the IPCC is to be applauded for working the difference out. The advantage is that we can use the same terminology; if you do not want to follow this terminology, this is fine, but if others do, they are not "false".
Andreas, a single run can very well be a prediction, namely, when we think we know both the initial state (what we hardly do, in particular when looking at the ocean, but efforts are underway to do just that) and the drivers (which may be described by mere inertia for the next few years - in terms of GHGs and aerosols.)
An ensemble of simulations describes a set of possible developments, and can not be framed as constituting a probability density function of an overall set of admissible scenarios; actually, the IPCC is also not doing this.
Hi Hans - I have had to post my response to your comment at http://www.climatedialogue.org/are-regional-models-ready-for-prime-time/ since my reply was too large to be accepted by your weblog. I have e-mailed you to see if you can permit it to appear here.
Best regards
Roger Sr.
At Hans's suggestion, I have segemented into three comments.
"Hi Hans –
"The attempt to distinguish between the terms “projection”and “prediction”, whether by the IPCC or others, has introduced an unnecessary confusion to the impacts and policy communities regarding the skill of regional and local multi-decadal climate model runs.
"The2007 IPCC defines“projection”,“climate projection” and“climate prediction” as [http://www.ipcc.ch/publications_and_data/ar4/wg1/en/annexes.html]
“Projection -A projection is a potential future evolution of a quantity or set of quantities, often computed with the aid of a model. Projections are distinguished from predictions in order to emphasize that projections involve assumptions concerning, for example, future socioeconomic and technological developments that may or may not be realised, and are therefore subject to substantial uncertainty.
Climate prediction - A climate prediction or climate forecast is the result of an attempt to produce an estimate of the actual evolution of the climate in the future, for example, at seasonal, interannual or long-term time scales. Since the future evolution of the climate system may be highly sensitive to initial conditions, such predictions are usually probabilistic in nature.
Climate projection - A projection of the response of the climate system to emission or concentration scenarios of greenhouse gases and aerosols, or radiative forcing scenarios, often based upon simulations by climate models. Climate projections are distinguished from climate predictions in order to emphasize that climate projections depend upon the emission/concentration/radiative forcing scenario used, which are based on assumptions concerning, for example, future socioeconomic and technological developments that may or may not be realised and are therefore subject to substantial uncertainty”
Clearly a "climate projection" IS a "climate prediction, using the IPCC definitions, when the emission/concentration/radiative forcing scenario actually has occurred. This can be done using hindcast model runs, and these model runs can be quantitatively tested.
The source of much of this confusion with respect to the use of the terminology "prediction" and "projection" appears to be in the assumption by the IPCC and others, that while weather prediction is an initial value problem, climate prediction (on multi-decadal time periods) is a boundary value problem;the term “projection”then being reserved for the later.
However, as succinctly stated by F. Giorigi
F. Giorgi, 2005 : Climate Change Prediction: Climatic Change (2005) 73: 239. DOI: 10.1007/s10584-005-6857-4
“….because of the long time scales involved in ocean, cryosphere and biosphere processes a first kind predictability component also arises. The slower components of the climate system (e.g. the ocean and biosphere) affect the statistics of climate variables (e.g. precipitation) and since they may feel the influence of their initial state at multi decadal time scales, it is possible that climate changes also depend on the initial state of the climate system (e.g. Collins, 2002; Pielke, 1998). For example, the evolution of the THC in response to GHG forcing can depend on the THC initial state, and this evolution will in general affect the full climate system. As a result, the climate change prediction problem has components of both first and second kind which are deeply intertwined.”
You clearly adopt the assumption that the multi-decadal climate runs represent a boundary value problem when you use the term "simulation" and write
“Technically, most if not all projections are conditional/what-if forward simulations, as we run a GCM or RCM with assumed forcing and a random initial state consistent with climatology.”
The 2007 IPCC Glossary has no definition for "simulation".
The AMS Glossary defines a “numerical simulation” as [http://glossary.ametsoc.org/wiki/Numerical_simulation]
“A numerical integration in which the goal is typically to study the behavior of the solution without regard to the initial conditions (to distinguish it from a numerical forecast).
Thus the integrations are usually for extended periods to allow the solution to become effectively independent of the initial conditions.”
This is a view of this definition “simulation” that is being shown to be flawed with respect to the real world climate; e.g. see
S. Lovejoy, 2013: What Is Climate? EOS 2 JAN 2013 DOI: 10.1002/2013EO010001. http://onlinelibrary.wiley.com/doi/10.1002/2013EO010001/pdf
and
Rial, J., R.A. Pielke Sr., M. Beniston, M. Claussen, J. Canadell, P. Cox, H. Held, N. de Noblet-Ducoudre, R. Prinn, J. Reynolds, and J.D. Salas, 2004: Nonlinearities, feedbacks and critical thresholds within the Earth's climate system. Climatic Change, 65, 11-38.http://pielkeclimatesci.wordpress.com/files/2009/10/r-260.pdf
Much more important, however, than the name we use is how do you test the quantitative accuracy of the multi-decadal climate model results, regardless of whether they are called “projections”, “predictions”,“simulations” or “forecasts”?
I have presented evidence from the peer reviewed literature; see my post at
http://www.climatedialogue.org/are-regional-models-ready-for-prime-time/
that shows they are not yet robust tools to be used to make multi-decadal climate (predictions; projections) of changes in regional and local climate statistics.
Do you have peer reviewed results that counter this conclusion? I also would value your response to summarize how you conclude we should assess the skill of multi-decadal climate predictions of changes in climate statistics.
Best Regards
Roger Sr.
Hi Hans
I have another query for you to address.
You write
“But the definition [of a projection] is more of the sort "description of a possible outcome, independent of the method". A prediction, also independent of the methodology, is an ‘effort for describing a probable outcome’.
This distinction, possible vs. probable is an important difference for dealing with the objects, which may look rather similar, and the IPCC is to be applauded for working the difference out. The advantage is that we can use the same terminology; if you do not want to follow this terminology, this is fine, but if others do, they are not ‘false’.”
What you are telling us, in my view, are that the impact and policy communities are basing their studies and decisions on model results that are “possible” based on the multi-decadal climate model runs. This means, of course, that any of the results I present in my papers on land use change, as just one example, are “possible” and should be used as part of the assessment of risk. I have stated, however, that such model runs are “sensitivity studies” only.
This framework that the results are “possible” and thus are a subset, perhaps a small subset of what will occur in the coming decades, is not a view that is communicated to the impacts and policy communities. Indeed, a review of multi-decadal climate impact papers makes frequent use of verbs such as “will” (rather than “could”), etc when presenting the model results for the coming decades. Figures are usually presented as, for example, the decade 2050-2059, etc.
If you agree that a “projection” is a “sensitivity study” than we are actually in closer agreement than I thought. :-)
However, you also write that, with respect to the distinction between a “projection” and a “prediction”,
“the IPCC is to be applauded for working the difference out. The advantage is that we can use the same terminology; if you do not want to follow this terminology, this is fine, but if others do, they are not "false".
It is fine, of course, to use a different word (i.e. “projection” instead of a “sensitivity study”) as long as the meaning is clear and they mean the same thing.
The IPCC, unfortunately, has failed to properly communicate that the “projections” are only providing, at best, what is possible, and does not in any way, provide a demonstration of quantitative likelihood or evidence of any actual regional and local prediction skill on multi-decadal time periods.
I appreciate you engaging in this discussion, as you have permitted a clear statement of what the IPCC is actually doing when they introduce the terminology “projection”.
With Best Regards
Roger Sr.
I posted the following comment in Climate Dialogue in reply to Roger Sr.'s comment, also posted there:
Dear Roger,
I applaud your comment re. projection vs. prediction and I offer the following paragraph from the paper Koutsoyiannis et al. (2011), which is related to this issue:
“In fact, it is the IPCC that uses climate model outputs as predictions. Calling these by another name, such as “credible quantitative estimates of future climate change” (Randall et al., 2007, p. 591) does not change the essence. For example, in IPCC (2007, Fourth Assessment Report—AR4; Summary for policymakers, p. 15), we read (our emphasis): “It is very likely that hot extremes, heat waves and heavy precipitation events will continue to become more frequent”. This is one of a total of six occurrences of the word “will” in a similar context (in the three next pages of the section “Projections of future changes in climate”), the last one being “...anthropogenic carbon dioxide emissions will continue to contribute to warming and sea level rise for more than a millennium”—not to mention the over 20 appearances of expressions such as “it is expected”, “it would”, etc. The same style is adopted in other IPCC documents, including the Freshwater Chapter (Kundzewicz et al. 2007). The conviction that climate model outputs are credible predictions for the future propagates beyond IPCC texts, often without mentioning their origin (for which we cannot imagine anything else but climate models). Taking as an example the most cited “climate change” document (as seen by a Google Scholar search), the so-called Stern Review, we may see that in a single page (Stern 2006, p. vi of the Executive Summary) the word “will” appears ten times. The same is also obvious in many papers and conference talks, where sometimes the “projections” are presented as facts. “
Best regards,
Demetris
Reference
Koutsoyiannis, D., A. Christofides, A. Efstratiadis, G. G. Anagnostopoulos, and N. Mamassis (2011), Scientific dialogue on climate: is it giving black eyes or opening closed eyes?, Hydrological Sciences Journal, 56 (7), 1334–1339.
Post a Comment (pop-up window,non-moderated)