Statistical significance of climate change ensembles?

The issue of statistical significance of an ensemble of climate change simulations has been examined in a paper by Hans von Storch and Francis Zwiers: Testing ensembles of climate change scenarios for"statistical significance" now published by Climatic Change 117 (2013): 1-9 DOI: 10.1007/s10584-012-0551-0. The article was published as open access, and a copy can be downlaoded from academia.edu. Earlier on-line versions were available.



The applicability of the concept of "statistical significance" is discussed for the case of expected future climate change. The concept is sometimes sloppily used, associating the impression that scenarios would be representative of the future (and not mere possible futures), while this is hardly the case. Indeed, almost always, apart of a very limited case, the concept is not applicable. One reason is that the population of valid scenarios can not be described, so that a random variable can not be defined.



When applying the concept of "statistical significance",a (null) hypothesis is needed to be formulated, which features a well-defined random variable. Significance then means that the hypothesis may be rejected with a pre-defined probability, which exploits the random structure of the considered random varibale (under the null hypothesis). For further details refer to the text book "Statistical analysis in climate research" by Hans von Storch and Francis Zwiers.

In the article, different cases of random variables, or populations of events conceptualized as random, are considered:

a) “All climate scenarios”—in which case, we would have to determine what is meant by “all”. This presumably means climate scenarios produced with all conceivable models, all conceivable emission scenarios, and all conceivable downscaling approaches, plus an understanding of how the available 11 scenarios were selected from that broad population. Moreover, this would need qualification. All emission scenarios that are deemed valid by contemporary economists? Or all followers of a certain school of thought? All climate models? Note that in the IPCC Special Report on Emissions Scenarios probabilities or relative likelihoods were not assigned to the various scenarios that were described.

There is likely no way to make an assertion about “all climate scenarios”, because that set is simply not definable. Nevertheless, there have been attempts to quantify uncertainty from models, forcing scenarios and downscaling, for example, using complex hierarchical Bayesian models, and serious thought has been given to the basis for the interpretation of statistics calculated from multimodel ensembles.

 

b) “Climate scenarios based on a specific emissions scenario”—in which case we would want to make statements that are specific to individual emission scenarios. This is the approach that was used by the Intergovernmental Panel on Climate Change (IPCC) in its 4th Assessment Report, in which it assessed a likely range for global projections of future change under each of several different emissions scenarios. The IPCC was also confronted by the question of the interpretation of the available ensemble of models, and thus the assessments of global projections included an aspect of expert judgement that drew on physical understanding as well as the spread of available ensembles of climate change simulations.

 

c) “Climate scenarios based on a specific emission scenario and produced with a restricted class of models”—this is closer to being a tractable problem if the class of models is sufficiently restricted, albeit still a very large problem. For example, one might consider models that share the same code, but in which parameter settings have been varied systematically using a well designed sampling scheme, such as is used in the climateprediction.net-project.

 

d) “All available climate scenarios”—in which case this is no longer a problem of statistical inference because the population is completely known. Therefore we can simply state: summer rainfall decreases in most locations in all n = 11 cases, with some exceptions at a few locations in m  = 3 or four cases. No attempt is made to quantify statistically how additional, so far unknown scenarios, would play out. Nevertheless, the lessons learned from the responses to forcing simulated by a limited collection of physically based climate models under a limited range of forcing scenarios, together with our understanding of physics and how changes in the composition of the atmosphere affect the planet’s energy balance, should provide considerable insight about the general nature of the responses that can reasonably be expected from other physically based climate models and under other forcing scenarios.