A Few Things Ill Considered

A layman's take on the science of Global Warming featuring a guide on How to Talk to a Climate Sceptic.

Monday, April 03, 2006

send this to... Digg it! | Technorati | Del.icio.us | Reddit | Furl | Spurl

The Modelers Won't Tell Us How Confident the Models Are

(Part of the How to Talk to a Climate Sceptic guide)

There is no indication of how much confidence we should have in the models. How are we supposed to know if it is a serious prediction or just a wild guess?

This is a pretty sure indicator your objector hasn't ever looked at an actual climate model prediction.

There is indeed a lot of uncertainty in what the future will be, but this is not all because of an imperfect understanding of how the climate works. A large part of it is simply not knowing how the human race will react to this danger and/or how the world economy will develope. Since these factors control what emissions of CO2 will accumulate in the atmosphere, which in turn influences the temperature, there is really no way for a climate model to predict what the future will be.

What modelers can do, however, is talk about and estimate the climate's sensitivity to CO2, usually in terms of how high the temperature will rise given a doubling of CO2. See the Real Climate glossary entry for climate sensitivity.

So how much certainty is there? This varies from model to model but typically a projection is given as a most likely temperature together with a range that encompasses all the likely values. In the IPCC report "likely" is defined as a 70% probability. If you want a specific number and range, you must chose a specific scenario of emissions over time and a specific model. The IPCC does note that:

The globally averaged surface temperature is projected to increase by 1.4 to 5.8°C over the period 1990 to 2100. These results are for the full range of 35 SRES scenarios, based on a number of climate models.

But what about the certainty of this summary of model predictions? Well, a recent paper by James Annan et al. has attempted to clarify this question by statistically combining the certainties of a wide variety of models in a variety of situations. Focusing on climate sensitivity, they conclude that in terms of the climate's response to a doubling of CO2, the model's say:

"The resulting distribution can be represented by (1.7,2.9,4.9) in the format used throughout this paper. That is to say, it has a maximum likelihood value of 2.9oC, and, using the IPCC terminology for confidence levels, we find a likely range of 2.2-3.9oC (70% confidence) and a very likely range of 1.7-4.9oC (95%). We can also state that climate sensitivity is very likely to lie below 4.5oC(95%). These results represent a substantial decrease in uncertainty over those originally presented in NAS [1979] and in subsequent research. They also imply that the sensitivity range of modern GCMs (2.1-4.4oC) is likely to include the correct value (with greater than 80% confidence)"

So, most likely value is 2.9oC with a 95% probability of falling between 1.7oC and 4.9oC.

There is a summary and discussion of this paper at Real Climate.

Other Guides, by Category



  • At April 04, 2006 1:44 AM, Blogger Peter Hearnden said…


    Linked to this, I'm not sure anyone answered you question in post #3 here http://www.realclimate.org/index.php/archives/2006/03/climate-sensitivity-plus-a-change/#more-274 , which I thought interesting because I'd also ask the same question :). If you look at the RC explaination for climate sensitivity ( http://www.realclimate.org/index.php?p=116 ) it seems to me to be saying CO2 climate sensitivity is CO2 alone. So, bascially, I don't know...

  • At April 04, 2006 9:25 AM, Blogger Dano said…

    CMIP is a good place to start. Or here.

    Most bots haven't been told this exists, BTW.



  • At April 04, 2006 12:09 PM, Blogger coby said…

    Hi Peter,

    I asked James on his blog here and got a nice direct answer (finally!) upon which I based what I said above. Not sure why that one seems kind of hard to pin down, maybe one of those things that is too obvious when you are on the inside to realize that is what people are wondering about...

  • At May 02, 2006 9:44 PM, Anonymous Anonymous said…

    Yet another abuse of statistics. The certainty figure that you have arrived at is dependent upon the correctness of the studies.

    You can't state from this analysis anything about the world with "95% certainty", you can only infer from this result predictions about what results further climate models will arrive at.

  • At May 03, 2006 9:15 PM, Blogger coby said…

    Hi Sam,

    You are largely correct in that the way some people ask this question implies a misunderstanding of what a model is and does. I did however try to turn it into a useful form in my answer.

    You can also state with 95% certainty that this is what will happen if the models are correct! Well, I should say "usefully correct" as after all, all models are wrong, but some are useful.

  • At November 15, 2006 8:26 PM, Blogger jo_bobtwc said…


    "You can also state with 95% certainty that this is what will happen if the models are correct! Well, I should say "usefully correct" as after all, all models are wrong, but some are useful."

    This statement is accurate, but in my opinion does nothing to answer the question at hand. The question you stated was that we don't know how confident the models are. The question you answered is how confident we are of the results of the models if we assume with 100% confidence that the models are accurate. By your own admission, we simply don't have enough data to verify past weather...or climate conditions rendering it impossible to say how valid the models actually are. And, since I don't believe anyone thinks the models are 100% accurate, the overall confidence level must be reduced from the stated 95% confidence. If the models only have an 80% confidence level, the results of the model only have a 76% confidence level. If the model has a 60% confidence level, the model results have a mere 57% confidence. If my made up numbers are accurate (and I have seen no evidence to refute them) then there is anywhere from a 24%-43% chance that global warming is even an issue (by that I mean man-made issue, I'm not saying the earth is not warming). So the question of how confident we may be in the models remains unanswered in my opinion.

  • At November 15, 2006 10:24 PM, Blogger coby said…

    This article probably needs a good rewrite, but really the point is that the question is ill posed. How can you quantify how confident you are in your theories? It is kind of like asking for error bounds on e=mc^2, because after all, new theories may find that this equation is incomplete.

  • At November 16, 2006 8:22 PM, Blogger jo_bobtwc said…

    My numbers were completely made up on the spot, my point is thaat nowhere have I seen a study which validates the reliability of the models themselves, only the results of the models. It seems ill founded to me to implement policy based on models that nobody can quantify as to accuracy. There are too many variables at stake to simply put your trust in a model that may be close but may have gotten lucky (because after all we don't even know all the factors contributing to weather and climate let alone account for them in the models). I am not opposed to models, I would just like some confidence studies to be done on the models themselves before policy is put into effect based on the result of the models. I/we just don't know how much we can trust the models...they may be extremely accurate but until we find out, that isn't good enough.

  • At November 16, 2006 11:11 PM, Blogger coby said…

    What you are asking for just does not make sense. How can one quantify the accuracy of a theory? If you ask "how old is the universe" you will get an answer with error bounds due to uncertainty in measuring red shifts and things like that, but it will not (and how could it), include a factor quantifying our confidence in quantum mechanics as a model.

    The only way to really find out if the models are correct is wait one hundred years, but it will be too late by then to do anything useful with the knowledge.

  • At November 17, 2006 3:34 PM, Blogger jo_bobtwc said…

    I think climate models which are computer programs and quantum mechanics as a model are not very comparable, and I disagree that what I ask for doesn't make sense. I do agree that the only way to truly test the models is to wait and see. The very fact that as you say it is impossible to quantify the models is what I think makes it dangerous to implement policy based on those models. FWIW whether or not it will be too late in 100 years is a matter of opinion and I don't agree with yours. Reducing emissions is certainly not the only way to remove CO2 from the air, we already use CO2 scrubbers in space craft to recycle air quality, and I imagine if mankind decided to do so, something of that nature could be constructed on a large scale assuming it turns out to be necessary. This is by no means all inclusive, just one thought I had. In the end, I think we will have to agree to disagree about the usefulness of climate models because our discussion is becoming increasingly less productive beyond winning a debate.

  • At February 02, 2007 5:57 PM, Anonymous Anonymous said…

    There is no indication of how much confidence we should have in the models. How are we supposed to know if it is a serious prediction or just a wild guess?


    Geez, you are doing all of the work for us Critics. Your making me lazy. Thanks.

    I couldn't agree more.

  • At February 02, 2007 6:06 PM, Blogger coby said…

    Umm..your not supposed to stop reading after the objection.

  • At February 20, 2007 6:06 PM, Anonymous Anonymous said…

    Umm..your not supposed to stop reading after the objection.


    You have Republican arguements that I have never heard before, thanks, I am taking notes.

  • At February 22, 2008 8:42 PM, Anonymous Anonymous said…

    This article is rubbish. Take DePreSys, supposedly one of the latest and best models. It forecasts a 10 year warming in the range 0.1 - 0.5 degrees C to 90% confidence levels.

    Firstly, normal science uses 95% confidence bounds. Pretty clearly doing that would not exclude zero warming. How embarrassing.

    Secondly, as many have said in different ways above, the error estimate is derived as I understand it mainly by multiple runs of the model with variations in the initial conditions and parameters. That does not and cannot account for errors due to fundamental deficiencies in the model. So the actual error range is substantially greater.

    Finally, considering the above the model prediction is uncertain to within a factor of ten. And this is for a ten year prediction. Extrapolating for a century can only increase the uncertainty.

    Basically, the conclusion of present models is that we haven't a clue what will happen. Anything else is not science, it is religious belief.

  • At July 11, 2008 11:10 PM, Anonymous Anonymous said…

Post a Comment

<< Home