Notes on CRASSH debate, Clare College, Cambridge, 29 Sep 2010

Is Building Bigger and 'Better' Climate Change Models a Poor
Investment?

My position is that we need more investment in climate science, but
that building bigger climate models is a poor investment compared to
the alternative, which I will describe.  In climate science we can
distinguish between what we might term 'academic' climate science and
'policy' climate science.  These both have their place, but
difficulties arise when science funders think they are paying for
policy climate science but actually get academic climate science.
This is what I want to talk about, and bigger climate /simulators/ are
at the heart of the issue.

In a nutshell, academic climate science is about understanding how the
large-scale regularities that we see in the weather, particularly the
mean or average weather, emerge as a result of microscale processes in
the atmosphere and oceans.  Climate simulators based on microscale
principles can be used to demonstrate---through computation---the
emergence of these large-scale regularities.  This requires a big
climate simulator with a high resolution in space and time---such
simulators are known as General Circulation Models, or GCMs.  There
have been some notable successes with the latest generation of GCMs,
such as a much better understanding of the El Nino Southern
Oscillation, and the way that this affects the mean weather on a
planetary scale.  There are still some problems, though, such as an
unrealistic representation of the Inter-Tropical Convergence Zone.  It
is thought that building yet bigger GCMs with yet higher resolution
will resolve some of these problems.

*Academic* climate science is a fascinating area, and someone more
knowledgeable than myself, for example Prof Tim Palmer, would be able
to keep you spellbound for several hours with a description of what
GCMs can and cannot do, and why, and what this tells us about the
nature of the climate system.  **For *policy* climate science,
however, the focus is different.** Policy climate science is about
*climate as the /distribution/ of weather*.  This focus on the whole
distribution (rather than mainly the mean) is important because we
humans are affected directly by extremes: heatwaves, for example.  We
are also affected indirectly through the impact of extreme weather on
our infrastructure, and our environment.  So understanding the
distribution of weather, especially the tails of the distribution, is
crucial for policy.

Now a prediction of the *distribution of future weather* has three
stages.

* First, there is the distribution of weather within a GCM.  This
  requires multiple evaluations of the GCM under different initial
  conditions, because weather is a manifestation of the chaotic nature
  of the underlying equations.  An analogy would be my journey into
  work.  Making the journey again and again, where each trip is subtly
  different, allows me to infer a /distribution/ of journey times.
  With GCMs, I would expect the distribution of weather to be
  strangely-shaped, owing to the well-known features of forced
  dynamical systems, for example that they oscillate and bifurcate.

* Second, we must account for the limitations of the GCMs.  For all
  the progress that has been made in the last twenty years, GCMs
  provide a very limited representation of a system as complex as
  climate.  This is not going to change in the foreseeable future,
  unless there is a total revolution in computing.  To capture the
  effect of these limitations on uncertainty we need to repeat the
  weather calculation in the first stage, many times.  This repetition
  is across the range of all GCM versions that are consistent with our
  understanding of the climate dynamics and physics.  For any given
  GCM, there is a huge number of versions meeting these conditions.

* Third, we need to account for uncertainty about the future.  This is
  typically represented by alternative scenarios.  So we need to
  repeat the first two stages for each of the future scenarios that we
  can envisage.  The latest thinking proposes /four/ Representative
  Concentration Pathways (RCPs), out to 2100, so these would be our
  future scenarios.

Repeated runs of a GCM at different choices of initial conditions, for
different versions, and over different future scenarios would be
loosely termed 'replications' in Statistics, and that is how I will
refer to them here.

So, how many GCM replications might we need to estimate a distribution
for future weather?  As a gross approximation, my guess is that we
will need about 1000.  The crude number is probably much larger, but I
am assuming that we can reduce it to 1000 by using careful sequential
experimental design.  The design and analysis of experiments is
something that Statisticians have studied for nearly 100 years, and it
has had profound effects in fields such as agriculture, engineering,
and medicine.  The last 20 years has seen the development of methods
specifically for computer experiments.  So far, these methods have
been little used in climate science, although I think that is
beginning to change.

So, how long would it take to do 1000 replications?  GCMs currently
run at about one hundred simulated years per calendar month on the Met
Office supercomputers, and we might be able to run in parallel batches
of ten without compromising the daily weather forecast.  Allowing for
the cost of also running a regional climate simulator (for downscaling
to policy-relevant spatial and temporal scales), 1000 replications
might take twelve years.  Twelve years is quite a long time to wait,
so can we reduce it?  If

  (i) Moore's Law continues to hold, and

 (ii) the Met Office gets a new supercomputer every three years, and,
      crucially,

(iii) there is /no/ increase in GCM resolution,

then this comes down to seven years.  I should add that we are not
starting from scratch with this kind of experiment: many parts of it
are already in place, and UK scientists have been at the forefront of
this effort, for example in the UKCIP study.

So my case in a nutshell is that *bigger climate simulators* are
better for *academic* climate science, but *more replications of
current climate simulators* are better for *policy climate science*.
This is because it is a basic principle that replications are
necessary to quantify uncertainty, and there is a /lot/ of uncertainty
about future weather.

To return to my first point, science funders have to be clear about
which type of climate science they want.  To make a /policy/ case for
bigger GCMs it must be shown that they lead to an improved assessment
of uncertainty, particularly in the tails of the weather distribution.
And this must be despite reducing the number of replications that are
possible for the same computing budget.  To my knowledge, such a case
has yet to be made.  I would certainly support more investment in
climate science, but I do not think bigger GCMs are the right
investment for /policy/ climate science, particularly when there is a
pressing need for policy decisions.

Thank you.

Jonathan Rougier
Department of Mathematics
University of Bristol, UK