PROJECT OUTLINE
ABOUT UNCERTAINTY IN MODELS
HOME PAGEProject NewsPROJECT OUTLINE About Uncertainty in Models Background Project Aims Outputs of Project Facts Themes People & Structure CONTACTS DISSEMINATION Papers & Presentations Events DownloadsRESOURCES ToolkitReading ListRelated Projects Glossary MEMBERS ONLY Project Team Management Board Advisory Panel WIKI
Modelling is a vital part of research and development in almost every sphere of modern life, as a few examples will suffice to demonstrate:
Climate models, incorporating very complex equations of atmospheric and sea surface processes (and even more complex models that widen the scope to include full ocean modelling) are used for weather and climate forecasting. Some of the largest computers in the world are used to run weather forecasting models. Running such models to forecast long-term climate change and its impacts is even more computationally demanding, yet hugely important for national and international policy-making.
In order to predict the behaviour of nuclear power reactors, nuclear waste storage facilities and high-energy physics experiments, very complex models are used incorporating the latest nuclear physics theory. The safety of such installations depends in part on the accuracy of these models, and the models are an integral part of their monitoring and regulation..
In developing large engineering projects, it is standard practice to build a theoretical model of the proposed equipment in order to predict its behaviour and to set the design parameters to obtain optimal results. This avoids the need to make many expensive prototypes, and is used for everything from car engines to aircraft wings to the hulls of ocean racing yachts.
Those who rely on models to understand complex processes, and for prediction, optimisation and many kinds of decision-and policy-making, increasingly wish to know how much they can trust the model outputs. Uncertainty and inaccuracy in the outputs arises from numerous sources, including error in initial conditions, error in model parameters, imperfect science in the model equations, approximate solutions to model equations and errors in model structure or logic. The nature and magnitudes of these contributory uncertainties are often very difficult to estimate, but it is vital to do so. All the while, for instance, different models produce very different predictions of the magnitude of global warming effects, with no credible error bounds, sceptics can continue to ignore them and pressure groups will seize upon the most pessimistic predictions.
The MUCM project will develop a technology that is capable of addressing all sources of uncertainty in model predictions and to quantify their implications efficiently, even in the most complex models. It has the potential to revolutionise scientific debate by resolving the contradictions in competing models. It will also have a radical effect on everyday modelling and model usage by making the uncertainties in model outputs transparent to modellers and end users alike.
In particular, this technology will provide a unified framework in which to address the following tasks that frequently arise in the use of models.
Uncertainty Analysis. This is the task of quantifying the overall uncertainty in model outputs. In principle, this involves taking account of all sources of uncertainty, although it has often in the past been interpreted more narrowly as quantifying the uncertainty in outputs due to input uncertainty.
Sensitivity Analysis and value of information. A related task is that of identifying how the model output responds to individual inputs (or other uncertain factors). Variance-based sensitivity analysis summarises the sensitivity by identifying how much of the output variance is due to each contributory source of uncertainty. Value of information analysis identifies how much each source of uncertainty impacts on decision-making. These tools give modellers insight into how their models behave (often pointing to bugs in coding or model failures) and allow model users to prioritise research to reduce uncertainty.
Calibration and data assimilation. Calibration, the process of adjusting uncertain model parameters to fit the model to observed data, is typically a very demanding task that can involve many man months or even years of effort. Data assimilation, in which data are used to adjust the state vector of a dynamic model, is equally demanding and the subject of quite intensive research in its own right. Gaussian process methods can not only perform these tasks more efficiently, but also properly characterise how they reduce uncertainty about those parameters and state vector (and hence reduce uncertainty in model outputs).
Validation and structural uncertainty. It is often said that models cannot be validated since no model is perfect. Nevertheless, it is possible to validate the combination of a model with a description of uncertainty, simply by computing implied probability distributions for test data and then verifying that they lie within the bounds of those distributions. However, this requires all forms of uncertainty to be accounted for, including uncertainty in model structure, and cannot be addressed by conventional Monte Carlo analyses. Bayesian statistical methods are able to tackle this problem, and indeed a model for model discrepancy underlies the calibration techniques.