Methodologies for uncertainty assessment

Uncertainty Matrix

Description

The uncertainty matrix (Walker et al, 2003, Janssen et al., 2003) can be used to identify and prioritise the most important uncertainties in a given model study. The matrix shown below is an example of a project specific adaptation of the more general uncertainty matrix shown in Fig. 5, above.

For a specific application the different sources of uncertainty are listed in the rows and the type of uncertainty associated to each source is noted and characterised. This may be done either quantitatively or, as shown in the above figure, qualitatively. The importance of each source may then be characterised by a weighting depending on its impact on the modelling study in question. The Sum of uncertainty may then be assessed, e.g. by use of the error propagation equation (Section 4.2).

It may not be possible to identify all sources of uncertainty and/or assigning correct weightings from the project start. The matrix may thus be used interactively by adding or reassigning weights during the modelling process as more insight into the system is gained.

Resources required

The uncertainty matrix can be used as a heuristic instrument and screening tool. One can draft a gross list of uncertainties to be addressed and give a priority (e.g. high, medium, low) to each cell by going over all the cells of the matrix systematically and reflecting on the question whether that cell may be relevant to include in an uncertainty assessment of the case at hand. This can be done by an individual researcher; in a dialogue in a research team; or in a dialogue between a research team and the client; or, in a wider dialogue between the research team, the client and a group of stakeholder representatives.

The resources required will depend on the way the matrix is used in the process. The broader the dialogue is in which the tool is used, the more resources will be required.

Strengths and limitations

+         Forces the analyst to systematically consider a broad range of possible sources and types of uncertainty. It thereby reduces the chance that important uncertainties are overlooked.

+         Can be used quick and dirty/back of the envelope or more sophisticated and comprehensive

+         Facilitates and structures a dialogue on uncertainty, both within a research team and in communication with clients, peers and stakeholders

-         Strongly relies on expert judgement, the result is likely to be sensitive to the composition of the group of analysts involved in applying the matrix

-         Yields mainly qualitative insights, has limited value if it is not complemented by some form of quantitative analysis

References

Walker WE, Harremoës P, Rotmans J, Van der Sluijs JP, Van Asselt MBA, Janssen P and Krayer von Krauss MP (2003) Defining Uncertainty A Conceptual Basis for Uncertainty Management in Model-Based Decision Support, Integrated Assessment, 4(1), 5-17.

Janssen PHM, Petersen AC, van der Sluijs JP, Risbey JS and Ravetz JR (2003) RIVM/MNP Guidance for Uncertainty Assessment and Communication: Quickscan Hints & Actions List. RIVM/MNP, ISBN 90-6960-105-2, 2003 (available from www.nusap.net)