When is uncertainty assessment required?

Uncertainty aspects

Uncertainty aspects are important throughout the modelling process. Thus, uncertainty is considered explicitly in 14 out of the 45 tasks in Fig. 3. But uncertainty is treated in different ways at different stages of the process. The four main actions of dealing with uncertainty may be characterised as:

A. Identify and characterise sources of uncertainty (Fig 3)

The various sources of uncertainty need to be identified and characterised in connection with the tasks Describe Problem and Context and Determine Requirements in Step1 Model Study Plan (yellow arrows in Fig. 3). This should be done by the water manager but typically after a dialogue with relevant stakeholders. Depending on the framing of the model study some of these uncertainties may be located as external non-controllable sources. It is crucial that uncertainty is considered explicitly so early in the definition phase of the model study. Here uncertainties are seldom quantified. It is also at this early stage that the first analyses are made on the acceptable level of uncertainty and the expected model performance.

B. Modeller reconsiders uncertainty and performance criteria  (Fig 3 )

The modeller has to reconsider several times during the modelling process if the accuracy performance criteria formulated by the water manager are realistic for the particular model study. Most importantly (full green arrows in Fig. 3), the modeller has to do this in connection with preparation of a proposal, and when, after having made the first model test run, s/he re-evaluates the performance criteria before the model validation tests in the next step. Furthermore (dotted green arrows in Fig. 3) in connection with reporting at the end of each step, the modeller has the opportunity to reconsider whether the originally promised performance criteria are still realistic, given the new information produced during the previous tasks in the respective step.

C. Reviews – dialogue- decisions (Fig 3)

The last task in each step is a dialogue or decision task where a dialogue between water manager and modeller takes place. Often independent reviews are conducted as a basis for the decision and stakeholders and/or the general public are involved in the dialogue. As part of this dialogue, uncertainty aspects become important, e.g. when discussing whether there are sufficient data to proceed with the modelling, or whether the uncertainty of the model simulations are at a level where the results can be expected to be useful. The reviews and the stakeholder dialogues are also important platforms for a reflection on whether the assumptions made in the model are realistic and on how the study outcome may be influenced by the implicit and explicit assumptions made in the model. In many cases, more than one assumption is scientifically tenable. If such assumptions influence the model outcome, then the ignorance regarding which assumption is the best assumption can be an important source of uncertainty.

D. Uncertainty assessment and propagation  (Fig 3)

Towards the end of the step Calibration and Validation and the step Simulation and Evaluation there are two tasks dealing exclusively with uncertainty assessment. In the first (Uncertainty Analysis of Calibration and Validation) an assessment is made of the model uncertainty related to simulations in the validation test cases. This is used for evaluating possible biases in model simulations and assessing if the model performance is good enough compared to the agreed accuracy requirements. In the second task (Uncertainty Analysis of Simulation) the uncertainties in the problem framing (the context) and the management scenarios are also taken into account.

The uncertainty assessment and propagation tasks (item D above) are the traditional uncertainty tasks often conducted in connection with model studies. These tasks are often comprehensive and may involve a lot of model calculations and are often limited to quantitative uncertainty. These tasks, which are both located towards the end of the modelling process, are very important. However, it is equally important to introduce uncertainty in the introductory phase of a model study. Therefore, the identification and characterisation of all uncertainty sources recommended in the task Determine Requirements under Step 1 (item A above) is crucial. The uncertainty aspects mentioned under items B and C are “less heavy” and may be seen as a follow up to item A.

Fig. 3 The five modelling steps and 45 tasks in the HarmoniQuA modelling protocol (Refsgaard et al., 2004)