Formal Hypothesis Testing for Prospective Hydrological Model Improvements

Abstract

New algorithms for simulating hydrological processes are regularly proposed in the hydrological literature. However, the tests used to evaluate the effectiveness of these algorithms are typically no more than history matching – an improved model hydrograph is (often inappropriately) interpreted as an improved model. These tests ignore the considerable uncertainty inherent to hydrological models which may obscure the results of any comparisons.

In this work, a simple and more stringent method is proposed for comparing two model algorithms in terms of their ability to provide distinguishably different validation results under the impact of uncertainty in observation data and forcings. by generating distributions of performance indicators (e.g. Nash Sutcliffe) which can then be compared using basic statistical methods. The results show that at times modelling decisions are indistinguishable even when a single performance indicator shows improvement. As may be expected, our ability to identify the preferred hydrologic algorithm is significantly diminished when increased model/data uncertainty is incorporated into the evaluation process. This suggests that more robust testing is needed than what is typically reported in literature proposing model enhancements.

The thesis goes on to provide an example of how the new model comparison procedure can be include multiple performance measure and hydrological signatures to provide a diagnostic evaluation of modelling decisions. Diagnostics approaches to model evaluation can be found in the literature, but this work shows that data uncertainty affects each signature differently, and often introduces significant variability in model performance. Finally, it is demonstrated how the testing procedure can be used to examine the interactions between modelling decisions for better understanding of model deficiencies and the underlying hydrology.