Developing Methodology for Assessing Medium and Large Scale Environmental
Models
E. David Ford, Rie Komuro, Marianne Turley and Joel Reynolds
We have developed a new
method, and associated software, for assessment and inference of complex
models. It uses multiple outputs from a model as goodness-of-fit
measures that are assessed, simultaneously, against data or standards
defined from a theory. The software, Poreto_Evolve, uses an evolutionary
optimization algorithm to calculate the model's Pareto Optimal Set - the
parameterizations that simulate unique non-dominated groups of output
criteria effectively. Failure to achieve all criteria simultaneously is
used as a diagnostic of deficiencies, with investigation of the Pareto
Optimal Set providing guidance into deficiency sources. Our use of the
Pareto concept as an analytical assessment tool is novel; the usual
application is to define tolerance ranges in engineering design.
Currently we are applying this method to process-based models in biology
and ecology, where complexity is normal. Scientists reviewing ecological
modeling consider that inadequate general progress has been made in this
field. We suggest this is due to lack of effective assessment, and that
multiple criteria assessment provides an effective method. Our recent
experience has also shown that this approach to multiple criteria
optimization can provide an effective alternative to traditional methods
of parameter inference for non-linear and high-order statistical models
for complex data.
Our objective is to further develop the shareware, Pareto_Evolve, extend
its range of application, and publicize its availability and
applications through our web-site and discussion board. We propose a
number of investigations and benchmarking exercises (additional to those
already completed) to improve the search algorithm, the user-guidelines
for applying the heuristic search software, and to further extend the
methodology. We also propose a plan for integrating the approach into
graduate education [and the local research community?] by creating a
consulting center for ecological model assessment. |
|
|