Comparing fire spread algorithms using equivalence testing and neutral landscape models

We demonstrate a method to evaluate the degree to which a meta-model approximates spatial disturbance processes represented by a more detailed model across a range of landscape conditions, using neutral landscapes and equivalence testing. We illustrate this approach by comparing burn patterns produced by a relatively simple fire spread algorithm with those generated by a more detailed fire behavior model from which the simpler algorithm was derived. Equivalence testing allows objective comparisons of the output of simple and complex models, to determine if the results are significantly similar. Neutral landscape models represent a range of landscape conditions that the model may encounter, allowing evaluation of the sensitivity and behavior of the model to different landscape compositions and configurations. We first tested the model for universal applicability, then narrowed the testing to assess the practical domain of applicability. As a whole, the calibrated simple model passed the test for significant equivalence using the 25% threshold. When applied to a range of landscape conditions different from the calibration scenarios, the model failed the tests for equivalence. Although our particular model failed the tests, the neutral landscape models were helpful in determining an appropriate domain of applicability and in assessing the model sensitivity to landscape changes. Equivalence testing provides an effective method for model comparison, and coupled with neutral landscapes, our approach provides an objective way to assess the domain of applicability of a spatial model.