Incorporating Parameter Estimability Into Model Selection

Jake M. Ferguson, Mark L. Taper, Rosana Zenil-Ferguson, Marie Jasieniuk, Bruce D. Maxwell

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

We investigate a class of information criteria based on the informational complexity criterion (ICC), which penalizes model fit based on the degree of dependency among parameters. In addition to existing forms of ICC, we develop a new complexity measure that uses the coefficient of variation matrix, a measure of parameter estimability, and a novel compound criterion that accounts for both the number of parameters and their informational complexity. We compared the performance of ICC and these variants to more traditionally used information criteria (i.e., AIC, AICc, BIC) in three different simulation experiments: simple linear models, nonlinear population abundance growth models, and nonlinear plant biomass growth models. Criterion performance was evaluated using the frequency of selecting the generating model, the frequency of selecting the model with the best predictive ability, and the frequency of selecting the model with the minimum Kullback-Leibler divergence. We found that the relative performance of each criterion depended on the model set, process variance, and sample size used. However, one of the compound criteria performed best on average across all conditions at identifying both the model used to generate the data and at identifying the best predictive model. This result is an important step forward in developing information criterion that select parsimonious models with interpretable and tranferrable parameters.

Original languageEnglish
Article number427
JournalFrontiers in Ecology and Evolution
Volume7
DOIs
StatePublished - Nov 14 2019

Bibliographical note

Funding Information:
We thank R. Boik, Mark Greenwood, Steve Cherry, Subhash Lele, Jose Ponciano, Taal Levi, Kristen Sauby, and Robert Holt for comments on earlier drafts of this work. MT's understanding of the problem of model identification has been greatly enhanced over the years by discussions with Subhash Lele, Brian Dennis, and Jose Ponciano. Funding. MT has been supported in part by MTFWP contract #060327 while JF and BM were supported in part by USDA NRI Weedy and Invasive Plants Program Grant No. 00-35320-9464.

Funding Information:
MT has been supported in part by MTFWP contract #060327 while JF and BM were supported in part by USDA NRI Weedy and Invasive Plants Program Grant No. 00-35320-9464.

Publisher Copyright:
© Copyright © 2019 Ferguson, Taper, Zenil-Ferguson, Jasieniuk and Maxwell.

Keywords

  • AIC
  • BIC
  • coefficient of variation
  • covariance
  • ICOMP
  • informational complexity
  • prediction
  • variable selection

ASJC Scopus subject areas

  • Ecology, Evolution, Behavior and Systematics
  • Ecology

Fingerprint

Dive into the research topics of 'Incorporating Parameter Estimability Into Model Selection'. Together they form a unique fingerprint.

Cite this