Spatial Analysis of Model Performance, Illustrated by Soil Carbon Dioxide Emissions.
M.J. Pringle and R. Murray Lark. Environmetrics Group, Bioinformatics and Biomathematics Division, Rothamsted Research, Harpenden, United Kingdom
Carbon dioxide emissions from soil to the atmosphere an are an important component of the global carbon cycle. Models of carbon dioxide emissions from soil are necessary for the assessment of sequestration rates, and to predict the effects of alternative management scenarios. However, the evaluation of model performance usually relies on the comparison of the observed variable with corresponding model predictions. This is generally done in a non-spatial context. However, when the observations and predictions are attached to locations in space, they become spatial variables. It is our opinion that when the observations and predictions are spatial variables, a spatial model evaluation in required; non-spatial statistics do not suffice. This is be because: (i) a model may reproduce observed variations better at some spatial scales than others; (ii) the model may be more or less successful at reproducing the relative variability of a process at different spatial scales; (iii) the spatial pattern of model error may contain information about its possible sources; and, iv) a consequence of spatial dependence is that the correlation of the aggregated model output with the corresponding true value will depend on the scale of aggregation, and this has implications for the use and presentation of model output. A geostatistical analysis can address these issues. We developed a rate-limited model of carbon dioxide emissions from soil. Soil was sampled at 256 regularly spaced intervals on a 1024-m transect at Silsoe, UK. The model was calibrated on a subset of 100 randomly selected locations; the remaining data were used for a non-spatial and spatial evaluation of model performance. In a non-spatial context, the mean error of the model was -0.13 ln(g ha-1 d-1), which indicated that the model generally overestimated the flux. The root-mean-square-error of 0.53 ln(g ha-1 d-1) was large relative to the mean error. The correlation of the observations and predictions was r = 0.34, which is weak. A spatial analysis of the model's performance was based on a (cross-validated) linear model of coregionalization (LMCR). The LMCR showed that the model predictions underestimated the overall variance of the flux, and also underestimated the relative magnitude of fine-scale variability. Model predictions were uncorrelated with observed emissions at the finest spatial scales, but strongly correlated at spatial scales up to 175 m. This reflects the general rule that coarse-scale processes are easier to predict than fine-scale processes (“model decoherence”). The inability of the model to predict fine-scale variations may have been due to localized hot-spots of microbial activity. The LMCR was used with factorial cokriging to estimate and visualize the components of variation that occur at different scales; examination of these components revealed that, at a coarse scale, model overestimation was associated with areas of relatively heavy-textured soil. We used the LMCR, in conjunction with estimates of inter-block correlations, to quantify the effect of a change of support on the correlation of observations and model predictions. The spatial analysis of model performance revealed details not readily apparent with a non-spatial analysis: i) the model reproduced best the components of variation at relatively coarse spatial scales — because the model could not predict the short-range variation, but could predict well the variation spatially dependent to about 175 m, it may be useful for management within or between fields; ii) the model did not accurately reproduce the relative magnitude of the variability of CO2 emissions at different spatial scales, underestimating the short-range variation; iii) the spatial pattern of model error suggested that the model performed poorly on heavy-textured soil; and iv) the best pixel size with which to present the predicted soil CO2 emissions was 10-20 m. It is probable that many models of soil processes are scale-dependent in both the spatial and temporal domains, and this could be investigated with the techniques we have discussed.