External validity
(or generalizability) refers to the validity of the model in
other populations, e.g. patients from other centers, or treated
in a more recent time (Justice
et al., 1999).
Studying external
validity can be done when a data set is available where non-random
splits can be made, e.g. according to place or year of treatment.
Also, the predictive model can first be developed and published,
while external validity is studied after some years, either
in the same center (temporal validation) or at another center
(spatial validation).
Note that external
validity is more relevant than internal validity when a previously
developed prediction model is applied in another clinical setting
in a more recent time. Estimates of internal validity might
be interpreted as what can maximally be expected of external
validity. The aim of the analysis is to develop a model that
is at least internally valid; external validity can only be
a secondary aim.
Clinical and statistical
validity are distinct in model development (Altman
and Royston, 2000). Compare and contrast features
below.
| Clinical
Validity
| Statistical
Validity
|
|---|
| Refers
to:
| - Whether
the performance is good enough for the problem
addressed
| - Whether
model assumptions are met ("goodness-of-fit")
- Whether
the predictions are well-calibrated and discrimnate
low risk from high risk patients
|
|---|
Determined
by:
| - Intrinsic
prognostic information, i.e. whether strong predictors
are available
| - Continuous
predictors adequately modeled
- Interaction
terms not missed
- Outcome
adequately modeled
|
|---|
|
An illustration
of clinical validity is evident when we aim for identification
of a high-risk group, but only 1% of the patients can be identified
with a clearly elevated risk. We may judge such a model "clinically
invalid."
For additional information
on determinants of statistical validity including assumptions
of linearity, additivity, and distribution, see Theoretical
Aspects of Predictive Modeling.