Skip to Content
Interactive Textbook on Clinical Symptom Research Logo


Home Button

Statistical Models for Prognostication
Author Bio
Introduction
Predictions: Statistical Models
Insight: Statistical Models
Ingredients: Statistical Models
Theoretical Aspects
Central Concepts
Currently selected section: Regression Models
Problems: Regression
Practical Advice
Example 1
Example 2
Chapter 8: Statistical Models for Prognostication: Development of Regression Models
        

Stepwise Selection

After this variable reduction step, a more limited number of covariables will remain for predictive modeling. A frequently applied method to achieve variable reduction is stepwise selection of covariables.

Stepwise selection is usually applied in a forward or backward way.

  • Forward selection starts with inclusion of the most significant candidate covariable in a regression model.
  • Backward selection starts with elimination of the least significant one from a regression model that includes all covariables (a full model).

If stepwise selection is applied, it is generally agreed upon that backward selection is preferable to forward selection (Harrell et al., 1996). The stopping rule for inclusion or exclusion usually applies the standard significance level for testing of hypotheses (alpha=0.05). It has, however, been demonstrated that alpha=0.05 is too small in relatively small data sets. The power is too low to identify important predictors as statistically significant. Therefore, if stepwise selection is used in a small data set, it should be used in a backward manner with a high p-value (e.g. 0.20 or 0.50) (Steyerberg et al., 2000a).

QUESTION 7.3

Stepwise selection is a method aimed at:

Selection AImprovement of predictions from a model.
Selection BReduction of the number of covariables in a model.

The one advantage of stepwise selection is that a small, readily interpretable model arises, which contains the most important predictors in a prediction problem.

The numerous disadvantages of stepwise selection are, however, known from separate studies (Chatfield, 1995) (Harrell et al., 1996) (Steyerberg et al., 1999). Disadvantages include:

  1. The selection is unstable; adding or deleting relatively few patients may substantially change the selection.
  2. The statistical power of a study may be insufficient to select true predictors, whereas multiple comparisons (almost) increase the risk that noise variables are included. Failure to select true predictors leads to a loss in predictive performance.
  3. The variance of the estimated regression coefficients is estimated as if the selection of covariables was predetermined. This biases the calculation of confidence intervals.
  4. The selection is based on the fact that a covariable had a relatively extreme p-value (usually: < 5%). This biases the p-values of selected covariables to extreme values.
  5. Extreme p-values correspond to relatively extreme regression coefficients. The estimated regression coefficients are therefore biased to more extreme values.

These problems assert that predictions from models that were based on stepwise selection will generally be sub-optimal: important predictors may not be included, and predictions are too extreme.

 

Previous Page