Skip to Content
Interactive Textbook on Clinical Symptom Research Logo


Home Button

Statistical Models for Prognostication
Author Bio
Introduction
Predictions: Statistical Models
Insight: Statistical Models
Ingredients: Statistical Models
Theoretical Aspects
Central Concepts
Regression Models
Currently selected section: Problems: Regression
Practical Advice
Example 1
Example 2
Chapter 8: Statistical Models for Prognostication: Problems with Regression Models
        

Model Uncertainty

Model uncertainty refers to the problem that the structure of a model is often not known beforehand, but is specified based on the findings in the data set under study (Chatfield, 1995).

Examples of model aspects that are uncertain include:

  • The coding of predictors (regrouping of categorical covariables, inclusion of non-linear terms for continuous variables), and
  • The selection of predictors (main effects and interaction terms).

Standard statistical methods, e.g. to estimate the regression coefficients, assume that the model is pre-specified. The estimated variability (standard error) and statistical significance (p-value) are biased when the model is not pre-specified. The variability may increase substantially when model uncertainty is taken into account. In principle, this can be accomplished by bootstrapping procedures, which include the model specification phase (Efron and Tibshirani, 1993). This means that the model specification is replayed in every bootstrap sample. See for example: (Altman and Andersen, 1989).

QUESTION 8.2

The specification of a predictive model may be guided by the data under study. Compared to a fully pre-specified model, the p-values and regression coefficients will:

Selection AStill be unbiased estimates.
Selection BBe biased to less extreme values.
Selection CBe biased to more extreme values.

QUESTION 8.3

A model consisting of 4 predictors is selected from a set of 10 candidate predictors with an all-subset algorithm (all combinations of predictors are examined). How many models are actually considered in the selection process?

Selection A1
Selection B10
Selection C210
Selection D1024

Previous Page