Identifying
Most Important Predictors
The most
important predictors can be identified from a regression model
by consideration of the regression coefficients and/or p-values.
However, it is quite common to identify the most important predictors
through a stepwise
selection process, where only statistically significant
predictors are included. Note that the deletion
of weaker predictors from a model may introduce
some confounding in the effects of the included predictors.
When correlations
exist between predictors, the effects of the included predictors
are mixed up with those of omitted predictors. The magnitude
of the confounding depends on the strength of the correlations
among predictors and the strength of the predictive relationship
of the predictors, i.e. the correlation between predictors and
the outcome. Although the latter correlation will be small for
omitted, statistically non-significant, predictors, we judge
stepwise selection inappropriate when one is interested in the
importance of predictors.