| |
Reliability of
Measurement: Clinical and Self-report Measures
Ensuring the reliability (or repeatability) of measurement when
examining subjects with TMD during the trial is required if a
sample of clinical TMD subjects is desired, and if these measures
are to be used as outcome measures in the trial.
For example, if a study
is seeking to demonstrate that experimental group TMD patients
who receive a flat-plane hard acrylic splint appliance will demonstrate
increased range of vertical jaw motion after three months compared
to a randomly chosen comparable control group who received no
such appliance, the study outcome hinges on the ability to measure
vertical jaw opening with accuracy and consistency. It is self
evident that measuring vertical opening with a ruler is a valid
measure upon which to base such a statistical test. What may not
be so self-evident is that the reliability of clinical measurements
across dentistry and medicine is known to be susceptible to poor
reliability--different examiners will not come up with the same
findings when examining the same patient using the same examining
methods.
Other fields, such
as periodontics, have devoted much energy to developing standardized
protocols and training procedures to insure that periodontal investigators
conducting RCTs investigating treatment efficacy be shown to be
highly reliable. Because reliability of clinical examinations
has proven so problematic, the WHO requires all dental epidemiologic
studies to include data demonstrating the reliability of their
clinical examiners.
Reliability of clinical
examinations is absolutely critical to any RCT investigating clinical
intervention outcomes because unreliable data is, in principle,
statistically invalid--that is, no valid statistical testing of
hypotheses and no statistical inference about the meaning of findings
in any RCT will be valid if data instruments and/or data collection
methods are unreliable. In this context, it is interesting to observe
that questionnaire data such as is contained in RDC/TMD Axis II
is not confronted with similar reliability issues. The questionnaires
and measurement scales that assess pain severity, depression, somatization
and level of psychosocial functioning have been demonstrated to
show adequate to excellent reliability, validity, and clinical usefulness
(Von
Korff et al., 1992; Dworkin
et al., 2000b).
Reliability of clinical
TMD examinations has similarly received much attention. It has
been possible to demonstrate that certain clinical measurements
in a TMD examination are more reliable--e.g. range of motion--
than other measurements, such as the assessment of joint sounds.
It has also been possible to demonstrate, as Table 16.4 depicts,
that the reliability of most commonly gathered TMD clinical examination
measures can be improved with the use of standardized examination
specifications and training sessions for TMD clinical examiners
preparing to conduct RCTs and other clinical TMD epidemiologic
studies.
| Table
16.4 Effects of Conducting Training and Calibration Sessions
on
Inter-Examiner Reliability for Assessing Components of
the RDC/TMD Axis I Clinical Examination
(Data from University of Washington Department of Oral
Medicine, Orofacial Pain and Dysfunction Clinic.Dworkin
et al. Clin J Pain 4:89-99, 1988
|
|---|
| Clinical
Assessments
| Initial
Reliability
| Post
Re-Calibration Reliability
|
|---|
| Opening-Unassisted* | .72
| .90
|
| Opening-Maximum* | .90
| .96
|
| Opening-Max.
Assisted | .92
| .98
|
| Vertical
overbite* | .81
| .85
|
| Horizontal
overjet | .79
| .88
|
| Occlussal
classification** | .40
| .78
|
| Palpation
pain**: Extra-oral Muscles | .47
| .65
|
|
Intra-oral Muscles | .27
| .61
|
| TMJ | .47
| .52
|
| *Intra
Class Correlation (ICC)>0.80=acceptable; >0.90=excellent
**k>0.4=acceptable;
0.6-0.8=good; >0.8=excellent
|
Note that the RDC/TMD
Axis I clinical examination, in particular, has been the focus
of multiple reliability studies and has been employed extensively
in TMD clinical research around the world.
The minimal requirements
for assessing TMD examination reliability include:
- Use of detailed
examination specifications
- Use of symptomatic
and asymptomatic subjects
- Clinical examiners
randomly sequenced to all subjects
- Distinguishing examiner
variability from normal or biologic variability
- Use of appropriate
statistical methods: we presently recommend intraclass correlation
coefficients as the most efficient measure of TMD clinical examiner
reliability
|