5.3 How do rehabilitation providers know if an outcome measure will be useful in practice?

Reliability

The consistency of the measure and whether a measure (or questionnaire) is free from error.¹

It is important that measures are reliable (or consistent) and able to differentiate measure scores between clients.

Validity

How well the measure really measures what it is supposed to measure.¹

Cronbach’s alpha is a measure of internal consistency reliability, otherwise referred to as homogeneity of the scale. This is a reflection of how well the items in the scale are measuring different aspects of the same concept.¹ Nunnally suggests that a Cronbach’s alpha of > 0.9 is defined as acceptable for an instrument used with individual patients and a Cronbach alpha > 0.80 is defined as acceptable for a clinical instrument used with a group of patients (i.e. research).²

For example, does the HIV Symptom Index,³ developed to measure symptom presence and severity, really measure this construct or are there other HIV symptoms that people living with HIV might experience not captured in this questionnaire?

Responsiveness

Ability for a measure to detect any change in a client over time if a change has occurred.¹ This property is relevant to evaluative types of measures.

A sensitive, or responsive, assessment enables the healthcare provider to detect small to large changes in the construct of interest.

For example, a rehabilitation provider might be interested in knowing whether participation in a six-week aerobic exercise program has an impact on the Health-Related Quality of Life of a client.

Interpretability

Meaning of the scores or values associated with the outcome measures, i.e., what do the numbers really mean?

For example, what does a score of 82 on the Mental Health Summary Score of the Medical Outcomes Study Short Form (SF-36)^4,5 mean for clients? What does it mean for treatment decisions in clinical practice?

Terms such as the minimal detectable change (MDC), or minimal clinically important difference (MCID) refer to interpretability, specifically the minimum score that reflects an important or clinical change (improvement or worsening) for a given measure.^1,6 

For example, the MCID for the six-minute walk test is 25 meters among people living with Chronic Obstructive Pulmonary Disease.⁷ If an individual improves their score on the test by 30 metres, this can be interpreted as a clinically important improvement in functional capacity.

Often measures do not have a clear MCID or MDC and rehabilitation providers are left trying to interpret what the scores on a given measure mean for specific clients and what the scores mean for decision-making in clinical practice.