Artikel
Development and validation of a measurement instrument for appraising indicator quality: appraisal of indicators through research and evaluation (AIRE) instrument
Suche in Medline nach
Autoren
Veröffentlicht: | 6. September 2007 |
---|
Gliederung
Text
Background: In recent years, performance measurement in health care has become top priority for all major stakeholders in health care systems worldwide and large numbers of quality indicators have emerged from many sources. However, despite the availability of hundreds of indicators, the quality of many of them often is questioned.
Aim: The aim of this study was to develop and validate an instrument for assessing the quality of indicators.
Method: The AIRE Instrument was developed and tested through a multi-staged process including; literature review, item generation, selection and grouping, consultation rounds with experts in the field of performance measurement in health care, investigation of reliability and validity by an expert panel (n=8) using a set of clinical indicators, and a questionnaire survey among these experts on the instrument’s relevance and usability.
Results: The AIRE Instrument consists of 20 quality criteria (items) grouped into four quality domains: 1. Purpose, relevance and organizational context, 2. Stakeholder involvement, 3. Scientific evidence, 4. Additional evidence, formulation and usage. All panellists found the instrument useful for appraising indicator quality. Reliability was acceptable for most domains (Cronbach’s alpha 0.69–0.94). Intraclass correlations (ICC) were calculated to assess the reliability within each domain. All domains were rated consistently, however for different numbers of appraisers. As some items could be interpreted differently, particularly in domain 1 and domain 2, further refinement of these items was needed. Quality indicators produced as part of an established indicator program or organizations specialized in this field had significantly higher scores on domain 1 (p=,016), domain 3 (p=,046), and domain 4 (p=,001).
Conclusion: The AIRE Instrument can be used consistently to appraise indicator quality and is sensitive to differences in important aspects of clinical indicators. pplication of this instrument by individuals or institutions developing quality indicators enhances consistency in indicator development procedures and indicator quality.