Developing an instrument for evaluating implementation of clinical practice guidelines: a test-retest study

Document type: Journal Articles
Article type: Original article
Peer reviewed: Yes
Author(s): Christel Bahtsevani, Ania Willman, Azzam Khalaf, Margareta Östman
Title: Developing an instrument for evaluating implementation of clinical practice guidelines: a test-retest study
Translated title: Utveckling av ett instrument för att utvärdera införandet at kliniska riktlinjer - en test- återtest studie
Journal: Journal of Evaluation in Clinical Practice
Year: 2008
Volume: 14
Issue: 5
Pagination: 839-846
ISSN: 1356-1294
URI/DOI: 10.1111/j.1365-2753.2007.00916.x
ISI number: 000260544400034
Organization: Blekinge Institute of Technology
Department: School of Health Science (Sektionen för hälsa)
School of Health Science S-371 79 Karlskrona
+46 455 38 50 00
Authors e-mail:
Language: English
Abstract: this study focuses on the development of an instrument for the evaluation of clinical practice guidelines and is one part of a research project about the implementation and use of such guidelines among hospitals in the southern region of Sweden. The aim of the present paper was to investigate the test -retest reliability of a questionnaire. A questionnaire was designed to gather data about guidelines that have been implemented as well as information about factors, which, according to the promoting Action on Research Implementation in Health Services (PARIHS)-model, influence the success of implementation. Thirty-nine health professionals at one of the hospitals included in the survey completed the questionnaire on two occasions within a mean time of 5.5 weeks. The test-retest reliability was analysed by means of Cohen´s kappa and percentage concordance. Eight items had good agreement in terms of strength and high percentage concordance. With regard to the kappa values, 13 items show moderate and two fair agreement. The test-retest reliability scores show mainly acceptable results indicating a reasonable stability, thus suggesting the possibility of further developing the instrument. The factors described in the PARIHS-model seem relevant for use in evaluating implementation and use of guidelines. The instrument could benefit from a revision of the language in order to enhance clarity and make it less abstract.
Summary in Swedish: Studien handlar om utvecklingen av ett instrument/enkät för att utvärdera implementeringen av kliniska riktlinjer. Studien är en del av ett större forskningsprojekt. Syftet med denna artikel är att åpresentera resultatet av en test-retest studie av instrumentet. Frågeformuläret besvarades av 39 vårdverksamma personer med 5,5 veckoras mellanrum vid ett sjukhus i södra Sverge. Reliabiliteten analyserades med bl.a Cohen´s Kappa och resultatet visar relativ stabilitet för instrumenten men ett behov av fortsatt utveckling främst avseende språk, tydlighet och konkretisering. Nyckelord: kliniska riktlinjer, test-retest, PARIHS-modellen
Subject: Nursing & Caring Sciences\General
Keywords: clinical practice guidelines, evaluating, implementation, test-retest