The psychometric properties of a tool to assess brief behaviour change counselling in South Africa

Abstract

Background: Primary care providers should be competent in brief behaviour change counselling (BBCC). A new model of BBCC was developed in South Africa. Tools are needed for training and research to evaluate BBCC.

Aim: To evaluate the validity and reliability of a tool to assess BBCC.

Setting: Primary care providers in Western Cape, South Africa.

Methods: Exploratory sequential mixed methods included initial qualitative feedback from an expert panel to assess validity, followed by quantitative analysis of internal consistency, inter- and intra-rater reliability. Six raters assessed 33 randomly selected audiotapes from a repository of 123 tapes of BBCC at baseline and 1 month later.

Results: Changes to the existing tool involved item changes, added items and grammatical as well as layout changes. The ‘Assessment of Brief Behavioural Change Counselling’ tool (ABC tool) had good overall internal consistency (Cronbach’s alpha 0.955), inter-rater (intra-class correlation coefficient [ICC] 0.813 at follow-up) and intra-rater reliability (Pearson’s correlation 0.899 and p < 0.001). Sub-scores for the Assist (ICC 0.784) and Arrange (ICC 0.704) stages had lower inter-rater reliability than the sub-scores for Ask (ICC 0.920), Alert (ICC 0.925) and Assess (ICC 0.931) stages.

Conclusion: The ABC tool is sufficiently reliable for the assessment of BBCC. Minor revisions may further improve the reliability of the tool, particularly for the sub-scores measuring Assist and Arrange. The ABC tool can be used in clinical training or research studies to assess fidelity to this model of BBCC.