Can a brief training intervention help improve mental health service delivery in South Africa?

Background South Africa faces a number of significant challenges apropos mental health service delivery, including a large treatment gap, a high rate of readmission, over-burdened specialist tertiary facilities, and slow integration of mental health into general health services. The South African National Mental Health Education Programme implemented between February 2019 and December 2019, aimed to upskill health workers to diagnose and manage mental disorders at primary and secondary levels of care. Aim This study aimed to assess the evolution of training participants’ self-reported competency in mental health care and the number of referrals made to higher levels of care as well as to reflect on the possible broader effects of the training. Setting The programme and study were conducted in South Africa with Medical Officers and Professional Nurses working at public sector primary and secondary level health care facilities. Methods A descriptive observational study collected data from training participants through a pre- and post-course, and 3-month follow-up survey. Results The average confidence ratings for performing mental health care activities and managing mental health conditions increased from pre- to post-course, and was either maintained or increased further at 3-month follow-up. A decrease in the self-reported percentage of patients being referred to a higher level of care was observed 3-months after the training. Conclusion The evaluation suggests that a brief training intervention such this can go a long way in increasing the confidence of primary and secondary level health care workers in managing common mental health conditions and adhering to the provisions of legislation.

services at primary, secondary and tertiary levels to improve access to mental health services and reduce the stigma associated with mental illness. 10 This would require that primary and secondary health facilities have appropriately trained staff, facilities and access to medication, a reality that is still a work in progress. 11 The Mental Health Care Act also sets out strict procedures that need to be followed by health care workers in the admission of mentally ill patients, especially in cases of assisted and involuntary care. Szabo and Kaliski noted that the administrative burden placed on health care workers has increased and they need to complete a 'host' of documents throughout the admission process to ensure that all decisions that are made are well thought through and defensible in court. 11 To support countries in the shift towards decentralised care, the World Health Organization (WHO) identified 10 key principals for mental health integration, one of which was the adequate training of primary care workers in the diagnosis and treatment of mental health conditions. 12 Research into the outcomes and effectiveness of short mental health training interventions for non-specialist health care workers have found significant improvements in knowledge, confidence, attitude towards mental health, clinical practice and skills, and patient outcomes. 13,14,15,16 A study on a 5-day training intervention for primary health care workers in Ethiopia (medical officers [MO] and nurses) on the management of mental disorders found significant improvements in participants' knowledge (42% average increase), attitude (43% average increase) and practice (45% average increase) after the training. 14 Fewer studies have evaluated the effects of training on patient outcomes. A study conducted in Kenya found no significant improvement in the detection of mental health problems, but did find significant improvements in patient health, disability, and quality of life measures, 12-weeks after the training intervention. 17 A systematic review of literature on interventions to improve provider recognition and management of mental disorders in primary care, found improved diagnosis of mental disorders reported in 18 of 23 studies examining this outcome, improved treatment in 14 of 20 studies, and clinical improvement in psychiatric symptoms in 4 of 11 studies. 18 The South African National Mental Health Education Programme (SANMHEP) was a National Department of Health (NDoH) initiative to strengthen mental health services in South Africa in collaboration with the Foundation for Professional Development (FPD), SANOFI and the various provincial departments of health. The training programme was delivered using a blended learning approach consisting of a face-to-face workshop and e-learning.
Since the implementation of this programme, the coronavirus disease 2019 (COVID-19) pandemic has taken hold of the world. Initial data are suggesting that the incidence of mental disorders is increasing and that during the pandemic the condition of many people with pre-existing mental disorders is worsening. Social distancing, quarantine, lockdown and other restrictions implemented, as well as their subsequent economic and social consequences are leading to an increase in the burden of mental disorders. 19 It is important, now more than ever, to equip health care workers with the tools needed to respond to this threat.

Study design
A descriptive observational outcome evaluation was designed to determine if the intervention had an influence on training participants' self-reported competency and referral behaviour. The objectives of the evaluation were to: (1) determine if there was an increase in training participants' self-reported competency in mental health care immediately after the course and 3 months later; (2) determine if there was a decrease in the number of self-reported mental health referrals made to higher levels of care; (3) describe the challenges faced by the training participants in providing mental health care once they were back at work; (4) describe the training participants' overall experience of the training and how it has affected their work; (5) describe the training participants' reflections on the possible broader, long-term effects of the training; and (6) describe how the course can be improved. For the purposes of this research, findings related to objectives 1, 2, 4 and 5 will be discussed. The other objectives of the evaluation may be reported on in future research.

Study setting
The evaluation was conducted within the context of the training programme. Workshops were conducted in training venues across the country, including venues at Regional Training Centres and public health facilities, from 20 February 2019 to 29 November 2019.

Study population
The training programme specifically targeted MO and professional nurses (PN) working within public facilities that are listed to conduct 72-h assessments (primary health care [PHC] clinics and psychiatric units attached to district and regional hospitals). The MOs and PNs from Correctional Service Centres and university PHC clinics were also included in the training.

Sample size and selection of participants
There was no sampling of participants for the first part of the evaluation (pre-and post-workshop survey) as all trainees were asked to complete a questionnaire before and after the workshop. A requirement to be included in the second part of the evaluation (the follow-up survey) was to complete the online component of the training. All trainees who completed the online component of the training were invited to complete the follow-up survey.

Intervention
The SANMHEP programme aimed to upskill MOs and PNs to diagnose mental disorders and provide quality care, treatment and rehabilitation within the facility. It also aimed to streamline referral indications and processes so that only those mental health care users who require specialised services are referred. The programme was implemented through a short course using a blended-learning approach consisting of a 3-day face-to-face workshop, provided by psychiatrists and mental health practitioners with significant experience in the field, and an online component delivered by the FPD. Students had to complete both components to be awarded with a certificate of completion. The course covered treatment guidelines for common mental disorders; conducting psychiatric interviews and conducting mental health screening, assessment and treatment; the Mental Health Care Act; referral pathways; as well as guidelines, flowcharts, and other resources.
A total of 1120 health care workers, the majority of which were PNs and medical doctors, were trained through 36 workshops between February and November 2019. Workshops took place across all nine provinces of South Africa, with Gauteng having the highest number of students, followed by KwaZulu-Natal and Mpumalanga. The majority of students were female (69%) and worked in a district hospital (53.4%).
Approximately one-third of the workshop attendees (337; 30%) completed the online component of the training that was required to receive a certificate.

Data collection
Data were collected from training participants through a preand post-course survey which participants completed before and after the workshop, and a 3-month follow-up survey. Only those students who completed the online component of the training were invited to complete the follow-up survey. The pre-course, post-course and follow-up survey asked the training participants to rate their competency in performing 11 mental health care activities and in managing 30 mental health conditions, to estimate the percentage of mental health patients they refer to higher levels of care, and to describe the mental health care resources available to them, as well as challenges faced in providing mental health care.

Data analysis
Only those students who had completed all three surveys were included in the analysis. Quantitative data were coded and initially descriptively analysed in MS Excel ® . Data from the three surveys were compared to determine if there was an increase in training participants' self-reported competency in mental health care, and if there was a decrease in the number of self-reported mental health referrals made to higher levels of care immediately after the course and 3 months later. Among the participants who completed the pre-, post-, and follow-up course assessment, one-way repeated measures analysis of variance (ANOVA) was performed with STATA 13.1 software (Stata Corp, College Station, Texas, United States [US]) to determine if there was a statistically significant difference in the confidence rating of performing mental health activities and managing mental health activities from the pre-course assessment.

Ethical considerations
Research ethics approval for this evaluation was obtained from the Foundation for Professional Development's Research Ethics Committee (registration number: REC-03711-033-RA Level 1) in October 2019.

Results
Of the 337 students invited to participate in the follow-up survey, 53 responded (15.7% response rate).
The majority of the 53 participants were female (75.5%), PNs (71.7%), working in a district hospital (45.3%). Eight of the nine South African provinces were represented in the final sample with 16% from the Free State and Mpumalanga, respectively, 14% from the Eastern Cape and Gauteng, respectively, 13% from KwaZulu-Natal, 11% from Limpopo, 7% from the Western Cape, and 4% from the North-West.
Competency ratings for the 11 mental health care activities and 30 mental health conditions were from 1 to 10 with 1 being not at all confident and 10 being extremely confident.
The overall average confidence rating for performing mental health care activities increased significantly from 5.8 before the course to 8.2 immediately after the course, and was maintained at 8.2 at 3-month follow-up (p < 0.001) ( Table 1). The confidence ratings of all activities significantly increased from the pre-course assessment ( Table 1). The activities with the highest post-course and follow-up ratings were: conducting a psychiatric interview; conducting a mental status examination; post-test human immunodeficiency virus (HIV) counselling; and placing a patient under involuntary care and treatment. The biggest increase from pre-course to 3-month follow-up was observed for the activity of placing a patient under involuntary care and treatment (pre: 5.7, follow-up: 8.8).
The overall average confidence rating for managing mental health conditions increased significantly from 5.8 before the course, to 7.6 immediately after the course, and to 7.9 at 3-month follow-up (p < 0.001) ( Table 2). The confidence ratings for all mental health conditions significantly increased from the pre-course assessment except for managing disturbances in intelligence and feeding/eating disorders ( Table 2). The conditions with the highest post-course and follow-up confidence ratings were depressive disorders, schizophrenic disorders, psychotic disorders, and bipolar disorders. The biggest increase from pre-course to follow-up was for dissociative disorders, psychiatric emergencies, and violent patients.
In the pre-course and follow-up survey, the participants were asked to select the percentage of mental health patients that they usually referred to higher levels of care with the options being: ≤ 10%; 11% -50%; 51% -75%; or 76% -100%. Before the workshop, only 17.3% of the participants selected ≤ 10%. At 3-month follow-up 61% of the participants selected ≤ 10% (Figure 1) implying that health care professionals felt more confident and equipped to assess patients and provide appropriate treatment at lower entry points to the health care system.

Key findings
The participants' confidence in performing mental health care related activities and in managing mental health conditions increased immediately after the course and increased at follow-up. The analysis of variance showed that the pre-course and follow-up overall average ratings for performing activities and managing mental health conditions were significantly different from one another.
The percentage of participants who reported that they referred less than 10.0% of their mental health patients to higher levels of care increased by 43.7%.

Discussion of key findings
The findings suggest that the training programme increased participants' confidence in their ability to manage mental health conditions in their facility. The increases in confidence observed at follow-up may be because of the students putting what they have learnt into practice.
The self-reported decrease in the percentage of mental health care patients referred to higher levels of care found at followup provides an indication of the possible medium-term effects of the training programme.  Percentage of mental health paƟents referrred to higher level of care 17 The ability to manage and monitor mental health care patients' conditions at primary care level may reduce the risk of relapse and subsequent need for re-admission, thereby contributing to the easing of the revolving door syndrome.

Limitations
The low number of students who completed both the workshop and the online components of the course resulted in a small sample size for the evaluation. The low response rate to the follow-up survey further reduced the sample for the statistical analyses. This selection bias may positively bias the results because students who are continually engaged in the course generally experience larger benefits from the course. The reasons for noncompletion of the online component will be investigated in further studies.

Implications and recommendations
Training on the management of common mental health conditions in primary and secondary care settings should be made available to more MOs and PNs as it has been shown to improve health care workers' confidence and the number of referrals made to higher levels of care.
Several important lessons were learnt during the implementation of this programme which may be valuable to others who are implementing similar programmes. Using a blended learning approach was found to be an effective way of delivering this training as it minimised the number of days that participants would need to be away from the facility. However, the low completion rate of the e-learning component of the course highlighted the challenges of providing online learning to this population. It is recommended that courses such as this continue to use traditional modes of delivery (e.g. workshops), supplemented by online engagement. Another solution to increase completion rates may be to require participants to complete the online component of the training before attending the workshops. This will also enable students to work through the course material before the workshop, allowing the facilitator to solely focus on application and areas that need clarification.

Conclusion
The evaluation provided evidence to show that a brief training intervention such as a short course can go a long way in increasing the capacity and confidence of primary and secondary level health care workers in managing common mental health conditions and adhering to the provisions of the Mental Health Care Act.
These outcomes may lead to broader system level improvements including a decrease in the burden placed on tertiary mental health care hospitals and an improvement in the revolving door syndrome. Furthermore, treating patients closer to where they live will save them and their families the time and costs of travelling to further away tertiary facilities.
Further research on the impact of similar training interventions needs to be conducted including determining impact on the number of admissions to tertiary facilities and the revolving door syndrome. Other areas of future research could include a cost-benefit analysis to determine if the medium-and longterm outcomes of the intervention justify the resources needed to implement it and a compliance audit to determine the extent to which the provisions of the Mental Health Care Act are adhered to.