<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1d1 20130915//EN" "http://jats.nlm.nih.gov/publishing/1.1d1/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" article-type="research-article" xml:lang="en">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">PHCFM</journal-id>
<journal-title-group>
<journal-title>African Journal of Primary Health Care &#x0026; Family Medicine</journal-title>
</journal-title-group>
<issn pub-type="ppub">2071-2928</issn>
<issn pub-type="epub">2071-2936</issn>
<publisher>
<publisher-name>AOSIS</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">PHCFM-14-3744</article-id>
<article-id pub-id-type="doi">10.4102/phcfm.v14i1.3744</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Original Research</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Evaluating postgraduate family medicine supervisor feedback in registrars&#x2019; learning portfolios</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<contrib-id contrib-id-type="orcid">https://orcid.org/0000-0001-6216-7189</contrib-id>
<name>
<surname>Erumeda</surname>
<given-names>Neetha J.</given-names>
</name>
<xref ref-type="aff" rid="AF0001">1</xref>
<xref ref-type="aff" rid="AF0002">2</xref>
</contrib>
<contrib contrib-type="author">
<contrib-id contrib-id-type="orcid">https://orcid.org/0000-0002-9042-2279</contrib-id>
<name>
<surname>George</surname>
<given-names>Ann Z.</given-names>
</name>
<xref ref-type="aff" rid="AF0003">3</xref>
</contrib>
<contrib contrib-type="author">
<contrib-id contrib-id-type="orcid">http://orcid.org/0000-0002-3221-8683</contrib-id>
<name>
<surname>Jenkins</surname>
<given-names>Louis S.</given-names>
</name>
<xref ref-type="aff" rid="AF0004">4</xref>
<xref ref-type="aff" rid="AF0005">5</xref>
<xref ref-type="aff" rid="AF0006">6</xref>
</contrib>
<aff id="AF0001"><label>1</label>Department of Family Medicine and Primary Care, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa</aff>
<aff id="AF0002"><label>2</label>Gauteng Department of Health, Ekurhuleni District Health Services, Germiston, South Africa</aff>
<aff id="AF0003"><label>3</label>Centre of Health Science Education, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa</aff>
<aff id="AF0004"><label>4</label>Division of Family Medicine and Primary Care, Department of Family and Emergency Medicine, Faculty of Health Sciences, Stellenbosch University, Cape Town, South Africa</aff>
<aff id="AF0005"><label>5</label>George Hospital, Western Cape Department of Health, George, South Africa</aff>
<aff id="AF0006"><label>6</label>Primary Health Care Directorate, Department of Family, Community and Emergency Care, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa</aff>
</contrib-group>
<author-notes>
<corresp id="cor1"><bold>Corresponding author:</bold> Neetha Erumeda, <email xlink:href="Neetha.Erumeda@wits.ac.za">Neetha.Erumeda@wits.ac.za</email></corresp>
</author-notes>
<pub-date pub-type="epub"><day>20</day><month>12</month><year>2022</year></pub-date>
<pub-date pub-type="collection"><year>2022</year></pub-date>
<volume>14</volume>
<issue>1</issue>
<elocation-id>3744</elocation-id>
<history>
<date date-type="received"><day>12</day><month>07</month><year>2022</year></date>
<date date-type="accepted"><day>25</day><month>10</month><year>2022</year></date>
</history>
<permissions>
<copyright-statement>&#x00A9; 2022. The Authors</copyright-statement>
<copyright-year>2022</copyright-year>
<license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by/4.0/">
<license-p>Licensee: AOSIS. This work is licensed under the Creative Commons Attribution License.</license-p>
</license>
</permissions>
<abstract>
<sec id="st1">
<title>Background</title>
<p>Postgraduate supervision forms a vital component of decentralised family medicine training. While the components of effective supervisory feedback have been explored in high-income countries, how this construct is delivered in resource-constrained low- to middle-income countries has not been investigated adequately.</p>
</sec>
<sec id="st2">
<title>Aim</title>
<p>This article evaluated supervisory feedback in family medicine registrars&#x2019; learning portfolios (LPs) as captured in their learning plans and mini-Clinical Evaluation Exercise (mini-CEX) forms and whether the training district or the year of training affected the nature of the feedback.</p>
</sec>
<sec id="st3">
<title>Setting</title>
<p>Registrars&#x2019; LPs from 2020 across five decentralised sites affiliated with the University of the Witwatersrand in South Africa were analysed.</p>
</sec>
<sec id="st4">
<title>Methods</title>
<p>Two modified tools were used to evaluate the quantity of the written feedback in 38 learning plans and 57 mini-CEX forms. Descriptive statistics, Fisher&#x2019;s exact and Wilcoxon rank-sum tests were used for analysis. Content analysis was used to derive counts of areas of feedback.</p>
</sec>
<sec id="st5">
<title>Results</title>
<p>Most learning plans (61.2&#x0025;) did not refer to registrars&#x2019; clinical knowledge or offer an improvement strategy (86.1&#x0025;). The &#x2018;extent of supervisors&#x2019; feedback&#x2019; was rated as &#x2018;poor&#x2019; (63.2&#x0025;), with only 14.0&#x0025; rated as &#x2018;good.&#x2019; The &#x2018;some&#x2019; and &#x2018;no&#x2019; feedback categories in the mini-CEX competencies (<italic>p</italic> &#x003C; 0.001 to <italic>p</italic> = 0.014) and the &#x2018;extent of supervisors&#x2019; feedback&#x2019; (<italic>p &#x003C;</italic> 0.001) were significantly associated with training district. Feedback focused less on clinical reasoning and negotiation skills.</p>
</sec>
<sec id="st6">
<title>Conclusion</title>
<p>Supervisors should provide specific and constructive narrative feedback and an action plan to improve registrars&#x2019; future performance.</p>
</sec>
<sec id="st7">
<title>Contribution</title>
<p>Supervisory feedback in postgraduate family medicine training needs overall improvement to develop skilled family physicians.</p>
</sec>
</abstract>
<kwd-group>
<kwd>decentralised training</kwd>
<kwd>family physician</kwd>
<kwd>feedback</kwd>
<kwd>individualised learning plan</kwd>
<kwd>learning portfolio</kwd>
<kwd>postgraduate supervision</kwd>
<kwd>mini-Clinical Evaluation Exercise</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec id="s0001">
<title>Introduction</title>
<p>Feedback is a vital aspect of supervision in clinical settings.<sup><xref ref-type="bibr" rid="CIT0001">1</xref>,<xref ref-type="bibr" rid="CIT0002">2</xref></sup> Feedback is defined as &#x2018;a way the trainee comes to know about the gaps in their current level of knowledge and the desired goal&#x2019;.<sup><xref ref-type="bibr" rid="CIT0003">3</xref></sup> For maximum effectiveness, feedback should be provided timeously, in small, manageable quantities, relate to observable behaviours<sup><xref ref-type="bibr" rid="CIT0004">4</xref></sup> and should be tailored to the trainee&#x2019;s learning needs and non-judgemental.<sup><xref ref-type="bibr" rid="CIT0005">5</xref>,<xref ref-type="bibr" rid="CIT0006">6</xref></sup> Feedback must also be clearly articulated based on trainee&#x2019;s observed performance,<sup><xref ref-type="bibr" rid="CIT0007">7</xref></sup> promote self-assessment,<sup><xref ref-type="bibr" rid="CIT0008">8</xref></sup> be context-specific, enable interactions between the trainee and supervisor<sup><xref ref-type="bibr" rid="CIT0003">3</xref></sup> and focus on adequate performance and areas for improvement.<sup><xref ref-type="bibr" rid="CIT0009">9</xref>,<xref ref-type="bibr" rid="CIT0010">10</xref></sup> Supervisors should assist trainees in developing written action plans specifying the supervisor&#x2019;s role in supporting trainees to achieve the goals outlined in the learning plans.<sup><xref ref-type="bibr" rid="CIT0009">9</xref>,<xref ref-type="bibr" rid="CIT0010">10</xref></sup> More recent definitions regard feedback as a coaching activity where supervisors facilitate trainees&#x2019; self-reflection to improve performance.<sup><xref ref-type="bibr" rid="CIT0010">10</xref>,<xref ref-type="bibr" rid="CIT0011">11</xref></sup> However, the acceptability of feedback by trainees still depends on who provides it, whether it is constructive, the trainee&#x2019;s assessment literacy and the nature of supervisor-trainee relationships.<sup><xref ref-type="bibr" rid="CIT0005">5</xref>,<xref ref-type="bibr" rid="CIT0011">11</xref></sup></p>
<sec id="s20002">
<title>The role of feedback in postgraduate family medicine training</title>
<p>Postgraduate family medicine decentralised clinical training of FPs involves educational and clinical supervision, each of which can be demonstrated in trainees&#x2019; learning portfolios (LPs). Educational supervision includes identifying trainees&#x2019; learning objectives and providing feedback on their personal development plans and progress in their LPs.<sup><xref ref-type="bibr" rid="CIT0012">12</xref>,<xref ref-type="bibr" rid="CIT0013">13</xref></sup> Clinical supervision involves giving feedback on trainees&#x2019; relationships with staff and patients, their management of clinical conditions and their professionalism.<sup><xref ref-type="bibr" rid="CIT0014">14</xref></sup> The LPs provide evidence of trainees&#x2019; observed performance by clinical supervisors using workplace-based assessments (WPBAs),<sup><xref ref-type="bibr" rid="CIT0015">15</xref></sup> using tools such as the mini-Clinical Evaluation Exercise (mini-CEX) form.<sup><xref ref-type="bibr" rid="CIT0016">16</xref></sup></p>
</sec>
<sec id="s20003">
<title>Educational supervision in learning plans</title>
<p>Individualised learning plans in LPs are recognised as essential components of self-regulated lifelong learning,<sup><xref ref-type="bibr" rid="CIT0017">17</xref></sup> which refers to trainees&#x2019; ability to modify their thinking behaviour and level of motivation.<sup><xref ref-type="bibr" rid="CIT0018">18</xref></sup> Learning plans are essential tools for postgraduate trainees, assisting them with understanding their strengths, professional goals and the family medicine speciality requirements needed to make individual adjustments and identify resources to progress.<sup><xref ref-type="bibr" rid="CIT0019">19</xref></sup> The educational supervisor&#x2019;s role in the learning plans is to orientate trainees to the learning environment, help them set goals and objectives and plan feedback sessions.<sup><xref ref-type="bibr" rid="CIT0006">6</xref></sup></p>
<p>Learning plans can also promote self-directed learning, which differs from self-regulated learning. In self-directed learning, trainees develop their learning goals, identify activities and resources to achieve them and get external feedback to modify learning. By comparison, in self-regulated learning, the trainers set the goals, and trainees regulate their learning, influenced by their cognitive domain, which primarily happens in an academic environment.<sup><xref ref-type="bibr" rid="CIT0020">20</xref></sup> However, there is a lack of literature on self-directed learning,<sup><xref ref-type="bibr" rid="CIT0020">20</xref></sup> especially on the potential for learning plans to promote self-directed learning based on the quality of the feedback provided. The self-directed learning goals trainees develop in these plans must be meaningful, specific, measurable, accountable, realistic and include specified timelines.<sup><xref ref-type="bibr" rid="CIT0021">21</xref></sup> While internal accountability facilitates self-reflection, trainees&#x2019; external accountability requires regular supervisor feedback.<sup><xref ref-type="bibr" rid="CIT0021">21</xref></sup></p>
</sec>
<sec id="s20004">
<title>Clinical supervision in the mini-Clinical Evaluation Exercise</title>
<p>The mini-CEX tool<sup><xref ref-type="bibr" rid="CIT0016">16</xref></sup> is widely used in HICs to assess clinical competencies.<sup><xref ref-type="bibr" rid="CIT0022">22</xref>,<xref ref-type="bibr" rid="CIT0023">23</xref>,<xref ref-type="bibr" rid="CIT0024">24</xref></sup> Although this tool was modified to accommodate several scoring systems, clinical disciplines and settings,<sup><xref ref-type="bibr" rid="CIT0005">5</xref>,<xref ref-type="bibr" rid="CIT0022">22</xref></sup> more contextualised studies are needed to understand its effectiveness in resource-limited settings. One of the primary purposes of the mini-CEX form is to provide structured feedback on observed performance.<sup><xref ref-type="bibr" rid="CIT0005">5</xref>,<xref ref-type="bibr" rid="CIT0022">22</xref></sup> The educational impact of this tool<sup><xref ref-type="bibr" rid="CIT0024">24</xref></sup> is enhanced when combined with well-written feedback<sup><xref ref-type="bibr" rid="CIT0025">25</xref></sup> and negatively impacted by inadequate feedback.<sup><xref ref-type="bibr" rid="CIT0005">5</xref>,<xref ref-type="bibr" rid="CIT0023">23</xref></sup> Supervisors often provide inadequate feedback by, for example, not appreciating or being aware of the role of feedback as an educational tool, not being skilled enough to provide helpful feedback, and tending to give high numerical scores (superior or satisfactory) on each mini-CEX competency,<sup><xref ref-type="bibr" rid="CIT0025">25</xref></sup> without explaining why this score is justified. Trainees value narrative feedback more than numerical scores-narrative feedback helps them to distinguish between performance deemed &#x2018;good&#x2019; and &#x2018;not so good&#x2019;, promotes reflection on their performance,<sup><xref ref-type="bibr" rid="CIT0026">26</xref></sup> and develop an action plan.<sup><xref ref-type="bibr" rid="CIT0027">27</xref></sup></p>
<p>In sub-Saharan Africa, clinical supervisors play a crucial role in postgraduate training in the workplace.<sup><xref ref-type="bibr" rid="CIT0028">28</xref></sup> Clinical supervisors are expected to provide constructive, good-quality verbal and written feedback during WPBAs. An evaluation of supervisors&#x2019; feedback in registrars&#x2019; LPs at a South African university showed that feedback was better documented in e-portfolios than in paper-based ones but was still not sufficiently specific.<sup><xref ref-type="bibr" rid="CIT0029">29</xref></sup> A satisfactory LP is a prerequisite for registrars (trainees) to sit the national exit examination to qualify as an FP.<sup><xref ref-type="bibr" rid="CIT0030">30</xref></sup> One of the challenges of tools such as learning plans or mini-CEX used in LPs is that they assess &#x2018;hard&#x2019; skills (history-taking, physical examination and management) and &#x2018;soft&#x2019; skills (problem-solving, communication, collaboration and professionalism).<sup><xref ref-type="bibr" rid="CIT0020">20</xref></sup> While some received training in providing feedback in South Africa,<sup><xref ref-type="bibr" rid="CIT0031">31</xref></sup> many FPs may not know what constitutes adequate feedback, which is more complex for &#x2018;soft&#x2019; skills.</p>
<p>Evaluating feedback in learning plans or the mini-CEX tool is still in its infancy in various high and low-middle income countries in clinical settings, including sub-Saharan Africa. It has been identified as an area for further research.<sup><xref ref-type="bibr" rid="CIT0020">20</xref>,<xref ref-type="bibr" rid="CIT0027">27</xref></sup> This article evaluated both the quantity and quality of supervisory feedback and the scores given in registrars&#x2019; LPs as captured in their learning plans and mini-CEX forms. The article also reports whether the training district or year of training affected the nature of the feedback across the decentralised clinical training sites. The results could help develop recommendations for improving educational and clinical supervision feedback in similar settings nationally and across sub-Saharan Africa.</p>
</sec>
</sec>
<sec id="s0005">
<title>Research method and design</title>
<sec id="s20006">
<title>Study design and setting</title>
<p>This study is part of a broader cross-sectional, convergent mixed-methods study that evaluated postgraduate family medicine training at five decentralised training sites affiliated with the University of the Witwatersrand in South Africa, using the logic model. The logic model is causal, based on the reductionist theory that links the inputs, processes, outputs and outcomes.<sup><xref ref-type="bibr" rid="CIT0032">32</xref></sup> A previous publication from the broader study reported evaluating postgraduate supervision as a process in the logic model.<sup><xref ref-type="bibr" rid="CIT0033">33</xref></sup> This article reports evaluating supervisors&#x2019; feedback as a proxy measure for effective supervision as an output in the logic model. In the setting for this study, registrar training is primarily based in primary or community healthcare clinics or a district hospital. During clinical rotations, registrars join various disciplines for 3 months or less in regional hospitals to gain specific knowledge and skills. Family medicine registrars are primarily supervised by FPs but may be supervised by non-family physician (non-FP) supervisors during clinical rotations. The non-FP supervisors could be specialists, registrars in other disciplines, or medical officers. Medical officers are general practitioners working at hospitals or clinics without any specialist registration.</p>
<p>Family physician training for their supervisory role varies. Some FP supervisors had attended &#x2018;train the trainer&#x2019; courses conducted by the South African Academy of FPs. Others had opportunities to attend the courses organised by Wits University but were not trained formally in medical education. The background of the non-FP supervisors in medical education training was not well understood at the time of this study. Family physicians act as educational supervisors and mentors, assess registrars&#x2019; learning plans and provide feedback during clinical rotations. In addition, as part of WPBAs, FP supervisors complete mini-CEX forms during directly observed consultations by registrars with patients in clinical settings. Supervisors&#x2019; written feedback in the learning plans and mini-CEX forms provides evidence for the quality of clinical and educational supervision.</p>
<p>This study evaluated the written feedback in registrars&#x2019; LPs from the five decentralised training sites, four of which are in Gauteng province and the fifth in the Northwest province.</p>
</sec>
<sec id="s20007">
<title>Study population and sampling</title>
<p>All LPs submitted in 2020 by the 20 registrars across all 3 years of training were eligible for evaluation, constituting a purposive sample.<sup><xref ref-type="bibr" rid="CIT0034">34</xref></sup> Thirty-six learning plans and 57 mini-CEX forms were evaluated from 19 eligible LPs. One portfolio was excluded as there was no consent. The learning plans and mini-CEX forms represented all 5 training districts (D1&#x2013;D5) and 3 years of training (Y1&#x2013;Y3).</p>
</sec>
<sec id="s20008">
<title>Data collection</title>
<p>The authors evaluated the learning plans and mini-CEX forms between October and December 2020.</p>
<sec id="s30009">
<title>Learning plans</title>
<p>At the beginning of a clinical rotation, the registrar develops goals in their learning plan. The supervisor then provides initial feedback on these goals. At the end of the rotation, the supervisor assigns a global score for the registrar&#x2019;s knowledge, skills, professional values, and attitudes and provides final written feedback. A validated tool developed at a South African university<sup><xref ref-type="bibr" rid="CIT0029">29</xref></sup> was modified to assess supervisors&#x2019; final feedback in the learning plans. The tool was adapted in consultation with one of the co-authors, who developed the validated tool: variables were added to make it more comprehensive on each of the feedback categories extracted from the data, thus increasing the construct validity of the tool.</p>
<p>The modified tool consists of seven categories:</p>
<list list-type="bullet">
<list-item><p>feedback on registrars&#x2019; professional behaviour</p></list-item>
<list-item><p>feedback on registrars&#x2019; knowledge and skills</p></list-item>
<list-item><p>the valence of feedback</p></list-item>
<list-item><p>the presence of an improvement strategy</p></list-item>
<list-item><p>a rating of the extent of supervisors&#x2019; feedback</p></list-item>
<list-item><p>the signatory (FP or other clinical supervisors) on the initial feedback</p></list-item>
<list-item><p>the signatory (FP or other clinical supervisors) on the final feedback.</p></list-item>
</list>
<p>The written feedback given by supervisors in the general feedback section was captured in an MS Excel spreadsheet to analyse the quality. The spreadsheet also recorded the training district, year of training and global numerical scores from each learning plan were also recorded in Microsoft Excel. The rating of the extent of feedback was classified as &#x2018;poor&#x2019;, &#x2018;average&#x2019;, or &#x2018;good&#x2019; by the researchers, depending on the feedback aspects provided. The feedback was rated &#x2018;poor&#x2019; if (1) feedback was absent or irrelevant (2) specific feedback was not provided for the following categories in the tool: professional behaviour, knowledge and skills, valence of feedback and improvement strategy present, (3) forms were not signed and completed by the supervisor. A rating of &#x2018;average&#x2019; was assigned when specific feedback was provided on one or two of the categories (registrar professional behaviour, knowledge and skills, valence of feedback provided or improvement strategy) and forms were signed and completed. The rating was &#x2018;good&#x2019; when feedback was provided for three or more categories (registrar professional behaviour, knowledge and skills, valence of feedback and improvement strategy) and the form was signed and completed.</p>
</sec>
<sec id="s30010">
<title>Mini-Clinical Evaluation Exercise forms</title>
<p>The mini-CEX form requires supervisors to allocate individual scores for the competencies and provide a total score. The form includes sections for the supervisor&#x2019;s feedback and supervisor and registrar signatures.</p>
<p>We developed a feedback assessment tool to evaluate written feedback in the mini-CEX forms, whether explicitly related to the seven competencies or appearing in the general feedback section. The tool assessed the extent and adequacy of the supervisors&#x2019; feedback, the training district, the year of training, and whether both FPs and registrars signed the form. The &#x2018;extent of feedback&#x2019; rating was classified as &#x2018;poor&#x2019;, &#x2018;average&#x2019;, or &#x2018;good&#x2019; depending on the areas of mini-CEX competencies addressed, whether supervisors had completed the general feedback section and whether the supervisor and registrar signed the forms. The scores for each of the seven competencies and total scores given in mini-CEX forms were recorded in Microsoft Excel. The narrative feedback provided in the general feedback section was captured in a separate MS Excel spreadsheet to analyse the quality.</p>
</sec>
</sec>
<sec id="s20011">
<title>Data analysis</title>
<p>The data were analysed for descriptive statistics using STATISTICA 13.5.0.17 and for associations using STATA 14.2. For the inferential statistics, the feedback categories were collapsed into &#x2018;some feedback&#x2019; and &#x2018;no feedback&#x2019; in both learning plans and mini-CEX forms. For both tools, the absence of feedback was categorised as &#x2018;no feedback&#x2019;, and all other feedback categories were grouped as &#x2018;some feedback&#x2019;. Similarly, if the rating of &#x2018;extent of supervisor&#x2019;s feedback&#x2019; was &#x2018;poor&#x2019;, this was categorised as &#x2018;no feedback&#x2019;, and the &#x2018;average&#x2019; and &#x2018;good&#x2019; categories were grouped into a single category labelled &#x2018;some feedback&#x2019;. All associations between &#x2018;no feedback&#x2019; and &#x2018;some feedback&#x2019; categories and the training district or the year of training were determined using Fisher&#x2019;s exact tests (<italic>p</italic> &#x003C; 0.05). Fisher&#x2019;s exact tests were used rather than chi-squared tests because of the small sample size.<sup><xref ref-type="bibr" rid="CIT0035">35</xref></sup> Where there was a statistically significant association, Fisher&#x2019;s exact tests were used to test each training district and year with the &#x2018;some feedback&#x2019; and &#x2018;no feedback&#x2019; categories to determine the strength of association.</p>
<p>Kruskal&#x2013;Wallis tests were used to compare the global and total scores in the learning plans and mini-CEX forms across the training district and year of training. Two-sample Wilcoxon rank-sum tests were used to test the strength of association. Bonferroni correction was applied for 2X2 training district and year of training comparison with feedback categories and scores and significance for the <italic>p</italic>-value was set at &#x003C; 0.005 and &#x003C; 0.017, respectively.</p>
<p>The feedback was first characterised according to length, that is, the number of words, overall quality and specificity to evaluate the quality of written narrative feedback in both forms. The characteristics assessed for the overall feedback quality in this study were extracted from validated tools used in previous studies. The following feedback quality characteristics were adapted from De Swardt et al. (2019) the number of words in the feedback, whether proper sentences were used, strengths and weaknesses highlighted and areas of improvement specified.<sup><xref ref-type="bibr" rid="CIT0029">29</xref></sup> The following were derived from Pilgrim et al.: whether an action plan was provided and whether the feedback was specific.<sup><xref ref-type="bibr" rid="CIT0036">36</xref></sup> For example, if the feedback simply read &#x2018;satisfactory&#x2019; or &#x2018;good&#x2019;, it was coded under non-specific feedback. The second level of analysis involved content analysis<sup><xref ref-type="bibr" rid="CIT0037">37</xref></sup> to derive counts for specific feedback areas. For example, the feedback &#x2018;the joint examination was performed well&#x2019; was coded as &#x2018;used proper sentences&#x2019; and &#x2018;provided strengths&#x2019; under the overall quality. This feedback example was coded in the category &#x2018;physical examination&#x2019; during the content analysis. The main author and co-authors initially separately, coded the data set, then compared their coding systems. Any differences were discussed until agreement on the naming of codes and categories and the categorisation of the codes was reached. The coding process was iterative, involving several cycles of discussions until the data analysis was completed. The iterative process improved the interrater reliability of the findings.<sup><xref ref-type="bibr" rid="CIT0038">38</xref></sup></p>
</sec>
<sec id="s20012">
<title>Ethical considerations</title>
<p>Ethical approval for the broader project was obtained from the Human Research Ethics Committee (Medical) of the University of the Witwatersrand (certificate number: M191140). Permission to conduct the research was obtained from the University Registrar and the Head of the Department of Family Medicine. Informed written consent to access the LPs was obtained from registrars and supervisors.</p>
</sec>
</sec>
<sec id="s0013">
<title>Results</title>
<p>When the feedback in the 36 learning plans was analysed, most (61.2&#x0025;) did not refer to registrars&#x2019; clinical knowledge or skills, and 86.1&#x0025; did not offer an improvement strategy. The &#x2018;extent of supervisors&#x2019; feedback&#x2019; was mainly rated as &#x2018;poor&#x2019; (47.2&#x0025;) or &#x2018;average&#x2019; (47.2&#x0025;); only a few (5.6&#x0025;) were &#x2018;good&#x2019;. In one-third of the forms, the final feedback was provided by non-FP supervisors from other clinical departments (<xref ref-type="table" rid="T0001">Table 1</xref>). The data were not uniformly distributed, so the global score median of 8.0 (7.7&#x2013;9.0) out of 10.0 in the learning plans was calculated.</p>
<table-wrap id="T0001">
<label>TABLE 1</label>
<caption><p>Learning plan feedback characteristics (<italic>N</italic> = 38).</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left">Variable</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" colspan="3"><bold>Training district</bold></td>
</tr>
<tr>
<td align="left">District 1</td>
<td align="center">10</td>
<td align="center">27.7</td>
</tr>
<tr>
<td align="left">District 2</td>
<td align="center">6</td>
<td align="center">16.7</td>
</tr>
<tr>
<td align="left">District 3</td>
<td align="center">6</td>
<td align="center">16.7</td>
</tr>
<tr>
<td align="left">District 4</td>
<td align="center">8</td>
<td align="center">22.2</td>
</tr>
<tr>
<td align="left">District 5</td>
<td align="center">6</td>
<td align="center">16.7</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Year of training</bold></td>
</tr>
<tr>
<td align="left">Year 1</td>
<td align="center">10</td>
<td align="center">27.8</td>
</tr>
<tr>
<td align="left">Year 2</td>
<td align="center">16</td>
<td align="center">44.4</td>
</tr>
<tr>
<td align="left">Year 3</td>
<td align="center">10</td>
<td align="center">27.8</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Reference to registrar&#x2019;s professional behaviour</bold></td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">15</td>
<td align="center">41.7</td>
</tr>
<tr>
<td align="left">General feedback</td>
<td align="center">8</td>
<td align="center">22.2</td>
</tr>
<tr>
<td align="left">Specific feedback</td>
<td align="center">13</td>
<td align="center">36.1</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Reference to registrars&#x2019; clinical knowledge or skills</bold></td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">22</td>
<td align="center">61.2</td>
</tr>
<tr>
<td align="left">General feedback</td>
<td align="center">7</td>
<td align="center">19.4</td>
</tr>
<tr>
<td align="left">Specific feedback</td>
<td align="center">7</td>
<td align="center">19.4</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Statement indicating valence of feedback</bold></td>
</tr>
<tr>
<td align="left">No feedback and/or unclear</td>
<td align="center">15</td>
<td align="center">41.7</td>
</tr>
<tr>
<td align="left">Mostly positive or negative</td>
<td align="center">20</td>
<td align="center">55.6</td>
</tr>
<tr>
<td align="left">Constructive feedback (positive balanced with negative)</td>
<td align="center">1</td>
<td align="center">2.7</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Comments offering an improvement strategy</bold></td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">31</td>
<td align="center">86.1</td>
</tr>
<tr>
<td align="left">General feedback</td>
<td align="center">3</td>
<td align="center">8.3</td>
</tr>
<tr>
<td align="left">Specific feedback</td>
<td align="center">2</td>
<td align="center">5.6</td>
</tr>
<tr>
<td align="left">National unit standards</td>
<td align="center">0</td>
<td align="center">0.0</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Initial feedback given and completed by</bold></td>
</tr>
<tr>
<td align="left">Family physician supervisor</td>
<td align="center">24</td>
<td align="center">66.7</td>
</tr>
<tr>
<td align="left">Academic coordinator</td>
<td align="center">8</td>
<td align="center">22.2</td>
</tr>
<tr>
<td align="left">Other specialists</td>
<td align="center">1</td>
<td align="center">2.8</td>
</tr>
<tr>
<td align="left">Medical officer</td>
<td align="center">0</td>
<td align="center">0.0</td>
</tr>
<tr>
<td align="left">Not signed</td>
<td align="center">3</td>
<td align="center">8.3</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Final feedback given and completed by</bold></td>
</tr>
<tr>
<td align="left">Family physician supervisor</td>
<td align="center">16</td>
<td align="center">44.4</td>
</tr>
<tr>
<td align="left">Academic coordinator</td>
<td align="center">8</td>
<td align="center">22.2</td>
</tr>
<tr>
<td align="left">Other specialists</td>
<td align="center">8</td>
<td align="center">22.2</td>
</tr>
<tr>
<td align="left">Medical officer</td>
<td align="center">4</td>
<td align="center">11.2</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Rating of the extent of supervisors&#x2019; feedback</bold></td>
</tr>
<tr>
<td align="left">Poor (No feedback; irrelevant feedback; no specific feedback on at least one aspect; not signed and completed)</td>
<td align="center">17</td>
<td align="center">47.2</td>
</tr>
<tr>
<td align="left">Average (average or good feedback on one to two aspects; signed and completed)</td>
<td align="center">17</td>
<td align="center">47.2</td>
</tr>
<tr>
<td align="left">Good (gives feedback on three or more aspects; signed and completed)</td>
<td align="center">2</td>
<td align="center">5.6</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>In 17 of the 19 LPs, the mini-CEX forms were completed by more than one FP from the same district, while the same FP completed the remaining two. For the &#x2018;extent of supervisors&#x2019; feedback&#x2019;, 63.2&#x0025; were classified as &#x2018;poor&#x2019; and only 14.0&#x0025; as &#x2018;good&#x2019; (<xref ref-type="table" rid="T0002">Table 2</xref>). The median of the total scores in the mini-CEX forms was 6.8 (range: 6.1&#x2013;7.3). Most of the mini-CEX competency scores given by FP supervisors ranged between 5&#x2013;7 or 8&#x2013;10, with very few scores of 1&#x2013;4 (<xref ref-type="table" rid="T0003">Table 3</xref>).</p>
<table-wrap id="T0002">
<label>TABLE 2</label>
<caption><p>Mini-Clinical Evaluation Exercise feedback characteristics (<italic>N</italic> = 57).</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left">Variable</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" colspan="3"><bold>Training district</bold></td>
</tr>
<tr>
<td align="left">District 1</td>
<td align="center">15</td>
<td align="center">26.3</td>
</tr>
<tr>
<td align="left">District 2</td>
<td align="center">12</td>
<td align="center">21.1</td>
</tr>
<tr>
<td align="left">District 3</td>
<td align="center">9</td>
<td align="center">15.7</td>
</tr>
<tr>
<td align="left">District 4</td>
<td align="center">12</td>
<td align="center">21.1</td>
</tr>
<tr>
<td align="left">District 5</td>
<td align="center">9</td>
<td align="center">15.8</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Year of training</bold></td>
</tr>
<tr>
<td align="left">Year 1</td>
<td align="center">18</td>
<td align="center">31.6</td>
</tr>
<tr>
<td align="left">Year 2</td>
<td align="center">21</td>
<td align="center">36.8</td>
</tr>
<tr>
<td align="left">Year 3</td>
<td align="center">18</td>
<td align="center">31.6</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>General Feedback</bold></td>
</tr>
<tr>
<td align="left">No feedback (1)</td>
<td align="center">10</td>
<td align="center">17.5</td>
</tr>
<tr>
<td align="left">Feedback on areas performed well (2)</td>
<td align="center">3</td>
<td align="center">5.3</td>
</tr>
<tr>
<td align="left">Feedback on areas not performed well (3)</td>
<td align="center">4</td>
<td align="center">7.0</td>
</tr>
<tr>
<td align="left">Areas performed well and not well (4)</td>
<td align="center">5</td>
<td align="center">8.8</td>
</tr>
<tr>
<td align="left">Offer strategies for improvement (5)</td>
<td align="center">11</td>
<td align="center">19.3</td>
</tr>
<tr>
<td align="left">General or reference to national unit standards (6)</td>
<td align="center">2</td>
<td align="center">3.5</td>
</tr>
<tr>
<td align="left">Combinations of 2&#x2013;4 and 5</td>
<td align="center">15</td>
<td align="center">26.3</td>
</tr>
<tr>
<td align="left">Combinations of 2&#x2013;4 and 6</td>
<td align="center">7</td>
<td align="center">12.3</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Rating of the extent of supervisors&#x2019; feedback</bold></td>
</tr>
<tr>
<td align="left">Poor (feedback on &#x2264; 2 areas of competencies; no general feedback)</td>
<td align="center">36</td>
<td align="center">63.2</td>
</tr>
<tr>
<td align="left">Average (feedback on 3&#x2013;4 areas of competencies; general feedback completed)</td>
<td align="center">13</td>
<td align="center">22.8</td>
</tr>
<tr>
<td align="left">Good (feedback on five or more areas of competencies; general feedback completed)</td>
<td align="center">8</td>
<td align="center">14.0</td>
</tr>
<tr>
<td align="left" colspan="3"><bold>Signed by registrar and supervisor</bold></td>
</tr>
<tr>
<td align="left">Completed</td>
<td align="center">29</td>
<td align="center">50.9</td>
</tr>
<tr>
<td align="left">Not completed</td>
<td align="center">28</td>
<td align="center">49.1</td>
</tr>
</tbody>
</table>
</table-wrap>
<table-wrap id="T0003">
<label>TABLE 3</label>
<caption><p>Feedback characteristics and scores on mini-Clinical Evaluation Exercise competencies (<italic>N</italic> = 57).</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left" rowspan="2">Mini-CEX competencies</th>
<th valign="top" align="center" colspan="2">Establish relationship<hr/></th>
<th valign="top" align="center" colspan="2">Gathering information<hr/></th>
<th valign="top" align="center" colspan="2">Physical examination<hr/></th>
<th valign="top" align="center" colspan="2">Clinical judgement<hr/></th>
<th valign="top" align="center" colspan="2">Explaining and planning<hr/></th>
<th valign="top" align="center" colspan="2">Shows organised approach<hr/></th>
<th valign="top" align="center" colspan="2">Overall competence<hr/></th>
</tr>
<tr>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" colspan="15"><bold>Feedback categories</bold></td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">40</td>
<td align="center">70.2</td>
<td align="center">29</td>
<td align="center">50.9</td>
<td align="center">35</td>
<td align="center">61.4</td>
<td align="center">39</td>
<td align="center">68.4</td>
<td align="center">40</td>
<td align="center">70.2</td>
<td align="center">44</td>
<td align="center">77.2</td>
<td align="center">48</td>
<td align="center">84.2</td>
</tr>
<tr>
<td align="left">Feedback on areas performed well</td>
<td align="center">10</td>
<td align="center">17.5</td>
<td align="center">12</td>
<td align="center">21.1</td>
<td align="center">8</td>
<td align="center">14.0</td>
<td align="center">10</td>
<td align="center">17.6</td>
<td align="center">10</td>
<td align="center">17.5</td>
<td align="center">4</td>
<td align="center">7.0</td>
<td align="center">3</td>
<td align="center">5.7</td>
</tr>
<tr>
<td align="left">Feedback on areas not performed well</td>
<td align="center">2</td>
<td align="center">3.5</td>
<td align="center">11</td>
<td align="center">19.3</td>
<td align="center">12</td>
<td align="center">21.1</td>
<td align="center">7</td>
<td align="center">12.3</td>
<td align="center">4</td>
<td align="center">7.0</td>
<td align="center">7</td>
<td align="center">12.3</td>
<td align="center">4</td>
<td align="center">7.0</td>
</tr>
<tr>
<td align="left">Areas performed well and not performed well</td>
<td align="center">1</td>
<td align="center">1.8</td>
<td align="center">4</td>
<td align="center">7.0</td>
<td align="center">2</td>
<td align="center">3.5</td>
<td align="center">1</td>
<td align="center">1.8</td>
<td align="center">1</td>
<td align="center">1.8</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">0</td>
<td align="center">0.0</td>
</tr>
<tr>
<td align="left">Feedback general or inadequate</td>
<td align="center">4</td>
<td align="center">7.0</td>
<td align="center">1</td>
<td align="center">1.8</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">2</td>
<td align="center">3.5</td>
<td align="center">2</td>
<td align="center">3.5</td>
<td align="center">2</td>
<td align="center">3.5</td>
</tr>
<tr>
<td align="left" colspan="15"><bold>Score categories</bold></td>
</tr>
<tr>
<td align="left">Score (1&#x2013;4)</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">1</td>
<td align="center">1.8</td>
<td align="center">4</td>
<td align="center">7.1</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">1</td>
<td align="center">1.8</td>
<td align="center">0</td>
<td align="center">0.0</td>
</tr>
<tr>
<td align="left">Score (5&#x2013;7)</td>
<td align="center">34</td>
<td align="center">59.7</td>
<td align="center">44</td>
<td align="center">77.2</td>
<td align="center">43</td>
<td align="center">75.4</td>
<td align="center">43</td>
<td align="center">75.4</td>
<td align="center">43</td>
<td align="center">75.4</td>
<td align="center">43</td>
<td align="center">75.4</td>
<td align="center">46</td>
<td align="center">80.7</td>
</tr>
<tr>
<td align="left">Score (8&#x2013;10)</td>
<td align="center">22</td>
<td align="center">38.6</td>
<td align="center">12</td>
<td align="center">21.1</td>
<td align="center">9</td>
<td align="center">15.8</td>
<td align="center">14</td>
<td align="center">24.6</td>
<td align="center">14</td>
<td align="center">24.6</td>
<td align="center">13</td>
<td align="center">22.8</td>
<td align="center">11</td>
<td align="center">19.3</td>
</tr>
<tr>
<td align="left">No score</td>
<td align="center">1</td>
<td align="center">1.8</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">1</td>
<td align="center">1.8</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">0</td>
<td align="center">0.0</td>
<td align="center">0</td>
<td align="center">0.0</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<fn><p>Mini-CEX, mini-Clinical Evaluation Exercise.</p></fn>
</table-wrap-foot>
</table-wrap>
<p>Further analysis showed a statistically significant association of the &#x2018;some feedback&#x2019; and &#x2018;no feedback&#x2019; categories with all Mini-CEX competencies (<italic>p</italic> &#x003C; 0.001 to <italic>p</italic> = 0.014), the rating of &#x2018;extent of supervisors&#x2019; feedback&#x2019; (<italic>p</italic> &#x003C; 0.001) and the &#x2018;signed by registrar and supervisor&#x2019; (<italic>p</italic> &#x003C; 0.001) categories with the training district (<xref ref-type="table" rid="T0004">Table 4</xref>). However, there was no association with the year of training. There was no significant association of feedback categories in the learning plans with either the training district or year of training.</p>
<table-wrap id="T0004">
<label>TABLE 4</label>
<caption><p>Association of mini-Clinical Evaluation Exercise feedback categories with training district and year of training.</p></caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left" rowspan="3">Mini-CEX competencies</th>
<th valign="top" align="center" colspan="10">Training district<hr/></th>
<th valign="top" align="center" rowspan="3"><italic>p</italic><xref ref-type="table-fn" rid="TFN0001">&#x2020;</xref></th>
<th valign="top" align="center" colspan="6">Year of training<hr/></th>
<th valign="top" align="center" rowspan="3"><italic>p</italic><xref ref-type="table-fn" rid="TFN0001">&#x2020;</xref></th>
</tr>
<tr>
<th valign="top" align="center" colspan="2">D1<hr/></th>
<th valign="top" align="center" colspan="2">D2<hr/></th>
<th valign="top" align="center" colspan="2">D3<hr/></th>
<th valign="top" align="center" colspan="2">D4<hr/></th>
<th valign="top" align="center" colspan="2">D5<hr/></th>
<th valign="top" align="center" colspan="2">Y2<hr/></th>
<th valign="top" align="center" colspan="2">Y3<hr/></th>
</tr>
<tr>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
<th valign="top" align="center"><italic>n</italic></th>
<th valign="top" align="center">&#x0025;</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" colspan="11"><bold>Establish relationship</bold></td>
<td align="center" rowspan="3">&#x003C; 0.001</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.128</td>
</tr>
<tr>
<td align="left">Some feedback</td>
<td align="center">1</td>
<td align="center">4.5</td>
<td align="center">4</td>
<td align="center">3.6</td>
<td align="center">8</td>
<td align="center">2.7</td>
<td align="center">3</td>
<td align="center">3.6</td>
<td align="center">1</td>
<td align="center">2.7</td>
<td align="center">2</td>
<td align="center">5.4</td>
<td align="center">8</td>
<td align="center">6.3</td>
<td align="center">7</td>
<td align="center">5.4</td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">14</td>
<td align="center">10.5</td>
<td align="center">8</td>
<td align="center">8.4</td>
<td align="center">1</td>
<td align="center">6.3</td>
<td align="center">9</td>
<td align="center">8.4</td>
<td align="center">8</td>
<td align="center">6.3</td>
<td align="center">16</td>
<td align="center">12.6</td>
<td align="center">13</td>
<td align="center">14.7</td>
<td align="center">11</td>
<td align="center">12.6</td>
</tr>
<tr>
<td align="left" colspan="11"><bold>Gathering information</bold></td>
<td align="center" rowspan="3">0.009</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.241</td>
</tr>
<tr>
<td align="left">Some feedback</td>
<td align="center">3</td>
<td align="center">7.4</td>
<td align="center">8</td>
<td align="center">5.9</td>
<td align="center">8</td>
<td align="center">4.4</td>
<td align="center">6</td>
<td align="center">5.9</td>
<td align="center">3</td>
<td align="center">4.4</td>
<td align="center">6</td>
<td align="center">8.8</td>
<td align="center">13</td>
<td align="center">10.3</td>
<td align="center">9</td>
<td align="center">8.8</td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">12</td>
<td align="center">7.6</td>
<td align="center">4</td>
<td align="center">6.1</td>
<td align="center">1</td>
<td align="center">4.6</td>
<td align="center">6</td>
<td align="center">6.1</td>
<td align="center">6</td>
<td align="center">4.6</td>
<td align="center">12</td>
<td align="center">9.2</td>
<td align="center">8</td>
<td align="center">10.7</td>
<td align="center">9</td>
<td align="center">9.2</td>
</tr>
<tr>
<td align="left" colspan="11"><bold>Physical examination</bold></td>
<td align="center" rowspan="3">&#x003C; 0.001</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.466</td>
</tr>
<tr>
<td align="left">Some feedback</td>
<td align="center">3</td>
<td align="center">5.8</td>
<td align="center">9</td>
<td align="center">4.6</td>
<td align="center">7</td>
<td align="center">3.5</td>
<td align="center">2</td>
<td align="center">4.6</td>
<td align="center">1</td>
<td align="center">3.5</td>
<td align="center">7</td>
<td align="center">6.9</td>
<td align="center">10</td>
<td align="center">8.1</td>
<td align="center">5</td>
<td align="center">6.9</td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">12</td>
<td align="center">9.2</td>
<td align="center">3</td>
<td align="center">7.4</td>
<td align="center">2</td>
<td align="center">5.5</td>
<td align="center">10</td>
<td align="center">7.4</td>
<td align="center">8</td>
<td align="center">5.5</td>
<td align="center">11</td>
<td align="center">11.1</td>
<td align="center">11</td>
<td align="center">12.9</td>
<td align="center">13</td>
<td align="center">11.1</td>
</tr>
<tr>
<td align="left" colspan="11"><bold>Clinical judgement</bold></td>
<td align="center" rowspan="3">0.014</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.287</td>
</tr>
<tr>
<td align="left">Some feedback</td>
<td align="center">3</td>
<td align="center">4.7</td>
<td align="center">4</td>
<td align="center">3.8</td>
<td align="center">7</td>
<td align="center">2.8</td>
<td align="center">1</td>
<td align="center">3.8</td>
<td align="center">3</td>
<td align="center">2.8</td>
<td align="center">3</td>
<td align="center">5.7</td>
<td align="center">8</td>
<td align="center">6.6</td>
<td align="center">7</td>
<td align="center">5.7</td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">12</td>
<td align="center">10.3</td>
<td align="center">8</td>
<td align="center">8.2</td>
<td align="center">2</td>
<td align="center">6.2</td>
<td align="center">11</td>
<td align="center">8.2</td>
<td align="center">6</td>
<td align="center">6.2</td>
<td align="center">15</td>
<td align="center">12.3</td>
<td align="center">13</td>
<td align="center">14.4</td>
<td align="center">11</td>
<td align="center">12.3</td>
</tr>
<tr>
<td align="left" colspan="11"><bold>Explaining and planning</bold></td>
<td align="center" rowspan="3">&#x003C; 0.001</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.765</td>
</tr>
<tr>
<td align="left">Some feedback</td>
<td align="center">0</td>
<td align="center">4.5</td>
<td align="center">6</td>
<td align="center">3.6</td>
<td align="center">8</td>
<td align="center">2.7</td>
<td align="center">2</td>
<td align="center">3.6</td>
<td align="center">1</td>
<td align="center">2.7</td>
<td align="center">6</td>
<td align="center">5.4</td>
<td align="center">7</td>
<td align="center">6.3</td>
<td align="center">4</td>
<td align="center">5.4</td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">15</td>
<td align="center">10.5</td>
<td align="center">6</td>
<td align="center">8.4</td>
<td align="center">1</td>
<td align="center">6.3</td>
<td align="center">10</td>
<td align="center">8.4</td>
<td align="center">8</td>
<td align="center">6.3</td>
<td align="center">12</td>
<td align="center">12.6</td>
<td align="center">14</td>
<td align="center">14.7</td>
<td align="center">14</td>
<td align="center">12.6</td>
</tr>
<tr>
<td align="left" colspan="11"><bold>Show organised approach</bold></td>
<td align="center" rowspan="3">&#x003C;0.001</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.794</td>
</tr>
<tr>
<td align="left">Some feedback</td>
<td align="center">0</td>
<td align="center">3.4</td>
<td align="center">2</td>
<td align="center">2.7</td>
<td align="center">8</td>
<td align="center">2.1</td>
<td align="center">3</td>
<td align="center">2.7</td>
<td align="center">0</td>
<td align="center">2.1</td>
<td align="center">3</td>
<td align="center">4.1</td>
<td align="center">5</td>
<td align="center">4.8</td>
<td align="center">5</td>
<td align="center">4.1</td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">15</td>
<td align="center">11.6</td>
<td align="center">10</td>
<td align="center">9.3</td>
<td align="center">1</td>
<td align="center">6.9</td>
<td align="center">9</td>
<td align="center">9.3</td>
<td align="center">9</td>
<td align="center">6.9</td>
<td align="center">15</td>
<td align="center">13.9</td>
<td align="center">16</td>
<td align="center">16.2</td>
<td align="center">13</td>
<td align="center">13.9</td>
</tr>
<tr>
<td align="left" colspan="11"><bold>Overall competence</bold></td>
<td align="center" rowspan="3">&#x003C; 0.001</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.902</td>
</tr>
<tr>
<td align="left">Some feedback</td>
<td align="center">0</td>
<td align="center">2.4</td>
<td align="center">3</td>
<td align="center">1.9</td>
<td align="center">6</td>
<td align="center">1.4</td>
<td align="center">0</td>
<td align="center">1.9</td>
<td align="center">0</td>
<td align="center">1.4</td>
<td align="center">3</td>
<td align="center">2.8</td>
<td align="center">4</td>
<td align="center">3.3</td>
<td align="center">2</td>
<td align="center">2.8</td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">15</td>
<td align="center">12.6</td>
<td align="center">9</td>
<td align="center">10.1</td>
<td align="center">3</td>
<td align="center">7.6</td>
<td align="center">12</td>
<td align="center">10.1</td>
<td align="center">9</td>
<td align="center">7.6</td>
<td align="center">15</td>
<td align="center">15.2</td>
<td align="center">17</td>
<td align="center">17.7</td>
<td align="center">16</td>
<td align="center">15.2</td>
</tr>
<tr>
<td align="left" colspan="11"><bold>Rating of the extent of supervisors&#x2019; feedback</bold></td>
<td align="center" rowspan="3">&#x003C; 0.001</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.932</td>
</tr>
<tr>
<td align="left">Some feedback</td>
<td align="center">2</td>
<td align="center">5.5</td>
<td align="center">9</td>
<td align="center">4.4</td>
<td align="center">8</td>
<td align="center">3.3</td>
<td align="center">2</td>
<td align="center">4.4</td>
<td align="center">0</td>
<td align="center">3.3</td>
<td align="center">7</td>
<td align="center">6.6</td>
<td align="center">8</td>
<td align="center">7.7</td>
<td align="center">6</td>
<td align="center">6.6</td>
</tr>
<tr>
<td align="left">No feedback</td>
<td align="center">13</td>
<td align="center">9.5</td>
<td align="center">3</td>
<td align="center">7.6</td>
<td align="center">1</td>
<td align="center">5.7</td>
<td align="center">10</td>
<td align="center">7.6</td>
<td align="center">9</td>
<td align="center">5.7</td>
<td align="center">11</td>
<td align="center">11.4</td>
<td align="center">13</td>
<td align="center">13.3</td>
<td align="center">12</td>
<td align="center">11.4</td>
</tr>
<tr>
<td align="left" colspan="11"><bold>Signed by registrar and supervisor</bold></td>
<td align="center" rowspan="3">&#x003C; 0.001</td>
<td align="left" colspan="6"></td>
<td align="center" rowspan="3">0.944</td>
</tr>
<tr>
<td align="left">Completed</td>
<td align="center">1</td>
<td align="center">6.8</td>
<td align="center">6</td>
<td align="center">5.8</td>
<td align="center">4</td>
<td align="center">4.3</td>
<td align="center">7</td>
<td align="center">5.8</td>
<td align="center">9</td>
<td align="center">4.3</td>
<td align="center">9</td>
<td align="center">8.2</td>
<td align="center">10</td>
<td align="center">10.1</td>
<td align="center">8</td>
<td align="center">8.7</td>
</tr>
<tr>
<td align="left">Not completed</td>
<td align="center">13</td>
<td align="center">7.3</td>
<td align="center">6</td>
<td align="center">6.2</td>
<td align="center">5</td>
<td align="center">4.7</td>
<td align="center">5</td>
<td align="center">6.2</td>
<td align="center">0</td>
<td align="center">4.7</td>
<td align="center">8</td>
<td align="center">8.8</td>
<td align="center">11</td>
<td align="center">10.9</td>
<td align="center">10</td>
<td align="center">9.3</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<fn><p>Mini-CEX, mini-Clinical Evaluation Exercise; D, district; Y, year.</p></fn>
<fn id="TFN0001"><label>&#x2020;</label><p>, Fisher&#x2019;s exact test, <italic>p</italic>&#x003C; 0.05.</p></fn>
</table-wrap-foot>
</table-wrap>
<p>When the strength of association of D1&#x2013;D5 with the feedback categories was tested further, D3 feedback rated higher when compared with other districts in all mini-CEX competencies with a statistical significance (<italic>p</italic> &#x003C; 0.001 to <italic>p</italic> = 0.003). The &#x2018;extent of supervisors&#x2019; feedback was also rated higher in D2 and D3 than D1, D4 and D5, which was significantly different (<italic>p</italic> &#x003C; 0.001 to <italic>p</italic> = 0.002). D1 had significantly more signed and completed forms than D5 (<italic>p</italic> &#x003C; 0.001).</p>
<p>Comparing the global scores in learning plans and total scores in mini-CEX forms with the year of training revealed significant differences in the median values of the global score (<italic>p</italic> = 0.012) and the total score (<italic>p</italic> = 0.049). On further analysis with two-sample Wilcoxon rank-sum tests, the differences were between Y1 and Y3 in both global score (<italic>p</italic> = 0.003) and the total score (<italic>p</italic> = 0.016). There were no significant differences between Y1 and Y2, or Y2 and Y3, across both forms. There were no significant differences in the median global and total scores across D1&#x2013;D5.</p>
<p>The FP supervisors had provided the feedback in 22/36 learning plans, non-FP supervisors in 12/36 and 2/36 had no feedback. Non-FP supervisors provided more extended feedback (average of &#x2212;25.8 words) than FP supervisors (average of &#x2212;13.7 words). The feedback focused mainly on &#x2018;highlighting strengths&#x2019; or was &#x2018;non-specific&#x2019;, with less emphasis on &#x2018;specific areas for improvement&#x2019; and &#x2018;providing action plans&#x2019; (<xref ref-type="fig" rid="F0001">Figure 1</xref>). The FP feedback focused only on registrars&#x2019; &#x2018;work ethic&#x2019; and &#x2018;clinical skills&#x2019;. In contrast, non-FP feedback was more comprehensive because it referred to various areas, including &#x2018;professional behaviour&#x2019; and &#x2018;teamwork&#x2019; (<xref ref-type="fig" rid="F0002">Figure 2</xref>).</p>
<fig id="F0001">
<label>FIGURE 1</label>
<caption><p>Overall quality of feedback in learning plans and mini-Clinical Evaluation Exercise forms.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="PHCFM-14-3744-g001.tif"/>
</fig>
<fig id="F0002">
<label>FIGURE 2</label>
<caption><p>Areas of feedback in learning plans by family physicians and non-family physicians.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="PHCFM-14-3744-g002.tif"/>
</fig>
<p>The FPs mini-CEX feedback focused on history-taking, comprehensive management and organisational efficiency. There was less emphasis on negotiation skills and comprehensive assessment (<xref ref-type="fig" rid="F0003">Figure 3</xref>). The average feedback by FPs in the mini-CEX forms (16.3 words) was longer than the learning plans (13.7 words). There was more emphasis on specifying &#x2018;areas of improvement&#x2019; (per the overall quality evaluation) compared with the learning plans (<xref ref-type="fig" rid="F0001">Figure 1</xref>).</p>
<fig id="F0003">
<label>FIGURE 3</label>
<caption><p>Areas of feedback in mini-clinical Evaluation Exercise forms by family physicians.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="PHCFM-14-3744-g003.tif"/>
</fig>
</sec>
<sec id="s0014">
<title>Discussion</title>
<p>This is one of the first sub-Saharan studies evaluating feedback in registrars&#x2019; learning plans and mini-CEX forms as evidence of educational and clinical supervision. The general quantity and quality of the feedback across both forms were inadequate, which has implications for the registrars&#x2019; ability to engage in self-directed learning.</p>
<p>The feedback in the learning plans lacked elements of effectiveness such as context specificity, being tailored to the trainee&#x2019;s needs and documented action plans. Other deficiencies were failure to focus on areas needing improvement, such as gaps in clinical knowledge, skills and professional behaviour and not specifying how the supervisor would assist the registrar in addressing these gaps. These findings were similar to previous studies where supervisors struggled to point out registrars&#x2019; weaknesses or offer suggestions for improvement or action plans.<sup><xref ref-type="bibr" rid="CIT0029">29</xref></sup> Given feedback&#x2019;s vital role in promoting self-directed learning,<sup><xref ref-type="bibr" rid="CIT0018">18</xref></sup> the lack of focus on registrars&#x2019; learning needs deprives them of opportunities to reflect on how they could improve their performance. As found previously, the written narrative feedback also minimally reflected registrar responses to feedback.<sup><xref ref-type="bibr" rid="CIT0009">9</xref>,<xref ref-type="bibr" rid="CIT0018">18</xref></sup></p>
<p>Several studies evaluating mini-CEX forms in HIC<sup><xref ref-type="bibr" rid="CIT0023">23</xref>,<xref ref-type="bibr" rid="CIT0039">39</xref></sup> found that feedback focused more on adequately and less on inadequately performed areas and did not always include documented action plans. The findings of this study were similar, with a tendency for feedback to focus on adequate performance, highlighting strengths, and neglecting areas that needed improvement, with few action plans included. In addition, written feedback was lacking in many of the forms. Mini-CEX forms have descriptors that guide structured feedback on observed performance,<sup><xref ref-type="bibr" rid="CIT0008">8</xref></sup> but these were not used effectively. Possible reasons for supervisors not providing effective feedback could be their multiple roles and responsibilities<sup><xref ref-type="bibr" rid="CIT0033">33</xref></sup> and the lack of protected time for educational activities,<sup><xref ref-type="bibr" rid="CIT0040">40</xref></sup> especially for direct observations or their lack of skills or training.<sup><xref ref-type="bibr" rid="CIT0005">5</xref>,<xref ref-type="bibr" rid="CIT0017">17</xref>,<xref ref-type="bibr" rid="CIT0039">39</xref></sup> The supervisors in this study often appeared reluctant to provide honest and negative feedback and grade trainee performance when it is poor, as has been seen in previous studies.<sup><xref ref-type="bibr" rid="CIT0001">1</xref>,<xref ref-type="bibr" rid="CIT0005">5</xref></sup> Other possible reasons for the poor supervisory feedback reported in this study could be deficiencies in trainers&#x2019; knowledge and clinical competence, facilitation or interpersonal skills, and awareness of trainee&#x2019;s learning needs &#x2013; all factors that have been described in other studies.<sup><xref ref-type="bibr" rid="CIT0002">2</xref>,<xref ref-type="bibr" rid="CIT0033">33</xref></sup> Organisational factors such as work demands, poor recognition for teaching activities and a lack of a conducive environment could also negatively affect feedback provision.<sup><xref ref-type="bibr" rid="CIT0001">1</xref></sup> Some FP supervisors participated in training courses conducted by the academy or the university but ongoing training opportunities could capacitate them better than once off attending training.<sup><xref ref-type="bibr" rid="CIT0041">41</xref></sup></p>
<p>In the mini-CEX forms, instead of using competency descriptors, supervisors mainly provided global feedback or non-specific judgements such as &#x2018;satisfactory&#x2019;, which are vague and do not offer any educational value, as reported previously.<sup><xref ref-type="bibr" rid="CIT0022">22</xref>,<xref ref-type="bibr" rid="CIT0023">23</xref>,<xref ref-type="bibr" rid="CIT0042">42</xref></sup> The inadequate space for written feedback on each competency resulted in supervisors recording feedback anywhere on the forms. The WPBA forms can be modified to so-called &#x2018;supervised learning events&#x2019;, as done in the United Kingdom,<sup><xref ref-type="bibr" rid="CIT0043">43</xref></sup> in which text boxes can replace rating scales with tick boxes to provide narrative feedback. Modified mini-CEX forms with separate spaces to record areas performed well, those that need improvement, and action plans improve educational impact because they act as reminders for supervisors.<sup><xref ref-type="bibr" rid="CIT0036">36</xref>,<xref ref-type="bibr" rid="CIT0044">44</xref></sup> Although general descriptors offer a rough guide for providing feedback, modifying the forms to include the categories &#x2018;competent&#x2019;, &#x2018;not competent&#x2019;, and &#x2018;good&#x2019; can guide supervisors to provide more specific feedback.<sup><xref ref-type="bibr" rid="CIT0045">45</xref></sup></p>
<p>Most FP supervisors scored highly on all mini-CEX competencies and total and global scores across all years of training. The lenient scoring was evident from the first year of training and did not exhibit the regular progression as seen in previous studies.<sup><xref ref-type="bibr" rid="CIT0045">45</xref>,<xref ref-type="bibr" rid="CIT0046">46</xref></sup> The global and total scores progression was evident only between the first and third years, in contrast to previous studies demonstrating score progression across all 3 years.<sup><xref ref-type="bibr" rid="CIT0046">46</xref></sup> Ideally, feedback should be provided in the areas where registrars scored low. Insufficient supervisor feedback on the areas that scored low could lead to the registrar&#x2019;s self-assessing by over- or under-estimating their performance against expected standards.<sup><xref ref-type="bibr" rid="CIT0008">8</xref></sup> Feedback should improve and be more holistic during the later years compared with the first year, where it could be limited to fewer competencies and be more specific. This avoids overloading junior registrars with feedback, allowing them to apply it in similar situations and enhancing transferability.<sup><xref ref-type="bibr" rid="CIT0006">6</xref></sup> More than half of the mini-CEX forms were not signed by FPs or registrars, suggesting a lack of accountability from supervisors and registrars.</p>
<p>At the end of each clinical rotation, FPs and not non-FPs are expected to assess and provide feedback on whether registrars met the initial learning needs identified in their learning plans. Feedback provided by specialists and medical officers who functioned as educational supervisors may not have been aligned with training expectations. This could have been because of non-FP supervisors being unclear about the registrar&#x2019;s initially identified expected learning outcomes. A previous SA study also reported that registrars received more feedback from non-FP supervisors, which they found useful.<sup><xref ref-type="bibr" rid="CIT0047">47</xref></sup> The role of non-FP supervisors in registrar supervision needs further research.</p>
<p>The reliability of WPBAs increases with multiple assessments conducted at various times by different assessors.<sup><xref ref-type="bibr" rid="CIT0022">22</xref>,<xref ref-type="bibr" rid="CIT0045">45</xref></sup> In this study, registrars had multiple assessors in some districts, but, in others, the mini-CEX forms were completed only by their immediate supervisor. Clinicians functioning as supervisors in clinical settings are neither always trained in medical education<sup><xref ref-type="bibr" rid="CIT0008">8</xref></sup> nor equipped to give meaningful feedback.<sup><xref ref-type="bibr" rid="CIT0005">5</xref>,<xref ref-type="bibr" rid="CIT0006">6</xref></sup> This could negatively impact registrar training, even more so in districts where only one supervisor provides feedback. Regular faculty development is needed to understand the educational value of feedback.<sup><xref ref-type="bibr" rid="CIT0022">22</xref>,<xref ref-type="bibr" rid="CIT0027">27</xref></sup> Identifying a core group of well-trained supervisors to provide feedback when conducting WPBAs<sup><xref ref-type="bibr" rid="CIT0005">5</xref></sup> could be another strategy to address the reliability of mini-CEX assessments.<sup><xref ref-type="bibr" rid="CIT0027">27</xref></sup></p>
<p>Feedback in the learning plans focused on ethics, professionalism, clinical knowledge and skills and learning behaviours. This was in contrast to a study where professionalism was the learning goal least identified by trainees.<sup><xref ref-type="bibr" rid="CIT0048">48</xref></sup> Feedback provided by specialists and medical officers was much more comprehensive, covered many competencies and was aligned to registrars&#x2019; learning needs (<xref ref-type="fig" rid="F0002">Figure 2</xref>). Supervisors&#x2019; ability to provide feedback on appropriate history-taking and physical examination skills is critical in workplace learning, as clinical diagnosis depends on these registrar skills.<sup><xref ref-type="bibr" rid="CIT0005">5</xref></sup> Mini-CEX feedback mostly covered history-taking, physical examination, comprehensive management and organisational efficiency. Clinical reasoning was the least covered. This differed from previous studies where feedback focused least on professionalism and organisational efficiency.<sup><xref ref-type="bibr" rid="CIT0049">49</xref></sup> As reported previously, there was no alignment of many feedback areas to the registrars&#x2019; learning needs.<sup><xref ref-type="bibr" rid="CIT0049">49</xref></sup> Family physicians provided scanty feedback on soft skills, such as clinical reasoning or patient negotiation skills (<xref ref-type="fig" rid="F0003">Figure 3</xref>). High scores given by FP supervisors on all competencies suggested insufficient skills to evaluate registrars critically, especially for soft skills. Ideally, more narrative feedback should be offered on identified learning needs where supervisors gave low scores.</p>
<p>This is the first sub-Saharan study to explore the significance of training sites and supervisor characteristics on the quality of feedback. While one district (D3) had fewer FPs involved in training, registrars received more comprehensive feedback than other districts with more FPs. This infers that feedback does not depend on the number of supervisors but is context-specific, dependent on willingness and effort, understanding of feedback characteristics<sup><xref ref-type="bibr" rid="CIT0005">5</xref></sup> and assessment literacy.<sup><xref ref-type="bibr" rid="CIT0007">7</xref>,<xref ref-type="bibr" rid="CIT0022">22</xref></sup> All supervisors and registrars involved in decentralised clinical training require ongoing faculty development training on what constitutes adequate feedback to maximise this essential element of workplace-based training and assessments in decentralised settings. There are several training options that could supplement the existing training. One option is to conduct practical sessions using videos of actual registrar consultations and the supervisory feedback that could be critiqued by pairs or groups of FPs as done in other settings.<sup><xref ref-type="bibr" rid="CIT0050">50</xref></sup> Another strategy could capitalise on the peer coaching or co-teaching<sup><xref ref-type="bibr" rid="CIT0001">1</xref></sup> opportunities provided by observed consultations &#x2013; pairs of junior and senior FPs should provide feedback to registrars after these sessions. Similarly, feedback sessions between FPs and registrars in combined sessions with medical students or interns at clinics or hospitals provide opportunities for registrars to practise and internalise effective feedback. Finally, and most importantly, FPs should promote an empathic learning culture based on constructive feedback, even at the lowest family medicine training level, at the primary healthcare facilities.</p>
<sec id="s20015">
<title>Limitations</title>
<p>Only written feedback was evaluated; verbal feedback during direct observations was not included. Scores and feedback were compared across the different sets of registrars in the three years but not for the same registrar across training years. Although it is difficult to make high-quality assessments in postgraduate training because of low numbers,<sup><xref ref-type="bibr" rid="CIT0027">27</xref></sup> all registrars&#x2019; portfolios were included to maximise participation. The principal author constantly engaged with co-authors on coding and analysis, improving internal and external validity. The study was conducted in the decentralised sites of one university, which could affect the generalisability of the results.</p>
</sec>
<sec id="s20016">
<title>Recommendations</title>
<p>Given the integral role of feedback in effective clinical and educational supervision, we suggest there is a need to improve the feedback supervisors provide to trainees and how trainees utilise the feedback. Supervisors and registrars need regular faculty development on various aspects of effective feedback. Training for supervisors should focus on how to give specific feedback according to registrar performance, especially on softer skills such as clinical reasoning and professionalism, along with other hard skills such as history-taking and physical examination, how to develop action plans and assist in implementing the action plans. For the registrars, the training should focus on utilising the written feedback to develop as self-directed lifelong learners.</p>
</sec>
</sec>
<sec id="s0017">
<title>Conclusion</title>
<p>This study evaluated supervisors&#x2019; feedback in family medicine registrars&#x2019; LPs. Supervisor feedback was shown to be inadequate, often very general and not very helpful in developing action plans to improve registrars&#x2019; skills. The lack of detail tailored to individual needs could negatively impact the training of health workers. Family physicians&#x2019; supervisory role is vital to ensure the training of skilled family physicians to strengthen primary healthcare and district health systems, which should translate to better health outcomes in communities. Future research on feedback in postgraduate family medicine training programmes in similar contexts could help to improve the quality of this cadre of health professionals.</p>
</sec>
</body>
<back>
<ack>
<title>Acknowledgements</title>
<p>The authors would like to express their special thanks to the registrars and family physicians of the University of the Witwatersrand, who consented to access their learning plans and mini-CEX forms. The authors acknowledge the statistical assistance from Prof E.L. and Dr Z.M.Z. from the University of the Witwatersrand.</p>
<sec id="s20018" sec-type="COI-statement">
<title>Competing interests</title>
<p>The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.</p>
</sec>
<sec id="s20019">
<title>Authors&#x2019; contributions</title>
<p>N.J.E. conceptualised the research, collected and analysed the data and wrote the first draft of the manuscript. N.J.E., A.Z.G. and L.S.J. contributed to the data analysis, and revising the subsequent drafts. All authors contributed to the development of the manuscript and approved the final version.</p>
</sec>
<sec id="s20020">
<title>Funding information</title>
<p>This research work is supported by the faculty Research Committee Individual Research Grants 2021, University of the Witwatersrand.</p>
</sec>
<sec id="s20021">
<title>Data availability</title>
<p>The data that support the findings of this study can by made available by the corresponding author, N.J.E., upon reasonable request.</p>
</sec>
<sec id="s20022">
<title>Disclaimer</title>
<p>The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.</p>
</sec>
</ack>
<ref-list id="references">
<title>References</title>
<ref id="CIT0001"><label>1</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Ramani</surname> <given-names>S</given-names></string-name>, <string-name><surname>Leinster</surname> <given-names>S</given-names></string-name></person-group>. <article-title>AMEE Guide No. 34: Teaching in the clinical environment</article-title>. <source>Med Teach</source>. <year>2008</year>;<volume>30</volume>(<issue>4</issue>):<fpage>347</fpage>&#x2013;<lpage>364</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/01421590802061613">https://doi.org/10.1080/01421590802061613</ext-link></comment></mixed-citation></ref>
<ref id="CIT0002"><label>2</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Kilminster</surname> <given-names>S</given-names></string-name>, <string-name><surname>Cottrell</surname> <given-names>D</given-names></string-name>, <string-name><surname>Grant</surname> <given-names>J</given-names></string-name>, <string-name><surname>Jolly</surname> <given-names>B</given-names></string-name></person-group>. <article-title>AMEE Guide No. 27: Effective educational and clinical supervision</article-title>. <source>Med Teach</source>. <year>2007</year>;<volume>29</volume>(<issue>1</issue>):<fpage>2</fpage>&#x2013;<lpage>19</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/01421590701210907">https://doi.org/10.1080/01421590701210907</ext-link></comment></mixed-citation></ref>
<ref id="CIT0003"><label>3</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Wood</surname> <given-names>DF</given-names></string-name></person-group>. <chapter-title>Formative assessment: Assessment for learning</chapter-title>. In: <person-group person-group-type="editor"><string-name><surname>Swanwick</surname> <given-names>T</given-names></string-name>, <string-name><surname>Forrest</surname> <given-names>K</given-names></string-name>, <string-name><surname>O&#x2019;Brien</surname> <given-names>BC</given-names></string-name>, editors</person-group>. <source>Understanding medical education: Evidence, theory and practice</source>. <edition>3rd</edition> ed. <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>Wiley-Blackwell</publisher-name>; <year>2019</year>, p. <fpage>366</fpage>&#x2013;<lpage>369</lpage>.</mixed-citation></ref>
<ref id="CIT0004"><label>4</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Kogan</surname> <given-names>JR</given-names></string-name>, <string-name><surname>Hatala</surname> <given-names>R</given-names></string-name>, <string-name><surname>Hauer</surname> <given-names>KE</given-names></string-name>, <string-name><surname>Holmboe</surname> <given-names>E</given-names></string-name></person-group>. <article-title>Guidelines: The do&#x2019;s, don&#x2019;ts and don&#x2019;t knows of direct observation of clinical skills in medical education</article-title>. <source>Perspect Med Educ</source>. <year>2017</year>;<volume>6</volume>(<issue>5</issue>):<fpage>286</fpage>&#x2013;<lpage>305</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1007/s40037-017-0376-7">https://doi.org/10.1007/s40037-017-0376-7</ext-link></comment></mixed-citation></ref>
<ref id="CIT0005"><label>5</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Norcini</surname> <given-names>J</given-names></string-name>, <string-name><surname>Burch</surname> <given-names>V</given-names></string-name></person-group>. <article-title>Workplace-based assessment as an educational tool: AMEE Guide No. 31</article-title>. <source>Med Teach</source>. <year>2007</year>;<volume>29</volume>(<issue>9&#x2013;10</issue>):<fpage>855</fpage>&#x2013;<lpage>871</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/01421590701775453">https://doi.org/10.1080/01421590701775453</ext-link></comment></mixed-citation></ref>
<ref id="CIT0006"><label>6</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Wood</surname> <given-names>BP</given-names></string-name></person-group>. <article-title>Feedback: A key feature of medical training</article-title>. <source>Radiology</source>. <year>2000</year>;<volume>215</volume>(<issue>1</issue>): <fpage>17</fpage>&#x2013;<lpage>19</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="http://doi.org/10.1148/radiology.215.1.r00ap5917">http://doi.org/10.1148/radiology.215.1.r00ap5917</ext-link></comment></mixed-citation></ref>
<ref id="CIT0007"><label>7</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Prentice</surname> <given-names>S</given-names></string-name>, <string-name><surname>Benson</surname> <given-names>J</given-names></string-name>, <string-name><surname>Kirkpatrick</surname> <given-names>E</given-names></string-name>, <string-name><surname>Schuwirth</surname> <given-names>L</given-names></string-name></person-group>. <article-title>Workplace-based assessments in postgraduate medical education: A hermeneutic review</article-title>. <source>Med Educ</source>. <year>2020</year>;<volume>54</volume>(<issue>11</issue>): <fpage>981</fpage>&#x2013;<lpage>992</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1111/medu.14221">https://doi.org/10.1111/medu.14221</ext-link></comment></mixed-citation></ref>
<ref id="CIT0008"><label>8</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Burgess</surname> <given-names>A</given-names></string-name>, <string-name><surname>Mellis</surname> <given-names>C</given-names></string-name></person-group>. <article-title>Feedback and assessment for clinical placements: Achieving the right balance</article-title>. <source>Adv Med Educ Pract [serial online]</source>. <year>2015</year> [cited 2021 Jun 12];<volume>6</volume>: <fpage>373</fpage>&#x2013;<lpage>381</lpage>. <comment>Available from: <ext-link ext-link-type="uri" xlink:href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4445314/">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4445314/</ext-link></comment></mixed-citation></ref>
<ref id="CIT0009"><label>9</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Weallans</surname> <given-names>J</given-names></string-name>, <string-name><surname>Roberts</surname> <given-names>C</given-names></string-name>, <string-name><surname>Hamilton</surname> <given-names>S</given-names></string-name>, <string-name><surname>Parker</surname> <given-names>S</given-names></string-name></person-group>. <article-title>Guidance for providing effective feedback in clinical supervision in postgraduate medical education: A systematic review</article-title>. <source>Postgrad Med J</source>. <year>2022</year>;<volume>98</volume>(<issue>1156</issue>):<fpage>138</fpage>&#x2013;<lpage>149</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1136/postgradmedj-2020-139566">https://doi.org/10.1136/postgradmedj-2020-139566</ext-link></comment></mixed-citation></ref>
<ref id="CIT0010"><label>10</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Lefroy</surname> <given-names>J</given-names></string-name>, <string-name><surname>Watling</surname> <given-names>C</given-names></string-name>, <string-name><surname>Teunissen</surname> <given-names>PW</given-names></string-name>, <string-name><surname>Brand</surname> <given-names>P</given-names></string-name></person-group>. <article-title>Guidelines: The do&#x2019;s, don&#x2019;ts and don&#x2019;t knows of feedback for clinical education</article-title>. <source>Perspect Med Educ</source>. <year>2015</year>;<volume>4</volume>(<issue>6</issue>):<fpage>284</fpage>&#x2013;<lpage>299</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.10.1007/s40037-015-0231-7">https://doi.10.1007/s40037-015-0231-7</ext-link></comment></mixed-citation></ref>
<ref id="CIT0011"><label>11</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Ekpenyong</surname> <given-names>A</given-names></string-name>, <string-name><surname>Zetkulic</surname> <given-names>M</given-names></string-name>, <string-name><surname>Edgar</surname> <given-names>L</given-names></string-name>, <string-name><surname>Holmboe</surname> <given-names>ES</given-names></string-name></person-group>. <article-title>Reimagining feedback for the milestones era</article-title>. <source>J Grad Med Educ</source>. <year>2021</year>;<volume>13</volume>(<supplement>2s</supplement>):<fpage>109</fpage>&#x2013;<lpage>112</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4300/JGME-D-20-00840.1">https://doi.org/10.4300/JGME-D-20-00840.1</ext-link></comment></mixed-citation></ref>
<ref id="CIT0012"><label>12</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Bartlett</surname> <given-names>J</given-names></string-name>, <string-name><surname>Silver</surname> <given-names>ND</given-names></string-name>, <string-name><surname>Rughani</surname> <given-names>A</given-names></string-name>, <string-name><surname>Rushforth</surname> <given-names>B</given-names></string-name>, <string-name><surname>Selby</surname> <given-names>M</given-names></string-name>, <string-name><surname>Mehay</surname> <given-names>R</given-names></string-name></person-group>. <chapter-title>MRCGP in nutshell</chapter-title>. In: <person-group person-group-type="editor"><string-name><surname>Mehay</surname> <given-names>R</given-names></string-name>, editor</person-group>. <source>The essential handbook for GP training and education</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Radcliff Publishing</publisher-name>; <year>2013</year>, p. <fpage>365</fpage>&#x2013;<lpage>367</lpage>.</mixed-citation></ref>
<ref id="CIT0013"><label>13</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Launer</surname> <given-names>J</given-names></string-name></person-group>. <chapter-title>Supervision, mentoring, and coaching</chapter-title>. In: <person-group person-group-type="editor"><string-name><surname>Swanwick</surname> <given-names>T</given-names></string-name>, <string-name><surname>Forrest</surname> <given-names>K</given-names></string-name>, <string-name><surname>O&#x2019;Brien</surname> <given-names>BC</given-names></string-name>, editors</person-group>. <source>Understanding medical education: Evidence, theory and practice</source>. <edition>3rd</edition> ed. <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>Wiley-Blackwell</publisher-name>; <year>2019</year>, p. <fpage>179</fpage>&#x2013;<lpage>189</lpage>.</mixed-citation></ref>
<ref id="CIT0014"><label>14</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Kumar</surname> <given-names>M</given-names></string-name>, <string-name><surname>Charlton</surname> <given-names>R</given-names></string-name>, <string-name><surname>Mehay</surname> <given-names>R</given-names></string-name></person-group>. <chapter-title>Effective clinical supervision</chapter-title>. In: <person-group person-group-type="editor"><string-name><surname>Mehay</surname> <given-names>R</given-names></string-name>, editor</person-group>. <source>The essential handbook for GP training and education</source>. <publisher-loc>London</publisher-loc>: <publisher-name>Radcliff Publishing</publisher-name>; <year>2013</year>, p. <fpage>370</fpage>&#x2013;<lpage>374</lpage>.</mixed-citation></ref>
<ref id="CIT0015"><label>15</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Tartwijk</surname> <given-names>JV</given-names></string-name>, <string-name><surname>Driessen</surname> <given-names>EW</given-names></string-name></person-group>. <article-title>Portfolios for assessment and learning: AMEE Guide no. 45</article-title>. <source>Med Teach</source>. <year>2009</year>;<volume>31</volume>(<issue>9</issue>):<fpage>790</fpage>&#x2013;<lpage>801</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/01421590903139201">https://doi.org/10.1080/01421590903139201</ext-link></comment></mixed-citation></ref>
<ref id="CIT0016"><label>16</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Norcini</surname> <given-names>JJ</given-names></string-name>, <string-name><surname>Blank</surname> <given-names>LL</given-names></string-name>, <string-name><surname>Arnold</surname> <given-names>GK</given-names></string-name>, <string-name><surname>Kimball</surname> <given-names>HR</given-names></string-name></person-group>. <article-title>The mini-CEX (clinical evaluation exercise): A preliminary investigation</article-title>. <source>Ann Intern Med</source>. <year>1995</year>;<volume>123</volume>(<issue>10</issue>):<fpage>795</fpage>&#x2013;<lpage>799</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="http://doi.org/10.7326/0003-4819-123-10-199511150-00008">http://doi.org/10.7326/0003-4819-123-10-199511150-00008</ext-link></comment></mixed-citation></ref>
<ref id="CIT0017"><label>17</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Lockspeiser</surname> <given-names>TM</given-names></string-name>, <string-name><surname>Kaul</surname> <given-names>P</given-names></string-name></person-group>. <article-title>Using individualized learning plans to facilitate learner-centered teaching</article-title>. <source>J Pediatr Adolesc Gynecol</source>. <year>2016</year>;<volume>29</volume>(<issue>3</issue>):<fpage>214</fpage>&#x2013;<lpage>217</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.jpag.2015.10.020">https://doi.org/10.1016/j.jpag.2015.10.020</ext-link></comment></mixed-citation></ref>
<ref id="CIT0018"><label>18</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Nicol</surname> <given-names>DJ</given-names></string-name>, <string-name><surname>Macfarlane-Dick</surname> <given-names>D</given-names></string-name></person-group>. <article-title>Formative assessment and self-regulated learning: A model and seven principles of good feedback practice</article-title>. <source>Stud High Educ</source>. <year>2006</year>;<volume>31</volume>(<issue>2</issue>):<fpage>199</fpage>&#x2013;<lpage>218</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/03075070600572090">https://doi.org/10.1080/03075070600572090</ext-link></comment></mixed-citation></ref>
<ref id="CIT0019"><label>19</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Edgar</surname> <given-names>L</given-names></string-name>, <string-name><surname>McLean</surname> <given-names>S</given-names></string-name>, <string-name><surname>Hogan</surname> <given-names>SO</given-names></string-name>, <string-name><surname>Hamstra</surname> <given-names>S</given-names></string-name>, <string-name><surname>Holmboe</surname> <given-names>ES</given-names></string-name></person-group>. <source>The milestones guidebook [homepage on the Internet]</source>. <publisher-name>Accreditation Council for Graduate Medical Education</publisher-name>; <year>2020</year> <comment>[cited 2021 Jun 12]. Available from: <ext-link ext-link-type="uri" xlink:href="https://www.acgme.org/What-We-Do/Accreditation/Milestones/Resources">https://www.acgme.org/What-We-Do/Accreditation/Milestones/Resources</ext-link></comment></mixed-citation></ref>
<ref id="CIT0020"><label>20</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Robinson</surname> <given-names>JD</given-names></string-name>, <string-name><surname>Persky</surname> <given-names>AM</given-names></string-name></person-group>. <article-title>Developing self-directed learners</article-title>. <source>Am J Pharm Educ</source>. <year>2020</year>;<volume>84</volume>(<issue>3</issue>):<fpage>847512</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.5688/ajpe847512">https://doi.org/10.5688/ajpe847512</ext-link></comment></mixed-citation></ref>
<ref id="CIT0021"><label>21</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Li</surname> <given-names>S-TT</given-names></string-name>, <string-name><surname>Paterniti</surname> <given-names>DA</given-names></string-name>, <string-name><surname>Co</surname> <given-names>JPT</given-names></string-name>, <string-name><surname>West</surname> <given-names>DC</given-names></string-name></person-group>. <article-title>Successful self-directed lifelong learning in medicine: A conceptual model derived from qualitative analysis of a national survey of pediatric residents</article-title>. <source>Acad Med</source>. <year>2010</year>;<volume>85</volume>(<issue>7</issue>):<fpage>1229</fpage>&#x2013;<lpage>1236</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="http://doi.org/10.1097/ACM.0b013e3181e1931c">http://doi.org/10.1097/ACM.0b013e3181e1931c</ext-link></comment></mixed-citation></ref>
<ref id="CIT0022"><label>22</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Burch</surname> <given-names>VC</given-names></string-name></person-group>. <article-title>The changing landscape of workplace-based assessment</article-title>. <source>J Appl Test Technol [serial online]</source>. <year>2019</year> [cited 2021 Aug 20];<volume>20</volume>(<supplement>S2</supplement>):<fpage>37</fpage>&#x2013;<lpage>59</lpage>. <comment>Available from: <ext-link ext-link-type="uri" xlink:href="http://www.jattjournal.net/index.php/atp/article/view/143675">http://www.jattjournal.net/index.php/atp/article/view/143675</ext-link></comment></mixed-citation></ref>
<ref id="CIT0023"><label>23</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Saedon</surname> <given-names>H</given-names></string-name>, <string-name><surname>Salleh</surname> <given-names>S</given-names></string-name>, <string-name><surname>Balakrishnan</surname> <given-names>A</given-names></string-name>, <string-name><surname>Imray</surname> <given-names>CH</given-names></string-name>, <string-name><surname>Saedon</surname> <given-names>M</given-names></string-name></person-group>. <article-title>The role of feedback in improving the effectiveness of workplace based assessments: A systematic review</article-title>. <source>BMC Med Educ</source>. <year>2012</year>;<volume>12</volume>(<issue>1</issue>):<fpage>25</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/1472-6920-12-25">https://doi.org/10.1186/1472-6920-12-25</ext-link></comment></mixed-citation></ref>
<ref id="CIT0024"><label>24</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>L&#x00F6;rwald</surname> <given-names>AC</given-names></string-name>, <string-name><surname>Lahner</surname> <given-names>F-M</given-names></string-name>, <string-name><surname>Nouns</surname> <given-names>ZM</given-names></string-name>, <string-name><surname>Berendonk</surname> <given-names>C</given-names></string-name>, <string-name><surname>Norcini</surname> <given-names>J</given-names></string-name>, <string-name><surname>Greif</surname> <given-names>R</given-names></string-name>, <etal>et al</etal></person-group>. <article-title>The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis</article-title>. <source>PLoS One</source>. <year>2018</year>;<volume>13</volume>(<issue>6</issue>):<fpage>e0198009</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1371/journal.pone.0198009">https://doi.org/10.1371/journal.pone.0198009</ext-link></comment></mixed-citation></ref>
<ref id="CIT0025"><label>25</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Weller</surname> <given-names>JM</given-names></string-name>, <string-name><surname>Jolly</surname> <given-names>B</given-names></string-name>, <string-name><surname>Misur</surname> <given-names>MP</given-names></string-name>, <string-name><surname>Merry</surname> <given-names>AF</given-names></string-name>, <string-name><surname>Jones</surname> <given-names>A</given-names></string-name>, <string-name><surname>Crossley</surname> <given-names>JGM</given-names></string-name>, <etal>et al</etal></person-group>. <article-title>Mini-clinical evaluation exercise in anaesthesia training</article-title>. <source>Br J Anaesth</source>. <year>2009</year>;<volume>102</volume>(<issue>5</issue>):<fpage>633</fpage>&#x2013;<lpage>641</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1093/bja/aep055">https://doi.org/10.1093/bja/aep055</ext-link></comment></mixed-citation></ref>
<ref id="CIT0026"><label>26</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Jenkins</surname> <given-names>LS</given-names></string-name>, <string-name><surname>Von Pressentin</surname> <given-names>K</given-names></string-name></person-group>. <article-title>Family medicine training in Africa: Views of clinical trainers and trainees</article-title>. <source>Afr J Prim Health Care Fam Med</source>. <year>2018</year>;<volume>10</volume>(<issue>1</issue>):<fpage>e1</fpage>&#x2013;<lpage>e4</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4102/phcfm.v10i1.1638">https://doi.org/10.4102/phcfm.v10i1.1638</ext-link></comment></mixed-citation></ref>
<ref id="CIT0027"><label>27</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Norcini</surname> <given-names>JJ</given-names></string-name>, <string-name><surname>Zaidi</surname> <given-names>Z</given-names></string-name></person-group>. <chapter-title>Workplace assessment</chapter-title>. In: <person-group person-group-type="editor"><string-name><surname>Swanwick</surname> <given-names>T</given-names></string-name>, <string-name><surname>Forrest</surname> <given-names>K</given-names></string-name>, <string-name><surname>O&#x2019;Brien</surname> <given-names>BC</given-names></string-name>, editors</person-group>. <source>Understanding medical education: Evidence, theory and practice</source>. <edition>3rd</edition> ed. <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>Wiley-Blackwell</publisher-name>; <year>2019</year>, p. <fpage>319</fpage>&#x2013;<lpage>333</lpage>.</mixed-citation></ref>
<ref id="CIT0028"><label>28</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Mash</surname> <given-names>RJ</given-names></string-name>, <string-name><surname>Ogunbanjo</surname> <given-names>G</given-names></string-name>, <string-name><surname>Naidoo</surname> <given-names>SS</given-names></string-name>, <string-name><surname>Hellenberg</surname> <given-names>D</given-names></string-name></person-group>. <article-title>The contribution of family physicians to district health services: A national position paper for South Africa</article-title>. <source>S Afr Fam Pract [serial online]</source>. <year>2015</year> [cited 2021 May 05];<volume>57</volume>(<issue>3</issue>):<fpage>54</fpage>&#x2013;<lpage>56</lpage>. <comment>Available from: <ext-link ext-link-type="uri" xlink:href="https://scholar.sun.ac.za/handle/10019.1/99785">https://scholar.sun.ac.za/handle/10019.1/99785</ext-link></comment></mixed-citation></ref>
<ref id="CIT0029"><label>29</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>De Swardt</surname> <given-names>M</given-names></string-name>, <string-name><surname>Jenkins</surname> <given-names>LS</given-names></string-name>, <string-name><surname>Von Pressentin</surname> <given-names>KB</given-names></string-name>, <string-name><surname>Mash</surname> <given-names>R</given-names></string-name></person-group>. <article-title>Implementing and evaluating an e-portfolio for postgraduate family medicine training in the Western Cape, South Africa</article-title>. <source>BMC Med Educ</source>. <year>2019</year>;<volume>19</volume>(<issue>1</issue>):<fpage>251</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/s12909-019-1692-x">https://doi.org/10.1186/s12909-019-1692-x</ext-link></comment></mixed-citation></ref>
<ref id="CIT0030"><label>30</label><mixed-citation publication-type="web"><source>The Colleges of Medicine of South Africa: Fellowship of the College of Family Physicians of South Africa [home page on internet]</source>. <year>2022</year> <comment>[cited 2022 Jul 15]. Available from: <ext-link ext-link-type="uri" xlink:href="https://www.cmsa.co.za/view_exam.aspx?QualificationID=9">https://www.cmsa.co.za/view_exam.aspx?QualificationID=9</ext-link></comment></mixed-citation></ref>
<ref id="CIT0031"><label>31</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Blitz</surname> <given-names>J</given-names></string-name>, <string-name><surname>Edwards</surname> <given-names>J</given-names></string-name>, <string-name><surname>Mash</surname> <given-names>B</given-names></string-name>, <string-name><surname>Mowle</surname> <given-names>S</given-names></string-name></person-group>. <article-title>Training the trainers: Beyond providing a well-received course</article-title>. <source>Educ Prim Care</source>. <year>2016</year>;<volume>27</volume>(<issue>5</issue>):<fpage>375</fpage>&#x2013;<lpage>379</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/14739879.2016.1220237">https://doi.org/10.1080/14739879.2016.1220237</ext-link></comment></mixed-citation></ref>
<ref id="CIT0032"><label>32</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Lovato</surname> <given-names>C</given-names></string-name>, <string-name><surname>Peterson</surname> <given-names>L</given-names></string-name></person-group>. <chapter-title>Programme evaluation</chapter-title>. In: <person-group person-group-type="editor"><string-name><surname>Swanwick</surname> <given-names>T</given-names></string-name>, <string-name><surname>Forrest</surname> <given-names>K</given-names></string-name>, <string-name><surname>O&#x2019;Brien</surname> <given-names>BC</given-names></string-name>, editors</person-group>. <source>Understanding medical education: Evidence, theory and practice</source>. <edition>3rd</edition> ed. <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>Wiley-Blackwell</publisher-name>; <year>2019</year>, p. <fpage>443</fpage>&#x2013;<lpage>455</lpage>.</mixed-citation></ref>
<ref id="CIT0033"><label>33</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Erumeda</surname> <given-names>NJ</given-names></string-name>, <string-name><surname>Jenkins</surname> <given-names>LS</given-names></string-name>, <string-name><surname>George</surname> <given-names>AZ</given-names></string-name></person-group>. <article-title>Perceptions of postgraduate family medicine supervision at decentralised training sites, South Africa</article-title>. <source>Afr J Prim Health Care Fam Med</source>. <year>2022</year>;<volume>14</volume>(<issue>1</issue>):<fpage>a3111</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4102/phcfm.v14i1.3111">https://doi.org/10.4102/phcfm.v14i1.3111</ext-link></comment></mixed-citation></ref>
<ref id="CIT0034"><label>34</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Rowley</surname> <given-names>J</given-names></string-name></person-group>. <article-title>Designing and using research questionnaires</article-title>. <source>Manag Res Rev</source>. <year>2014</year>;<volume>37</volume>(<issue>3</issue>):<fpage>308</fpage>&#x2013;<lpage>330</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1108/MRR-02-2013-0027">https://doi.org/10.1108/MRR-02-2013-0027</ext-link></comment></mixed-citation></ref>
<ref id="CIT0035"><label>35</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Kim</surname> <given-names>H-Y</given-names></string-name></person-group>. <article-title>Statistical notes for clinical researchers: Chi-squared test and Fisher&#x2019;s exact test</article-title>. <source>Restor Dent Endod</source>. <year>2017</year>;<volume>42</volume>(<issue>2</issue>):<fpage>152</fpage>&#x2013;<lpage>155</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.5395/rde.2017.42.2.152">https://doi.org/10.5395/rde.2017.42.2.152</ext-link></comment></mixed-citation></ref>
<ref id="CIT0036"><label>36</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Pelgrim</surname> <given-names>EA</given-names></string-name>, <string-name><surname>Kramer</surname> <given-names>AW</given-names></string-name>, <string-name><surname>Mokkink</surname> <given-names>HG</given-names></string-name>, <string-name><surname>Van der Vleuten</surname> <given-names>CP</given-names></string-name></person-group>. <article-title>Quality of written narrative feedback and reflection in a modified mini-clinical evaluation exercise: An observational study</article-title>. <source>BMC Med Educ</source>. <year>2012</year>;<volume>12</volume>(<issue>1</issue>):<fpage>97</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/1472-6920-12-97">https://doi.org/10.1186/1472-6920-12-97</ext-link></comment></mixed-citation></ref>
<ref id="CIT0037"><label>37</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Krippendorf</surname> <given-names>K</given-names></string-name></person-group>. <source>Content analysis: An introduction to its methodology</source>. <edition>3rd</edition> ed. <publisher-loc>Thousand Oaks, CA</publisher-loc>: <publisher-name>Sage</publisher-name>; <year>2013</year>.</mixed-citation></ref>
<ref id="CIT0038"><label>38</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Cohen</surname> <given-names>L</given-names></string-name>, <string-name><surname>Lawrence</surname> <given-names>M</given-names></string-name>, <string-name><surname>Morrison</surname> <given-names>K</given-names></string-name></person-group>. <source>Research methods in education</source>. <edition>8th</edition> ed. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Routledge</publisher-name>; <year>2018</year>, p. <fpage>269</fpage>.</mixed-citation></ref>
<ref id="CIT0039"><label>39</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Gauthier</surname> <given-names>S</given-names></string-name>, <string-name><surname>Cavalcanti</surname> <given-names>R</given-names></string-name>, <string-name><surname>Goguen</surname> <given-names>J</given-names></string-name>, <string-name><surname>Sibbald</surname> <given-names>M</given-names></string-name></person-group>. <article-title>Deliberate practice as a framework for evaluating feedback in residency training</article-title>. <source>Med Teach</source>. <year>2015</year>;<volume>37</volume>(<issue>6</issue>):<fpage>551</fpage>&#x2013;<lpage>557</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.3109/0142159X.2014.956059">https://doi.org/10.3109/0142159X.2014.956059</ext-link></comment></mixed-citation></ref>
<ref id="CIT0040"><label>40</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Gawne</surname> <given-names>S</given-names></string-name>, <string-name><surname>Fish</surname> <given-names>R</given-names></string-name>, <string-name><surname>Machin</surname> <given-names>L</given-names></string-name></person-group>. <article-title>Developing a workplace-based learning culture in the NHS: Aspirations and challenges</article-title>. <source>J Med Educ Curric Dev</source>. <year>2020</year>;<volume>7</volume>:<fpage>2382120520947063</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1177/2382120520947063">https://doi.org/10.1177/2382120520947063</ext-link></comment></mixed-citation></ref>
<ref id="CIT0041"><label>41</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Mash</surname> <given-names>R</given-names></string-name>, <string-name><surname>Blitz</surname> <given-names>J</given-names></string-name>, <string-name><surname>Edwards</surname> <given-names>J</given-names></string-name>, <string-name><surname>Mowle</surname> <given-names>S</given-names></string-name></person-group>. <article-title>Training of workplace-based clinical trainers in family medicine, South Africa: Before-and-after evaluation</article-title>. <source>Afr J Prim Health Care Fam Med</source>. <year>2018</year>;<volume>10</volume>(<issue>1</issue>):<fpage>1589</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4102/phcfm.v10i1.1589">https://doi.org/10.4102/phcfm.v10i1.1589</ext-link></comment></mixed-citation></ref>
<ref id="CIT0042"><label>42</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Canavan</surname> <given-names>C</given-names></string-name>, <string-name><surname>Holtman</surname> <given-names>MC</given-names></string-name>, <string-name><surname>Richmond</surname> <given-names>M</given-names></string-name>, <string-name><surname>Katsufrakis</surname> <given-names>PJ</given-names></string-name></person-group>. <article-title>The quality of written comments on professional behaviors in a developmental multisource feedback program</article-title>. <source>Acad Med</source>. <year>2010</year>;<volume>85</volume>(<supplement>10 Suppl.</supplement>):<fpage>S106</fpage>&#x2013;<lpage>S109</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org.10.1097/ACM.0b013e3181ed4cdb">https://doi.org.10.1097/ACM.0b013e3181ed4cdb</ext-link></comment></mixed-citation></ref>
<ref id="CIT0043"><label>43</label><mixed-citation publication-type="web"><article-title>Recommendations for specialty trainee assessment and review: Incorporating lessons learnt from the workplace-based assessment pilot</article-title>. <source>Joint Royal College of Physicians Training Board [home page on Internet]</source>. <year>2014</year> <comment>[cited 2021 Jun 20]. Available from: <ext-link ext-link-type="uri" xlink:href="https://www.jrcptb.org.uk/sites/default/files/April&#x0025;202014&#x0025;20Recommendations&#x0025;20for&#x0025;20specialty&#x0025;20trainee&#x0025;20assessment&#x0025;20and&#x0025;20review.pdf">https://www.jrcptb.org.uk/sites/default/files/April&#x0025;202014&#x0025;20Recommendations&#x0025;20for&#x0025;20specialty&#x0025;20trainee&#x0025;20assessment&#x0025;20and&#x0025;20review.pdf</ext-link></comment></mixed-citation></ref>
<ref id="CIT0044"><label>44</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Djajadi</surname> <given-names>RM</given-names></string-name>, <string-name><surname>Claramita</surname> <given-names>M</given-names></string-name>, <string-name><surname>Rahayu</surname> <given-names>GR</given-names></string-name></person-group>. <article-title>Quantity and quality of written feedback, action plans, and student reflections before and after the introduction of a modified mini-CEX assessment form</article-title>. <source>Afr J Health Prof Educ</source>. <year>2017</year>;<volume>9</volume>(<issue>3</issue>):<fpage>148</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="http://doi.10.7196/AJHPE.2017.v9i3.804">http://doi.10.7196/AJHPE.2017.v9i3.804</ext-link></comment></mixed-citation></ref>
<ref id="CIT0045"><label>45</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><surname>Tartwijk</surname> <given-names>JV</given-names></string-name>, <string-name><surname>Driessen</surname> <given-names>EW</given-names></string-name></person-group>. <chapter-title>Portfolios in personal and professional development</chapter-title>. In: <person-group person-group-type="editor"><string-name><surname>Swanwick</surname> <given-names>T</given-names></string-name>, <string-name><surname>Forrest</surname> <given-names>K</given-names></string-name>, <string-name><surname>O&#x2019;Brien</surname> <given-names>BC</given-names></string-name>, editors</person-group>. <source>Understanding medical education: Evidence, theory and practice</source>. <edition>3rd</edition> ed. <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>Wiley-Blackwell</publisher-name>; <year>2018</year>, p. <fpage>255</fpage>&#x2013;<lpage>262</lpage>.</mixed-citation></ref>
<ref id="CIT0046"><label>46</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Hejri</surname> <given-names>SM</given-names></string-name>, <string-name><surname>Jalili</surname> <given-names>M</given-names></string-name>, <string-name><surname>Masoomi</surname> <given-names>R</given-names></string-name>, <string-name><surname>Shirazi</surname> <given-names>M</given-names></string-name>, <string-name><surname>Nedjat</surname> <given-names>S</given-names></string-name>, <string-name><surname>Norcini</surname> <given-names>J</given-names></string-name></person-group>. <article-title>The utility of mini-clinical evaluation exercise in undergraduate and postgraduate medical education: A BEME review: BEME Guide No. 59</article-title>. <source>Med Teach</source>. <year>2020</year>;<volume>42</volume>(<issue>2</issue>):<fpage>125</fpage>&#x2013;<lpage>142</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1080/0142159X.2019.1652732">https://doi.org/10.1080/0142159X.2019.1652732</ext-link></comment></mixed-citation></ref>
<ref id="CIT0047"><label>47</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Jenkins</surname> <given-names>L</given-names></string-name>, <string-name><surname>Mash</surname> <given-names>B</given-names></string-name>, <string-name><surname>Derese</surname> <given-names>A</given-names></string-name></person-group>. <article-title>The national portfolio for postgraduate family medicine training in South Africa: A descriptive study of acceptability, educational impact, and usefulness for assessment</article-title>. <source>BMC Med Educ</source>. <year>2013</year>;<volume>13</volume>(<issue>1</issue>):<fpage>1</fpage>&#x2013;<lpage>11</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/1472-6920-13-101">https://doi.org/10.1186/1472-6920-13-101</ext-link></comment></mixed-citation></ref>
<ref id="CIT0048"><label>48</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Chitkara</surname> <given-names>MB</given-names></string-name>, <string-name><surname>Satnick</surname> <given-names>D</given-names></string-name>, <string-name><surname>Lu</surname> <given-names>W-H</given-names></string-name>, <string-name><surname>Fleit</surname> <given-names>H</given-names></string-name>, <string-name><surname>Go</surname> <given-names>RA</given-names></string-name>, <string-name><surname>Chandran</surname> <given-names>L</given-names></string-name></person-group>. <article-title>Can individualised learning plans in an advanced clinical experience course for fourth year medical students foster self-directed learning?</article-title> <source>BMC Med Educ</source>. <year>2016</year>;<volume>16</volume>(<issue>1</issue>):<fpage>232</fpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/s12909-016-0744-8">https://doi.org/10.1186/s12909-016-0744-8</ext-link></comment></mixed-citation></ref>
<ref id="CIT0049"><label>49</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Montagne</surname> <given-names>S</given-names></string-name>, <string-name><surname>Rogausch</surname> <given-names>A</given-names></string-name>, <string-name><surname>Gemperli</surname> <given-names>A</given-names></string-name>, <string-name><surname>Berendonk</surname> <given-names>C</given-names></string-name>, <string-name><surname>Jucker-Kupper</surname> <given-names>P</given-names></string-name>, <string-name><surname>Beyeler</surname> <given-names>C</given-names></string-name></person-group>. <article-title>The mini-clinical evaluation exercise during medical clerkships: Are learning needs and learning goals aligned?</article-title> <source>Med Educ</source>. <year>2014</year>;<volume>48</volume>(<issue>10</issue>):<fpage>1008</fpage>&#x2013;<lpage>1019</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1111/medu.12513">https://doi.org/10.1111/medu.12513</ext-link></comment></mixed-citation></ref>
<ref id="CIT0050"><label>50</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><surname>Holmboe</surname> <given-names>ES</given-names></string-name>, <string-name><surname>Yepes</surname> <given-names>M</given-names></string-name>, <string-name><surname>Williams</surname> <given-names>F</given-names></string-name>, <string-name><surname>Huot</surname> <given-names>SJ</given-names></string-name></person-group>. <article-title>Feedback and the mini clinical evaluation exercise</article-title>. <source>J Gen Intern Med</source>. <year>2004</year>;<volume>19</volume>(<issue>5</issue>):<fpage>558</fpage>&#x2013;<lpage>561</lpage>. <comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1111/j.1525-1497.2004.30134.x">https://doi.org/10.1111/j.1525-1497.2004.30134.x</ext-link></comment></mixed-citation></ref>
</ref-list>
<fn-group>
<fn><p><bold>How to cite this article:</bold> Erumeda NJ, George AZ, Jenkins LS. Evaluating postgraduate family medicine supervisor feedback in registrars&#x2019; learning portfolios. Afr J Prm Health Care Fam Med. 2022;14(1), a3744. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.4102/phcfm.v14i1.3744">https://doi.org/10.4102/phcfm.v14i1.3744</ext-link></p></fn>
</fn-group>
</back>
</article>