St Vincent 04/01/2019 03/11/2019 Final Review Update Report

U.S. Department of Education

Update Report for the Government of St Vincent and the Grenadines


Prepared February, 2019

Background

The Government of St Vincent and the Grenadines is a member of the Caribbean Community (CARICOM), which has officially designated the Caribbean Accreditation Authority for Education in Medicine and other Health Professions (CAAM-HP) as the legally constituted body empowered to determine and prescribe standards and to accredit medical programs on behalf of the contracting parties in CARICOM. The country is providing an update report based upon its use of the CAAM-HP standards for the evaluation and ongoing monitoring of medical education programs offered through its medical schools.


Summary of Findings

None

Staff Analysis


Current status of medical schools


Country Narrative

There are 3 offshore medical schools in St Vincent. Below is a list of these schools currently operating in St Vincent and their status with the Caribbean Accreditation Authority for Education in Medicine and Other Health Professions (CAAM-HP): 1. Trinity School of Medicine (TSOM) – This is an offshore medical school which was established in 2008. It was last visited by CAAM-HP in April 2017 and was accorded accreditation with conditions for two years, 2017-2019 in July 2017. See Appendix 1: CAAM-HP 2017 Assessment Letter to TSOM. 2. Saint James School of Medicine (SJSM), St Vincent and the Grenadines (SVG) – This is a newly established offshore medical school, one of a two-campus institution with the other campus on the island of Anguilla. It was surveyed by CAAM-HP in June 2017. However, a decision on accreditation was deferred pending clarification as to whether SJSM was a single institution with two campuses. In July 2018, the CAAM-HP extended Initial Provisional Accreditation on Probation for one year, applicable to both campuses. See Appendix 2: CAAM-HP 2018 Assessment Letter to SJSM. 3. All Saints University (ASU) – This offshore medical school was established in 2011. The CAAM-HP surveyed this school in June 2017. However, the school was denied accreditation in July 2017 when the report of the visit was considered by CAAM-HP at its annual meeting. See Appendix 3: CAAM-HP 2018 Assessment Letter to ASU.


Analyst Remarks to Narrative

There are 3 medical schools operating in St. Vincent. The Caribbean Accreditation Authority for Education in Medicine and Other Health Professions (CAAM-HP) has conducted recent reviews at all 3 schools and has granted conditional accreditation to St. Trinity, initial provisional accreditation to Saint James School of Medicine (SJSM), St Vincent and the Grenadines (SVG), and denied accreditation to All Saints University. CAAM-HP has included documentation of its assessment letters to each of the medical schools demonstrating application of the review.


Overview of accreditation activities


Country Narrative

ACCREDITATION REVIEWS The CAAM-HP conducted full site visits to the following medical schools in St Vincent: 1. TSOM: In April 2017, CAAM-HP conducted a full site visit to TSOM’s basic science campus in St Vincent in addition to the following clinical affiliates in Baltimore: • Bon Secours Hospital • Providence Hospital • Willoughby Beach Paediatrics • UM Baltimore Washington See Appendix 4: TSOM Site Visit Report, 2017. At the July 2017 meeting of the CAAM-HP, the MD programme at TSOM was accredited with conditions for two years, 2017-2019. See Appendix 1: CAAM-HP 2017 Assessment Letter to TSOM. Clinical affiliates of TSOM which were visited by CAAM-HP in 2015 were: • Milton Cato Memorial Hospital, St Vincent • Northwest Hospital, Baltimore • Brookhaven Memorial Hospital, New York • Southampton Hospital, New York 2. SJSM: In June 2017, CAAM-HP visited the basic science campus in St Vincent. The report of that visit can be found at Appendix 5: SJSM SVG Site Visit Report, 2017. At its July 2018 meeting, CAAM-HP, after considering an update report from SJSM, extended the Initial Provisional Accreditation on Probation for one year. See Appendix 2: CAAM-HP 2018 Assessment Letter to SJSM. MONITORING The CAAM-HP monitors the medical programmes of TSOM and SJSM during the period of accreditation by way of progress reports which are submitted annually. This is in keeping with the conditions of the schools’ accreditation. These reports outline steps taken to address issues highlighted by CAAM-HP in the site visit reports. Since the site visits in 2017, both TSOM and SJSM provided the CAAM-HP with progress reports which are appended as follows: Appendix 6: TSOM 2018 Progress Report; Appendices 7A & 7B: SJSM 2018 Progress Report, Parts 1 & 2. These reports were considered by CAAM-HP at its annual meeting in 2018. The assessments are appended as follows: Appendix 8: TSOM 2018 Progress Report Assessment; Appendix 9: SJSM 2018 Progress Report Assessment. CAPACITY BUILDING WORKSHOPS CAAM-HP hosted a successful and well received Capacity Building Workshop in Barbados, March 19-22, 2017. Participants came from 24 medical schools in the region including St George’s University. The workshop was designed to provide guidance to schools and strengthen their capacity to prepare for the accreditation process and evaluate themselves against the Standards. Presentations focused on themes such as Aspects of the Accreditation Process; Focusing on the Standards and The Standards in Practice. One significant objective of the Workshop was to introduce the revised accreditation standards for medicine to the schools and help them develop a comprehensive understanding of their interpretation, application and assessment. Through presentations and group deliberations using case studies, the workshop provided participants with an interactive forum to engage with CAAM-HP and medical education experts. Among the outcomes of the Workshop was that suitable participants were identified and served as site visitors for CAAM-HP during the 2017-2018 administrative year. The programme and presentations can be accessed on the CAAM-HP website at http://www.caam-hp.org/workshop.html. CAAM-HP is also pleased to share its 2016-2017 Annual Report (Appendix 10) in which a review of the Workshop was documented on page 16 through to 22. CAAM-HP hosted another Capacity Building Workshop from July 25-28, 2018 in Trinidad and Tobago. That Workshop was attended by approximately one hundred (100) representatives from just under thirty (30) schools in the English-speaking, Spanish-speaking and Dutch territories in the Caribbean. Information about the workshop can be found at https://www.caam-hp.org/news-details/Capacity+Building+Workshop%2C+July+2018+Aug+18+2018/661b9f14-a334-11e8-8584-f23c911818f6


Analyst Remarks to Narrative

Caribbean Accreditation Authority for Education in Medicine and Other Health Professions (CAAM-HP), the agency who oversees medical schools in St. Vincent, has reported information and documentation of its most recent accreditation reviews of school campuses and clinical clerkship sites affiliated with the country. Documentation includes dates of the on-site visits (exhibits 4 and 5) to all of the locations, decision meeting dates (exhibit 2), and training conducted by the agency. The training for the capacity building of the agency was held in March 2018 and the agency references its website that explains the details about the topics that were covered during this event. In addition to the information required in this section, CAAM-HP conducts additional monitoring of its medical schools through annual reports and this documentation is included in exhibits 6-9.


Laws and regulations


Country Narrative

There have been no changes to the laws and regulations that affect the accreditation of medical schools in St Vincent.


Analyst Remarks to Narrative

The country has attested that there have been no changes to their laws or regulations that would affect the accreditation of the its medical schools.


Standards


Country Narrative

In keeping with its Procedures for revision of standards the CAAM-HP Secretariat commissioned a sub-committee to review its accreditation standards for medicine over the period January 2016 and March 2017. Membership of the committee comprised: • 1 representative from an accredited medical school • 1 CAAM-HP member • 1 representative of the Caribbean Association of Medical Councils (CAMC) • 1 person from outside of the region with experience with the CAAM-HP accreditation standards and processes • 1 experienced CAAM-HP site visitor • The Executive Director as an ex-officio member The committee: • maintained the current format of the CAAM-HP standards • considered the existing standards in the light of the recently revised LCME standards • compared the CAAM-HP standards against the revised LCME standards and identified discrepancies resulting in the addition of a number of new standards to the current CAAM-HP standards • changed some standards from “should” to “must” • amalgamated some standards where they overlapped • renumbered the standards in the various sections to take account of the additions and amalgamations • was mindful that the standards must be applicable both to regional and to “off shore” medical schools In considering the standards the Committee unanimously came to the view that the Internship and Continued Professional Education (CPE) sections did not fit well with the process of accrediting medical schools. CPE is the responsibility of registering bodies and the individual professional bodies and are not a part of an undergraduate medical programme. The Committee took into consideration the fact that Internship is not part of the undergraduate medical education programme nor is it the responsibility of the schools and hence should not be a part of the assessment process. The question of quantitative standards has been controversial for CAAM-HP. Internationally, the trend seems to be that less is more in this area; it was the Committee’s view that it is more important to capture a small number of key, robust and verifiable data than to ask for vast amounts of data which are difficult to collect, verify and impossible to manage effectively. The Committee felt that external examinations and residency rates could not be made compulsory standards and that students who choose not to sit the USMLE and apply for residency must not be put at a disadvantage. However, as many of the schools in the region are geared to successful USMLE and residency placements it was accepted that data should be collected and evaluated for individual schools and be included in the database. If a school has these as important goals for their programme they should form part of the accreditation judgement on that school. For schools where these are not major programme goals and where student rates of sitting these examinations are low these data would not be an important part of the assessment. External examinations are not necessary for graduates of the University of the West Indies (UWI) or the University of Guyana (UG) as the successful completion of the undergraduate course is followed by a period of internship which, when successfully completed, allows the doctor the privilege of Full Registration. There is no requirement for sitting further examinations. The vast majority of these graduates remain in the region and either enter postgraduate residency programmes or General Practice in the region. The differences between the regional and the “off shore” schools mean that this flexibility is required. Finally, since CAAM-HP does not insist that a proportion of students at all schools must obtain external examinations or residency placements the Committee determined that these cannot be requirements. Should there be questions concerning the methods of assessment or standards of the assessment programme then it may be necessary to seek a benchmark examination and then require the school to mandate their students to sit this benchmark examination before graduating. The draft revised standards were presented to CAAM-HP’s Annual General Meeting held in Jamaica, July 27-29, 2016. With respect to quantitative standards the meeting agreed with the Committee’s recommendation to use only objective quantitative data such as examination pass rates, attrition, and placement in postgraduate training. The meeting also discussed the issue of including measures for student performance outcomes in the draft revised standards. The NCFMEA had suggested that CAAM-HP should base its accreditation decisions, in part, on the effective use of data in evaluating the performance of students after graduation from the medical school. CAAM-HP should, therefore, establish student performance outcome standards such as acceptable numbers of graduates from the school passing a licensing examination and an acceptable percentage of all students graduated during the preceding year that obtained placement in an accredited US postgraduate medical training programme to determine whether to grant accreditation. While acknowledging that such data would be useful, the meeting expressed concern about including performance outcome data in the standards and noted that this was not a requirement of the LCME. The meeting agreed that this matter should be considered carefully at the 2017 Annual General Meeting. While there was general agreement in principle with the draft revised accreditation standards members were given two weeks to comment further following which any suggestions/amendments would be forwarded to the Committee for consideration. The majority of the members responded indicating agreement with the proposed new standards, strongly supported the removal of the Internship and Continued Professional Education standards as they are not a required part of the undergraduate medical education programme. Members were also pleased that they would not have to adopt additional quantitative standards as these would have been not only difficult to provide but also somewhat useless without internationally accepted benchmarks. The main purpose of the March 2017 Capacity Building Workshop was to introduce the draft revised accreditation standards for medicine, their interpretation, application and assessment. This involved Group deliberations using case studies which helped participants to identify the relevant accreditation standards. The programme and presentations can be accessed on the CAAM-HP website at: http://www.caam-hp.org/workshop.html. The two areas of significant concern during the extended discussion of the new standards were: the examination results data particularly from the schools whose graduates do not sit external examinations and the dropping of the Internship standards. However, a number of schools wanted to be able to supply some data on the participation of their graduates in Internship and residency programmes. Consequently, changes were made to Medical Education Database-Section III-Educational Programme Part A and Standard ED-46. Where revisions have taken place, these are highlighted in bold and underlined (Appendices 11-16). These were approved at the CAAM-HP Annual General Meeting in July 2017 and came into effect on August 1, 2017. Schools have been advised.


Analyst Remarks to Narrative

The country has included documentation of the Caribbean Accreditation Authority for Education in Medicine and Other Health Professions (CAAM-HP) standards for accreditation (exhibit 12). These standards include how CAAM-HP conducts reviews of its medical school's administration, faculty, curriculum, admissions procedures, student services, methods for evaluating student achievement, and facilities. The standards were reviewed and revised if necessary to ensure that they still are relevant and reflect best practice. In response to previous NCFMEA requests that CAAM-HP reconsider methods for evaluating student achievement. CAAM-HP has determined that capturing a small number of key, robust and verifiable data was a better solution than to ask for vast amounts of data which are difficult to collect, verify and impossible to manage effectively. Additionally, CAAM-HP established student performance outcome standards, such as licensing exam rates and student placement percentages as a data point for consideration in whether to grant accreditation to a medical school.


Processes and procedures


Country Narrative

DOCUMENTS UPDATED CAAM-HP has updated some of its documents to simplify processes and procedures and make the documents more user friendly. These documents are: • Procedures of the CAAM-HP, Appendix 16 • Summary of Accreditation Procedures (formerly Accreditation Procedures of the CAAM-HP), Appendix 17 • Accreditation Guidelines for New and Developing Schools, Appendix 18 • Students’ Role in the CAAM-HP, Appendix 19 • Guidelines for Accreditation Survey Visits, Appendix 20 • Guide for Writing a Report on a Visit of a CAAM-HP Survey Team, Appendix 21 • Guide to the Institutional Self-Study (ISS) for Programmes of Education in Medicine, Appendix 22 • Instructions for Completing the CAAM-HP Medical Education Database and ISS Summary, Appendix 23 • Annual Medical School Questionnaire, Appendix 24 These can also be found on the website at https://www.caam-hp.org/documents. In addition, changes were made to CAAM-HP’s Appeals Policy and States of Accreditation are outlined as follows: APPEALS POLICY REVISED The revised Policy and Procedures for Appeal were presented to the meeting (CAAM-HP 2017 AGM) for approval. Attention was given to the following key elements: • the appeal would be decided by an independent 3-member panel; • the decision of the panel would be binding on CAAM-HP and would be implemented immediately; • the inclusion of specific grounds for appeal; • the administrative dismissal of defective appeals; • the facilitation of electronic hearings; • the costs of the appeal would be borne by the school; • the school would be required to notify the relevant government authorities in its jurisdiction of any change in its accreditation status. The meeting emphasised that the school must notify all students, including potential students of an adverse accreditation decision and update its accreditation status on all its public documents and website. The meeting considered whether the composition of the panel should be specified, but agreed that this could be limiting and noted that the school already had the right to object to panel members. In the discussion with respect to the specific grounds for appeal, the meeting questioned the relevance of including the citing of factually incorrect information in making a determination, given that the school had an opportunity to review the report and correct errors of fact. The meeting agreed to remove citing of factually incorrect information as one of the grounds for appeal. The Authority approved the revised Policy and Procedures for Appeal subject to the change noted above. STATES OF ACCREDITATION REVISED The revised States of Accreditation were presented to the CAAM-HP 2017 AGM for approval. The review was designed to: clarify terminology; address inconsistencies; reduce the likelihood of legal challenge; and make the document more user-friendly. The following major changes were highlighted: • extending Probation for a maximum of three (3) years; • granting Provisional Accreditation for three (3) years; • increasing the period before re-application to a minimum of three (3) years; • changing terminology from “Candidacy Withdrawn” to “Candidacy Terminated”; • including “failure to submit annual progress reports” as one of the reasons for Candidacy Terminated; • making all adverse accreditation decisions subject to appeal. The Authority approved the revised States of Accreditation. The Revised States of Accreditation and the Revised Appeals Policy can be found in the Procedures of the CAAM-HP (Appendix 16) at Appendix A and Appendix H, respectively.


Analyst Remarks to Narrative

The country has included documentation of the Caribbean Accreditation Authority for Education in Medicine and Other Health Professions (CAAM-HP) processes for accreditation. These documents include how CAAM-HP conducts on-site visits (exhibit 22), selects and trains individuals who conduct site evaluations, as well as those who make accreditation decisions (exhibit 17), and periodically reevaluates and monitors medical schools (exhibits 25 and 26). In addition, CAAM-HP has included documentation of the agency's own procedures that provide for reviewing substantive changes reported by medical schools, ensuring the accrediting/approval body has effective controls against conflicts of interest and the inconsistent application of accreditation standards, and ensuring that accreditation decisions are based, in part, on the evaluation of student performance after graduation from medical school (exhibit 19). The data that is collected on an annual basis from the medical school is reviewed to ensure ongoing compliance with the agency's standards.


Schedule of upcoming accredidation activities


Country Narrative

TSOM and SJSM are both due for full accreditation surveys in 2019. The site visit to TSOM and clinical affiliates will take place the week of April 7, 2019. A date has not yet been set for the visit to SJSM.


Analyst Remarks to Narrative

CAAM-HP has indicated that two of three medical schools will have full reviews conducted in 2019. It is Department's staff understanding that based on previous activities the full survey review would encompass visits to the clinical sites as well.