Accreditation and State Liason Image Skip navigation US Department of Education, Promoting educational excellence for all Americans.
|  OPE Home Page | ASL Home Page | NACIQI | User Guide
Back

U.S. Department of Education

 

Final Staff Analysis of the Report Submitted by Dominica

 
Prepared March 2012
 
Background
 
In October 1997, the National Committee on Foreign Medical Education Accreditation (NCFMEA or the Committee) first determined that the accreditation standards used by the Medical Board of Dominica (the Board or Dominica) to evaluate medical schools on Dominica were comparable to those used in the United States. In 2001 and again in 2007, the NCFMEA reaffirmed its prior determination that the standards and processes used by the Board for the evaluation of medical schools remained comparable. The NCFMEA also requested that the Board submit periodic reports describing its continuing accreditation activities.

The NCFMEA met in March 2009 to review the report submitted by the Board regarding its accrediting activities. The NCFMEA formally accepted the Board's report and requested that it submit a report for review at the spring 2010 NCFMEA meeting updating the Committee on its accrediting activities and addressing, specifically, the relationship between the Board and the Bahamas and its activities related to All Saints University School of Medicine and any additional activities related to Ross University School of Medicine. The NCFMEA did not meet in spring 2010 due to the pending appointment of new NCFMEA committee members and the designation of a committee chair.

At the fall 2011 meeting, the NCFMEA reviewed and formally accepted the Board's report, and requested that the Board submit a report for review at the spring 2012 NCFMEA meeting updating the Committee on the Board's action regarding the public accreditation information disclosed by All Saints University School of Medicine, and the Board's collection and evaluation of USMLE data for Ross University School of Medicine.
 
Summary of Findings
 
Based on its review of the information submitted by the country, Department staff concludes that Dominica provided information in response to the NCFMEA's request for a report of the issues requested at the fall 2011 meeting. This report specifically addressed the Board's action regarding the public accreditation information disclosed by All Saints University School of Medicine, and the Board's collection and evaluation of USMLE data for Ross University School of Medicine (RUSM).

With regard to All Saints University School of Medicine, Department staff
noted that the Board took action to request the correction of incorrect or
misleading information released by All Saints University School of Medicine on
its website. Department staff confirmed that the incorrect or misleading information has been removed from the website, and no additional information is required.

Regarding the USMLE data for RUSM, Department staff reviewed the information and documentation provided by the Board to include the ECFMG-prepared and RUSM-prepared USMLE pass rate data. Although the reported data is not identical, Department staff noted that difference between the RUSM-prepared USMLE pass rate data and the ECFMG-reported data is not significant, and no additional information is required.

With regard to the Board's review of the data in the assessment of RUSM, the Board indicated that it will use the independently-audited learning outcomes report in conjunction with the RUSM- and ECFMG-prepared USMLE pass rate data in the assessment of RUSM and its recertification in December 2012. In addition, Department staff noted that the RUSM- and ECFMG-prepared USMLE pass rate data include the performance comparison of students who attend RUSM with that of examinees from all medical schools outside the United States and Canada. However, the Board did not provide any information or documentation that it has used the the data obtained in the assessment of RUSM. Therefore, Department staff suggests that the Board be asked to provide additional information and documentation concerning the evaluation of RUSM using the USMLE pass rate data and learning outcomes report provided.
 
Staff Analysis
 
 
Outstanding Issues
Accreditation activities related to All Saints University School of Medicine, with specific regards to the correspondence and/or actions taken concerning the accuracy of the information published on the school's website.
 
Country Narrative
As previously reported, All Saints University School of Medicine (“All Saints”) has not pursued Dominica Medical Board (“Board”) accreditation, and the Board has not recognized or approved All Saints. All Saints remains chartered by the Government of the Commonwealth of Dominica.

By letter dated October 14, 2011, the Board advised All Saints to remove promptly from its website an inaccurate and misleading statement regarding All Saint’s accreditation status. See Letter from D. Shillingford to J. Yusuf (Oct. 14, 2011) (Exhibit A). Specifically, All Saints had posted the following statement on its website: “The standard of the University MD degree program is periodically assessed to be in conformity with the accreditation standards of the Medical Board of Dominica.” As of January 20, 2012, All Saints had removed that statement from its website.
 
Analyst Remarks to Narrative
The Board provided information and documentation to indicate that it requested that All Saints University School of Medicine correct the incorrect or misleading information regarding its accreditation on its website. Department staff confirmed that the incorrect or misleading information has been removed from the website. Therefore, no additional information is needed concerning the relationship between the Medical Board of Dominica and All Saints University School of Medicine.
 
USMLE data and collection practices for Ross University School of Medicine, to specifically include the following data: • The number of students who entered Ross University School of Medicine during the same academic year (the two most recent complete cohorts); • The number of students who took each Step of the USMLE from each cohort; • The number of students who passed each Step of the USMLE on the first attempt from each cohort; and • The number of students who re-took and passed each Step of the USMLE from each cohort. Provide the data for each cohort from internally prepared data and ECFMG-reported data.
 
Country Narrative
Educational Commission for Foreign Medical Graduates (“ECFMG”)-prepared data:

Ross University School of Medicine (“RUSM”) has supplied to the Board two reports that it obtained from ECFMG. The reports are at Exhibits B and C. The Board understands that 2010 is the most recent calendar year for which ECFMG is able to supply such reports. RUSM has informed the Board that it expects to receive by June 2012 an ECFMG report for the period January 1, 2011 through December 31, 2011.

RUSM-prepared data:

In connection with this report to the NCFMEA, the Board asked RUSM to supply (1) information regarding RUSM’s “USMLE data and collection practices” and (2) the NCFMEA-requested data, as prepared by RUSM, namely (a) the number of students who entered RUSM during the same academic year (the two most recent complete cohorts); (b) the number of students who took each Step of the USMLE from each cohort; (c) the number of students who passed each Step of the USMLE on the first attempt from each cohort; and (d) the number of students who re-took and passed each Step of the USMLE from each cohort. RUSM’s response is at Exhibit D.

Board-requested data:

The Board has asked RUSM to supply extensive audited data pertinent to learning outcomes and to create a statistically valid predictive model for specified dependent variables (“Outcomes Report”). The Board’s request is at Exhibit E. RUSM has engaged Ernst & Young to perform the audit and to develop the predictive model. The Board expects to receive RUSM’s Outcomes Report on February 6, 2012. The Board will supply the Outcomes Report to the NCFMEA.
 
Analyst Remarks to Narrative
The Board provided information and documentation of the USMLE data and collection practices for Ross University School of Medicine (RUSM). Specifically, the Board provided ECFMG-prepared USMLE pass rate data for calendar years 2009 and 2010, as well as the RUSM-prepared data for two complete cohorts (academic years 2004-2005 and 2005-2006).

Even though the Board provided the information and documentation as requested, the information and documentation are not in a format that is comparable. Therefore, Department staff requests the re-submission of the RUSM-prepared data in a format that would allow it to be compared with the ECFMG-prepared data. The RUSM-prepared data needs to reflect the same reporting period. More specifically, the data, instead of being broken down by starting cohort, needs to be broken down by attempts, passes, and re-takes at each Step of the USMLE for the calendar years 2009 and 2010.

Department staff noted that the ECFMG-prepared data includes the performance comparison of students who attend RUSM with that of examinees from all medical schools outside the United States and Canada. This comparison indicates that RUSM students performed better than those from all medical schools outside the United States and Canada on the Step 1 and Step 2 Clinical Skills, but not for Step 2 Clinical Knowledge.

In addition to the data request of the NCFMEA, the Board also included information concerning an independently-audited learning outcomes report. The Board indicated that the learning outcomes report is not yet available, but would be provided in response to this report.
 
Country Response
As indicated in Dominica’s original submission, the Dominica Medical Board (Board) asked Ross University School of Medicine (RUSM) to supply extensive audited data pertinent to learning outcomes and to create a statistically valid predictive model for specified dependent variables (“Outcomes Report”) (See Exhibit E). The data that RUSM provided in connection with the Outcomes Report is at Exhibits 1 and 2. The predictive models are at Exhibit 3. Dominica is supplying the data in the form in which RUSM supplied the data to the Board, except that Dominica has removed student names in the interest of privacy. The correspondence and narrative that accompanied the Outcomes Report are at Exhibit 4.

In addition, the Board asked RUSM to supply the RUSM-prepared USMLE data in a form that allows comparison to the ECFMG-prepared USMLE data. RUSM’s response is at Exhibit 5.
 
Analyst Remarks to Response
The Board provided updated information and documentation of the USMLE data and collection practices for Ross University School of Medicine (RUSM). Specifically, the Board provided the updated RUSM-prepared data for calendar years 2009 and 2010, as well as the independently-audited learning outcomes report described in the previous submission.

Since the format of the RUSM-prepared USMLE pass rate data was revised to reflect the same reporting period as the ECFMG-reported data, Department staff is able to compare the information. Although the reported data is not identical, Department staff notes that difference between the RUSM-prepared USMLE pass rate data and the ECFMG-reported data is not significant.

Department staff noted that both sets of data include the performance comparison of students who attended RUSM with that of examinees from all medical schools outside the United States and Canada. Those comparisons indicate that RUSM students performed better than those from all medical schools outside the United States and Canada on the Step 1 and Step 2 Clinical Skills, but not for Step 2 Clinical Knowledge.

The Board also included the independently-audited learning outcomes report it required RUSM produced as part of its recertification process. The Board provided an overview of the learning outcomes report and the raw data, the data analysis, and the predictive model to identify attributes of students who successfully complete the program.
 
Staff Conclusion: Comprehensive response provided
 
Based on the data collected, provide information on how the data is analyzed and used in assessing the school.
 
Country Narrative
The Board extended until December 16, 2012 RUSM’s current certification, pending receipt of certain information. See Letter from D. Shillingford to J. Flaherty (Dec. 15, 2011) (Exhibit F). The Board plans to analyze RUSM’s Outcomes Report, referenced in response to Question 2, in connection with its consideration as to whether and under what conditions to extend RUSM’s certification beyond December 16, 2012.
 
Analyst Remarks to Narrative
The Board indicated that it will use an independently-audited learning outcomes report, which is not yet available, in the assessment of RUSM and its certification process with the Board.

Previously, Department staff noted that the ECFMG-prepared USMLE pass rate data includes a performance comparison which indicates that RUSM students performed better than those from all medical schools outside the United States and Canada on the Step 1 and Step 2 Clinical Skills, but not for Step 2 Clinical Knowledge. The Board did not provide any information or documentation concerning the evaluation of the ECFMG-prepared data in the assessment of RUSM.

Therefore, Department staff requests the submission of the learning outcomes report, as well as information and documentation specifically on how the ECFMG-reported data is used in assessing RUSM.
 
Country Response
The Board plans to analyze the RUSM- and ECFMG-prepared data, as well as the Outcomes Report, in connection with its consideration as to whether and under what conditions to extend RUSM’s certification beyond December 16, 2012. The Board will consider such data in connection with its assessment of RUSM’s compliance with accreditation standards related to, for example:

• geographically separate sites (e.g., “If some components of the educational programme are conducted at sites that are geographically separated from the main campus of the medical school, the school must have appropriate mechanisms in place to ensure that (a) the educational experiences at all geographically separated sites are comparable in quality to those at the main campus and to each other, and (b) there is consistency in student evaluations at all sites.”);

• clinical education (e.g., “Medical schools must ensure that their students possess the knowledge and clinical abilities to enter any field of graduate medical education.”);

• evaluation of student achievement (e.g., “The administration and the faculty should have knowledge of methods for measurement of student performance in accordance with stated educational objectives and national norms. . . . The varied measures utilized must determine whether or not students have attained the school’s standards of performance, as measured by licensing examinations, acceptance into residency programmes, and other valid assessments.”); and

• program assessment (e.g., “There must be an ongoing review and if necessary revision of the curriculum and the delivery of the curriculum. The medical school must regularly evaluate the effectiveness of its medical programme by documenting achievement of its students and graduates in verifiable ways that show the extent to which institutional and programme purposes are met. The school should use a variety of measures to evaluate programme quality, such as data on student performance, academic progress and graduation, acceptance into residency programmes, and postgraduate performance; the licensure of graduates, particularly in relation to any national norms; and any other measures that are appropriate and valid in light of the school's mission and objectives. A medical school must consider student evaluations of their courses and teachers in assessing programme quality.").
 
Analyst Remarks to Response
The Board provided (in the previous section) the independently-audited learning outcomes report that it plans to use in conjunction with the RUSM- and ECFMG-prepared USMLE pass rate data in the assessment of RUSM and its recertification in December 2012. Although the Board described how it will use the data in the assessment of RUSM and RUSM's compliance with the Board's standards, the Board has not demonstrated that it has used the data in the assessment of RUSM.

Also, in the previous section, Department staff noted that the USMLE pass rate data, prepared by ECFMG and RUSM respectively, include the performance comparison which indicate that RUSM students performed better than those from all medical schools outside the United States and Canada on the Step 1 and Step 2 Clinical Skills, but not for Step 2 Clinical Knowledge. The Board still did not provide any information or documentation concerning the evaluation of this specific data.

Therefore, Department staff suggests that the Board be asked to provide additional information and documentation concerning the evaluation of RUSM using the USMLE pass rate data and learning outcomes report provided.
 
Staff Conclusion: Additional Information requested