Accreditation and State Liason (ASL) E-Recognition Web Site US Department of Education, Promoting educational excellence for all Americans.
Skip to main content | Home | OPE Home | ASL Home | NACIQI | NCFMEA | User Guide

Back

U.S. Department of Education

 

Staff Analysis of the Redetermination Submitted by Australia and New Zealand

 
Prepared April 2017
 
Background
 
At its Spring 1995 meeting, the National Committee on Foreign Medical Education and Accreditation (NCFMEA) initially determined that the standards of the Australian Medical Council (AMC) used to evaluate medical schools in Australia and New Zealand were comparable to those used to evaluate medical schools in the United States. The AMC's standards were again found comparable in 2001, 2007, and 2009. Due to delays in the review schedule, the country was most recently reviewed for a redetermination of comparability at the Spring 2016 NCFMEA meeting. At that time, the Committee determined that additional information was required before it could make a decision regarding the country's continued comparability and the country was requested to provide additional information and supporting documentation related to specific criteria at the Spring 2017 NCFMEA meeting. The country's submission in response to the NCFMEA's Spring 2016 request is the subject of the current review.
 
Summary of Findings
 
Additional information is requested for the following questions. These issues are summarized below and discussed in detail under the Staff Analysis section.

-- The Committee may wish to request that the country provide additional information regarding the adoption of a conflict of interest standard after it resumes its standards review process later in 2017.

-- The Committee may wish to request additional information as to whether the country would allow completely new sites to be established at institutions that the country had not already reviewed for accreditation, as well as the country's timeframe for reviewing such candidates for initial accreditation.

-- The Committee may wish to request that the country provide additional information regarding the comprehensive review of its one overseas institution after the school is reviewed in July 2017.

-- The Committee may wish to request additional information regarding the country's revision of its standard related to this area after the standards review process resumes in May 2017.

 
Staff Analysis
 
 
Outstanding Issues
Mission & Objective Q3: Please provide clarification on how it is determined that the objectives of the educational program will be formally adopted by the faculty, as a whole, and through its recognized governance process if the process is not formally in place.
 
Country Narrative
The AMC accreditation standards concerning governance of the education provider (medical school), management of the medical program, stakeholder consultation and evaluation collectively lead the AMC to understand the extent of faculty involvement in the development and approval of the objectives of the medical program and the program itself, and to determine if this is sufficient involvement. Under the law that enables the AMC to operate as the accreditation authority for medical programs, it is able to set conditions on the accreditation of any program and provider that do not meet the standards.

The AMC understands that this question relates to the Guideline “(a) The educational mission of the medical school must serve the general public interest, and its educational objectives must support the mission. The medical school’s educational program must be appropriate in light of the mission and objectives of the school.”

The AMC has no one, single standard that addresses this NCFMEA guideline. The AMC believes that its standards and processes provide an effective alternate to ensure that faculty involvement is sufficient to meet this guideline.

See APPENDIX 1: Standards for Assessment and Accreditation of Primary Medical Programs by the AMC 2012.

The relevant accreditation standards are:

1.3.1 The medical education provider has a committee or similar entity with the responsibility, authority and capacity to plan, implement and review the curriculum to achieve the objectives of the medical program.

1.1.2 The governance structures set out, for each committee, the composition, terms of reference, powers and reporting relationships, and allow relevant groups to be represented in decision-making.

1.1.3 The medical education provider consults relevant groups on key issues relating to its purpose, the curriculum, graduate outcomes and governance.

The AMC glossary of terms used in the accreditation standards (see page iv and v of the Standards) defines the term “relevant groups” as internal and external stakeholders, and the definition of internal stakeholders includes faculty - program directors, academic staff, supervisors and committees. The accreditation standards listed above make it clear that faculty staff are one of the relevant groups to be represented in decision making of the education provider including in setting the objectives of the medical program, and that they must be consulted on key issues such as the curriculum and the objectives of the medical program.

Together, by applying these standards and these defined terms in the assessment of a medical program, the AMC assesses whether the faculty is engaged in and formally adopt the objectives of the medical program – through their roles as members of advisory and decision making committees.

The AMC has provided information that demonstrates that it does consider the effectiveness of the education provider’s governance structures, and whether critical matters such as the objectives of the medical program, and the design of the curriculum to meet these objectives are formally considered through these processes. It does also consider the faculty input to these governance processes.

The AMC’s information is provided as:

APPENDIX 2: an extract from the legislation that covers the AMC accreditation process. This demonstrates that the AMC has the power to accredit programs and to set conditions on programs where the accreditation standards are not met or only substantially met (rather than fully met). Under the legislation that covers the AMC’s accreditation, the AMC is able to require that conditions are met by the education provider in a reasonable timeframe, and if not to revoke accreditation or place new conditions on the accreditation. This information was provided in 2016. These powers have not changed.

APPENDIX 3: An extract from the list of accreditation conditions the AMC has placed on medical schools accreditations since 2010. This extract shows the conditions we have set on medical schools relating to standards 1.3.1, 1.1.2 and 1.1.3. The conditions placed on medical schools that we believe are relevant to this NCFMEA guideline are highlighted.

APPENDIX 4: A document that collates extracts from three accreditation reports to show the findings made by individual accreditation teams that relate to the conditions placed on the accreditation (that is the conditions highlighted in APPENDIX 3). The relevant text is underlined.

 
Analyst Remarks to Narrative
In response to the question of how the objectives of the educational program are formally adopted by the faculty as a whole or through its governance process, the country has responded that its standards require the institution to have a committee that is tasked with the planning, implementation, and review of the curriculum in order to achieve the objectives of the medical program. The country's standards further require the institution's governance structures to specify the composition, powers, reporting channels, and the relevant groups that are to be represented on such committees. Finally, the country's standards require that the institution consult relevant groups on issues related to the curriculum. The country has defined "relevant groups" to include internal stakeholders, including faculty, program directors, academic staff, and supervisors. Therefore, it appears that rather than requiring that the curriculum be reviewed and approved by the faculty as a whole, the country has in place a governance process whereby the faculty has input into the curriculum development process by having faculty participate on committees that are tasked with curriculum development.

As documentation, the country provided copies of its standards applicable to this question (Ex. 1), as well as evidence of findings and accreditation reports demonstrating the enforcement of the country's standards (Exs. 3 and 4).

ED staff accepts the country's response, and no additional information is requested.
 
Faculty Q1 Please provide more details regarding the process for reviewing the size and qualifications of faculty members.
 
Country Narrative
NCFMEA has asked for more information to confirm how the AMC ensures consistency in the review of the size and qualifications of faculty members. In 2016 the AMC had provided one example from an accreditation report to show how it had completed this assessment.

NCFMEA had asked for documentation that shows how information is consistently collected about the faculty qualifications and how the AMC verifies the information, whether through assessments or onsite inspection team reviews of the institution.

The AMC has taken the question to refer to the size of the faculty group as well as the qualifications of the faculty.

See APPENDIX 1: Standards for Assessment and Accreditation of Primary Medical Programs by the AMC 2012.

The relevant accreditation standards is:
1.8.1 The medical education provider has the staff necessary to deliver the medical program.
The AMC glossary of terms used in the accreditation standards (see page iv and v of the Standards) defines the term staff as academic, professional and administrative staff.

The AMC provides a standard accreditation submission guide for all medical schools undergoing accreditation. This ensures all medical schools respond to the same questions and provide similar data for review by the AMC. The AMC also provides a guide for all medical schools to use when they prepare their regular (annual or two-yearly) progress reports to the AMC.

The AMC also requires the medical school to provide a written critical analysis of the adequacy of their staffing in their submission and reports (using data such as the staff benchmarking data produced by the Medical Deans Australia and New Zealand).

APPENDIX 5: Provides the AMC’s accreditation submission guide for medical schools preparing for accreditation. Note in response to the NCMFEA’s question in 2016, the AMC now includes in this guide an explicit request to medical schools to provide a staff list, with qualifications shown, so that the AMC is better able to judge the adequacy of the school’s response to this accreditation standard. See page 10 of the submission guide.

APPENDIX 6: Provides the AMC’s guide to the information education providers must submit in their annual or biennial monitoring (progress) reports. Note in response to the NCMFEA’s question in 2016, the AMC now includes in this guide an explicit request to medical schools to provide a staff list, and to provide more detailed information than has previously been requested on vacancies and recruitment plans, so that the AMC is better able to judge the adequacy of the school’s response to this accreditation standard. See page 5 of the submission guide.

APPENDIX 7: This is the Procedures for Assessment and Accreditation of Medical Schools by the Australian Medical Council. Section 4 of the Procedures describes the AMC’s processes for monitoring of accredited programs and providers which are referred to in the previous paragraph.

How the AMC considers the data and information provided by the medical school: The data and the medical school’s analysis of the data is considered by the AMC accreditation team in its on-site inspection. If the AMC is considering the medical program through a progress report, the medical school’s information is considered by the Medical School Accreditation Committee. The team or Committee consider the stage of development of the medical school and program, the curriculum model, and any agreements to share the curriculum model with another medical school, to arrive at a decision on the adequacy of the staff to deliver the medical program.

Medical schools provide staff lists with their submissions for accreditation by the AMC. AMC staff can verify the accuracy of the list and the current qualifications of staff by reference to University publications and staff handbooks, as well as by referring to the staff lists of neighbouring medical schools. This checking is undertaken routinely when two or more schools are located nearby, and academic staff may be working across more than one school. It also occurs routinely when the AMC has previously identified concerns about the academic staff profile and when a new school is established.

APPENDIX 8: Shows a response by a new medical school to the AMC request for information concerning the size and qualifications of the academic staff.

APPENDIX 9: Is the accreditation report that was produced as a result of the assessment of this new medical school. Section 1.1.8 provides the accreditation team’s findings concerning the size and appropriateness of the academic staff profile, and that the team considered the stage of development of the medical school and program, the curriculum model, and the agreement to share the curriculum model with another medical school, to arrive at a decision on the adequacy of the staff to deliver the medical program. The report executive summary shows that in response to this review, the AMC set conditions on the accreditation to address perceived gaps in the staff profile.


 
Analyst Remarks to Narrative
At the time of its last submission, the country provided one example of an accreditation report to demonstrate institutional compliance with its standards. However, the country was requested to provide additional examples of documentation to demonstrate compliance with this question. In its response, the country provided additional documentation in the form of a self-study document (Ex. 8) and an on-site review report (Ex. 9) to demonstrate that institutions are required to address the country's standards related to this question and that this compliance is reviewed by on-site review teams.

ED staff accepts the country's additional documentation, and no additional information is requested.
 
Faculty Q2 Please provide updated information of the progress that is being made to address having a standard for preventing conflict of interest.
 
Country Narrative
In 2016, the AMC advised that it was completing a minor review of its accreditation standards, and that it intended to introduce a new accreditation standard on conflict of interest . The proposed wording of the standard is based on a similar accreditation standard that has been introduced into the standards for the AMC’s other accreditation processes, which cover specialist medical training programs.

The proposed wording is “new standard 1.1.6 The education provider has developed and follows procedures for identifying, recording and managing conflicts of interest in its training and education functions, governance and decision-making.”

The AMC began a minor review of the Standards for Assessment and Accreditation of Primary Medical Programs by the Australian Medical Council in the second half of 2016. The first meeting of the standards review working group took place at the end of July 2016.

The draft consultation paper concerning the proposed changes to the standards is provided at APPENDIX 10. The description of the proposal for a new standard about conflict of interests is provided on the top of page 4 of this document.

The AMC has delayed formally consulting on these changes, because a review of the accreditation system for the regulated health professions in underway. This review is not enquiring specifically about the accreditation of medical programs, but the system for accreditation in general. Nevertheless, the AMC decided to delay new work on the accreditation standards until the after the accreditation systems review launches its discussion paper in February 2017. Evidence of this review is provided in APPENDIX 11. In addition, the AMC appointed a new chair of its Medical School Accreditation Committee in November 2016, and the new chair will now take responsibility for leading the review of the standards.

WE expect the review of the standards will be complete during 2017.
 
Analyst Remarks to Narrative
The country was previously requested to develop a standard to address conflicts of interest. In its response, the country notes that it is currently in the process of reviewing and revising its standards, which it does on a five-year cycle. The country provided a copy of a paper it has developed to gain feedback on proposed changes to its standards. The draft indicates that conflicts of interest related to the management and delivery of the medical program will be included under a revised Standard 1(Ex. 10, p. 4). The country noted at the time of its submission that its proposed revisions were scheduled to be reviewed in February 2017, and that review of the revisions was expected to be completed by September 2017 (Ex. 11).

Additional information is requested from the country regarding the progress that has been made to date with respect to the inclusion of conflicts of interest in the country's revised standards.




 
Country Response
As noted in the AMC’s submission, the AMC intends to introduce a new standard that will require medical schools to have procedures for identifying, recording and managing conflicts of interest in its training and education functions, governance and decision-making. This change is scheduled to be introduced when the AMC completes its scheduled minor review of the standards.

The AMC has slowed the progress of its review of the medical school accreditation standards because of a national accreditation systems review. When the AMC begins consultation, it wants to ensure that relevant stakeholders focus on the review of its standards, and thought there would be less confusion if we postponed consultation on the review of our standards until the consultation period on the national accreditation systems review is finished. The period of consultation on the accreditation systems review ends on 1 May 2017. While the AMC waits for this period of consultation to end, it has divided the changes it wishes to make to the accreditation standards for medical programs into high priority (to be introduced in 2017) and longer term changes. The introduction of a standard requiring medical schools to have procedures for identifying, recording and managing conflicts of interest in its training and education functions, governance and decision-making is considered a high priority change. The AMC still plans to complete consultations on this change and introduce the change this year. The AMC will provide evidence of the completion of the review of standards when this occurs.
 
Analyst Remarks to Response
In its response to the drafts staff analysis, the country states that the review of its standards is currently on hold because the country is conducting a national accreditation systems review, which has been given priority. The systems review will end on May 1, 2017, at which time the standard review will resume. The country reports that it has assessed the changes that need to be made to its accreditation standards. Some standard have been designated as high priority and others as long term changes. The country notes that the conflict of interest standard associated with this section has been designated as having a high priority and should therefore be introduced for consideration later in 2017.

Additional information is requested. The Committee may wish to request that the country provide additional information regarding the adoption of a conflict of interest standard after it resumes its standards review process later in 2017.
 
Staff Conclusion: Additional Information requested
 
Clinical Teaching Facility Q1 Please provide the official copies of the documentation to verify the information in the affiliation agreement.
 
Country Narrative
In 2016, as part of its response to this question the AMC provided a screenshot as documentation from one of its medical schools displaying an affiliation agreement. NCFMEA indicated that the actual affiliation agreement must be uploaded to be included in the official record of documentation submitted with the AMC’s comparability petition.

At APPENDIX 12, the AMC has provided an affiliation agreement as part of its comparability submission.

 
Analyst Remarks to Narrative
At the time of its prior review, the country had provided a screen shot of an affiliation agreement from one of its medical schools as documentation of its compliance with this question. Because the screen shot could not be uploaded into the system to serve as documentation, the country was requested to instead provide a copy of the affiliation agreement so that it could be included in the official record. In response to the Committee's request, the country has provided a sample affiliation agreement to serve as more complete documentation of its compliance in this area (Ex. 12).

ED staff accepts the country's response and additional documentation, and no additional information is requested.
 
Onsite Review Q3 Please provide clarification regarding how soon new sites are visited once it id determined that they are candidates for accreditation.
 
Country Narrative
The AMC would expect to visit a proposed new major teaching site within 12 months of the medical school providing notification of the development of the site.

The process is described in two sections of APPENDIX 7, the Procedures of Assessment and Accreditation of Medical Schools by the Australian Medical Council.

The medical school would provide notice to the AMC of the development. It can provide notice either in a specific letter/ notice of a proposed major change or in the regular progress reports that accredited providers must submit.

In Section 3.2 of the Procedures, the process for assessing a major change to a medical program, and the way in which the education provider gives the AMC notice of the change is described.

The definition of a major change includes “a substantial change in ….institutional setting”. As section 3.2 of the Procedures describes, the AMC expects to be told prospectively by medical schools of any such planned changes.

The medical school can also provide notice in one of the regular progress reports that it is required to submit to AMC. This process is explained in section 4 of the Procedures of Assessment and Accreditation of Medical Schools by the Australian Medical Council.

APPENDIX 6: This is the AMC’s guide to the information education providers must submit in their annual or biennial monitoring (progress) reports. Note that in response to NCFMEA’s questions in 2016, the AMC has strengthened the list of information it asks schools to provide about changes to teaching sites. The questions the AMC requires medical schools to answer are listed in section 8 of the guide.

The AMC Medical School Accreditation Committee considers the progress report and decides whether the program and the provider continue to meet the accreditation standard, or whether additional information is required before it makes a decision or whether this development represents a major change to the program. If it is a major change, the AMC decides when the onsite accreditation team visit will occur to review the new site.

It is not common for medical schools to change their major teaching sites. Most health care facilities in Australian cities and regional centres have an agreement with a medical school for teaching.

APPENDIX 13 is an extract from a progress report that shows a medical school providing information on a new clinical site which has become affiliated with the School.

APPENDIX 14 is a copy of the agreement between the School and the health service that is discussed in the extract from the progress report at APPENDIX 13.

APPENDIX 15 is a copy of the AMC’s next accreditation report on this medical school. This shows the school was visited within 12 months of progress report which advised the AMC of the new agreement with the clinical site (health service).

 
Analyst Remarks to Narrative
During its prior review, the country was asked to clarify how soon new sites are visited once it is determined that they are candidates for initial accreditation. In its response, the country states that candidates for accreditation are visited within 12 months after the country is notified of the new site. The country references Ex. 7 for a description of the review process for candidates for initial accreditation. Although Ex. 7 was not attached to this section, ED staff viewed the exhibit through the Table of Contents and was unable to find a policy specifying that candidates for initial accreditation will be reviewed within 12 months. Section 3.3.1 (Ex. 7, p. 15) indicates that candidates for reaccreditation will be notified 24 months prior to the expiration of their current grant of accreditation, and staff will again contact the institution 12 months in advance of the on-site visit. However, the guidelines do not appear to provide any timelines for the review of candidates for initial accreditation.

Additional information and documentation are requested to demonstrate that the country has a written policy specifying that institutions that are candidates for initial accreditation will be reviewed within 12 months.
 
Country Response
Additional information and documentation was requested to demonstrate that the AMC has a written policy specifying that institutions that are candidates for initial accreditation will be reviewed within 12 months.

The AMC policy is that education providers (medical schools) provide 18 months’ notice of plans for making a major change, and that the AMC makes a decision about the change and its implications for the medical program’s accreditation status before the change is introduced. The AMC does not have a separate policy that explicitly states that new sites must be visited within twelve months of the institution/medical school providing notification of the development of the site but the policy on notification and assessment of new developments and major changes does lead to this occurring.
The relevant policy is provided in The Procedures for Assessment and Accreditation of Medical Programs by the Australian Medical Council (APPENDIX 7 to the 2017 redetermination submission).
The policy relating to major changes to medical programs addresses the introduction of new institutional settings (sites). The Procedures (APPENDIX 7) provide the definition of a major change to a program. “The AMC defines a major change in an accredited program or education provider as a change in the length or format of the program, including the introduction of new distinct streams; a significant change in educational outcomes; a substantial change in educational philosophy, emphasis or institutional setting; and/or a substantial change in student numbers relative to resources.”
Section 3.2.2 of the Procedures (APPENDIX 7) indicates “As many of the changes described …. (under the definition of major change) will need to be assessed by an AMC team before they are introduced, the AMC requests at least 18 months’ notice of the intended introduction of the change and longer for a proposed offshore program.”
The AMC asks for 18 months’ notice of a development that is likely to fit the category of major change so it can complete the following steps in the assessment process:
The AMC Medical School Accreditation Committee considers the notice about the change and decides if it needs more information about the change proposed, and when/how the assessment should occur (based on the stated Procedures).
The AMC tells the medical school how the assessment will be completed.
The AMC sets up an accreditation assessment team, if the assessment of the change does not align with the medical school’s accreditation assessment cycle and the AMC has decided that a team needs to assess the change. If the change is occurring in line with the medical school’s scheduled accreditation assessment visit, the change will be assessed by the team already established by the AMC to complete the accreditation assessment of the program.
The AMC assesses the change and makes a decision about it. The AMC aims to complete these assessments before the change is made. Section 3.2.4 of the Procedures (APPENDIX 7) indicates that “The AMC aims to complete the team’s assessment six months before the program begins, so that the education provider can demonstrate it has satisfied any conditions that must be met before commencement.”
If the AMC Medical School Accreditation Committee decides that the change can be implemented before it is assessed, the AMC completes the assessment in the first 12 months after introduction. This is not explicitly stated in the procedures, since this is rare and most changes are assessed before implementation. AMC records show two situations in the last five years where a new site was introduced and the AMC completed the assessment of the change within the next 12 months. These two situations are
1. The first situation is the Seventh Day Adventist Hospital becoming a new site for University of Sydney (which was described in the 2017 submission with the following documents providing evidence of assessment of the change within 12 months):
APPENDIX 13 as an extract from a progress report that shows a medical school providing information on a new clinical site which has become affiliated with the school.

APPENDIX 14 is a copy of the agreement between the School and the health service that is discussed in the extract from the progress report at APPENDIX 13.

APPENDIX 15 is a copy of the AMC’s next accreditation report on this medical school. This shows the school was visited within 12 months of progress report which advised the AMC of the new agreement with the clinical site (health service).

2. The second situation is the introduction of a new site for the University of Notre Dame Australia, School of Medicine Fremantle. As evidence of the visit occurring within 12 months the AMC has provided the letter it sent to the Dean of the School advising that the Medical School Accreditation Committee had considered the School’s information about the change and that the AMC would assess the change with a site visit (see new APPENDIX 34).

A copy of the site visit program is also provided as new APPENDIX 35.
 
Analyst Remarks to Response
In its response to the draft staff analysis, the country provided additional information regarding the review process for new sites, as well as the associated timelines. The country states that the addition of a new site is considered to be a" major change," which would appear to be similar to ED's use of the term "substantive change." As is typical with substantive changes, prior approval is required before the institution may move forward with the change, and the country therefore conducts a review prior to the implementation of the new site in most cases. The institution must request the substantive change at least 18 months prior to its planned implementation. The country considers the change request, determines if more information is needed, and forms an accreditation assessment team. If possible, the assessment will be aligned with the institution's regular review cycle. If that is not possible, the assessment team completes its review at least six months before the change is to be implemented. The country notes that in rare instances it allows the implementation of the change for a new site after reviewing the change request. In such cases, the country assesses the new site within 12 months after the new site is implemented. The country notes that it has had two such instances in the past five years.

As documentation, the country provided exhibits related to the two instances in which it conducted its review after the implementation of the new site (Exs. 1 and 2). In the first exhibit, a letter was sent to the institution's dean regarding an instance where a program was moved from one institution to another. The letter indicates that the change was not considered sufficiently substantive to require a review prior to its implementation. A review occurred after the program was moved in order to validate the successful transition of the program from the first institution to the second. In the second exhibit, the country provided an agenda from a one-day review of the other new site that it reviewed after the site had already been implemented.

The country has affirmed that it does not have a written policy requiring that new sites be evaluated prior to their implementation. However, it has also documented that most new sites are, in practice, reviewed prior to implementation and that only on rare occasions will sites be reviewed after a new site has been established. ED staff has a concern that the country's current policies and procedures apply to the addition of sites at previously established institutions, which may make such requests via a substantive change. However, the country's policies appear mute on the possibility that a new program might be established at an institution that does not already have the country's approval.

The Committee may wish to request additional information as to whether the country would allow completely new sites to be established at institutions that the country had not already reviewed for accreditation, as well as the country's timeframe for reviewing such candidates for initial accreditation.
 
Staff Conclusion: Additional Information requested
 
Onsite Review Q4 Please provide the official copies of the documentation to verify the information in the affiliation agreement.
 
Country Narrative
In 2016, as part of its response to this question the AMC provided a screenshot as documentation from one of its medical schools displaying an affiliation agreement. NCFMEA indicated that the actual affiliation agreement must be uploaded to be included in the official record of documentation submitted with the AMC’s comparability petition.

At APPENDIX 12 and APPENDIX 14 the AMC has provided the two affiliation agreements that were shown in the screen shot provided in 2016.
 
Analyst Remarks to Narrative
As noted under a previous section, at the time of its last review the country provided copies of screen shots of affiliation agreements, rather than uploading sample affiliation agreements. Because the screen shots could not serve as documentation, the country was requested to instead provide sample affiliation agreements for the official record. In response to the Committee's request, the country has provided two sample affiliation agreements as documentation (Exs. 12 and 14).

ED staff accepts the country's documentation, and no additional information is requested.
 
Qualifications of Evaluators, Decision-makers, Policy-makers Please provide additional details about the appeals panel members to ensure qualifications and backgrounds are appropriate.
 
Country Narrative
The NCFMEA Guidelines indicate that “The accreditation/approval process must use competent and knowledgeable individuals, who are qualified by experience and training in the basic or clinical sciences, for on-site evaluations of medical schools, policy-making, and decision-making.”

NCFMEA staff analysis indicated in 2016 that the AMC had provided documentation demonstrating the background and qualifications of those members currently serving roles within the AMC, but indicated that it was unclear to Department staff about the background and qualifications of those who are involved in the appeals process.

While the Procedures of Assessment and Accreditation of Medical Schools by the Australian Medical Council 2017 do describe the processes for a medical school to appeal against an accreditation decision by the AMC, the AMC has not had a formal appeal against an accreditation decision or process and therefore it only has a policy document that explains how it would conduct an appeal it this event occurred. It can not demonstrate how it has conducted an appeal.

APPENDIX 7: This is the Procedures for Assessment and Accreditation of Medical Schools by the Australian Medical Council. Section 3.3.11 concerns the formal process for review (appeal) of a decision to refuse accreditation. The Procedures state “the AMC will establish a review committee comprising members with appropriate qualifications and experience which will meet as required to consider any request for a review of a decision to refuse accreditation. The review committee will not include any person on the original assessment team.” The AMC has deliberately left this statement broad since in the absence of experience, it does not wish to define what constitutes appropriate qualifications and experience. The AMC expects that it would seek advice on appropriate membership from other accreditation authorities should this situation arise.


 
Analyst Remarks to Narrative
At the time of the country's previous appearance before the Committee, it was requested to provide additional information regarding the qualifications of its appeals panel members. In it's response, the country notes that it has not had any appeals and therefore has no sample materials to provide regarding the qualifications of people who have served on appeals panels. However, the country notes that its procedures require that appeals panel members will be appropriately qualified and further specify that no members of the appeals panel will have served on the original evaluation team whose review is being appealed (Ex. 7, p. 19).

ED staff accepts the country's narrative and established procedure and notes that no documentation is available specific to the qualifications of specific appeals panel members. No additional information is requested.
 
Substantive Change Please provide additional documentation including an institutional application for substantive change and AMC’s review of this information.
 
Country Narrative
NCFMEA had requested more information demonstrating the process used by the AMC to assess a major change to a medical program.

In the answer below, the AMC has provided a summary of relevant sections of APPENDIX 7 Procedures for Assessment and Accreditation of Medical Schools by the Australian Medical Council 2017. This document provides the AMC’s rules, including the process for assessing proposals for major changes to accredited medical programs. The AMC has also provided material drawn from an AMC assessment of plans for a major change to a medical program during 2015-2016 to demonstrate how the AMC completes this assessment.

In summary, the Procedures for Assessment and Accreditation of Medical Schools by the Australian Medical Council 2017 (section 3.3.2) require that the education provider must give the AMC adequate notice of its plans to introduce a change (18 months before the change is implemented). Depending on the timing, the provider can provide this advice through a regular progress report (all accredited providers must submit progress reports), or it can submit a specific notice of the change.

The Medical School Accreditation Committee considers the information and decides if the proposal is a major change. Section 3.2.2 of the Procedures includes the Australian Medical Council’s definition of a major change, which is: “a change in the length or format of the program, including the introduction of new distinct streams; a significant change in objectives; a substantial change in educational philosophy, emphasis or institutional setting; and/or a substantial change in student numbers relative to resources. Significant changes forced by a major reduction in resources leading to an inability to achieve the objectives of the existing program are also major changes. The gradual evolution of a medical program in response to initiatives and review would not be considered a major change.”

At APPENDIX 16, the AMC has provided a copy of an item prepared by AMC staff (June 2015) for the Medical School Accreditation Committee which is asking the Committee to make a decision on whether a change proposed in an accredited program is a major change.

At APPENDIX 17 is the analysis of the medical school’s proposal by an AMC reviewer (in this case the chair of the last AMC accreditation team to assess the medical programs in this medical school). The AMC seeks an expert review of the proposal to assist the Medical School Accreditation Committee’s discussion.

When it considers the initial advice from a medical school about planned changes, either through a specific notice of intent or through progress reports, the Medical School Accreditation Committee decides if it is a major change. If it is, the Committee decides whether the major change can be approved for introduction within the current accreditation of the program or is of comprehensive impact that would require reaccreditation of the whole program. The AMC advises the medical school of its decision, including whether the assessment will be carried out by correspondence or by visit.

At APPENDIX 18 is an extract from the minutes of June 2015 the meeting of the Medical School Accreditation Committee concerning the discussion on this item, and the decision to assess the proposal as a major change to an accredited program and provider.

At APPENDIX 19 is a copy of the AMC’s letter to the Dean of the program advising that the AMC has found the proposal to be a major change, and beginning the process for the accreditation assessment.

The Procedures for Assessment and Accreditation of Medical Schools by the Australian Medical Council 2017 (APPENDIX 7) section 3, concerning assessment by a team explains the AMC’s processes for establishing a team to assess a program, the work of the team, the nature of the on-site visit, and the final accreditation report.

At APPENDIX 20 is a copy of the accreditation report prepared by the AMC on the assessment of the plans for this change by on-site visit. The Executive Summary of the Report describes the assessment process, the options for an accreditation decision, and lists the decision made by the AMC.

 
Analyst Remarks to Narrative
In its prior appearance before the Committee, the country was requested to provide additional documentation related to the implementation of its substantive change review process. In response, the country provided a number of examples of documentation related to its review of substantive change. Meeting minutes indicate that the country considered information that became available as a result of an on-site review and recommended gathering more information from the institution to determine if certain changes would be considered a substantive change or would require separate accreditation (Ex. 16). The country provided information related to the ongoing examination of an intent to implement a substantive change (Ex. 17). Committee minutes provided additional documentation of the country's ongoing review of the proposed substantive changes that were being considered (Ex. 18). A letter to an institution's dean provides information to the institution regarding the possible actions related to a substantive change and requests that the institution provide additional information related to proposed changes (Ex. 19). The country provided a copy of the on-site review report regarding a review it conducted as a result of a proposed substantive change (Ex. 20). In addition to the requested documentation, the country also provided a copy of its procedures manual, which addresses the process for the country's review of substantive changes (Ex. 7).

ED staff accepts the country's documentation, and no additional information is requested.
 
Conflicts of Interest, Inconsistent Application of Standards, Q1 NCFMEA may wish to request have to develop a process for collection of signed statements from all of its decision makers and site visitors acknowledging that they are not acting in an unbiased way.
 
Country Narrative
Since its 2016 report to the NCFMEA, the AMC has changed its management of conflicts of interest so that the process is more transparent and robust.

The AMC’s general policy requirements have not changed. The following statement concerning conflicts of interest is taken from the Procedures for Assessment and Accreditation of Medical Schools by the Australian Medical Council 2017 (APPENDIX 7) section 2.7. This information was provided in the AMC redetermination application in 2016.

"Members of AMC committees are expected to make decisions responsibly, and to apply standards in a consistent and an impartial fashion.

The AMC recognises that there is extensive interaction between the organisations that provide medical education and training in Australia and New Zealand so that individuals are frequently involved with a number of programs. The AMC does not regard this, of itself, to be a conflict. Where a member of an AMC accreditation committee or an assessment team has given recent informal advice to an education provider on its program of study outside the AMC accreditation process, that member must declare this interest.

The AMC requires its Directors and members of its committees to complete standing notices of interest and to update these regularly. These declarations are available at each meeting of the committee. The agendas for AMC committee meetings begin with a ‘declaration of interests’, in which members are requested to declare any personal or professional interests which might, or might be perceived to, influence their capacity to undertake impartially their roles as members of the committee.

The committee will decide how the member’s interest in a particular item will be managed, for example by exclusion from the meeting or from discussion of the relevant item. Members will not vote on matters on which they have a declared personal or professional interest. All declared interests will be recorded in the committee minutes, as will the committee’s decision in relation to the interest."

The AMC requires proposed members of assessment teams to declare to the Medical School Accreditation Committee any relevant personal or professional interest that may be perceived to conflict with their ability to undertake impartially their duties as an assessor. A copy of the standard notice of interests which must be completed by all members of committees and accreditation assessment team members was provided in 2016 as APPENDIX 34 to the AMC submission).

The NCFMEA indicated that “While the AMC has provided a copy of the procedures related to Conflict of Interest and has also provided a blank copy of its Standing Notice of Interests form, it has not included signed documentation that demonstrates the application of its requirements.”

NCFMEA asked about the process for collection of signed statements from all of its decision makers and site visitors acknowledging that they are not acting in an unbiased way.

The AMC does collect signed statements by the members of its accreditation committee and the members of its site visit teams. The process used by the AMC to manage conflicts is as follows.

For the Medical School Accreditation Committee:
AMC staff keep a list of members’ declarations, which is available at each meeting of the relevant committee. From mid-2016, agendas for all AMC accreditation committees, including the Medical School Accreditation Committee, now include an amended statement concerning declarations of interest. The new statement provides a more transparent policy for management of declared conflicts, and for AMC staff to draw attention to any perceived conflicts in committee members and accreditation assessors.

Agendas for AMC committee meetings begin with a ‘declaration of interests’, in which members are requested to declare any personal or professional interests which might, or might be perceived to, influence their capacity to undertake impartially their roles as members of the committee. From mid-2016, the known interests of members are provided in the agenda papers in a summary compiled by AMC staff.

The committee decides how to manage interests declared either as per the declarations list or at the meeting. Members cannot vote on matters on which they have a personal or professional interest. All declared interests are recorded in the committee minutes, as is the committee’s decision in relation to the interest. From mid-2016, the decisions AMC committees are required to make concerning management of a members’ conflict of interest is also pre-explained to the committee, to ensure that conflicts of interest are managed consistently.

APPENDIX 21 is a list of all committee members’ major interests. This list is provided in the Committee meeting papers rather than simply held by AMC staff as reference material. This list is compiled by AMC staff from members’ declarations, and review of the members’ CVs.

APPENDIX 22 is AMC guidance to its committees on how specific sorts of conflicts of interest must be managed. This is now provided with every committee agenda.

The AMC has also revised its statement of expectations of members of its accreditation committees. The two documents sent to committee members concerning AMC expectations and arrangements are provided at APPENDIX 23 and APPENDIX 24. The changes made are highlighted.

In addition, from 2016, new members of the Medical School Accreditation Committee are invited to attend the annual AMC accreditation workshop. In the past, this has focussed on training for accreditation assessment teams, but now it also included training for new committee members. The primary reference material for this workshop is the AMC Accreditation Handbook. The 2017 Handbook is provided at APPENDIX 25.

Section 4.1 of the AMC Accreditation Handbook is written specifically for accreditation assessment team members. It provides detailed information on what may constitute a conflict of interest for an accreditation assessment team member. It describes possible conflicts as:

- Personal conflicts could include animosity, close friendship or kinship between a team member and the one or more senior officer of the education provider or accreditation authority.

- Professional conflicts could occur if the team member had been a failed applicant or was a current applicant for a position in the education provider or accreditation authority, was a senior adviser, examiner or consultant to the provider, or was with an institution that is strongly competing with an education provider or authority being accredited.

- Ideological conflict would be a team member’s lack of sympathy to the style, type or ethos of an education provider or authority.

This guidance is to assist team members, team chairs and AMC staff to understand the range of possible conflicts of interest and to ensure that potential conflicts are identified and appropriately managed.

The AMC deals with conflicts of interests of assessment team members by requiring them to fill in notice of interest forms, to declare any relevant personal or professional interest that may be perceived to conflict with their impartiality, and to provide their curriculum vitae to the AMC. The AMC discloses the declared interests to the medical school, provides the team members’ curriculum vitae, and seeks the medical schools feedback. The Committee makes a decision to appoint teams after reviewing this material.
 
Analyst Remarks to Narrative
At its last appearance before the Committee, the country was requested to develop a process for collecting signed conflict of interest forms from its decision-makers and on-site reviewers. In its response, the country notes that while it has not changed its general policies in this area, it has strengthened its process for tracking conflicts of interest. The country provided a chart declaring conflicts of interest for its board and committee members (Ex. 21). It developed a chart that lists situations in which certain participants must recuse themselves from participation in meetings (Ex. 22). It provided a copy of its notice concerning the declaration of conflicts of interest (Ex. 24). It provided a copy of its handbook for on-site review teams, which addresses conflicts of interest (Ex. 25). It provided a copy of its policies and procedures manual, which also addresses conflicts of interest (Ex. 7).

Although the country provided ample documentation demonstrating that it has appropriate policies and procedures related to conflicts of interest, it did not provide any examples of conflict of interest forms that had been signed by its committee members or on-site review team members to demonstrate the implementation of its policies and procedures. Additional information is requested. The country is requested to provide sample conflict of interest forms that have been signed by its decision-makers and on-site reviewers.
 
Country Response
NCFMEA analysis indicates that although the AMC provided ample documentation demonstrating that it has appropriate policies and procedures related to conflicts of interest, the AMC should provide examples of conflict of interest forms that had been signed by its committee members or on-site review team members to demonstrate the implementation of its policies and procedures.

Two examples have been provided:

New APPENDIX 36: A declaration of interests form by a Medical School Accreditation Committee member

New APPENDIX 37: A declaration of interests form by a member of an accreditation assessment team.

 
Analyst Remarks to Response
In response to the draft staff analysis, the country provided sample signed conflict of interest forms as documentation that both its Committee members and its on-site evaluation team members provide written attestation as to any conflicts of interest that they may have.

ED staff accepts the country's supplemental documentation, and no additional information is requested.
 
Staff Conclusion: Comprehensive response provided
 
Accrediting/Approval Decisions, Question 4 Please provide documentation that demonstrates the institution’s goals for compliance under this guideline.
 
Country Narrative
NCFMEA has asked for clarification of how the AMC, through its accreditation processes, reviews the appropriateness of information and data provided by medical schools about the student performance outcome standards or benchmarks for schools, such as acceptable numbers of graduates from the school passing a licensing exam, and an acceptable percentage of students graduating.

The AMC was asked to describe its collection and use of the data.

While the AMC does not state a specific acceptable number of graduates passing licensing examinations or percentages of students graduating, it does require all schools to provide data that show student progression through and attrition from the medical program. This data must be provided annually either through the accreditation submission, if the medical school is undergoing an onsite accreditation assessment, or through the progress report that the school must submit to the AMC.

APPENDIX 5: Provides the AMC’s accreditation submission guide for medical schools preparing for accreditation. See pages 26 and 29 of the submission guide for the questions the AMC requires medical education providers to address. The medical education provider is asked “How does the medical education provider evaluate graduate outcomes. (e.g. intern supervisor evaluations, surveys of recent graduates, success in vocational and other licensure examinations, career destinations or employer feedback)?

APPENDIX 6: Provides the AMC’s guide to the information education providers must submit in their annual or biennial monitoring (progress) reports. See page 10 and page 12 of the submission guide for the questions the AMC requires medical schools to address.

How the AMC considers the data and information provided by the medical school: The data and the medical school’s analysis of the data is considered by the AMC accreditation team in its on-site inspection. If the AMC is considering the medical program through a progress report, the medical school’s information is considered by the Medical School Accreditation Committee. The team or Committee reference the data and information the AMC holds concerning other medical schools and medical programs and makes a judgement about whether the medical school’s students are performing adequately and whether the medical school is collecting adequate data to make this assessment.

APPENDIX 26: Is an extract from a recent accreditation report by the AMC. This shows that the AMC team has assessed the data and information provided by the medical school concerning how it measures student performance, and has placed a condition on the accreditation, where the AMC has found the school’s processes to require improvement. Relevant sections of the report are underlined.

As the AMC indicated in the letter it submitted with its 2016 NCFMEA submission, Australia and New Zealand both have highly regulated higher education sectors, with additional regulations to establish a medical school and to offer student places in a medical program. Each medical school operates in a university established under an Act of Parliament. Each university is subject to audit and accreditation by a government authority against national standards and higher education requirements (in Australia the Higher Education Standards Framework and the Australian Qualifications Framework). Because of the layers of regulation and quality assurance in Australia and New Zealand, the AMC has been able to be assured that universities are engaged in benchmarking and reviewing student performance achievements because the law requires the university to meet additional standards.

See APPENDIX 27: Higher Education Standards Framework, which cover the standard of academic governance, course design and delivery, staffing, facilities and infrastructure, learning resources and educational support for each university as a whole. These standards were revised in 2015, for introduction in 2017. Section 5.3 of the Higher Education Standards are relevant
5.3 Monitoring, Review and Improvement
1. All accredited courses of study are subject to periodic (at least every seven years) comprehensive reviews that are overseen by peak academic governance processes and include external referencing or other benchmarking activities.
2. A comprehensive review includes the design and content of each course of study, the expected learning outcomes, the methods for assessment of those outcomes, the extent of students’ achievement of learning outcomes, and also takes account of emerging developments in the field of education, modes of delivery, the changing needs of students and identified risks to the quality of the course of study

4. Review and improvement activities include regular external referencing of the success of student cohorts against comparable courses of study, including:
a. analyses of progression rates, attrition rates, completion times and rates and, where applicable, comparing different locations of delivery, and
b. the assessment methods and grading of students’ achievement of learning outcomes for selected units of study within courses of study.

Since the universities are required to demonstrate they undertake these processes across their academic areas, the AMC accreditation process is a more focussed assessment against standards and requirements that are specific to the medical program.
 
Analyst Remarks to Narrative
When it last appeared before the Committee, it was noted that the country had not established any national standard for assessing student performance and that collection of such data was determined by the individual institutions. The country was requested to provide additional information about how it reviews performance data that is collected by the institutions.

In its response, the country states that it reviews the benchmark data collected by the schools via comprehensive on-site reviews that take place at least every seven years, as well as through annual/biennial progress reports that the schools must submit on a regular basis. The country provided an excerpt from an accreditation report in which the country instructed the institution to provide additional evidence related to graduate outcomes (Ex. 26, p. 6). It also provided copies of guidelines documents that indicate that it requires institutions to provide information related to their outcomes measures (Exs. 5 and 6).

Although it is apparent that the country requires its institutions to submit information related to student assessment, it is unclear to ED staff what the country does with this information once it has been submitted. In other words, what is the formal review process used to evaluate the data, how is the data that has been submitted determined to be satisfactory, what consequences are there to the institution if the assessment data is found to be unacceptable, etc.?

Additional information is requested. It is obvious that the country has a process to collect information related to student outcome data. However, simply collecting the information is not sufficient to satisfy the requirements of this section. The country is requested to provide information on its process for reviewing the outcomes data that it collects from its institutions. It should describe how the information that has been submitted is determined by the country to be acceptable and also provide information regarding the consequences to the institution if the country determines that student performance is not at an acceptable level.

 
Country Response
Additional information was requested. The AMC was asked to provide information on its process for reviewing the outcomes data that it collects from its institutions. It was asked to describe how the information that has been submitted is determined by the country to be acceptable and also provide information regarding the consequences to the institution if the country determines that student performance is not at an acceptable level.

The AMC review of student outcomes data is part of its standard accreditation processes.

The standards which relate to the AMC consideration of these data are the monitoring and evaluation standards (see section 6 of APPENDIX 1 to the AMC’S 2017 Redetermination Submission). Specifically:

6.1.1 The medical education provider regularly monitors and reviews its medical program including curriculum content, quality of teaching and supervision, assessment and student progress decisions. It manages quickly and effectively concerns about, or risks to, the quality of any aspect of medical program.
6.1.3 The medical education provider collaborates with other education providers in monitoring its medical program outcomes, teaching and learning methods, and assessment.
6.2.1 The medical education provider analyses the performance of cohorts of students and graduates in relation to the outcomes of the medical program.
6.2.3 The medical education provider examines performance in relation to student characteristics and feeds this data back to the committees responsible for student selection, curriculum and student support.

The AMC processes for considering the data are through accreditation assessments and monitoring processes. During the comprehensive on-site reviews that take place at least every ten years, these data form part of the medical school’s accreditation submission and are reviewed by the accreditation assessment team.

The data provided are also reviewed through annual/biennial progress reports that the schools must submit as part of the AMC’s monitoring of accredited programs. During these monitoring processes, the data are reviewed by AMC staff, a progress report reviewer (usually a member of the team that last assessed the program), and the Medical School Accreditation Committee.

The consequences to the institution if the AMC determines that student performance is not at an acceptable are the same as for failure to comply with any accreditation standard. Under the legislation that relates to the AMC accreditation process, the Health Practitioner Regulation National Law, the AMC must impose conditions if the provider (medical school) and the program of study do not meet an approved accreditation standard, and the imposition of conditions must ensure the program meets the standard within a reasonable time. The AMC sets a time frame for each condition to be met, and the medical school must take action and report on that action within that time frame.

The Procedures for Assessment and Accreditation of Medical Programs by the Australian Medical Council (APPENDIX 7 to the 2017 redetermination submission) also outline unsatisfactory progress procedures (see section 4.4).
These procedures relate to serious concerns about a provider not meeting standards. They relate to circumstances where the AMC, on the basis of progress reports or other material, considers the education provider and its program may no longer meet the accreditation standards or may have difficulty meeting the standards in the future. These procedures allow the AMC to seek an unscheduled review of a program. The AMC tells the provider what its concerns are, investigates the concerns by appointing a team, and the team reports to the AMC on the outcomes of its investigation. If this investigation leads the AMC reasonably to believe the program and the education provider no longer meet the accreditation standards, the AMC will either impose conditions on the accreditation or revoke the accreditation.
The AMC has had one instance where its routine monitoring processes identified concerns about student outcome data (particularly in the clinical years of the program), which led to an unsatisfactory progress review, and the imposition of additional conditions on the medical program. This situation occurred in 2007. The AMC reduced the accreditation of the program to one year. It required that there be remediation for the students (in the form of additional clinical training opportunities) to address the concerns raised. An extract from the accreditation report for the School is provided summarising the process and outcome is new APPENDIX 38.
 
Analyst Remarks to Response
In response to the draft staff analysis, the country reiterated that it reviews benchmark data through its ongoing comprehensive reviews, as well as through its required annual and biennial reports. The data are reviewed by the agency's staff, a progress report reviewer who is usually a member of the team that last assessed the program, and the country's Committee. The country states that institutions that have unsatisfactory outcomes data are considered to be out of compliance with the country's standards and are therefore subject to the same corrective measures associated with any area of noncompliance. In areas of noncompliance, the country notifies the institution, appoints an on-site review team to investigate, and the country's commission reviews the team's report and takes action. If the Committee finds that the institution no longer meets the country's standards, it will impose conditions on the institution or withdraw its accreditation.

The country reports that it has only had one instance in the past ten years in which its review processed identified unsatisfactory progress related to student outcomes. In response, the country shortened the institution's period of accreditation. As documentation of its processes and procedures in this area, the country provided the report associated with its review of that institution (Ex. 6).

ED staff accepts the country's narrative and supporting documentation, and no additional information is requested.
 
Staff Conclusion: Comprehensive response provided
 
Chief Academic Officer Q1 NCFMEA may wish to ask for the actual documentation of this job description to ensure compliance relative to this guideline.
 
Country Narrative
This response addresses the NCFMEA guideline that “The chief academic official of the medical school must be qualified by education and experience to provide leadership in medical education, scholarly activity, and patient care.” In response to the AMC’s 2016 report, NCFMEA requested documentation demonstrating the AMC’s review and assessment of the appropriateness of the qualification of the chief medical officer and other educational staff and faculty.

NCFMEA asked for clarification on the selection process of academic officials for medical schools. Its analysis in 2016 indicated that, “the agency has not demonstrated the onsite inspection team review of an institution for compliance relative to these standards.” NCFMEA also asked that the AMC upload the actual position description so it is included in the official record of documentation submitted with the agency’s comparability petition.

The AMC has provided additional material in 2017 that relates to an accreditation assessment of a new medical school.

APPENDIX 28: This is an extract from the accreditation submission provided by a new medical school accredited in 2016. This shows that the School, as requested by the AMC provided a position description for the dean of the new medical school.

APPENDIX 29: Is the position description for the academic head of the new school, that was provided with the accreditation submission.

 
Analyst Remarks to Narrative
At its last appearance before the Committee, the country was requested to provide additional documentation verifying that position descriptions are used in the selection of chief academic officers. In response, the country has provided a sample position description (Ex. 29), as well as documentation regarding the responsibilities of the chief academic officer at one of its institutions (Ex. 28).

ED staff accepts the country's supplemental documentation, and no additional information is requested.
 
Chief Academic Officer Q2 AMC states that they believe having an AMC standard relating to the qualifications of the chief academic official are not necessary. AMC relies on the information that is established by the Tertiary Education Quality and Standards Agency for standards on this topic. AMC has provided documentation that demonstrates the qualifications of the Deans of its medical schools. Please provide clarification on the selection process of academic officials for medical schools.
 
Country Narrative
There is not one common process for selection of academic officials for medical schools. All the Australian and New Zealand medical schools are located in universities which are established by law of the state, territory in which they are located, or in New Zealand under national law.

As the AMC has indicated earlier in this submission, Australia and New Zealand both have highly regulated higher education sectors, with additional regulations to establish a medical school and to offer student places in a medical program. Each medical school operates in a university established under an Act of Parliament. Each university is subject to audit and accreditation by a government authority against national standards and higher education requirements (in Australia the Higher Education Standards Framework and the Australian Qualifications Framework). Because of the layers of regulation and quality assurance in Australia and New Zealand, the AMC has not needed specific accreditation standards for medical schools in some areas addressed in NCFMEA Guidelines and LCME standards. One area of difference is in setting standards for the selection or qualifications of the chief academic official of a medical school.

The legislation establishing each university gives the university council the power to appoint the senior academic staff. Universities have detailed appointment policies that describe the processes for selection the senior academic staff.

Although there are not specific standards on these matters, AMC accreditation assessments do consider them.

The AMC has provided at APPENDIX 30 an extract from the review undertaken by a member of the AMC’s Medical School Accreditation Committee of the preliminary documentation provided by a new medical school to support its accreditation. The analysis shows that the AMC did consider the credentials of the senior leadership of the new school.



 
Analyst Remarks to Narrative
At its last appearance before the Committee, the country was requested to provide additional information regarding the selection process for academic officials for medical schools. In its response, the country notes that it does not specify the selection process for academic officials for its medical schools. All of the country's medical schools operate as components of larger universities that are subject to parliamentary approval and government review and accreditation. The country notes that although there are not separate standards specific to the selection of academic officials at medical schools, the country does review the qualifications of academic officials in reviewing its schools and provided information from one of its evaluations as documentation (Ex. 30).

ED staff accepts the country's narrative and supporting documentation, and no additional information is requested.
 
Faculty Please provide clarification to verify that faculty are involved in the decision making relative to this guideline
 
Country Narrative
The NCFMEA response to the AMC’s 2016 submission said that the AMC had not confirmed that faculty (rather than senior staff) are involved in decisions related to admissions, the curriculum, and the hiring, retention, promotion, and discipline of faculty as required by this guideline.

The NCFMEA staff analysis also indicates that the standards themselves the AMC had quoted in relation to this question were not specific to faculty and listed broad terms like 'relevant groups or committees'.

Rather than include in each relevant accreditation standard the requirements that staff or students must be involved in the policy or process described in the standard, the AMC has defined the terms “relevant groups” and “stakeholders” to ensure that these definitions cover academic staff and students. It took this approach to simplify the wording of specific standards (so they do not need to contain a long list of the groups that must be included in consultation or decision making on the topic).

See APPENDIX 1: Standards for Assessment and Accreditation of Primary Medical Programs by the AMC 2012.

The relevant accreditation standards that relate to staff involvement in decision making about the medical program are:

1.3.1 The medical education provider has a committee or similar entity with the responsibility, authority and capacity to plan, implement and review the curriculum to achieve the objectives of the medical program.

1.1.2 The governance structures set out, for each committee, the composition, terms of reference, powers and reporting relationships, and allow relevant groups to be represented in decision-making.

1.1.3 The medical education provider consults relevant groups on key issues relating to its purpose, the curriculum, graduate outcomes and governance.

The AMC glossary of terms used in the accreditation standards (see page iv and v of the Standards) defines the term “relevant groups” as internal and external stakeholders, and the definition of internal stakeholders includes faculty - program directors, academic staff, supervisors and committees. The accreditation standards listed above make it clear that faculty staff are one of the relevant groups to be represented in decision making of the education provider, and that they must be consulted on key issues such as the curriculum and the objectives of the medical program.

While the AMC approach of setting broad standards with the terminology used in the standards defined in the glossary may be different to the approach used by the LCME, the accredited providers (medical schools) are aware that this is how the AMC accreditation standards are designed.

In relation to staff involvement in the hiring, retention, promotion, and discipline of faculty, the AMC does not explicitly address this guideline in its accreditation standards. The AMC requires education providers undergoing accreditation to provide access to their selection, appointment, promotion and staff review policies. This is shown in the APPENDIX 5: Reaccreditation Submission Guide (questions included in section 1.9 of the Guide).

The AMC would welcome clarification of the principle that it being addressed in this guideline. Is it that the development of these systems and policies must have staff input (an industrial democracy issue) or that the accreditation authority should assure there is adequate discipline-specific input to these processes?
 
Analyst Remarks to Narrative
At the time of the country's last review, a concern was raised over whether faculty were involved in decisions related to admissions, the curriculum, and the hiring, retention, promotion, and discipline of faculty. At question was the requirement that "relevant groups or committees" would be involved in these decisions and whether such groups or committees would include faculty. In its response, the country has clarified that it used the phrase in order to avoid providing long lists of the constituents that would be involved in various decisions, and it clarifies that faculty are an integral part of decisions related to admissions, the curriculum, and the hiring, retention, promotion, and discipline of faculty. As documentation of faculty involvement, the country provided a copy of its standards for assessment and its self-study guidelines document (Exs. 1 and 5).

ED staff accepts the country's clarification of its terminology and supporting documentation, and no additional information is requested.


 
Remote Sites Q1 AMC has stated that they have one program that is offered outside of Australia and has previously reported this information to the NCFMEA. While AMC has explained that they have conducted reviews of the program, they have not provided documentation of this review. NCFMEA may wish to request a status update regarding this program and documentation that demonstrates how it reviews the curriculum to ensure that it is equivalent.
 
Country Narrative
The Australian Medical Council has a policy on the additional requirements to be met by Australian and New Zealand medical schools that wish to offer part of their medical program offshore. The AMC applies this additional policy together with the accreditation standards whenever it assesses proposals for the accreditation of Australian (or New Zealand) medical programs to be partly offered offshore.

APPENDIX 31 is the AMC Policy Primary medical programs provided offshore by Australian and/or New Zealand education providers.

The AMC has had such a policy since 2005, but it completed a review of the policy in 2016. The changes to the policy are highlighted in the document.

The policy sets out four criteria that Australian or New Zealand medical programs being offered offshore must meet in order to be accredited. Two of these criteria relate to program equivalence. The medical programs must be:

“Located in an Australian or New Zealand higher education provider, where the Australian or New Zealand provider has developed the program and has a responsibility for the academic standards”

Essentially the same as the program accredited by the AMC for delivery in Australia or New Zealand, in terms of learning outcomes, curriculum, educational process, and assessment outcomes.”

These requirements link to accreditation standard 2.2.3 “The medical program achieves comparable outcomes through comparable educational experiences and equivalent methods of assessment across all instructional sites within a given discipline.” See APPENDIX 1: Standards for Assessment and Accreditation of Primary Medical Programs by the AMC 2012.

The AMC also provides guidance to accreditation teams and the AMC Medical School Accreditation Committee on the specific questions that need to be addressed when considering an offshore program. (This is appended in the policy document Primary medical programs offered offshore by Australian and/or New Zealand education providers.)

These questions aim to ensure that the education provider has a comprehensive plan for quality assurance of teaching and learning across the different sites and also that the Australian/New Zealand education provider is responsible for overall quality assurance of the program, not the offshore provider of the program.

The AMC policy makes it clear that the accreditation of the entire program is at risk, if the offshore component fails to meet the standards.

In 2016, the AMC had indicated that one medical program in Australia, Monash University, has a medical program that allows a cohort of the medical students to complete a component of preclinical studies outside Australia. NCFMEA has asked for more information about the AMC’s reviews of this program. This program is subject to accreditation assessment by site visit in 2017. It was last assessed in 2013.

APPENDIX 32: is a copy of the report of the Monash accreditation assessment. The report shows that Monash was asked to provide specific information concerning the consistency of assessment outcomes for preclinical teaching (specifically anatomy). The report does comment on equivalent across the teaching sites. Relevant sections of the report are highlighted.
 
Analyst Remarks to Narrative
When the country last appeared before the Committee, it was noted that the country has one overseas program. Although the country stated that it had conducted reviews of this program, no supporting documentation was provided to demonstrate that such a review had taken place. The country was requested to provide documentation of its review of the one overseas program that it accredits. In its current response, the country provided information on its review process, but did not provide documentation of its implementation as had been previously requested.

Additional information is requested. The country is requested to provide documentation of the implementation of its policies and procedures, in the form of an on-site review report, that it has reviewed the overseas program that it accredits.
 
Country Response
Monash University offers its medical program through Monash University Malaysia. The program is taught largely in Malaysia, by Monash University staff, although all students must complete clinical training in Australia. The AMC last accreditation assessment report on the Monash University program was provided as APPENDIX 32 to the 2017 submission.

This report of the 2013 assessment relates to a limited assessment. The assessment did not include assessment visits to all sites, although the medical school’s submission covered teaching at all sites and the curriculum equivalence across teaching sites, and the AMC report did address equivalence across sites (as was reported in the 2017 submission). In 2017 the AMC is conducting a reaccreditation assessment of the Monash program. The visit is scheduled for July 2017 (Malaysia) and August 2017 (Australian sites). A copy of the AMC letter to the Monash Dean confirming visit dates is at new APPENDIX 39.

The 2017 submission outlined the additional criteria and considerations the AMC applies to considering a proposal to offer a program offshore. These criteria relate specifically to comparability of the off shore program to the Australian accredited program. These criteria have been strengthened in the review of the AMC Policy, Primary medical programs provided offshore by Australian and/or New Zealand education providers, which was provided as APPENDIX 31 to the 2017 submission.

Because of the scheduled assessment of the Monash program in 2017, the AMC’s annual accreditation workshop, which provides training for teams and new members of the Medical School Accreditation Committee on the accreditation process, had a focus on assessing equivalence of programs across different sites. A copy of the program for this workshop is provided as new APPENDIX 40.
 
Analyst Remarks to Response
In its response to the draft staff analysis, the country states that the assessment materials it had submitted regarding its one overseas program were related to a limited assessment, rather than a full review of the program. The country states that the overseas program is scheduled for a full review by the country in July 2017. As documentation, the country provided a copy of the letter to the program scheduling the July 2017 review.

Additional information is requested. The Committee may wish to request that the country provide additional information regarding the comprehensive review of its one overseas institution after the school is reviewed in July 2017.
 
Staff Conclusion: Additional Information requested
 
Program Length Q1 NCFMEA may wish to request that the agency develop a policy for a minimum number of weeks instruction to ensure compliance under this guideline.
 
Country Narrative
The AMC has not needed a standard regarding a minimum number of weeks of instruction in the medical program, and has not been planning to introduce such a standard.

NCFMEA Staff indicated in 2016 that they were concerned that without a standard for the amount of the program length, that a new program may submit a request to be less than that standard and that AMC will not have a way to enforce it if no standard exists. The AMC would be able to refuse to accredit a program that is too short on the basis that it would not meet the standard concerning duration of the program.

In its 29 years’ experience accrediting medical schools, the AMC has not had an instance where a medical school in Australia or New Zealand has sought accreditation of a program that the AMC considered to be too short. All the accredited medical programs in Australia and New Zealand include four years graduate medical programs and five to six years medical programs for school-leavers. There are no accredited programs of less than four years duration or less than 130 weeks of instruction.

The AMC believes that its standards and processes provide an effective alternate to an explicit standard specifying the minimum length of the program of instruction.

The AMC has used an “outcomes based approach” in which a medical school must show that it has a program that enables graduates to meet defined outcomes rather than setting prescriptive standards concerning time based requirements.

APPENDIX 1 Standards for Assessment and Accreditation of Primary Medical Programs by the Australian Medical Council provides the accreditation standards.

Accreditation standard 3.1 Duration of the medical program states: “The medical program is of sufficient duration to ensure that the defined graduate outcomes can be achieved”

The AMC specifies a list of comprehensive graduate outcomes (Section 2 of the standards). Medical schools must demonstrate that their program is structured so that the graduates can meet these objectives. The AMC considers that the NCFMEA minimum of 130 weeks would be insufficient to achieve the program objectives, and that setting a standard on this matter would be unnecessary.

The accreditation report at APPENDIX 33, concerns the newest medical school in Australia (Curtin University accredited in 2016). Please see section 3 of the report, page 33. This section shows the AMC’s assessment of the medical program against standard 3.1 concerning program duration to demonstrate that it does carefully consider program length relative to program capacity to meet required outcomes.

The AMC understands the NCFMEA statements are guidelines, and that a country's standards and evaluation processes may differ substantially from these Guidelines and the LCME standards and still be determined to be comparable to the standards and evaluation processes used in the United States. The AMC believes that its process for assessing program length have not resulted in the accreditation of a substandard program of medical education leading to the M.D. (or equivalent) degree.

 
Analyst Remarks to Narrative
When it last appeared before the Committee, it was noted that the country had not developed a policy for a minimum number of weeks instruction for program completion of the medical program, and it was suggested that the country should consider establishing such a policy. In its response, the country states that it has not established such a policy and does not feel that such a policy is necessary. It notes that, while it does not have a formal policy related to program length, it has no accredited programs that require less than 130 weeks of instruction or are less than four years in duration. The country further states that this has been the case during the 29 years that it has been accrediting medical schools and that it is unlikely to change in the future.

Staff accepts the country's information on the length of its medical programs and the history of their accreditation, and no additional information is requested.
 
Admissions, Recruiting & Pub Q1 Please provide additional information from AMC regarding if it plans to adopt policy to specifically address this guideline.
 
Country Narrative
NCFMEA requested additional information from AMC regarding if it plans to adopt policy to specifically address this guideline concerning using students' scores on the MCAT in its evaluation of the quality of the school's admission practices.

The AMC has not specifically considered how it would use this information in its evaluation of the quality of medical schools’ admission practices but in the minor review of accreditation standards that it began in mid-2016, it considered tightening the accreditation standards concerning admission. The proposed revised standard is “The medical education provider has clear selection policy and processes that can be implemented and sustained in practice, are fit for purpose that are consistently applied and that prevent discrimination and bias, other than explicit affirmative action.”

The draft consultation paper concerning the proposed changes to the standards is provided at APPENDIX 10. The description of the proposal for revision to the on page 4 of this document shows how the AMC proposes to define “fit for purpose” admission standards. As noted above, the AMC will consult on these proposals for revisions to the accreditation standards in 2017.

When it makes this change, the AMC will update the data and information it requests from medical schools. At that stage, it could consider using students' scores on the MCAT in its evaluation of the quality of the school's admission practices. This process, including consultation on plans to change the standards, will take about 8 months.
 
Analyst Remarks to Narrative
When it last appeared before the Committee, additional information was requested from the country about whether it plans to adopt a policy concerning the use of students' scores on the MCAT in its evaluation of the quality of the school's admission practices. In its response, the country states that it is currently in the process of revising its standards and is considering revising its standard related to student admissions. The standards revision process is currently underway during 2017, and no changes have yet been made to the standards. As documentation of the standards review process, the country provided a copy of the draft revisions to the admissions standards that have been published for stakeholder feedback (Ex. 10, pp. 4-5).

Additional information is requested. The Committee may wish to request more information from the country in the future regarding its revised standards after they have been finalized.
 
Country Response
As noted in response to a previous question, the AMC review of standards has been delayed to accommodate consultation on a national accreditation systems review. The period of consultation on the accreditation systems review ends on 1 May 2017. While we wait for this period of consultation to end, the AMC has divided the changes it wishes to make to the accreditation standards for medical programs into high priority and longer term changes.

The AMC is planning to revise the present standard of student admissions from

7.2.1 The medical education provider has clear selection policy and processes that can be implemented and sustained in practice, that are consistently applied and that prevent discrimination and bias, other than explicit affirmative action. (Current standard)

to: “The medical education provider has clear selection policy and processes that can be implemented and sustained in practice, are fit for purpose, are consistently applied and that prevent discrimination and bias, other than explicit affirmative action (new standard)”

We consider this a high priority change, which means we will complete our consultation on this change and introduce the change this year. The AMC is very willing to provide evidence when this change has been made.

 
Analyst Remarks to Response
As was noted in a previous section, the country has delayed its standards review process while it conducts a national accreditation systems review. The country is proposing changes to its standard related to this section and has assigned the revision of the standard a high priority. The standards review process is expected to resume after May 1, 2017.

Additional information is requested. The Committee may wish to request additional information regarding the country's revision of its standard related to this area after the standards review process resumes in May 2017.
 
Staff Conclusion: Additional Information requested
 
Admissions, Recruiting & Pub Q2 Please provide additional information from AMC regarding if they intend to develop standards for this criterion
 
Country Narrative
Analysis of the 2016 AMC response to this guidelines suggested NCFMEA request additional information from AMC regarding if they intend to develop standards for the criterion “ the faculty of each school must develop criteria and procedures for the selection of students that are readily available to potential applicants and to their advisors.”

The AMC does not intend to develop a specific standard in relation to this criterion, but the AMC staff propose to ask the working group reviewing the accreditation standards whether in revising the admissions standards, it could define “fit for purpose” admissions policy and processes as being developed by the faculty. The AMC would welcome further advice on why this standard seems necessary, and what principle it is intended.

As the AMC indicated in its 2016 submission to the NCFMEA, each Australian and New Zealand medical school has an admissions committee that is part of the school’s governance structure and that oversees the admission of students. The membership of these committees includes faculty academic staff and faculty professional/technical staff who manage the selection process. The AMC accreditation standards have not included a separate standard to require these committees, because the AMC has not identified a problem or concern that would suggest a standard is required. The AMC believes that medical schools follow good practice in this area without it requiring an explicit standard.
 
Analyst Remarks to Narrative
When it last appeared before the Committee, the country was requested to provide additional information about whether it intended to develop a standard requiring that the faculty of each school develop criteria and procedures for the selection of students that are readily available to potential applicants and to their advisors.

In its response, the country notes that while it is currently in the process of revising its standards, including standards related to student admissions, it has no plans to develop a standard that is specific to this criterion. The country also requested additional information from the Committee regarding the intent of this criterion.

ED staff notes that the country does have standards regarding the admission of medical students. The primary focus of this criterion is the requirement that admissions criteria and procedures should be readily available to applicants. In other words, the admissions criteria and procedures should be published and readily available so that prospective students and their advisors will be able to determine if the prospective applicant meets the admissions criteria. The admissions process should also be published and transparent to ensure that it is applied consistently and fairly to all prospective applicants.

Additional information is requested. The country is requested to provide additional information about how its standards ensure that accredited medical schools publish their admissions criteria and admissions procedures and make them readily available to prospective students.

 
Country Response
The AMC misunderstood the initial question, and is grateful for the ED staff clarification.

The AMC currently has an accreditation standard which is “7.2.4 Information about the selection process, including the mechanism for appeals is publicly available.”

As the AMC stated in 2017 submission, it provides a standard accreditation submission guide for all medical schools undergoing accreditation. This ensures all medical schools respond to the same questions and provide similar data for review by the AMC. The Reaccreditation Submission Guide is APPENDIX 5.

In linking accreditation standard 7.2.4, to the information the AMC asks medical schools to provide, the reaccreditation submission guide indicates that when addressing each of the standards in the admission section, medical schools should include the following: “Provide the information on selection and admission provided to prospective students. (see page 29 and 30 of the Guide).
 
Analyst Remarks to Response
In response to the draft staff analysis, the country states that it misinterpreted the question. The country does have a standard related to the requirements of this section. Although not attached to this section, the country's standards handbook, available through the Table of Contents as Ex. 1, addresses the requirements of this section by requiring that information about the student selection process be publicly available (Ex. 1, p. 12).

ED staff accepts the country's narrative and standard, and no additional information is requested.
 
Staff Conclusion: Comprehensive response provided
 
Admissions, Recruiting & Pub Q6 Please provide additional information to be provided for the NCFMEA regarding whether the AMC has any plans to establish written policy to address this question.
 
Country Narrative
The AMC is responding to the questions:
Do students have access to their academic records and are they able to challenge their accuracy?
What laws (if any) govern student access to records and the confidentiality of student records?

Yes students have access to their academic records and they are able to challenge their accuracy. Student access to the information that their medical school holds about them and the access by other people to students’ information is covered in privacy and personal information protection legislation in each state and territory of Australia or national legislation of Australia and New Zealand. In addition, specific legislation in each state and territory and national information requires organisations covered by the legislation to have privacy policies. Under these requirements universities must tell students what information the university holds on the student, how it can be accessed and how the student can seek review of the information.

Australia is a federation of eight states and territories. Depending on whether the organisation holding information on a student is a national or state/territory institution, the specific laws that cover access to information (records) will vary but the features will be similar.

The AMC does not have any plans to write specific accreditation standards requiring that medical schools grant students access to their academic records and provide students with opportunity to challenge the accuracy of the records since this requirements is covered by state, territory and national laws.

There are also standards and requirements that by law higher education providers must meet and continue to meet. In Australia, these are the Higher Education Standards Framework. These standards are at APPENDIX 27.
The AMC has mapped its accreditation standards to the Higher Education Standards Framework. University audit and accreditation is a separate, additional layer of regulation. In the interests of regulatory efficiency the AMC does not set accreditation standards for medical schools where the AMC has determined that one or more other process, such as the audit and accreditation of universities and their programs, ensures an appropriate standard. Concerning student access to their academic records, the Higher Education Standards require:

Standard 1.5 concerns accurate academic records. It requires provided to demonstrate that they are able to maintain accurate records:

The 1.5 (7). Records of results state correctly, in addition to the requirements for all certification documentation:
a. the full name of all courses and units of study undertaken and when they were undertaken and completed
b. credit granted through recognition of prior learning
c. the weighting of units within courses of study
d. the grades and/or marks awarded for each unit of study undertaken and, if applicable, for the course overall
e. where grades are issued, an explanation of the grading system used
….

2.4 concerns Student Grievances and Complaints. It states
1. Current and prospective students have access to mechanisms that are capable of resolving grievances about any aspect of their experience with the higher education provider, its agents or related parties.
2. There are policies and processes that deliver timely resolution of formal complaints and appeals against academic and administrative decisions without charge or at reasonable cost to students, and these are applied consistently, fairly and without reprisal.

Standard 7.3 concerns information management

7.3 (3) states. Information systems and records are maintained, securely and confidentially as necessary to:
a. maintain accurate and up-to-date records of enrolments, progression, completions and award of qualifications
b. prevent unauthorised or fraudulent access to private or sensitive information, including information where unauthorised access may compromise academic or research integrity .

AMC duplicates some the Higher Education Standards in the AMC standards when the AMC needs to check and assure itself that there are specific mechanisms or processes in the medical school as well as University-wide policies, standards and procedures. Examples of areas where the AMC sets standards in addition to the Higher Education Standards are resourcing and autonomy of the medical program, formalised relationships with health care facilities and training sites, student support and wellbeing (given the additional support needs for medical students) and policies to address the health care of Australia and New Zealand’s Indigenous people: Aboriginal and Torres Strait Islander and Maori, governance, content and quality of clinical training. The AMC has not set additional requirements or standards concerning student access to their academic record and policies to stop unauthorised access to these records, since this is covered in other accreditation processes.

The AMC has not uncovered concerns or problems during its accreditation assessments or monitoring of accredited programs that would necessitate additional accreditation standards on these matters.

 
Analyst Remarks to Narrative
When it last appeared before the Committee, the country was requested to provide additional information about whether it intended to develop additional policies regarding student access to their academic records, the ability to challenge their records, and laws governing confidentiality of student records.

In its response, the country states that there are laws at the state and federal level that govern access to, and confidentiality of, student records. These laws vary by state, but all require ensure that students have access to their records and that the records are confidential. The country adds that in addition to the state and federal laws, these areas are also addressed in higher education standards that are applicable to higher education institutions in their entirety, and not just to the institutions' medical schools. The country's medical education standards mirror the higher education standards in some instances. As documentation, the country provided a copy of the 2015 higher education standards, which specify that records be maintained confidentially and securely, be accurate and up to date, prevent fraudulent or unauthorized access, and document and record information related to complaints (Ex. 27, p. 25).

Staff accepts the country's narrative and supporting documentation, and no additional information is requested.