American Psychological Association 06/23/2016 05/16/2016 Final Review Renewal Petition

U.S. Department of Education

Staff Report

to the

Senior Department Offical

on

Recognition Compliance Issues


Recommendation Page



  1. American Psychological Association
  2. Renewal Petition
  3. The accreditation in the United States of doctoral programs in clinical, counseling, school and combined professional-scientific psychology; doctoral internship programs in health service psychology; and postdoctoral residency programs in health service psychology; and the preaccreditation in the United States of doctoral internship programs in health service psychology; and postdoctoral residency programs in health service psychology.
  4. Same as above.
  5. June 23, 2016
  6. Continue the agency's recognition as a nationally recognized accrediting agency at this time, and require the agency to come into compliance within 12 months with the criteria listed below, and submit a compliance report due 30 days thereafter that demonstrates the agency's compliance.
  7. It does not appear that the agency meets the following sections of the Secretary’s Criteria for Recognition. These issues are summarized below and discussed in detail under the Summary of Findings section. -- The agency must demonstrate that it has, and effectively applies, a set of monitoring and evaluation approaches that enables the agency to identify problems with a program's continued compliance with agency standards, specifically with regards to student achievement thresholds. [§602.19(b)] -- The agency must demonstrate that it either initiates immediate adverse action or allows a program a time period, not to exceed two years, to come into compliance with its standards and requirements, when the agency's review of a program under any standard indicates that the program is not in compliance with that standard. [§602.20(a)]

Executive Summary



Part I: General Information About The Agency

The American Psychological Association (APA), Commission on Accreditation (CoA or the agency) is a programmatic accreditor. It currently accredits over 900 professional education and training programs at the doctoral and postdoctoral level in psychology. The agency has identified multiple Federal programs that require the Secretary’s recognition of its accredited programs as a prerequisite for programs to participate in non-Title IV federal programs and/or federal employment. These include, for example-- •The Graduate Psychology Education (GPE) Program administered by the United States Department of Health and Human Services (DHHS), •The Federal Center for Medicare/Medicaid Services (CMS) program for postdoctoral residency programs in medical settings, and •The Predoctoral Fellowship offered by the Mental Health and Substance Abuse Services Administration (SAMSHA). In addition, the Department of Veterans Affairs and the Federal prison system cite the APA’s CoA accreditation as the standard both for admission to its internship training programs in professional psychology and for employment as a psychologist at all VA medical centers (VAMCs).

Recognition History

The American Psychological Association (APA), Commission on Accreditation (CoA or the agency) received initial recognition by the Secretary in 1970, and has received continued recognition since that time. The agency was last reviewed for renewal of recognition at the spring 2011 meeting of the National Advisory Committee on Institutional Quality and Integrity (NACIQI or the Committee). Both Department staff and NACIQI recommended to the senior Department official to continue the agency's recognition and require it to come into compliance within 12 months, and submit a compliance report that demonstrates the agency's compliance with the issues identified in the staff report. The senior Department official, Assistant Secretary Eduardo Ochoa, concurred with the recommendations. The compliance report was reviewed and accepted by both Department staff and NACIQI at the spring 2013 meeting. At the June 2015 NACIQI meeting, the agency submitted a petition to expand its scope to include the preaccreditation in the United States of doctoral internship programs in health service psychology; and postdoctoral residency programs in health service psychology. Both Department staff and NACIQI recommended to the senior Department official to expand the scope as requested. The senior Department official, Acting Assistant Secretary Jamienne S. Studley, concurred with the recommendations. Since the agency's last review, the Department has received no complaints and approximately 50) 3rd party comments. In conjunction with agency's petition, Department staff reviewed the agency’s supporting documentation and observed an on-site evaluation in January 2016. Although the review was thorough, it cannot be assumed to be all-inclusive. The absence of statements in the report concerning specific practices, procedures, policies and standards of the agency must not be construed as acceptance, approval, or endorsement of those specific practices, procedures, policies or standards. Furthermore, it does not relieve the agency of its obligation to comply with all of the statutory and regulatory provisions pertinent to recognition and the role of recognized accreditors as Title IV gatekeepers, including those statutory and regulatory provisions not discussed by the agency in its petition. This analysis does not address any changes in policies, procedures or standards that have not been disclosed to Department staff by the agency in this recognition proceeding, nor any policies, procedures or standards disclosed in prior recognition proceedings or to Department personnel outside of this recognition proceeding. This report reflects initial findings, listed by criterion. These findings are not final. The senior Department official will issue a final decision letter listing the criteria, if any, with which the Department finds the agency to be out of compliance.


Part II: Summary Of Findings

602.15(a)(2) Competency of Representatives

(2) Competent and knowledgeable individuals, qualified by education and experience in their own right and trained by the agency on their responsibilities, as appropriate for their roles, regarding the agency's standards, policies, and procedures, to conduct its on-site evaluations, apply or establish its policies, and make its accrediting and preaccrediting decisions, including, if applicable to the agency's scope, their responsibilities regarding distance education and correspondence education;


Qualifications of Commissioners As outlined in Implementing Regulation A.1, the Commission on Accreditation is composed of representatives from current education and training councils, private practitioners, practitioners in public and private agencies, members of the public and a student member. The agency provided documentation about its current commissioners in its petition to demonstrate that they are qualified to fulfill their assigned roles. Qualifications of Appeals Panel Members The agency currently has a standing pool of 20 individuals from which to convene an appeals panel. Implementing Regulation D.5-3 state that appeals panel members must be qualified by education and experience, and include the training of appeals panel members. The agency provided documentation about its current appeals panel pool in its petition, but the list does not include any information about the qualifications of the individuals. Therefore, the agency has not demonstrated that the appeals panel pool members are qualified for their role. Qualifications of Site Visitors The agency’s Implementing Regulation D.3-1 sets specific qualifications for site visitors. The agency provided a sample site visitor information sheet demonstrating that the agency obtains sufficient information to determine that individuals serving in this role meet the agency’s qualifications requirements. The agency provided extensive documentation of its comprehensive, on-going training program for all these individuals, to include site visitor workshops, new member orientation, mentoring, annual policy meeting, and online training and resources. However, the sign-in/out sheets for workshops and site visitor evaluation sheets provided were blank and therefore do not demonstrate implementation of such training tools.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided additional information and documentation concerning the qualifications of appeal panel pool members and the training of potential and current site visitors. Specifically, the agency provided information and documentation regarding the qualifications required for potential appeal panel pool members, as well as documentation that the current appeal panel pool members meet those requirements and are qualified for their role. Also, the agency provided information on the training workshops for site visitors that have occurred this year, as well as the many more planned. The agency provided documentation of those training sessions, as well as documentation of the additional training resources available online and the on-going evaluation of site visitors.



602.15(a)(4) Educator/Practitioner Representatives

(4) Educators and practitioners on its evaluation, policy, and decision-making bodies, if the agency accredits programs or single-purpose institutions that prepare students for a specific profession;


The agency states that the evaluation, policy, and decision-making bodies (site visitors, commission, and appeals panel pool) are comprised of both educators and practitioners and represent the types of programs accredited by the agency and the sponsoring institutions. While the agency did provide documentation that the commission and appeals panel include educators and practitioners, the agency did not provide sufficient documentation that its on-site evaluation teams are comprised of educators and practitioners, as required by this section. Department staff observed an on-site evaluation in January 2016 to a doctoral program that is seeking accreditation by the agency. The agency sent a team of three evaluators, but there was no notation as to the role each person fulfilled on the team. Through individual interviews and a review of biographical information provided, Department staff noted that all three team members were professors (one emeritus) of a psychology department at a large university, and it was unclear that any could serve in the role of practitioner for the review.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided additional information and documentation concerning the representation of both educators and practitioners on its site visit teams. Specifically, the agency provided its definitions of educator and practitioner, as well as documentation of their representation on site visit teams.



602.15(a)(5) Public Representatives

(5) Representatives of the public on all decision-making bodies; and


The agency's Implementing Regulations (A.1 and D.5-3) require that its decision-making bodies include public representation. The 32-member commission includes three public representatives as well as a graduate student representing consumers of professional education and training in psychology. When convened, the three-person appeals panel includes one representative of the public. The agency defines a public member as one outside the discipline of professional psychology and provided the nomination criteria for a representative of the public on the commission, as well as a call for nominations. The criteria included in the call for nominations differs from the nomination criteria. Significantly, the nomination criteria do not meet the requirements of the definition of a public representative as required by the Secretary's Criteria for Recognition. No specific criteria was provided for a public representative on the appeals panel. In addition, one of the public members on the commission is a faculty member at a psychology program accredited by the agency, which would not meet the definition of a public representative as required by the Secretary's Criteria for Recognition. The agency provided a roster of a recent appeals panel, but it is not clear that the individual listed as the public member meets the definition of a public representative as required by the Secretary's Criteria for Recognition. The agency provided no information or documentation on how it ensures that its public members meet both the agency's criteria, as well as the Department's definition, for that position.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided clarification to its policy regarding the requirements for public representatives on the commission. Specifically, the agency states there are only two public representatives on the commission, with an additional representative of the public interest. The representative of the public interest is not required the meet the more definition of a public representative, but is expected to be "a psychologist who brings scholarly expertise on issues of individual and cultural diversity in the context of advancing the science and practice of psychology in public service." The agency also provided its policy for public representatives on the commission and in the appeal panel pool. The agency provided documentation, in the form of attestations and curricula vitae, that the current public representatives on the commission and appeals panel, respectively, meet the agency's and Department's definition. Although the nomination criteria (provided in Exhibit 24) do not meet the requirements of the definition of a public representative as required by the Secretary's Criteria for Recognition, the agency has provided sufficient information and documentation that its current members meet the required definition and that it applies those requirements when reviewing nominations.



602.15(a)(6) Conflict of Interest

(6) Clear and effective controls against conflicts of interest, or the appearance of conflicts of interest, by the agency's--
(i) Board members;

(ii) Commissioners;

(iii) Evaluation team members;

(iv) Consultants;

(v) Administrative staff; and

(vi) Other agency representatives; and


The agency has comprehensive policies on conflict of interest for commissioners, site visitors, APA liaisons, other consultants, staff, and appeal panel members. The policies cite specific instances of conflict of interest and provide guidance on addressing conflict of interest, such as recusal from the deliberation and decision-making process, as included in its Implementing Regulations and Accreditation Operating Procedures. Although the agency has written policies for controls against conflict of interest or the appearance of conflict of interest for all entities to meet this section of the Secretary's Criteria for Recognition, it has not provided sufficient documentation to demonstrate implementation and compliance. The agency provided documentation of its use of recusal activities for commissioners, APA liaisons, and staff to demonstrate implementation of its controls against conflict of interest. However, it did not provide documentation of implementation of the controls against conflict of interest described for its site visitors or appeals panel members, nor of the signed code of conduct statements signed by commissioners and APA liaisons.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided documentation of implementation of the controls against conflict of interest described previously for its site visitors or appeals panel members. Specifically, the agency provided documentation of signed statements of impartiality by appeals panel members and the review for conflict of interest for site visitors. In addition, the agency provided documentation of the code of conduct statements signed by commissioners and APA liaisons.



602.16(a)(1)(i) Student Achievement

(a) The agency must demonstrate that it has standards for accreditation, and preaccreditation, if offered, that are sufficiently rigorous to ensure that the agency is a reliable authority regarding the quality of the education or training provided by the institutions or programs it accredits. The agency meets this requirement if -

(1) The agency's accreditation standards effectively address the quality of the institution or program in the following areas:
(i) Success with respect to student achievement in relation to the institution's mission, which may include different standards for different institutions or programs, as established by the institution, including, as appropriate, consideration of course completion, State licensing examination, and job placement rates.


The agency's standards for student achievement are found within Domain F of the Guidelines and Principles and its Implementing Regulations. The agency applies the requirements to each program and requires the program to address how effectively it meets its objectives and goals in its self-study. The agency expects the program to include in the self-study detailed information and documentation of its self-assessment processes, to include a description of on-going evaluation of student achievement and the use of the results of the evaluation in planning for improvement. The program must also describe its effectiveness thorough outcomes data and data related to the educational model, philosophy, goals, and objectives of the programs. With regards to outcomes data, the agency reviews attrition rates and internship acceptance rates (for doctoral programs only) within the accreditation process and annual report. The agency has set a benchmark of 7% for attrition rate and 50% for internship acceptance rate. In addition, the agency requires each doctoral program to report its licensure rate, but does not set a minimum standard that must be met in that area. The agency did not provide any information regarding the decision to not set a minimum standard with regard to the review of licensure rates. Although the agency reviews certain specific outcome rates in consideration of success in student achievement, the agency has not described how it determined that these rates are sufficiently rigorous to ensure that the agency is a reliable authority regarding the education provided by the programs it accredits. The agency provided significant documentation to demonstrate the application of its student achievement standards. Specifically, the agency provided documentation of the comprehensive review of doctoral, internship and postdoctoral programs for accreditation and preaccreditation, as well as the guidance provided to programs in addressing student achievement in the self-study. The on-site evaluation reports demonstrate that site visitors evaluated the program's student achievement assessment processes and procedures, as well as verified the reliability of student outcomes reported. However, the documentation does not clearly indicate that the benchmarks for outcomes data for doctoral programs are enforced. For example, the sample doctoral program reported and the on-site evaluation team noted that the attrition rate was 15% (much higher than the 7% benchmark published). The commission decision letter noted that the rate was high, but concluded that the "program is consistent with the provisions of this domain," renewed the program's accreditation for seven years, and did not require any follow-up action by the program on the issue. Therefore, the agency must provide additional information and documentation on how it enforces the student achievement outcomes benchmarks that it has set.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided additional information and documentation concerning how it determined that its outcomes rates for attrition and internship acceptance are sufficiently rigorous to ensure that the agency is a reliable authority regarding the education provided by the programs it accredits. Specifically, the agency noted that the benchmarks are derived from data collected in the annual report and are recalculated every three years. The benchmarks are submitted for public comment prior to formal adoption. With regards to licensure rates, the agency state that it does not set specific benchmarks since specific educational goals and objectives differ amongst individual programs. Instead, the agency uses the licensure rates in conjunction with other review factors to determine if the program is successful in preparing students for entry-level practice. The agency also provided additional information and documentation concerning its enforcement of its student achievement standards. The agency specifically addressed the example noted in the draft staff analysis and provided documentation that the program was reviewed against the agency's attrition rate benchmark for three years - not the program-reported rate of seven years - and found in compliance. In addition, the agency described in detail the process for how the benchmarks for outcomes data for doctoral programs are reviewed and enforced, and provided documentation of such review and enforcement.



602.19(b) Monitoring

(b) The agency must demonstrate it has, and effectively applies, a set of monitoring and evaluation approaches that enables the agency to identify problems with an institution's or program's continued compliance with agency standards and that takes into account institutional or program strengths and stability. These approaches must include periodic reports, and collection and analysis of key data and indicators, identified by the agency, including, but not limited to, fiscal information and measures of student achievement, consistent with the provisions of ยง602.16(f). This provision does not require institutions or programs to provide annual reports on each specific accreditation criterion.


The agency has multiple monitoring approaches that include the annual report (ARO), interim reports, substantive changes, complaints, early site visits, and focused visits. The agency stated that all programs must submit an annual report and provided the annual report guide and data prep sheets, as well as sample letters to programs. However, the agency did not provide any completed annual report s to demonstrate that it collects on a variety of reporting areas including fiscal information and measures of student achievement. As noted in Section 602.16(a)(1)(i), the agency reviews attrition rates and internship acceptance rates (for doctoral programs only) within the annual report, as described in the agency's Implementing Regulation D.4-7(a-c) .The agency has set a benchmark of 7% for attrition rate and 50% for internship acceptance rate. In addition, the agency requires each doctoral program to report its licensure rate, but does not set a minimum standard that must be met in that area. The agency states, and provided documentation, that if a program does not meet the published rates, the agency will request additional information, which could also lead to an earlier on-site evaluation to the program. However, the documentation of an accreditation review does not clearly indicate that the benchmarks for outcomes data for doctoral programs are enforced. The sample doctoral program reported and the on-site evaluation team noted that the attrition rate was 15% (much higher than the 7% benchmark published). The commission decision letter noted that the rate was high, but concluded that the "program is consistent with the provisions of this domain," renewed the program's accreditation for seven years, and did not require any follow-up action by the program on the issue. In addition, although the agency provided template follow-up letters to programs, the agency did not provide a detailed description of its processes for reviewing the information it collects via the annual report, the manner by which it identifies significant changes, the breadth of actions it could take as the result of its monitoring efforts, nor documentation of follow-up of a program. The template letters also appear to allow for a program to fail to meet the established benchmarks for three years without any action by the agency, beyond the request of an improvement plan. Therefore, the agency must provide additional information and documentation on how it enforces the student achievement outcomes benchmarks that it has set and which are reviewed in the annual report. The agency stated that it also uses interim reports, substantive changes, complaints, early site visits, and focused visits as part of its monitoring mechanism, however the agency did not provide any documentation of the review of such approaches.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided additional information and documentation concerning its monitoring approaches. For the annual report, the agency provided the "dashboard' that agency staff use to determine if a program is not meeting the agency's standards with regards to student achievement, as well as the summary of such data. The agency also provided a detailed description of its processes for reviewing the information it collects via the annual report, the manner by which it identifies significant changes, and the breadth of actions it could take as the result of its monitoring efforts, as well as examples of letters sent to programs that did not meet the agency's student achievement thresholds. Although the letters demonstrate that the agency collects and analyzes data related to student achievement and contacts programs that fail to meet the thresholds, the letters do not demonstrate that the agency meets the enforcement timelines as required in Section 602.20(a), as a program could be noncompliant for five years with a given threshold. Therefore, the agency has not demonstrated that it has, and effectively applies, a set of monitoring and evaluation approaches that enables the agency to identify problems with a program's continued compliance with agency standards, specifically with regards to student achievement thresholds. The agency also provided a detailed description of how it uses interim reports, substantive changes, complaints, early site visits, and focused visits as part of its monitoring mechanism, as well as documentation of the review of such approaches.



602.20(a) Enforcement Timelines

(a) If the agency's review of an institution or program under any standard indicates that the institution or program is not in compliance with that standard, the agency must--

(1) Immediately initiate adverse action against the institution or program; or

(2) Require the institution or program to take appropriate action to bring itself into compliance with the agency's standards within a time period that must not exceed--

(i) Twelve months, if the program, or the longest program offered by the institution, is less than one year in length;

(ii) Eighteen months, if the program, or the longest program offered by the institution, is at least one year, but less than two years, in length; or

(iii) Two years, if the program, or the longest program offered by the institution, is at least two years in length.


This criterion requires that an agency either initiate immediate adverse action or allow a program a time period, not to exceed two years, to come into compliance with its standards and requirements, when the agency's review of a program under any standard indicates that the program is not in compliance with that standard. The agency has a written policy within its Accreditation Operating Procedures (Sections 4.2 and 4.3) and its Implementing Regulation D.4-4 that provides a time period of two years for a doctoral program and one year for a postdoctoral residency program or an internship program to return to compliance once placed on probation. What is not clear within the agency's policies and procedures is that the agency has the ability to initiate an adverse action when the agency's review of a program under any standard indicates that the program is not in compliance with that standard, or that the time period provided to return to compliance does not exceed two years. (As defined in Section 602.3, an adverse action means denial, withdrawal, suspension, revocation, or termination of accreditation or preaccreditation, or a comparable action.) Specifically, Section 4.2 of the Accreditation Operating Procedures states that the "revocation of accreditation" action would occur only after a doctoral program was placed on "accredited, on probation" status and given two years to demonstrate compliance, which would only occur after the program had been previously given an opportunity to show cause why it should not be placed on probation and a year to demonstrate compliance. Based on this policy, it appears that a doctoral program that failed to demonstrate compliance with a standard would be given a minimum of three years to demonstrate compliance, during which time the agency would be precluded in taking an adverse action, which does not meet the requirements of this section. This policy also appears to limit such decisions to the review of a program within an accreditation review, as opposed to any review when the agency were to find a program is not in compliance with a standard. The agency provided an exhibit that listed the data regarding the agency's accreditation actions taken over a five year period, to include adverse actions. However, this data does not demonstrate that the agency met the requirements of this section - to either initiate immediate adverse action or allow a program a time period, not to exceed two years, to come into compliance with its standards and requirements. The agency must provide examples of the full cycle of review (i.e. first determination that the institution was out of compliance to final action) to demonstrate that it enforces the required time period.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided additional information and documentation concerning its enforcement actions and timelines. Specifically, the agency detailed how its current policies and procedures allow for the agency to initiate an adverse action when the agency's review of a program under any standard indicates that the program is not in compliance with that standard, or that the time period provided to return to compliance does not exceed two years. The agency also provided documentation of its accreditation actions and letters for programs that needed time to return to compliance with the agency's standards. The agency stated that it has not revoked a program's accreditation under its current policies. The agency also provided information and documentation to demonstrate that its policies and procedures do not limit adverse actions to the review of a program within an accreditation review, as opposed to any review when the agency were to find a program is not in compliance with a standard. However, as noted in Section 602.19(b), the agency provided examples of letters sent to programs that did not meet the agency's student achievement thresholds; however those letters included programs that failed to meet the agency's student achievement thresholds for up to five years. Therefore, the agency has not demonstrated that it either initiates immediate adverse action or allows a program a time period, not to exceed two years, to come into compliance with its standards and requirements, when the agency's review of a program under any standard indicates that the program is not in compliance with that standard.



602.20(b) Enforcement Action

(b) If the institution or program does not bring itself into compliance within the specified period, the agency must take immediate adverse action unless the agency, for good cause, extends the period for achieving compliance.


The agency has a written policy within Section 4.3 of its Accreditation Operating Procedures that meets the requirements of this section, and that makes clear the circumstances under which a good cause extension would be granted. Specifically, the policy states that a program on probation must have made significant progress on most of the probation issues, but demonstrated the need of additional time to implement changes, in order for a good cause extension to be granted. The policy limits a good cause extension to one year. Although the agency stated that it has revoked the accreditation of one program in the current review cycle, it did not provide documentation of the revocation, nor did it provide any information or documentation about any program that was granted an extension for good cause.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided additional information and documentation concerning an adverse action taken by the agency. Specifically, the agency stated that the revocation action was taken under prior policies and procedures that were not compliant with the Secretary's Criteria for Recognition and were revised. The agency provided documentation of the adverse action. In addition, the agency provided information and documentation of its use of its good cause extension policy.



602.27(a)(6-7),(b) Fraud and Abuse

(a)(6) The name of any institution or program it accredits that the agency has reason to believe is failing to meet its Title IV, HEA program responsibilities or is engaged in fraud or abuse, along with the agency's reasons for concern about the institution or program; and

(a)(7)If the Secretary requests, information that may bear upon an accredited or preaccredited institution's compliance with its Title IV, HEA program responsibilities, including the eligibility of the institution or program to participate in Title IV, HEA programs.

(b) If an agency has a policy regarding notification to an institution or program of contact with the Department in accordance with paragraph (a)(6) or (a)(7) of this section, it must provide for a case by case review of the circumstances surrounding the contact, and the need for the confidentiality of that contact. Upon a specific request by the Department, the agency must consider that contact confidential.


The agency must provide a response to this section as it is required for this review. If there have been no changes to the policies or practices since the last review by NACIQI, then the agency must provide such an attestation.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency provided its policies to address this section.



Part III: Third Party Comments


Staff Analysis of 3rd Party Written Comments

Over 50 written third-party comments were received regarding this agency. All of the comments reflected negative views regarding the agency, and the commenters all appear to be psychologists in the field or training to be such. There was also one comment from a State professional organization. Overall, the commenters did not tie their areas of alleged noncompliance to specific sections of the Secretary’s Criteria for Recognition. Most of the comments are related and from individuals in support of the California Institute of Integral Studies (CIIS) in San Francisco, whose accreditation was terminated by APA in August 2012. Based on the information provided, it appears that the comments are actually complaints against APA based on its treatment of this program. It is unclear based on the comments if the complainants utilized the agency's complaint policy and procedures prior to submitting these comments to the Department. The Department does not typically commence a review of an agency unless and until a complainant exhausts the agency’s published complaint procedures. In addition, it is not clear that the issues raised in the comments would indicate noncompliance with the Secretary's Criteria for Recognition by the agency. No matter, the agency may wish to respond to the comments related to its accreditation and termination of CIIS in its response to the draft staff analysis. Many other comments are related to APA-accredited internship outcomes requirements for APA-accredited programs, and the use of APA accreditation by States and Federal agencies (such as the VA) for the employment of psychologists. With regard to the use of internship outcomes in the review of programs, Federal regulations do not specify how an agency evaluates a program for success with respect to student achievement. The agency determines what outcomes to evaluate to demonstrate the quality of the programs that it accredits. As a voluntary membership organization, the agency makes its own standards, as approved by its members. In addition, the HEA allows for the agency to adopt standards not provided for in the Federal law. The Department has noted in this analysis that it has questions related to its review of student achievement in Section 602.16(a)(1)(i). With regard to compliance with State laws or the hiring requirements of Federal agencies, it is not the role of the Department to interpret State laws or enforce them, nor does the Department dictate Federal hiring policies. The Department cannot find an agency out of compliance with respect to issues not required by Federal law, and this issue is therefore outside the scope of the review for continued recognition for the agency. Another area raised relates to the factors used by the agency to determine "effective therapy " in the approval of graduate programs. The reasons for disagreement appear to be linked to theoretical/philosophical differences in the approach to graduate education in psychology. As stated previously, the agency makes its own standards, as approved by its members.

Agency Response to 3rd Party Comments

The CoA relies heavily upon feedback and input from its publics. Upon review of the third-party comment submitted, the CoA would like to address the comments related to the internship requirement; accreditation and termination of the Clinical PsyD program at the California Institute of Integral Studies (CIIS); licensure and employment; and independence of CoA. Internship Requirement Approximately 59% of the third-party comment noted concern about the CoA’s internship requirement. Domain B.4 of the Guidelines and Principles for Accreditation (G&P) stats that eligibility for accreditation by the CoA requires, “completion of an internship prior to awarding the doctoral degree.” Consistent with IR C-31(c) (Exhibit 119, p. 93), the internship experience must be a formal, organized experience. The CoA does not require that all students in an accredited doctoral program complete an APA accredited internship. Rather, if students complete non-APA or CPA-accredited internship programs, the doctoral program must demonstrate how it ensures the quality of the internship training experience including its process for monitoring the quality of internship training. The detail provided must be sufficient to demonstrate the adequacy and quality of these training experiences. Accreditation and Termination of CIIS It is important to note that when the CoA revoked the accreditation status of the Clinical PsyD program at CIIS it was operating under different Accreditation Operating Procedures (AOP). The procedures in place at that time are located in Exhibit 116, pp. 16-19. The CoA adopted new procedures in 2012 to ensure adherence to the Secretary’s criteria for length of time to revocation. The procedures went into effect for all periodic reviews that began in 2012. Approximately 55% of the comment submitted referenced the revocation of the Clinical PsyD program at CIIS. The program was initially accredited effective March 3, 2003 and was scheduled for its next periodic review in 2006 with a self-study due September 1, 2005. After being deferred for information and having a special site visit and then being placed on show cause, the CoA voted to place the program on “accredited, on probation status” at the 2008 Spring Program Review Meeting. Under Section 4.4 of the old AOP, a program placed on probation, the interval for the next review was no less than 1 year and no more than 2 years (Exhibit 116, p. 14). The program’s next self-study was scheduled for 2010. The program was then show caused prior to revocation in 2011. The CoA voted to revoke the program’s accreditation in 2011. The program elected to appeal the CoA’s decision and the subsequent appeal hearing took place in July 2012. The appeal panel voted to uphold the CoA’s decision to revoke the program’s accreditation. Consistent with Section 4.6 of the old AOP (Exhibit 116, p. 18), the original CoA decision took effect 30 days after the appeal panel hearing date and the program’s accreditation status was revoked effective September 2, 2012. A complete overview of the timeline to revocation is located in Exhibit 112. The CoA adhered to the due process procedures outlined in the AOP which was publically available on the agency’s website. Neither the APA nor the CoA received complaints about the accreditation decision making and appeals process. The public was notified that the program was placed on “accredited, on probation” status through the annual listing of accredited programs in the American Psychologist, the agency’s website, and the program’s website beginning in 2007. Independence of CoA Approximately 53% of the third-party comment noted concern about the degree to which CoA is independent from the American Psychological Association (APA) and/or the Association of Psychology Postdoctoral and Internship Centers (APPIC). The CoA is housed with the APA. While a part of the APA it maintains a firewall from the membership organization. Programs, both accredited and those seeking accreditation, and members of the CoA do not need to be members of APA to participate in the accreditation process. Accreditation staff are located in a section of the building that requires an extra level of security to access. The CoA has policies and procedures to ensure that all accreditation decisions are not influenced, either directly or indirectly, by the APA (Exhibit 119, pp. 8-13). As both APA and APPIC are comprised of members of the profession, 4 seats on the CoA represent nominations from various boards and committees of APA (Board of Educational Affairs, Board of Professional Affairs, Board for the Advancement of Psychology in the Public Interest, and the American Psychological Association of Graduate Students) and up to 4 seats representing nominations from APPIC. It is important to note that APPIC and the APPIC Match process are not a part of the APA. Licensure and Employment Approximately 45% of the third-party comment mentioned the CoA’s role in and impact on licensure and/or employment. The CoA’s accreditation process is intended to promote consistent quality and excellence in education and training in health service psychology. The CoA’s function does not include establishing requirements for licensure or employment. Each state is responsible for defining its own requirements for licensure in psychology. Employers also retain the right to establish their own requirements to meet the needs of their organization. When drafting new or revised policy the CoA values the opinion of both the state licensing boards, the Association of State and Provincial Psychology Boards (ASPPB), and employers. The CoA does not dictate policy for these groups and these groups do not dictate policy for the CoA. The corresponding appendices are attached.

Staff Analysis of Agency Response to 3rd Party Comments

In response to the draft staff analysis, the agency provided a response to the third-party comments. That response included the attachment of the agency's current policies and regulations, also submitted in many other sections of the petition. In addition, the agency provided additional information and documentation of the revocation of accreditation of CIIS, to include the operating procedures in effect at the time of revocation.