Puerto Rico State Agency for the Approval of Public Postsecondary Vocational, Technical Institutions and Programs 05/22/2018 04/16/2018 Final Review Renewal Petition

U.S. Department of Education

Staff Report

to the

Senior Department Offical

on

Recognition Compliance Issues


Recommendation Page



  1. Puerto Rico State Agency for the Approval of Public Postsecondary Vocational, Technical Institutions and Programs
  2. Renewal Petition
  3. Recognition as the state agency for the approval of public postsecondary vocational technical education in the Commonwealth of Puerto Rico.
  4. May 22, 2018
  5. Defer the agency's recognition for one year and for such additional six month increments as the agency continues to demonstrate to the Staff that due to the continuing impact of Hurricane Maria it remains unable to submit complete documentation which has been fully translated into English.
  6. It does not appear that the agency meets the following sections of the Secretary’s Criteria for Recognition. These issues are summarized below and discussed in detail under the Summary of Findings section. -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide a full set of multiple examples of documentation of a completed semi-annual report. Without all of the information in the semi-annual report, Department staff is unable to verify the information needed for this section. [§603.24(a)(1)(iii)] -- The agency must provide a site visit report that provides a roster of the members, to demonstrate its composition of members. [§603.24(a)(2)(iii)(A)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide documentation of all of the statuses granted by the agency. Without all of the information, Department staff is unable to verify the information needed for this section. [§603.24(a)(3)(i)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide the entire team report. Without all of the information in the report, Department staff is unable to verify the information needed for this section. [§603.24(a)(3)(ii)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide the entire team leader report in English. Without all of the information in the team report, Department staff is unable to verify the information needed for this section. [§603.24(a)(3)(ii)(B)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide a full set of multiple examples of documentation of a completed semi-annual report. Without all of the information in the semi-annual report, Department staff is unable to verify the information needed for this section. [§603.24(b)(1)(iv)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide documentation (in English) of its examples for innovation and experimental learning. Without all of the information, Department staff is unable to verify the information needed for this section. [§603.24(b)(1)(v)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide an entire team report. Without all of the information in the report, Department staff is unable to verify the information needed for this section. [§603.24(b)(1)(vi)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide details about its substantive change policy. Without all of the information about the policy and how it has been requested, reviewed, and approved by the agency, Department staff is unable to verify the information needed for this section. [§603.24(b)(1)(xi)] -- The agency must provide additional documentation demonstrating how it confers regularly with counterpart agencies that have similar responsibilities methods and techniques. [§603.24(b)(1)(xii)] -- The agency must provide an agenda that demonstrates that all of the parties in this section have been included in the onsite visit team's review. [§603.24(b)(2)(i)] -- The agency must demonstrate that it has provided a written report of strengths, areas of improvement, and/or areas of noncompliance to the institution. [§603.24(b)(2)(ii)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide a full set of multiple examples of documentation of a completed semi-annual report. Without all of the information in the semi-annual report, Department staff is unable to verify the information needed for this section. [§603.24(c)(1)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide a full set of multiple examples of documentation of a completed semi-annual report. Without all of the information in the semi-annual report, Department staff is unable to verify the information needed for this section. [§603.24(c)(2)] -- Due to the extenuating circumstances that Puerto Rico has faced, the agency was unable to provide a full set of multiple examples of documentation of a completed semi-annual report. Without all of the information in the semi-annual report, Department staff is unable to verify the information needed for this section. [§603.24(c)(4)] -- The agency must provide complete information about how its ethical standards are reviewed to ensure compliance. [§603.24(d)(1)]

Executive Summary



Part I: General Information About The Agency

The Puerto Rico State Agency for the Approval of Public Postsecondary Vocational, Technical Institutions and Programs (PRSAA) is the state agency in Puerto Rico for accrediting public postsecondary vocational education programs and institutions in the Commonwealth of Puerto Rico. The agency approves institutions and programs in public postsecondary institutions that are legally authorized to offer postsecondary programs that have been operating for a minimum of two years. These institutions and programs award both Associate Degrees and Certificates, and admit as regular students only individuals with a high school diploma or its equivalent or who are above the compulsory school age to attend high school, which in Puerto Rico is 16 years of age. According to the Accreditation Group's records, the Department has received no 3rd party comments or complaints about the agency during the current recognition cycle.

Recognition History

The PRSAA was granted authority to approve public postsecondary vocational technical education institutions and programs in Puerto Rico in 1982 by Puerto Rico's Secretary of Education. The agency was granted initial recognition as a state approval agency by the U.S. Secretary of Education in 1983 and has received continued recognition since that time. The agency's most recent petition for continued recognition was reviewed at the Spring 2014 NACIQI meeting. Following that meeting, the agency was required to submit a compliance report at the 2016 Meeting. In October 2016, the SDO issued a letter to continue the agency's recognition. The current review of the agency is for the regular scheduled review for renewal of the agency's recognition.


Part II: Summary Of Findings

603.24(a)(1)(iii) Approval of Varying Program Levels

(iii) Delineates the process by which it differentiates among and approves programs of varying levels.


The agency's Operational Manual states that the PRSAA has oversight on program length, specifically the agency is responsible for the review of the length of an institution's certificate level and associate degree programs. The agency has established that its credit hour policy, for certificate programs, matches the US Department of Education definitions for time in the classroom. The Operations Manual further explains how the agency utilizes clock hours and that it means one credit equals 37.5 clock hours. The agency has developed the semi-annual report which captures information collecting the existence and correct implementation of credit hours or clock hour policy. However, has not included documentation of multiple examples of a completed annual report.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided multiple examples of documentation (semi-annual reports). However, the semi-annual reports submitted as evidence are not complete. In the first example (exhibit 51b), the agency does include the number of contact hours for the programs, but does not specify the number of credits. The report is not explicit as to whether credits hours are applicable to this program. Specifically, (see exhibit 51c), there seems to be no graduates from the programs which may be why some of the sections were not complete. However, it was unclear whether the incompleteness was due to information not being reported, timing of when the report information was collected, or some other reason. The statement “Emergency Mary Hurricane” is written on the cover page of the semiannual report (exhibit 51c). Department staff assumes that the report is incomplete due to the lasting effects of the hurricane (cancelled classes due to lack of power). In addition, the semiannual report (see exhibit 51b) also appears to have missing information. Example, the report does include the number of contact hours for a program, but does not specify the number of credit hours or graduates. Based on the information provided by the agency, there were a number of unanswered question about the status of the semiannual reports. Department staff contacted the agency to get further clarity about the questions/concerns previously mentioned. It was explained by the agency that these semiannual reports contain information for the first half of the year, which in this case goes through February. The second half of the report would be completed in June and would fill in the rest of the information not currently collected in the reports. However, due to the emergency status of the schools, many of these schools were unable to complete the second report. Further, the agency has been dealing with aftermath of the hurricanes, which (as mentioned previously) has included numerous power outages. This has also prevented the complete translation of the reports and other documentation throughout the agencies petition for recognition. The information provided does not give multiple examples of documentation of a completed semi-annual report as requested in the draft staff analysis.



603.24(a)(2)(ii) Fiscal Capacity

(ii) Receives adequate and timely financial support, as shown by its appropriations, to carry out its operations;


The agency has provided a narrative description of its financial support and has highlighted the dependency of the Puerto Rico Department of Education for financial support. Specifically included was information regarding compensation of onsite visitors for expenses. However, the narrative also states that due to the economic situation and executive order by the Governor, these individuals are not to be compensated. The agency must explain how it is conducting site visits with qualified on site visitors if compensation is not a possibility, and that site visits are scheduled.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation of its 2017-2018 approved budget. The agency has also explained it would recruit and use voluntary evaluators to assist with the onsite review process. While not provided specifically in this section, onsite visit reports are provided in another section of the agency's submission. The inclusion of this information resolves the concerns raised in the draft staff petition.



603.24(a)(2)(iii)(A) Site Team Representatives

(A) to participate on visiting teams,


The agency has provided documentation of the memo that it uses to solicit site visit reviewers, as well as the selection criteria for those peer reviewers. A list of peer reviewers was included as well. The narrative explains that assignments are made based on a non-discriminatory practice that determines team consultants solely on availability and affidavits of no conflicts of interest. The agency has provided documentation that shows both the training that the reviewers must go through, as well as example of the reports that are written by the team. The agency explains that they select reviewers based on expertise and skill sets. However, it is not clear how this information is verified and how it ensures consistencies across various programs.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has explained that it verifies the skill sets for each evaluator by reviewing their resume and by an oral interview. The agency has also included documentation that requires site-teams includes program area representation, general education, and administrative evaluators. The summary of needed evaluators has been documented by a number count in the draft petition (exhibit 19). The agency's Operational manual states (section 2.3) that the evaluation committee is a body established by a group of volunteer professionals of great competence in the educational, occupational and technical areas, selected by the SAA. While it appears as though the agency has been consistent in stating that it needs to include one member from each of the three areas outlined in the agency (program area representation, general education, and administrative evaluators) the agency did not provide a site visit report that demonstrates it adheres to the composition of the site team as described as needed in exhibit 19.



603.24(a)(2)(iii)(B) Consultant Representatives

(B) to engage in consultative services for the evaluation and approval process, and


In response to this section, the agency has provided documentation regarding its Advisory Commission and its Board of Examiners. While this information is relevant to ensuring quality personnel including visiting teams and decision making bodies, the information in this section is pertinent to those that consult for the agency. The agency has provided documentation of its training of the Board of Visitors and the Advisory Commission as well. The agency however has not explained if they utilize consultative services for the evaluation and approval processes of the agency. If the agency does utilize these individuals, it must indicate so and also provide documentation of the training of these individuals.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has explained that it does not utilize consultative services for the evaluation and approval processes of the agency.



603.24(a)(3)(i) Approval Status and Written Procedures

(i) Maintains clear definitions of approval status and has developed written procedures for granting, reaffirming, revoking, denying, and reinstating approval status:


The agency has provided documentation for the policies, procedures and standards it follows for registration and re-registration of public postsecondary vocational and technical institutions and programs. The agency has also explained that a school may be approved (granted), revoked, denied, or reinstated to the approval process. The agency has included documentation examples of when it uses the various statuses that it may apply to the school. However, while the agency has explained its process, Department staff is unable to fully review all of the documentation as it has not yet be translated to English.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has included documentation examples of when it uses the various approval statuses that it may apply to a school. In exhibit 56b, an example of a schools status that has been extended for 18 months has been provided. In exhibit 56a, an example of an institution not being awarded accreditation (zero years) has been provided. The agency has explained to Department staff that they have not had to grant an revocation or reinstatement during this cycle of review. However, the additional documentation of the programs was included in Spanish. The agency must provide documentation of all of the statuses in English. Department Staff was unable to verify the status of the other programs that are written in Spanish. While the inclusion of some information demonstrates that efforts have been made to translate the decision letters about the programs and staff is sympathetic to the ongoing challenges that the country/agency has faced, the report provided is incomplete. Department staff cannot verify the contents of information that has not been translated. Further consultation with the agency has provided Department staff with insights about some of the challenges that they have been dealing with since aftermath of the hurricanes, which has included numerous power outages. The power outages have prevented the agency from being able to fully complete the translation of the reports.



603.24(a)(3)(ii) Self-Analysis and On-Site Review Requirements

(ii) Requires, as an integral part of the approval and re-approval process, institutional or program self-analysis and on-site reviews by visiting teams, and provides written and consultative guidance to institutions or programs and visiting teams.


The agency's Operational Manual stipulates the standards that a program must meet for registration and re-registration which are verified through a program self-analysis and an on-site review. The agency has provided documentation of a team leader report, as well as documentation of its onsite visit agendas, examples of the self study, and corrective action plans demonstrating how it applies these standards. Requirements from the agency include involving educators, practitioners, and administrators in the onsite review process, as well as ensuring that these individuals are appropriately trained for the roles in which they serve. Not all of the documentation provided by the agency is in English. This must be done in order to allow for validation of information presented.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation of the self-study and the onsite review checklists (in English) that are completed by the onsite reviewers. In addition, the agency has provided copies of the annual training that occurs. While the inclusion of some information demonstrates that efforts have been made to translate the team leader report and staff is sympathetic to the ongoing challenges that the country/agency has faced, the team leader report provided is incomplete. Department staff cannot verify the contents of information that has not been provided. Further consultation with the agency has provided Department staff with insights about some of the challenges that they have been dealing with since aftermath of the hurricanes, which has included numerous power outages. The power outages have prevented the agency from being able to fully complete the translation of the reports.



603.24(a)(3)(ii)(A) Self-Analysis Requirements

(A) Self-analysis shall be a qualitative assessment of the strengths and limitations of the instructional program, including the achievement of institutional or program objectives, and should involve a representative portion of the institution's administrative staff, teaching faculty, students, governing body, and other appropriate constituencies.


The agency documentation demonstrates that there are clear procedures for the gathering data, writing reports, and creation of the self-study that adhere to agency standards and the guidelines under this criteria. The agency provided a self-study template to demonstrate how it applies its requirements for an in-depth self-analysis. However, without the information from a completed self-study it is not clear how the information is collected and reviewed.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation of the self-study and the onsite review checklists that are completed by the onsite reviewers. The inclusion of this information resolves the concerns raised in the draft staff analysis.



603.24(a)(3)(ii)(B) On-Site Visit Requirements

(B) The visiting team, which includes qualified examiners other than agency staff, reviews instructional content, methods and resources, administrative management, student services, and facilities. It prepares written reports and recommendations for use by the State agency.


The agency provided a copy of a team leader report that demonstrates what it reviews during an on-site visit. The information provided in the narrative explains the inclusion of reviewers of multiple perspectives in this process. The template provides a clear picture of what is reviewed during the on-site visit. However, it is not clear whether the agency consistently reviews all of the items identified in the template for the on-site visit review. In addition, the provided documentation does not identify the individual examiners; therefore, Department staff cannot determine whether agency staff is participating on the site team which is not allowed by this criterion. The sample onsite visit agenda submitted by the agency is in Spanish.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation of onsite visit agenda and the one site review checklists that are completed by the onsite reviewers. In addition, the agency has provided a copy of the evaluators team leader report. It is noted by the agency that it was unable to provide a complete team leader report due to the ongoing electrical outages that Puerto Rico has been facing at the time of submission. While the inclusion of some information demonstrates that efforts have been made to translate the team leader report and staff is sympathetic to the ongoing challenges that the country/agency has faced, the report provided is incomplete. Department staff cannot verify the contents of information that has not been provided. Further consultation with the agency has provided Department staff with insights about some of the challenges that they have been dealing with since aftermath of the hurricanes, which has included numerous power outages. The power outages have prevented the agency from being able to fully complete the translation of the reports.



603.24(a)(3)(iii) Reevaluation

(iii) Reevaluates at reasonable and regularly scheduled intervals institutions or programs which it has approved.


The agency provided documentation to demonstrate that it has policies and procedures (exhibit 1, Operations Manual) in place for the reevaluation of its programs for continued accreditation. The agency's narrative identifies that it visits its institutions once a semester to reevaluate its programs on a regular and consistent basis. The agency provided its calendar for tracking site visits which indicates the status of each of its institutions or programs. However, the agency also has not provided sample site visit reports for an institution or program that demonstrates that the site was visited in accord with the agency's policy.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided sample semester follow up reports to demonstrate the onsite visit review of a program. The agency has further provided a copy of the template checklist that the agency would complete during the time of the visit and a template letter that it would use to notify the institution of the site visit to be conducted. The inclusion of this information resolves the concerns mentioned in the draft staff analysis.



603.24(b)(1)(i) Has an advisory body

(i) Has an advisory body which provides for representation from public employment services and employers, employees, postsecondary vocational educators, students, and the general public, including minority groups. Among its functions, this structure provides counsel to the State agency relating to the development of standards, operating procedures and policy, and interprets the educational needs and manpower projections of the State's public postsecondary vocational education system;


The agency has provided its operational manual that outlines its advisory commission responsibilities that includes the terms of its members, the function that the commission serves, and makes it clear that commission members should not have a conflict of interest. It has also included copies of the membership roster, that include employers, educators, students, and public members. However, it is not clear from the documentation when the advisory commission last met. The agency has not included copies of meeting minutes, information discussed, and/or an agenda from its last meeting.


Analyst Remarks to Response:

In response to the draft staff analysis, The agency has provided copies of meeting minutes, information discussed, and an agenda from its last meeting. The information provided demonstrates that the advisory board has involvement in reviewing the educational needs of the State's public postsecondary vocational education system. The inclusion of the additional information resolves the concerns mentioned in the draft staff analysis.



603.24(b)(1)(ii) Demonstrate meaningful contribution by the advisory body

(ii) Demonstrates that the advisory body makes a real and meaningful contribution to the approval process;


The agency has included documentation of the semi-annual report (exhibit 10) that each institution submits, which that the advisory commission suggested to change from reporting annually to reporting semiannually. While it has been explained that this was the process, there is not documentation such as meeting minutes that demonstrate that this suggestion came from the Advisory Commission. Further, the report that was included was blank; therefore, Department staff is not able to verify the information collected through the report. The narrative provided by the agency also explains that the Advisory Commission had a role in review and updating of the agency's handbooks. Similarly, no meeting minutes were provided that demonstrates the agency's review of this information.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation of its meeting minutes that demonstrates the suggestions for the operational manual updates came from the Advisory Commission (such as changing from annual to semiannual reports). Similarly, meeting minutes were provided that demonstrate the Advisory Commission's review and influence on decisions that impact the larger PRSAA organization. The inclusion of this information resolves the concerns raised in the draft petition.



603.24(b)(1)(iii) Revision of Standards

(iii) Provides advance public notice of proposed or revised standards or regulations through its regular channels of communications, supplemented, if necessary, with direct communication to inform interested members of the affected community. In addition, it provides such persons the opportunity to comment on the standards or regulations prior to their adoption;


The Advisory Commission leads the evaluation of standards for the agency. It is the Advisory Commission’s policy to provide 30 days notice of proposed or revised standards to all persons, institutions, and organizations that would be affected and provide an opportunity for comment. While it appears that this is policy of the agency, it has not provided documentation demonstrating any changes or that it has given notice and followed the 30 day time frame prescribed in its operation manual.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation of information that it has shared publicly in its local newspapers about the agency. The information published in 2018 appears to document the institutions and programs that it accredits. The information published in 2015 demonstrates that the agency has provided the public information about the PRSAA including its function and operations. In addition, the notice makes clear that comments about the PRSAA’s Rules and Procedures Handbook, Advisory Committee By-laws, and Board of Examiners By-laws are welcome. The agency states that they have not had any other changes since 2015, so this was the last notice that they provided. The 2015 announcement also captures the 30 day requirement that the agency's standards require. The inclusion of this information resolves the concerns mentioned in the draft staff analysis.



603.24(b)(1)(iv) Program Evaluation

(iv) Secures sufficient qualitative information regarding the applicant institution or program to enable the institution or program to demonstrate that it has an ongoing program of evaluation of outputs consistent with its educational goals.


The agency has information in its narrative that each institution must submit an annual report for assurance of continued compliance. The agency has provided documentation of its calendar stating that this information is reviewed and an example of a cover letter from a report that it has collected. However, the agency has not provided a copy of a completed annual report to verify the collection of this information.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided multiple examples of documentation (semi-annual reports). However, the semi-annual reports submitted as evidence are not complete. In the first example (exhibit 51b), the agency does include the number of contact hours for the programs, but does not specify the number of credits. The report is not explicit as to whether credits hours are applicable to this program. Specifically, (see exhibit 51c), there seems to be no graduates from the programs which may be why some of the sections were not complete. However, it was unclear whether the incompleteness was due to information not being reported, timing of when the report information was collected, or some other reason. The statement “Emergency Mary Hurricane” is written on the cover page of the semiannual report (exhibit 51c). Department staff assumes that the report is incomplete due to the lasting effects of the hurricane (cancelled classes due to lack of power). In addition, the semiannual report (see exhibit 51b) also appears to have missing information. Example, the report does include the number of contact hours for a program, but does not specify the number of credit hours or graduates. Based on the information provided by the agency, there were a number of unanswered question about the status of the semiannual reports. Department staff contacted the agency to get further clarity about the questions/concerns previously mentioned. It was explained by the agency that these semiannual reports contain information for the first half of the year, which in this case goes through February. The second half of the report would be completed in June and would fill in the rest of the information not currently collected in the reports. However, due to the emergency status of the schools, many of these schools were unable to complete the second report. Further, the agency has been dealing with aftermath of the hurricanes, which (as mentioned previously) has included numerous power outages. This has also prevented the complete translation of the reports and other documentation throughout the agencies petition for recognition. The information provided does not give multiple examples of documentation of a completed semi-annual report as requested in the draft staff analysis.



603.24(b)(1)(ix) Complaint Procedures

(ix) Has written procedures for the review of complaints pertaining to institutional or program quality as these relate to the agency's standards, and demonstrates that such procedures are adequate to provide timely treatment of such complaints in a manner fair and equitable to the complainant and to the institution or program;


The agency has provided its Operational Manual that explains the processes it utilizes for reviewing complaints. The complaint must adhere to the Agency's Ethical standards and be clear in its information about what information is in question. The response must be provided within 10 days. If it is determined to have merit, a fix must be implemented within 120 days, along with a corrective action plan. It is not clear based on the information submitted, if any complaints have been filed. The agency must provide a sample student complaint that demonstrates that it remedies such complaints in accordance with its written policies and the requirements under this section.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has indicated that they have not received any complaints.



603.24(b)(1)(v) Experimental and Innovative Programming

(v) Encourages experimental and innovative programs to the extent that these are conceived and implemented in a manner which ensures the quality and integrity of the institution or program;


The agency has included documentation of its Operational Manual that includes Section 6.10 about Motivation for Innovation. It requires all institutions to report regularly to the agency its strategic plans that include the short- and long-term plans and to establish/determine how to incorporate new strategies and practices that have been shown to be effective methods of learning. The agency has explained that one of their programs has been responsive to workforce demands by increasing its programs, but did not include specific details about these changes.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has explained in their narrative a couple of examples of innovation such as the certificate of Airplane Maintenance program at PRDE and the Personal Training program at Guayama Campus. While the agency explains the reasons for developing these programs and cites these programs as evidence of its innovative practices to assist with the development of a skilled workforce, all of the supporting documentation has not been translated into English. Since the reports are not in English, Department staff was unable to verify the content of the information provided. Upon consultation with the agency, Department staff has learned that the agency has still been dealing with aftermath of the hurricanes, which has included numerous power outages. Therefore, the translation of the reports did not occur.



603.24(b)(1)(vi) Institutional/Program Approval

(vi) Demonstrates that it approves only those institutions or programs which meet its published standards; that its standards, policies, and procedures are fairly applied; and that its evaluations are conducted and decisions are rendered under conditions that assure an impartial and objective judgment;


The agency has described the steps and provided the policy explaining how it establishes consistency related to the selection of examiners and an advisory committee to ensure that its evaluations are conducted and decisions are rendered under conditions that assure an impartial and objective judgment. While, the agency provided documentation showing that it provided a 3 year grant of accreditation to an institution, it did not provide evidence (such as a site visit report) of its application of these processes.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation demonstrating that it has granted 3 years of accreditation to an institution. However, the agency has not provide a copy of the site visit report that verifies that the information collected during the onsite visit warrants this grant of accreditation. In other sections of the petition, the agency has provided a partial team leader report that was written as part of the onsite visit. However, the information is incomplete. Based on the information provided by the agency, there were a number of unanswered question about the status of the team reports. When contacted by Department staff, the agency explained that they have been dealing with aftermath of the hurricanes, which has included numerous power outages. Therefore, the complete translation of the site reports did not occur. Without a complete report, Department staff is unable to verify the contents needed for this section of the report.



603.24(b)(1)(x) Publicly Available Information

(x) Annually makes available to the public (A) its policies for approval, (B) reports of its operations, and (C) list of institutions or programs which it has approved;


The agency's Operational Manual and list of approved institutions and program programs are made available in a variety of formats. Information is regularly shared with stakeholders of these programs, as well as through Public Notice, though no documentation of the Public notice was provided. In addition, this information is available through the web. The agency states that the link to the registered institutions and program can be found at: de.gov.pr/aea. However, Department staff was unable to open that link and verify that the information stated is included on that page.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation of the public notice that the agency issues regarding the list of institutions that it accredits. The agency has also provided a link to the agency's website that has a copy of the operations manual on the page. Department staff has verified that the website displays this information (screen shot attached). The inclusion of this information resolves the concerns raised in the draft staff analysis.



603.24(b)(1)(xi) School/Program Changes

(xi) Requires each approved school or program to report on changes instituted to determine continue compliance with standards or regulations;


The agency states that each program or institution is required to notify the agency if there are any changes that could make an institution or program noncompliant. The agency provided documentation to show that it has policies for substantive change (exhibit 43) but no documentation was included that demonstrates how they evaluate and process and approve a request for substantive change. The agency additionally states in its narrative that it collects information twice a year in a semi-annual report and reviews the information provided for its accuracy and follows up, if needed.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has explained that it has to invoke its substantive change policy through the institution's response to the disaster response. In exhibit 74, it is explained how the institution had to adjust its schedules related to credit hours. Exhibit 77 is not translated to English and therefore Department staff was unable to verify the contents. The information provided by the agency appears to be 'in the spirit' of a substantive change, but does not provide enough details about how the change was processed by the agency. For example, the agency should further explain how this information was requested, reviewed and approved by the agency. The agency must further provide all supporting documentation in English. Upon consultation with the agency, Department staff has learned that the agency has been dealing with aftermath of the hurricanes, which has included numerous power outages. Therefore, the translation of the reports did not occur. Due to the reports being not translated, Department staff is unable to verify the contents of the report. The information provided does not give information about the substantive change policy as requested in the draft staff analysis.



603.24(b)(1)(xii) Interagency Consultation

(xii) Confers regularly with counterpart agencies that have similar responsibilities in other and neighboring States about methods and techniques that may be used to meet those responsibilities.


The agency states that it has cooperative relationships with similar state agencies including the Oklahoma Department of Career and Technology Education, the Pennsylvania State Board for Vocational Technical Education and the New York State Board of Regents. While the agency also reported that it reviewed standards and guidelines for these organizations it did not provide supporting documentation demonstrating this review.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has stated that the example of working with the other three states agencies was in regards to the ethical standards. The agency indicated that this was done through reviewing the websites of other state agencies, but there does not appear to be a systematic process requiring consultation with other state agencies. The agency indicates that as a result of a Middle State Association change to its substantive change policy that the advisory committee reviewed the Middle States policy for impacts to PRSSA. The agency has provided meeting minutes to evidence that it discussed this topic at its February 2015 meeting. The minutes indicate that it agreed with sections 1,3,4,5,6,7,8,10,11,12 which is information extracted from "Substantive Change Policy Statement from Middle State Association." However, based on the information provided it is not clear about if the discussion was in regards to strictly the policy change by Middle States or if there was any direct communication from Middle States on this topic. This section requires that the agency confers regularly with counterpart agencies that have similar responsibilities in other and neighboring States about methods and techniques that may be used to meet those responsibilities. While it appears that the agency seeks out information from other state approval agencies and accrediting agencies, there does not appear to be a systematic process requiring such consultation or copies of any correspondence between the two agencies demonstrating consultation. It is also not clear to Department staff if the agency regularly confers with other agencies as this section requires.



603.24(b)(2)(i) Due Process -- On-Site Visits

(i) Provides for adequate discussion during the on-site visit between the visiting team and the faculty, administrative staff, students, and other appropriate persons;


The agency explains in its narrative about the discussions with various personnel during an on-site visit. The agency states that it requests the institutions or programs allow for the appropriate times with appropriate parties. Documentation included shows a sample visit agenda but is written in Spanish, so Department staff is unable to verify this information and to ensure that the relevant parties are met with during the visit.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided a copy of the onsite agenda in a former section (603.24(a)(3)(ii)B) (exhibit 44) that shows the agenda while visiting the school. While it appears that site visitors met with students and with the director, it is not clear if the faculty or other administrative staff were also interviewed during this visit. Without the information on the agenda, Department staff is unable to verify if this has occurred.



603.24(b)(2)(ii) Report on Strengths, Weaknesses, and Compliance

(ii) Furnishes as a result of the evaluation visit, a written report to the institution or program commenting on areas of strength, areas needing improvement, and, when appropriate, suggesting means of improvement and including specific areas, if any, where the institution or program may not be in compliance with the agency's standards;


The agency states in its narrative that it allows 5 days for the team members to submit their reports to the team leader. Once that has been completed, the team leader creates a final report utilizing the model (exhibit 32) for each institution. While the agency narrative mentions that it has included exhibit 45 for documentation of this process, it was not included in this section and therefore Department staff was unable to verify the information collected to ensure that it contained information regarding strengths, weaknesses, and compliance.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided a copy of the evaluators team leader report (exhibit 61a) in section 603.24(a)(3)(ii)(b). In addition, the agency has provided an example letter that it would send to an institution for follow up. The letter indicates that a visit would be made to continue to improve the educational quality at the institution. However, it is not clear from either of these documents that a full written report has been provided to the institution that outlines strengths, areas that need improvement, and/or areas of noncompliance.



603.24(b)(2)(iii) Response to Report

(iii) Provides the chief executive officer of the institution program with opportunity to comment upon the written report and to file supplemental materials pertinent to the facts and conclusions in the written report of the visiting team before the agency takes action on the report;


The agency has included in its narrative a description of the process that it utilize for the site visit reporting, including an opportunity for the institution’s director to respond to the report. It is indicated that they have 15 labor days to submit a response. However, the agency did not provide its written policy, in addition the agency did not provide documentation to demonstrate the application of its written policy relative to this criterion.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation of the checkup on compliance report (exhibit 73) that the Board of Examiners and Advisory Commission utilize to keep track of the institution's status on resolving those areas that have findings. The agency has also provided a copy of the newspaper notice (exhibit 68b) that shows the agency affords the opportunity for public comment. However, the documentation provided in this section does not demonstrate the policy or the communications that have happened between the institution and the agency. However, in Criteria: 603.24(a)(3)(i) Approval Status and Written Procedures the agency has included the documentation that demonstrates how they provide an opportunity for the institution to respond to the agency's onsite report (exhibits 32b). The inclusion of this information resolves the concerns raised in the draft staff analysis.



603.24(b)(2)(v) Published Appeals Procedures

(v) Publishes rules of procedure regarding appeals;


The agency has provided a link to its website demonstrating that it publishes rules of procedure regarding appeals. For purposes of public record, the agency must provide this information in the petition for the official record. In addition, it is not clear from the information if it is provided to institutions in any other formats.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided a link to its website demonstrating that it publishes rules of procedure regarding appeals. Department staff verified that the agency does have its operational manual listed on its public website and that a screenshot has been provided in this petition. The agency further explains in its narrative that the information is also made available through CD or USB as requested. The inclusion of this information resolves the concerns raised in the draft staff analysis.



603.24(b)(2)(vii) Notifications: Appeal Decisions

(vii) Furnishes the chief executive officer of the institution or program with a written decision of the appeal body including a statement of its reasons therefore.


The agency provides its procedures for furnishing the appeal body's final determination including a statement of reasons the decision was made. However, the agency has not demonstrated application of this policy or stated that it has not had the opportunity to apply it.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided documentation demonstrating that it has made a decision to not accredit an institution. It appears at that time of that decision, the agency made an onsite visit to confirm the findings. The Advisory Commission who was serving as the appeals decision makers decided to approve for conditioned accreditation for a period of 18 months. The agency is checking compliance on the conditions of the institution through the tracking documents provided in exhibit 73. From the documentation provided, it appears as though an appeals process was provided to the institution and the actions of the Advisory Commission serving in that capacity, changed the status of the institution. While it appears that the appeals was convened in accord with the agency's policies and the institution was provided a grant of accreditation appears for 18 months. The understanding is at the end of the 18 months if conditions are not met the agency will either take an adverse action against the institution or grant an extension for good cause to allow them to demonstrate the rest of the compliance



603.24(c)(1)

(1) The State agency meets this requirement if--

(i) It reviews the institution’s--

(A) Policies and procedures for determining the credit hours, as defined in 34 CFR 600.2, that the institution awards for courses and programs; and
(B)The application of the institution’s policies and procedures to its programs and coursework; and 

(ii) Makes a reasonable determination of whether the institution’s assignment of credit hours conforms to commonly accepted practice in higher education.


The agency's standards require that the credit hour compliance meet that in the federal definition, found in 34 CFR 600.2. The agency has provided copies of the documentation explaining the requirements for the credit hour. The review of credit hour policy and all standards compliance occurs through the review of the semi-annual report collected about the institution, as well as during the onsite visit. The agency has not provided copies of a completed annual report and/or a completed site visit report demonstrating the application of this standard.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided multiple examples of documentation (semi-annual reports). However, the semi-annual reports submitted as evidence are not complete. In the first example (exhibit 51b), the agency does include the number of contact hours for the programs, but does not specify the number of credits. The report is not explicit as to whether credits hours are applicable to this program. Specifically, (see exhibit 51c), there seems to be no graduates from the programs which may be why some of the sections were not complete. However, it was unclear whether the incompleteness was due to information not being reported, timing of when the report information was collected, or some other reason. The statement “Emergency Mary Hurricane” is written on the cover page of the semiannual report (exhibit 51c). Department staff assumes that the report is incomplete due to the lasting effects of the hurricane (cancelled classes due to lack of power). In addition, the semiannual report (see exhibit 51b) also appears to have missing information. Example, the report does include the number of contact hours for a program, but does not specify the number of credit hours or graduates. Based on the information provided by the agency, there were a number of unanswered question about the status of the semiannual reports. Department staff contacted the agency to get further clarity about the questions/concerns previously mentioned. It was explained by the agency that these semiannual reports contain information for the first half of the year, which in this case goes through February. The second half of the report would be completed in June and would fill in the rest of the information not currently collected in the reports. However, due to the emergency status of the schools, many of these schools were unable to complete the second report. Further, the agency has been dealing with aftermath of the hurricanes, which (as mentioned previously) has included numerous power outages. This has also prevented the complete translation of the reports and other documentation throughout the agencies petition for recognition. The information provided does not give multiple examples of documentation of a completed semi-annual report as requested in the draft staff analysis.



603.24(c)(2) Credit Hour Review

(2) In reviewing and evaluating an institution's policies and procedures for determining credit hour assignments, a State agency may use sampling or other methods in evaluation, sufficient to comply with paragraph (c)(1)(i)(B) of this section.


The agency has established its credit hour policy, for certificate programs, which matches the US Department of Education definitions for time in the classroom. The agency has provided a copy of its Guidance to Institutions and Accrediting Agencies Regarding a Credit Hour as Defined in the Final Regulations Published on October 29, 2010. The Operations Manual and agency's narrative further explains how the agency utilizes clock hours and completes the conversion for ensuring adequate skills and knowledge are gained based on the type of program completed. While it appears that the agency has a policy, it is not clear to Department staff how this information is monitored. In the prior program length section, it is stated this information is reviewed in the semi-annual report. However, that documentation has not been provided in this section.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided multiple examples of documentation (semi-annual reports). However, the semi-annual reports submitted as evidence are not complete. In the first example (exhibit 51b), the agency does include the number of contact hours for the programs, but does not specify the number of credits. The report is not explicit as to whether credits hours are applicable to this program. Specifically, (see exhibit 51c), there seems to be no graduates from the programs which may be why some of the sections were not complete. However, it was unclear whether the incompleteness was due to information not being reported, timing of when the report information was collected, or some other reason. The statement “Emergency Mary Hurricane” is written on the cover page of the semiannual report (exhibit 51c). Department staff assumes that the report is incomplete due to the lasting effects of the hurricane (cancelled classes due to lack of power). In addition, the semiannual report (see exhibit 51b) also appears to have missing information. Example, the report does include the number of contact hours for a program, but does not specify the number of credit hours or graduates. Based on the information provided by the agency, there were a number of unanswered question about the status of the semiannual reports. Department staff contacted the agency to get further clarity about the questions/concerns previously mentioned. It was explained by the agency that these semiannual reports contain information for the first half of the year, which in this case goes through February. The second half of the report would be completed in June and would fill in the rest of the information not currently collected in the reports. However, due to the emergency status of the schools, many of these schools were unable to complete the second report. Further, the agency has been dealing with aftermath of the hurricanes, which (as mentioned previously) has included numerous power outages. This has also prevented the complete translation of the reports and other documentation throughout the agencies petition for recognition. The information provided does not give multiple examples of documentation of a completed semi-annual report as requested in the draft staff analysis.



603.24(c)(3) Actions on Deficiences

(3) The State agency must take such actions that it deems appropriate to address any deficiencies that it identifies at an institution as part of its reviews and evaluations under paragraph (c)(1)(i) and (ii) of this section, as it does in relation to other deficiencies it may identify, subject to the requirements of this part.


The agency has responded to this section stating that action would be taken for credit hour deficiencies, while the larger topic of this section is for any deficiencies identified about the institution or program. The agency explains in their narrative, the use of the semi-annual report as a way of monitoring an institution or program, but has not provided a completed report. The agency has provided a sample template for how the site team reviews administrative areas (exhibit 21) but has not provided documentation demonstrating how it broadly reviews the institution or program. The agency should also provide examples of action taken against institutions and programs after review and reports or site visits.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided a checklist (exhibit 73) noting that the institution has findings of noncompliance. The information captured in this report demonstrates the agency’s method of tracking and monitoring and institutions or programs compliance. In earlier sections of the petition, the agency explained that this documentation was created when a decision had been made to not grant accreditation, through the appeals process when that decision was changed to a conditional accreditation for 18 months. The document captures the information collected by the first onsite team evaluation, as well as the information captured during the second onsite (appeals) visit. The tracking of this information demonstrates that the agency tracks findings of noncompliance over time. The inclusion of this information resolves the concerns raised in the draft staff analysis.



603.24(c)(4) Credit Hour Notifications

(4) If, following the institutional review process under this paragraph (c), the agency finds systemic noncompliance with the agency’s policies or significant noncompliance regarding one or more programs at the institution, the agency must promptly notify the Secretary.


The agency's Operational Manual explains that non-compliance by the institution with the credit hours policy will result in a negative recommendation and an immediate correction plan. The review of credit hour policy and all standards compliance occurs through the review of the semi-annual report collected by the institution. The agency has not provided copies of a completed annual report and/or a letter demonstrating the application of this standard.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided multiple examples of documentation (semi-annual reports). However, the semi-annual reports submitted as evidence are not complete. In the first example (exhibit 51b), the agency does include the number of contact hours for the programs, but does not specify the number of credits. The report is not explicit as to whether credits hours are applicable to this program. Specifically, (see exhibit 51c), there seems to be no graduates from the programs which may be why some of the sections were not complete. However, it was unclear whether the incompleteness was due to information not being reported, timing of when the report information was collected, or some other reason. The statement “Emergency Mary Hurricane” is written on the cover page of the semiannual report (exhibit 51c). Department staff assumes that the report is incomplete due to the lasting effects of the hurricane (cancelled classes due to lack of power). In addition, the semiannual report (see exhibit 51b) also appears to have missing information. Example, the report does include the number of contact hours for a program, but does not specify the number of credit hours or graduates. Based on the information provided by the agency, there were a number of unanswered question about the status of the semiannual reports. Department staff contacted the agency to get further clarity about the questions/concerns previously mentioned. It was explained by the agency that these semiannual reports contain information for the first half of the year, which in this case goes through February. The second half of the report would be completed in June and would fill in the rest of the information not currently collected in the reports. However, due to the emergency status of the schools, many of these schools were unable to complete the second report. Further, the agency has been dealing with aftermath of the hurricanes, which (as mentioned previously) has included numerous power outages. This has also prevented the complete translation of the reports and other documentation throughout the agencies petition for recognition. The information provided does not give multiple examples of documentation of a completed semi-annual report as requested in the draft staff analysis.



603.24(d)(1) Promote set of ethical standards

(1) Promotes a well-defined set of ethical standards governing institutional or programmatic practices, including recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services;


The agency's Operational Manual requires an institution's requirements to have standards and procedures that are ethical, for recruitment, advertising, transcripts, fair and equitable students' tuition refunds and students' placement service. While the agency has written policy relative to this criterion, it has not provided documentation demonstrating the application of its policy (such as a review of an institution or program relative to each of the element required by this criterion). As noted previously there is a policy for this review in place, however no information regarding how this information is reviewed to ensure compliance was provided.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has provided copies of the checklist that the onsite reviewers utilize for reviewing an institution against the agencies standards. While the checklist does appear to be a useful tool/method to guide evaluators in their review of an institution or program for compliance with the agency’s standards, the documentation provided seems to be incomplete. In the first example (exhibit 21), there are no details about what was actually observed, only a points value. The documentation provided in exhibit 22 and exhibit 23 does appear to evidence a review; however, the documentation provided appears to be incomplete, and does not clearly explain whether the institution is compliant or not. Exhibit 32 documents the institutions response to findings that were captured by the site evaluation team. This document is not explained and Department staff is unclear if this is the institution's response to the onsite visitors reports following their report. The documentation provided in this section appears to 'be in the spirit' of reviewing the institution for ethical standards, but the sections of the report relative to this criterion are incomplete and prevents Department staff from making a compliance determination at this time.



603.24(d)(2) Maintains review in relation to ethical practices

(2) Maintains appropriate review in relation to the ethical practices of each approved institution or program.


The agency' states that it will conduct a special review of an institution's or program's requirements for recruitment, admission, fair and equitable students’ tuition refunds, and students’ placement services if concerns in regards to ethics are brought to their attention. The agency has included documentation demonstrate their review of ethical standards, however they have not provided examples of the special reports.


Analyst Remarks to Response:

In response to the draft staff analysis, the agency has explained that while they have a policy that will allow them to conduct a special review, there has not been a need to do so. The agency indicates that they have not had any complaints or finding relative to ethical standards. The inclusion of this information resolves the issues raised in the draft staff analysis.



Part III: Third Party Comments


The Department did not receive any written third-party comments regarding this agency.