Oklahoma Department of Career & Technology Education 02/07/2018 11/10/2017 Final Review Compliance Report

U.S. Department of Education

Staff Report

to the

Senior Department Offical

on

Recognition Compliance Issues


Recommendation Page



  1. Oklahoma Department of Career & Technology Education
  2. Compliance Report
  3. Petition for renewal of recognition as the state agency for the approval of public postsecondary vocational education offered at institutions in the State of Oklahoma that are not under the jurisdiction of the Oklahoma State Regents for Higher Education, including the approval of public postsecondary vocational education offered via distance education.
  4. February 07, 2018
  5. That the agency be granted an extension of its recognition, for good cause, for a period of one-year and that the agency submit a report demonstrating its compliance with the cited criteria 30 days after the expiration of the one-year period, with reconsideration of recognition status thereafter, including review of the report and appearance by the agency at a NACIQI meeting to be designated by the Department.
  6. It does not appear that the agency meets the following sections of the Secretary’s Criteria for Recognition. These issues are summarized below and discussed in detail under the Summary of Findings section. -- The agency's guidelines do not meet the requirements of this section regarding the inclusion of students and governing bodies in the development of the self-study document. [§603.24(a)(3)(ii)(A)] -- The agency did not provide evidence that schools are given the opportunity to respond to the on-site review report, as required in the agency's guidelines. [§603.24(b)(2)(iii)] -- The agency did not demonstrate that it has policies, procedures, and standards that specifically address the requirements of this section regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services. [§603.24(d)(1)]

Executive Summary



Part I: General Information About The Agency

The Oklahoma Board of Career and Technology Education (OBCTE) is recognized for the approval of public postsecondary vocational education programs offered at institutions in the State of Oklahoma that are not under the jurisdiction of the Oklahoma State Regents for Higher Education. The OBCTE is vested with the power to govern and establish criteria and procedures for 29 technology center districts encompassing 58 campuses across the state. Approval (accreditation) by the OBCTE enables the technology centers to receive funding under Title IV, as well as under other federal programs related to vocational education.

Recognition History

The OBCTE was first recognized in 1976 and has held continued recognition since that time. The agency submitted its most recent full petition for renewed recognition for consideration at the Fall 2015 NACIQI meeting. At that time, its recognition was continued and it was requested to submit a compliance report on several issues. That compliance report is the subject of the current analysis.


Part II: Summary Of Findings

603.24(a)(1)(ii) Scope of Objectives and Activities

(ii) Clearly sets forth the scope of its objectives and activities, both as to kinds and levels of public postsecondary vocational institutions or programs covered, and the kinds of operations performed;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis the agency was requested to clarify how many institutions the agency currently accredits, whether all of those institutions appear on its five-year review calendar, and which of the institutions are being reviewed under the agency's traditional review model, as opposed to its Baldrige pilot model. In response to the concerns raised in the draft staff analysis, the agency states that all of its 29 technology centers remain on a five year review cycle. The 2013-2014 academic year was the last year that all schools were evaluated under the agency's old (i.e., 1976) review model. Since that time, some schools have been participating in the Baldrige pilot model, and some schools remained under the 1976 model. At this point, all schools are transitioning into the agency's new review model. The agency provided documentation regarding its review calendar (Ex. 1) and information about when its centers have been reviewed (Ex. 2). ED staff accepts the agency's additional narrative and supporting documentation, and no additional information is requested.



603.24(a)(3)(ii)(A) Self-Analysis Requirements

(A) Self-analysis shall be a qualitative assessment of the strengths and limitations of the instructional program, including the achievement of institutional or program objectives, and should involve a representative portion of the institution's administrative staff, teaching faculty, students, governing body, and other appropriate constituencies.


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to demonstrate that it has revised its Accreditation Guidelines to include a requirement that administrative staff, teaching faculty, students, governing bodies, and other appropriate constituencies be involved in the development of the self-study document. In its response to the concerns raised in the draft staff analysis, the agency referred ED staff to its Accreditation Guidelines, which specify that “The team may include personnel from the Oklahoma Department of Career and Technology Education, certified career and technology education teachers and/or administrators, teacher educators, business and industry representatives, and other classifications of individuals.” However, this section of the agency's guidelines refers to the team that is conducting the on-site review, not the on-campus team that conducts the institution's self-analysis and prepares the self-study prior to the agency's review. The agency's guidelines do not appear to include any requirements specifying that a representative portion of the institution's administrative staff, teaching faculty, students, governing body, and other appropriate constituencies must be involved in the self-analysis and in preparing the self study. The agency does not meet the requirements of this section. The agency's policies have not been corrected to address the requirements of this section, as was requested at the time of the Fall 2015 staff analysis.



603.24(b)(1)(iii) Revision of Standards

(iii) Provides advance public notice of proposed or revised standards or regulations through its regular channels of communications, supplemented, if necessary, with direct communication to inform interested members of the affected community. In addition, it provides such persons the opportunity to comment on the standards or regulations prior to their adoption;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide additional documentation regarding the January 2017 standards updates, including sample screenshots of the committee's online solicitation of feedback, information on whether any responses were received during the online solicitation, which constituent groups were represented in those responses, the blast emails to individuals and any responses that were received as a result of those emails, etc. Documentation was also requested, such as meeting minutes from the January 2017 meeting, to demonstrate how feedback was considered at the advisory committee meeting, and what changes were made to the agency's standards as a result of the feedback. In its response to the concerns raised in the draft staff analysis, the agency notes that it follows a review procedure that is specified by its state board and the legislature. The agency provided documentation (Ex. 6) to demonstrate that it held a public comment period that was advertised on its web site. However, no comments were received as a result of its solicitation. As a result, the agency followed its regular process, which an advisory committee reviewed the agency's standards and made suggestions for revisions, which were then accepted (Exs. 7, 8, 9, 10, and 11). ED staff accepts the agency's narrative and supporting documentation, and no additional information is requested.



603.24(b)(1)(iv) Program Evaluation

(iv) Secures sufficient qualitative information regarding the applicant institution or program to enable the institution or program to demonstrate that it has an ongoing program of evaluation of outputs consistent with its educational goals.


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide sample on-site review reports from at least three recent on-site reviews as documentation of its compliance with the requirements of this section. In its response to the concerns raised in the draft staff analysis, the agency provided additional samples of supporting documentation, as requested. ED staff accepts the agency's documentation, and no additional information is requested.



603.24(b)(1)(v) Experimental and Innovative Programming

(v) Encourages experimental and innovative programs to the extent that these are conceived and implemented in a manner which ensures the quality and integrity of the institution or program;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide additional information, as well as supporting documentation, regarding the adoption and implementation of the agency's new standards regarding innovation. In response to the concerns raised in the draft staff analysis, the agency reports that it receives funding from a state lottery fund and that those funds are dedicated to a competitive grant program to which schools are encouraged to apply in order to fund innovative “cutting edge” hardware and software, curriculum materials, machinery and equipment for new technologies, and/or instructor training. The agency provided supporting documentation regarding the lottery, the grant program, and the grant awards process (Exs. 33, 34, 35, 36, 37, 38, and 39). ED staff accepts the agency's narrative and supporting documentation, and no additional information is requested.



603.24(b)(1)(vi) Institutional/Program Approval

(vi) Demonstrates that it approves only those institutions or programs which meet its published standards; that its standards, policies, and procedures are fairly applied; and that its evaluations are conducted and decisions are rendered under conditions that assure an impartial and objective judgment;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the was requested to provide supporting documentation demonstrating the steps of the agency's approval process. In response to the concerns raised in the draft staff analysis, the agency provided information about the review process under its new review model, which is currently being implemented in all technology centers. In Year 1, the centers develop their applications and self-study documents, undergo review by the agency and peers, and receive feedback based upon on-site reviews. Centers must develop corrective action plans within 60 days of the review for any standards that are found to have not been met. Under the new review model, the center must also develop six action plans for improvement that are tied to the agency's six standards. These plans are in addition to the corrective action plans that are developed in response to the compliance issues that have been identified. In Year 3 the centers receive a midpoint monitoring visit, at which point the centers must give updates on their corrective action plans and their action plans for improvement. If the centers have addressed the issues identified in their improvement plans, they will be asked to develop an additional plan in order to foster continuous improvement. As documentation the agency provided an extensive number of documents related to its new review model for a number of different types of programs at the centers. ED staff accepts the agency's narrative and supporting documentation, and no additional information is requested.



603.24(b)(1)(vii) Review of Standards

(vii) Regularly reviews its standards, policies and procedures in order that the evaluative process shall support constructive analysis, emphasize factors of critical importance, and reflect the educational and training needs of the students;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide documentation of the steps it undertakes to obtain feedback related to the regular and ongoing review of its standards, including documentation of input from the constituencies specified in its published standards review procedures. As was noted under a previous section, the agency notes that although it published a public notice soliciting feedback in accordance with its established review process, no feedback was submitted. The agency therefore followed its normal review process whereby an advisory committee provided feedback regarding proposed changes to the agency's standards, and those changes were adopted. ED staff accepts the agency's narrative and supporting documentation, and no additional information is requested.



603.24(b)(2)(i) Due Process -- On-Site Visits

(i) Provides for adequate discussion during the on-site visit between the visiting team and the faculty, administrative staff, students, and other appropriate persons;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide documentation to demonstrate that it has revised its on-site review procedures in order to formally solicit feedback from students and other personnel during the course of the on-site review team's visit. In response to the concerns raised in the draft staff analysis, the agency notes its use of a list of standardized questions for use during the review process. On-site reviewers use these questions, but may naturally also ask additional questions based upon the circumstances of the review. The agency provided numerous copies of the standardized questions and information related to the review process that have been developed for administrators, students, support staff, and stakeholders as requested. ED staff accepts the agency's additional documentation as evidence of its compliance with the requirements of this section, and no additional documentation is requested.



603.24(b)(2)(ii) Report on Strengths, Weaknesses, and Compliance

(ii) Furnishes as a result of the evaluation visit, a written report to the institution or program commenting on areas of strength, areas needing improvement, and, when appropriate, suggesting means of improvement and including specific areas, if any, where the institution or program may not be in compliance with the agency's standards;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide supporting documentation in the form of a sample completed on-site review report. In response to the concerns raised in the draft staff analysis, the agency provided a copy of a completed on-site review report, as requested. The report, from the 206-2017 academic year, included information on the on-site review team, a summary, and the school's compliance with the agency's standards (Ex. 60). ED staff accepts the agency's supporting documentation as evidence of its compliance with the requirements of this section, and no additional information is requested.



603.24(b)(2)(iii) Response to Report

(iii) Provides the chief executive officer of the institution program with opportunity to comment upon the written report and to file supplemental materials pertinent to the facts and conclusions in the written report of the visiting team before the agency takes action on the report;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide supporting documentation demonstrating that it is implementing the requirement specified within its policies and procedures that an institution be given an opportunity to respond to the on-site review report. In response to the draft staff analysis, the agency provided additional narrative stating that schools are given the opportunity to respond to the on-site review report, but no supporting documentation demonstrating that the schools are given this opportunity was provided. The agency does not meet the requirements of this section. No documentation was provided as evidence that schools are given the opportunity to respond to the on-site review report as provided for in the agency's guidelines.



603.24(d)(1) Promote set of ethical standards

(1) Promotes a well-defined set of ethical standards governing institutional or programmatic practices, including recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services;


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide information and supporting documentation to demonstrate that the agency itself has requirements in place that are applicable to all of its accredited institutions governing institutional or programmatic practices, including recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services. In response to the concerns raised in the draft staff analysis, the agency notes that all of its technology centers are required to adhere to the Technology Center Code of Ethics Guidelines (Ex. 77). The agency provided information regarding the Code of Ethics in its narrative, and further states that it has employees in its office that can assist the technology centers regarding the ethical practices that are required under this section. Although the agency does have a code of ethics for its technology centers, the ethics do not specifically address the requirements of this section and is not student-centered. It does not appear that the agency has taken any steps to address the concerns that were raised at the time of the Fall 2015 staff analysis, namely that it demonstrate that it has formalized policies, procedures, and standards that specifically address the requirements of this section regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services. As noted in the following section, it appears that the agency includes a review of these required areas in the on-site review process, but it does not appear that the agency has formalized the requirements in its guidelines. The agency does not meet the requirements of this section. The agency has not demonstrated that it has formal policies, procedures, and standards that specifically address the requirements of this section regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services.



603.24(d)(2) Maintains review in relation to ethical practices

(2) Maintains appropriate review in relation to the ethical practices of each approved institution or program.


ASL_MVC_v1.Models.TblAslstep2Analyst??.Remarks


Analyst Remarks to Response:

In the draft staff analysis, the agency was requested to provide documentation that it conducts a review of its policies, procedures, and standards regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services, during the on-site review process. In response to the concerns raised in the draft staff analysis, the agency states that its technology centers are asked to respond to various questions related to the requirements of this section as a part of the self-study and on-site review process. The agency's review process does appear to address the areas required under this section during the course of the on-site reviews, despite its apparent lack of formal policies and standards regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services, as was noted in the previous section. As supporting documentation, the agency provided numerous examples of its review of these areas, despite its apparent lack of a formal policy addressing all of the requirements of this section. ED staff accepts the agency's narrative and supporting documentation that it is reviewing the technology centers' compliance with the areas of recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services during the on-site review process, despite its apparent lack of a formal policy requirement in these areas.



Part III: Third Party Comments


The Department did not receive any written third-party comments regarding this agency.