Accreditation and State Liason (ASL) E-Recognition Web Site US Department of Education, Promoting educational excellence for all Americans.
Skip to main content | Home | OPE Home | ASL Home | NACIQI | NCFMEA | User Guide

Back

U.S. Department of Education

Staff Report
to the
Senior Department Official
on
Recognition Compliance Issues

RECOMMENDATION PAGE

1.
Agency:   Oklahoma Department of Career & Technology Education (1976/2016)
                  (The dates provided are the date of initial listing as a recognized agency and the date of the agency’s last grant of recognition.)
 
2.
Action Item:   Compliance Report
 
3.
Scope of Recognition:   State agency for the approval of public postsecondary vocational education.
4.
Date of Advisory Committee Meeting:   February, 2018
 
5.
Staff Recommendation:   That the agency be granted an extension of its recognition, for good cause, for a period of one-year and that the agency submit a report demonstrating its compliance with the cited criteria 30 days after the expiration of the one-year period, with reconsideration of recognition status thereafter, including review of the report and appearance by the agency at a NACIQI meeting to be designated by the Department.


 
6.
Issues or Problems:   It does not appear that the agency meets the following sections of the Secretary’s Criteria for Recognition. These issues are summarized below and discussed in detail under the Summary of Findings section.

-- The agency's guidelines do not meet the requirements of this section regarding the inclusion of students and governing bodies in the development of the self-study document. [§603.24(a)(3)(ii)(A)]

-- The agency did not provide evidence that schools are given the opportunity to respond to the on-site review report, as required in the agency's guidelines. [§603.24(b)(2)(iii)]

-- The agency did not demonstrate that it has policies, procedures, and standards that specifically address the requirements of this section regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services. [§603.24(d)(1)]



EXECUTIVE SUMMARY

 
 

PART I: GENERAL INFORMATION ABOUT THE AGENCY

 
The Oklahoma Board of Career and Technology Education (OBCTE) is recognized for the approval of public postsecondary vocational education programs offered at institutions in the State of Oklahoma that are not under the jurisdiction of the Oklahoma State Regents for Higher Education. The OBCTE is vested with the power to govern and establish criteria and procedures for 29 technology center districts encompassing 58 campuses across the state. Approval (accreditation) by the OBCTE enables the technology centers to receive funding under Title IV, as well as under other federal programs related to vocational education.
 
 
Recognition History
 
The OBCTE was first recognized in 1976 and has held continued recognition since that time. The agency submitted its most recent full petition for renewed recognition for consideration at the Fall 2015 NACIQI meeting. At that time, its recognition was continued and it was requested to submit a compliance report on several issues. That compliance report is the subject of the current analysis.


PART II: SUMMARY OF FINDINGS

 
§603.24 Criteria for State agencies
The following are the criteria which the Secretary of Education will utilize in designating a State agency as a reliable authority to assess the quality of public postsecondary vocational education in its respective State.
(ii) Clearly sets forth the scope of its objectives and activities, both as to kinds and levels of public postsecondary vocational institutions or programs covered, and the kinds of operations performed;

 
In the Fall 2015 final staff analysis, the agency was requested to provide additional information as to whether and how technology centers that are not part of the Baldrige pilot project are being reviewed, and especially whether they are being reviewed within the specified five-year review cycle.

In response to the 2015 staff analysis, the agency provided detailed information on its traditional review process, as opposed to having already provided information on its Baldrige pilot review process. The agency provided a copy of its Accreditation Guidelines, which include a detailed description of the agency's review process (Ex. 2). The agency also provided sample action plans for four of its institutions that were developed as a result of its review of those institutions. The plans specify areas in need of improvement, the persons responsible for those areas, project timelines, resources, needed professional development or technical assistance to support the improvements, measures of progress, and the expected deliverables to show that the improvements have taken place (Exs. 3A, 3B, 3C, and 3D). The agency also provided a copy of its five-year review calendar, showing when various institutions are scheduled for review (Ex. 1).

Although the agency provided information regarding its review calendar, no information was provided showing which institutions are being reviewed under the Baldrige pilot and which institutions are still subject to the agency's traditional review process. Clarification is needed as to how many institutions the agency currently accredits, whether all of those institutions appear on its five-year review calendar, and which of the institutions are being reviewed under the agency's traditional review model, as opposed to its Baldrige pilot model.

Analyst Remarks to Response:
In the draft staff analysis the agency was requested to clarify how many institutions the agency currently accredits, whether all of those institutions appear on its five-year review calendar, and which of the institutions are being reviewed under the agency's traditional review model, as opposed to its Baldrige pilot model.

In response to the concerns raised in the draft staff analysis, the agency states that all of its 29 technology centers remain on a five year review cycle. The 2013-2014 academic year was the last year that all schools were evaluated under the agency's old (i.e., 1976) review model. Since that time, some schools have been participating in the Baldrige pilot model, and some schools remained under the 1976 model. At this point, all schools are transitioning into the agency's new review model. The agency provided documentation regarding its review calendar (Ex. 1) and information about when its centers have been reviewed (Ex. 2).

ED staff accepts the agency's additional narrative and supporting documentation, and no additional information is requested.
 

(A) Self-analysis shall be a qualitative assessment of the strengths and limitations of the instructional program, including the achievement of institutional or program objectives, and should involve a representative portion of the institution's administrative staff, teaching faculty, students, governing body, and other appropriate constituencies.

 
In the Fall 2015 staff analysis, the agency was requested to revise its Accreditation Guidelines document to include the requirement that administrative staff, teaching faculty, students, governing bodies, and other appropriate constituencies be involved in the development of the self-study document.

The agency's narrative and supporting documentation do not address the previous finding. The agency is requested to demonstrate that it has revised its Accreditation Guidelines to include a requirement that administrative staff, teaching faculty, students, governing bodies, and other appropriate constituencies be involved in the development of the self-study document.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to demonstrate that it has revised its Accreditation Guidelines to include a requirement that administrative staff, teaching faculty, students, governing bodies, and other appropriate constituencies be involved in the development of the self-study document.

In its response to the concerns raised in the draft staff analysis, the agency referred ED staff to its Accreditation Guidelines, which specify that “The team may include personnel from the Oklahoma Department of Career and Technology Education, certified career and technology education teachers and/or administrators, teacher educators, business and industry representatives, and other classifications of individuals.” However, this section of the agency's guidelines refers to the team that is conducting the on-site review, not the on-campus team that conducts the institution's self-analysis and prepares the self-study prior to the agency's review. The agency's guidelines do not appear to include any requirements specifying that a representative portion of the institution's administrative staff, teaching faculty, students, governing body, and other appropriate constituencies must be involved in the self-analysis and in preparing the self study.

The agency does not meet the requirements of this section. The agency's policies have not been corrected to address the requirements of this section, as was requested at the time of the Fall 2015 staff analysis.
 

(iii) Provides advance public notice of proposed or revised standards or regulations through its regular channels of communications, supplemented, if necessary, with direct communication to inform interested members of the affected community. In addition, it provides such persons the opportunity to comment on the standards or regulations prior to their adoption;

 
In the Fall 2015 staff analysis, the agency was requested to provide additional information regarding its formal
standards review process and procedures, and especially how feedback is actively solicited regarding changes to the standards, which constituent groups provide feedback, and how proposed changes to the standards are published for public comment.

In its narrative, the agency references its Accreditation Guidelines, which specify that an accreditation advisory committee provides advice to the agency regarding the development of standards (Ex. 2, p. 12). The agency's narrative states that the advisory committee's meetings are open to the public, that the committee solicits online feedback regarding the standards for a period of at least 30 days, and that the committee also solicits feedback from potential sources via an email blast that invites comment.

The agency provided a copy of a meeting agenda that appears to show that Accreditation Guidelines updates and recommendations were considered by the agency's advisory committee at a January 2017 meeting (Ex. 8). However, no information or supporting documentation was provided regarding the process leading to any updates that were adopted at that meeting.

The agency is requested to provide additional documentation regarding the January 2017 standards updates, including sample screenshots of the committee's online solicitation of feedback, information on whether any responses were received during the online solicitation, which constituent groups were represented in those responses, the blast emails to individuals and any responses that were received as a result of those emails, etc. Documentation is also requested, such as meeting minutes from the January 2017 meeting, to demonstrate how feedback was considered at the advisory committee meeting, and what changes were made to the agency's standards as a result of the feedback.



Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide additional documentation regarding the January 2017 standards updates, including sample screenshots of the committee's online solicitation of feedback, information on whether any responses were received during the online solicitation, which constituent groups were represented in those responses, the blast emails to individuals and any responses that were received as a result of those emails, etc. Documentation was also requested, such as meeting minutes from the January 2017 meeting, to demonstrate how feedback was considered at the advisory committee meeting, and what changes were made to the agency's standards as a result of the feedback.

In its response to the concerns raised in the draft staff analysis, the agency notes that it follows a review procedure that is specified by its state board and the legislature. The agency provided documentation (Ex. 6) to demonstrate that it held a public comment period that was advertised on its web site. However, no comments were received as a result of its solicitation. As a result, the agency followed its regular process, which an advisory committee reviewed the agency's standards and made suggestions for revisions, which were then accepted (Exs. 7, 8, 9, 10, and 11).

ED staff accepts the agency's narrative and supporting documentation, and no additional information is requested.
 

(iv) Secures sufficient qualitative information regarding the applicant institution or program to enable the institution or program to demonstrate that it has an ongoing program of evaluation of outputs consistent with its educational goals.

 
In the Fall 2015 staff analysis, the agency was requested to provide samples of completed on-site review team reports as documentation of the type of review team feedback that is provided to institutions following the on-site review.

No documentation was attached to the agency's response to this section. The agency did not provide sample on-site review reports as had been requested previously.

The agency is requested to provide sample on-site review reports from at least three recent on-site reviews.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide sample on-site review reports from at least three recent on-site reviews as documentation of its compliance with the requirements of this section.

In its response to the concerns raised in the draft staff analysis, the agency provided additional samples of supporting documentation, as requested. ED staff accepts the agency's documentation, and no additional information is requested.
 

(v) Encourages experimental and innovative programs to the extent that these are conceived and implemented in a manner which ensures the quality and integrity of the institution or program;

 
In the Fall 2015 staff analysis, the agency was requested to provide additional information and documentation as to how it encourages experimental and innovative programming.

In its narrative, the agency states that one of its institutions has recently applied to be an ED-approved experimental site for competency-based education. While this is commendable, it does not demonstrate that the agency routinely encourages experimental and innovative programs or that such programs are conceived and implemented in a manner that ensures the quality and integrity of the institution or program.

The agency states that it began implementation of a model in the 2016-2017 academic year that will address innovation, but no documentation was provided regarding that model. It is unclear to ED staff whether the information provided at the opening of the agency's narrative is drawn from those standards. Additional information, as well as supporting documentation, are requested about the adoption and implementation of the agency's new standards regarding innovation.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide additional information, as well as supporting documentation, regarding the adoption and implementation of the agency's new standards regarding innovation.

In response to the concerns raised in the draft staff analysis, the agency reports that it receives funding from a state lottery fund and that those funds are dedicated to a competitive grant program to which schools are encouraged to apply in order to fund innovative “cutting edge” hardware and software, curriculum materials, machinery and equipment for new technologies, and/or instructor training. The agency provided supporting documentation regarding the lottery, the grant program, and the grant awards process (Exs. 33, 34, 35, 36, 37, 38, and 39).

ED staff accepts the agency's narrative and supporting documentation, and no additional information is requested.
 

(vi) Demonstrates that it approves only those institutions or programs which meet its published standards; that its standards, policies, and procedures are fairly applied; and that its evaluations are conducted and decisions are rendered under conditions that assure an impartial and objective judgment;

 
In the Fall 2015 staff analysis, the agency was requested to provide supporting documentation demonstrating the steps of the agency's approval process. The agency was specifically requested to provide additional information about its timelines for requiring compliance, as well as documentation demonstrating the steps of the agency's approval process, such as: an on-site review report noting areas of non-compliance with the agency's standards; the action plans developed as a result of that on-site review report; minutes of Board meetings where that on-site review report, institutional response and action plan were discussed and an approval/accrediting decision was reached; the decision letter that was sent to the institution notifying it of the Board's decision and outlining any follow-up measures that must be taken; documentation that the institution took corrective action; and subsequent action by the Board accepting or rejecting the institution's corrective measures.

The agency did not provide the supporting documentation that was requested in the Fall 2015 staff analysis. The agency is again requested to provide the specified supporting documentation related to its approval process.

Analyst Remarks to Response:
In the draft staff analysis, the was requested to provide supporting documentation demonstrating the steps of the agency's approval process.

In response to the concerns raised in the draft staff analysis, the agency provided information about the review process under its new review model, which is currently being implemented in all technology centers. In Year 1, the centers develop their applications and self-study documents, undergo review by the agency and peers, and receive feedback based upon on-site reviews. Centers must develop corrective action plans within 60 days of the review for any standards that are found to have not been met. Under the new review model, the center must also develop six action plans for improvement that are tied to the agency's six standards. These plans are in addition to the corrective action plans that are developed in response to the compliance issues that have been identified. In Year 3 the centers receive a midpoint monitoring visit, at which point the centers must give updates on their corrective action plans and their action plans for improvement. If the centers have addressed the issues identified in their improvement plans, they will be asked to develop an additional plan in order to foster continuous improvement. As documentation the agency provided an extensive number of documents related to its new review model for a number of different types of programs at the centers.

ED staff accepts the agency's narrative and supporting documentation, and no additional information is requested.
 

(vii) Regularly reviews its standards, policies and procedures in order that the evaluative process shall support constructive analysis, emphasize factors of critical importance, and reflect the educational and training needs of the students;

 
In the Fall 2015 staff analysis, the agency was requested to provide documentation of the steps it undertakes to obtain feedback related to the regular and ongoing review of its standards, including documentation of input from the constituencies specified in its published standards review procedures.

The agency did not provide the requested supporting documentation related to its standards review process, and no exhibits were attached to the agency's response. The agency is again requested to provide documentation of the steps it undertakes to obtain feedback related to the regular and ongoing review of its standards, including documentation of input from the constituencies specified in its published standards review procedures.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide documentation of the steps it undertakes to obtain feedback related to the regular and ongoing review of its standards, including documentation of input from the constituencies specified in its published standards review procedures.

As was noted under a previous section, the agency notes that although it published a public notice soliciting feedback in accordance with its established review process, no feedback was submitted. The agency therefore followed its normal review process whereby an advisory committee provided feedback regarding proposed changes to the agency's standards, and those changes were adopted.

ED staff accepts the agency's narrative and supporting documentation, and no additional information is requested.
 

(i) Provides for adequate discussion during the on-site visit between the visiting team and the faculty, administrative staff, students, and other appropriate persons;

 
In the Fall 2015 staff analysis, the agency was requested to revise its on-site review procedures in order to formally solicit feedback from students and other personnel during the course of the on-site review team's visit.

In its response, the agency states that its on-site review teams are "encouraged" to talk to students and staff during the course of the on-site review process. The agency attached a sample agenda from an on-site review that indicates that the on-site review team was scheduled to speak with students and staff during the course of the on-site review (Ex. 11). However, the agency had been requested to revise its on-site review procedures to ensure that the requirement that on-site review teams were required (not merely encouraged) to speak to students and staff during the on-site review process. While it appears possible that on-site review teams are speaking to students and staff during the course of every on-site review, the agency provided no documentation to show that it has added this requirement to its on-site review procedures as had been requested in the Fall 2015 staff analysis.

The agency is requested to provide documentation to demonstrate that it has revised its on-site review procedures in order to formally solicit feedback from students and other personnel during the course of the on-site review team's visit.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide documentation to demonstrate that it has revised its on-site review procedures in order to formally solicit feedback from students and other personnel during the course of the on-site review team's visit.

In response to the concerns raised in the draft staff analysis, the agency notes its use of a list of standardized questions for use during the review process. On-site reviewers use these questions, but may naturally also ask additional questions based upon the circumstances of the review. The agency provided numerous copies of the standardized questions and information related to the review process that have been developed for administrators, students, support staff, and stakeholders as requested.

ED staff accepts the agency's additional documentation as evidence of its compliance with the requirements of this section, and no additional documentation is requested.
 

(ii) Furnishes as a result of the evaluation visit, a written report to the institution or program commenting on areas of strength, areas needing improvement, and, when appropriate, suggesting means of improvement and including specific areas, if any, where the institution or program may not be in compliance with the agency's standards;

 
In the Fall 2015 staff analysis, the agency was requested to provide supporting documentation in the form of a sample completed on-site review report.

Although the agency described its on-site review report process, it did not attach any documentation with its response to this section. The agency is again requested to provide supporting documentation in the form of a sample completed on-site review report.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide supporting documentation in the form of a sample completed on-site review report.

In response to the concerns raised in the draft staff analysis, the agency provided a copy of a completed on-site review report, as requested. The report, from the 206-2017 academic year, included information on the on-site review team, a summary, and the school's compliance with the agency's standards (Ex. 60).

ED staff accepts the agency's supporting documentation as evidence of its compliance with the requirements of this section, and no additional information is requested.
 

(iii) Provides the chief executive officer of the institution program with opportunity to comment upon the written report and to file supplemental materials pertinent to the facts and conclusions in the written report of the visiting team before the agency takes action on the report;

 
In the Fall 2015 staff analysis, the agency was requested to provide supporting documentation demonstrating that it is implementing the requirement specified within its policies and procedures that an institution be given an opportunity to respond to the on-site review report.

Although the agency described the process whereby an institution is given an opportunity to respond to its on-site review report, the agency did not attach any supporting documentation for this section. The agency is again requested to provide supporting documentation demonstrating that it is implementing the requirement specified within its policies and procedures that an institution be given an opportunity to respond to the on-site review report.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide supporting documentation demonstrating that it is implementing the requirement specified within its policies and procedures that an institution be given an opportunity to respond to the on-site review report.

In response to the draft staff analysis, the agency provided additional narrative stating that schools are given the opportunity to respond to the on-site review report, but no supporting documentation demonstrating that the schools are given this opportunity was provided.

The agency does not meet the requirements of this section. No documentation was provided as evidence that schools are given the opportunity to respond to the on-site review report as provided for in the agency's guidelines.
 

(1) Promotes a well-defined set of ethical standards governing institutional or programmatic practices, including recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services;

 
In the Fall 2015 staff analysis, the agency was requested to describe how its policies, procedures, and standards require that its accredited institutions follow ethical practices regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services, including documentation of the agency's requirements.

As was noted in the 2015 staff analysis, it appears that the agency is narrowly interpreting the requirements of this section to be limited to aspects of non-discrimination and affirmative action, which is not its intent. The purpose of the requirements of this section extends to many other areas, such as truthful and non-predatory recruitment practices, accurate advertising, policies addressing the availability of transcripts, adherence to Title IV requirements regarding students refunds, the provision of legitimate placement services, etc.

In its response, the agency states that its institutions develop their own affirmative action standards. It also states that all institutions are required to have a code of ethics on file and that the agency itself follows the state's code of ethics. No documentation was provided in support of this section. The agency is requested to provide information and supporting documentation to demonstrate that the agency itself has requirements in place that are applicable to all of its accredited institutions governing institutional or programmatic practices, including recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide information and supporting documentation to demonstrate that the agency itself has requirements in place that are applicable to all of its accredited institutions governing institutional or programmatic practices, including recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services.

In response to the concerns raised in the draft staff analysis, the agency notes that all of its technology centers are required to adhere to the Technology Center Code of Ethics Guidelines (Ex. 77). The agency provided information regarding the Code of Ethics in its narrative, and further states that it has employees in its office that can assist the technology centers regarding the ethical practices that are required under this section.

Although the agency does have a code of ethics for its technology centers, the ethics do not specifically address the requirements of this section and is not student-centered. It does not appear that the agency has taken any steps to address the concerns that were raised at the time of the Fall 2015 staff analysis, namely that it demonstrate that it has formalized policies, procedures, and standards that specifically address the requirements of this section regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services. As noted in the following section, it appears that the agency includes a review of these required areas in the on-site review process, but it does not appear that the agency has formalized the requirements in its guidelines.

The agency does not meet the requirements of this section. The agency has not demonstrated that it has formal policies, procedures, and standards that specifically address the requirements of this section regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services.
 

(2) Maintains appropriate review in relation to the ethical practices of each approved institution or program.

 
In the Fall 2015 staff analysis, the agency was requested to describe how it verifies that its institutions are implementing its policies, procedures, and standards regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services, including documentation of this verification.

As was noted in the previous section, as well as in the Fall 2015 staff analysis, the agency has misinterpreted the requirements of this section to narrowly apply to diversity and affirmative action, which is not the intent of this section. As was noted in the previous section, the agency must develop its own standards governing ethical institutional or programmatic practices, including recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services. To satisfy the requirements of this section, the agency must also provide documentation to demonstrate that it evaluates the implementation of its standards related to ethical practices during the course of its institutional review process.

Analyst Remarks to Response:
In the draft staff analysis, the agency was requested to provide documentation that it conducts a review of its policies, procedures, and standards regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services, during the on-site review process.

In response to the concerns raised in the draft staff analysis, the agency states that its technology centers are asked to respond to various questions related to the requirements of this section as a part of the self-study and on-site review process. The agency's review process does appear to address the areas required under this section during the course of the on-site reviews, despite its apparent lack of formal policies and standards regarding recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services, as was noted in the previous section. As supporting documentation, the agency provided numerous examples of its review of these areas, despite its apparent lack of a formal policy addressing all of the requirements of this section.

ED staff accepts the agency's narrative and supporting documentation that it is reviewing the technology centers' compliance with the areas of recruitment, advertising, transcripts, fair and equitable student tuition refunds, and student placement services during the on-site review process, despite its apparent lack of a formal policy requirement in these areas.
 
 

PART III: THIRD PARTY COMMENTS

 
The Department did not receive any written third-party comments regarding this agency.