CAEP Accountability Measures

The Radford University's Educator Preparation Program (EPP) has a systematic and multifaceted approach for collecting, analyzing and synthesizing data to understand completer effectiveness and impact of P-12 student learning and growth. The Virginia Department of Education (VDOE) does not provide EPPs with completer effectiveness and impact data; therefore, the EPP uses the following multiple measures to demonstrate that completers effectively and impactfully contribute to P-12 student learning and growth and apply the professional knowledge skills and dispositions they acquired in the program.

To collect the necessary data for assessment, the EPP incorporates two measures to locate completers. First, the EPP uses the VDOE Teacher Licensure Application and Annual Instructional Personnel (IPAL) report to follow its completers into the first year of employment. This report includes completer names, year of graduation, endorsement area, and employing school division and school in the Commonwealth of Virginia. Students on this list completed the program no more than three years ago and are employed in VA. 

Second, the EPP’s Office of Field Experience (OFEX) compiles a master dataset of student information during their graduating semester, including personal email, mailing address, phone number, and place of employment, if known. The OFEX updates the master completer list to include information gleaned from the IPAL report and conducts an extensive search of the internet to locate current contact information for missing data.  

The EPP uses the master student dataset to email completers, inviting them to provide feedback on their preparation, request annual evaluation data and observation feedback by their employers and to schedule formal observations of their teaching. The EPP collects from completers the following to assess completer effectiveness and impact on P-12 student learning and growth:

Completer Effectiveness on Student Learning and Growth:

School Principal Observations of Completers

The EPP uses formative observation feedback by school principal as a measure for demonstrating completer effectiveness on student learning and growth. The EPP requests observations annually from program completers by email. Like the summative annual teacher evaluation, the formal observation feedback is aligned to the eight Virginia Uniform Performance Standards (VUPS). Per the guidelines, formal observations are a required documentation source for teacher evaluation in VA; therefore, every new teacher should be formally observed and rated on each performance standard. The rating scales and rubrics vary depending upon the school division. 

For AY 2023-24, the EPP received four principal observations from completers. Consistent with principal observations in the past, the qualitative data received for AY 2023-24 indicate that completers continue to apply the professional knowledge, skills and dispositions learned in their program, demonstrating their effectiveness. Completers were described as effective in all seven of the VUPS.

University Supervisor Observations of Completers

The EPP utilizes structured observations as evidence of completer effectiveness. To ensure representativeness of all initial licensure teacher education preparation programs, the EPP created an annual rotation schedule for University Supervisors (USs; faculty) observations of program completers. Each year, a group of USs are assigned a list of completers in their first, second or third year of teaching to observe. The USs use the same formal observation rubric used to evaluate program candidates to document completer effectiveness in instruction and classroom management. Within these two broad areas, several items are individually rated on a 4-point Likert scale: (4) exemplary; (3) proficient; (2) developing; and (1) unacceptable.  

In AY 2023-24, the EPP collected data from 10 completers. The data show completers continue to apply in P-12 classrooms the professional knowledge, skills and dispositions they learned in the program. On average, the 10 completers were rated as proficient in 9 of the 10 areas of the observation rubric. Completers were rated slightly higher in the category of instruction in which the combined mean was 3.37.  Within the instruction category, completers were rated the highest in the area of content command (3.22). Within the classroom management category, completers scored highest on establishing clear and consistent expectations for learning with an overall mean of 3.50.  

VEAC  Initial Program Employer Survey of Completers AY 2023-24: Item IC

To understand employer satisfaction with the preparation of program completers, each spring the EPP surveys employers through the annual VEAC survey. The VEAC is a collaborative among Virginia EPPs created to:

• Develop a common assessment measure for gauging employer satisfaction with EPP graduates; and
• Provide a centralized assessment process for standardizing and reducing the complexity of data collection required for accreditation and program improvement. 

The survey consists of a 12-item questionnaire in which they rate program completers on a 4-point Likert scale: (4) exemplary, (3) proficient, (2) developing/needs improvement, or (1) unacceptable. The VEAC survey items are aligned to InTASC and VUPS standards. The EPP uses item IC (effectively engages students in learning by using a variety of research based instructional strategies) to evaluate completer effectiveness. The AY 2023-2024 employer response rate was 39 percent. For AY 2023-2024 our completers received an overall mean of 3.19/proficient. The overall VEAC mean for participating institutions was 3.25/proficient.

---

Completer Impact on P-12 Student Learning and Growth:

Annual Summative Teacher Evaluation by School Supervisor

The EPP uses the annual teacher summative evaluation as a measure for demonstrating completer impact on P-12 student learning and growth. The Virginia Board of Education’s (VBOE’s) Guidelines for Uniform Performance Standards and Evaluation Criteria for Teachers provides a uniform set of performance standards for the annual summative teacher evaluations. The are a total of eight standards for all annual teacher evaluations in Virginia. Collectively, these standards are called the Virginia Uniform Performance Standards (VUPS). Standards 1 through 7 relate to a teacher’s practice whereas standard 8 focuses on the results of the teacher’s work as evidenced by student growth. On the annual summative teacher evaluation, each performance standard is rated on a 4-point scale: (4) highly effective; (3) effective; (2) approaching effective; and (1) ineffective. Effective is the expected level of performance. 

The EPP’s rationale for using the annual summative evaluation as a measure for demonstrating completer impact and effectiveness is that it is aligned with the VUPS, the only common measure used in VA for evaluating teacher performance. All VA school divisions are required by law to use these eight standards in their annual evaluation of teachers. Since most program completers are employed and evaluated by these standards in VA public schools, it is appropriate for the EPP to use this to evidence their effectiveness and impact on P-12 student learning and growth. Data reflects that completers do apply the professional knowledge, skills, and dispositions learned in the program, demonstrating their impact on student learning and growth in P-12. Completers were rated as approaching effective or effective all seven of the VUPS with a mean score of  2.98 across standards 1 through 8. 

VEAC Initial Program Employer Survey of Completers AY 2023-24: Item IG 

To understand employer satisfaction with the preparation of program completers, each spring the EPP surveys employers through the annual VEAC survey. The VEAC is a collaborative among Virginia EPPs created to:

• Develop a common assessment measure for gauging employer satisfaction with EPP graduates; and
• Provide a centralized assessment process for standardizing and reducing the complexity of data collection required for accreditation and program improvement. 

The survey consists of a 12-item questionnaire in which they rate program completers on a 4-point Likert scale: (4) exemplary, (3) proficient, (2) developing/needs improvement, or (1) unacceptable. The VEAC survey items are aligned to InTASC and VUPS standards. The EPP uses item IG (engages in practices that result in acceptable, measurable, and appropriate student academic progress) to evaluate completer impact on P-12 student learning and growth. The AY 2023-24 employer response rate was 39 percent. For AY 2023-24 our completers received an overall mean of 3.17/proficient. The overall VEAC mean of participating institutions was 3.26/proficient.

Annually, the EPP surveys the employers of initial program completers. The EPP is a member of the Virginia Education Assessment Collaborative (VEAC), a statewide collaborative of 32 EPPs in Virginia designed to provide:

• EPP employer and completer satisfaction data about their initial licensure programs;
• EPP benchmark data regarding how they compare to other statewide EPPs; and
• Our P-12 partners with a more efficient process for providing EPPs across the state feedback about their graduates. 
   

As a participating VEAC institution, we use a common employer survey for our initial licensure programs administered annually to P-12 administrators across the Commonwealth. The survey is distributed in February/March. Survey results are posted online the following fall. The surveys are aligned to the VUPS and InTASC standards. The VDOE collects current employment and contact information on completers who are working in public schools within the Commonwealth. Additional contact information is collected by the EPP and submitted to VEAC to conduct the survey. The survey is distributed to P-12 school administrators to provide feedback on our completers who are within three years of completion of our teacher preparation program. 

VEAC Initial Program Employer Survey of Completers: Results: AY 2023-24

The EPP analyzed the VEAC survey qualitative and quantitative data for AY 2023-24. The EPP found that our student mean scores for the 13-question survey were at or above par with the VEAC average combined mean of participating institutions. The survey instrument uses a 4-point Likert scale for questions A through N, with a 4.0 as exemplary. The question related to overall employer satisfaction uses a 5-point Likert scale with a 5.0 as extremely satisfied. Our completers’ overall strengths were reflected in questions E (uses resources, routines and procedures), F (commitment to professional ethics) and J (multiple perspectives to instruction) with means of 3.37, 3.50 and 3.35, respectively. Areas for continuous improvement were reflected in questions C (effectively engages students in learning), D (use of data) and M (use of assessment results) with means of 3.15, 2.97 and 3.17, respectively. The EPP initial completers received a 4.31/satisfied on a 5-point Likert scale for: Based on your experience with this teacher, what best describes the extent to which they were ready to meet the needs of students in your school? 

The Title II and VDOE Biennial Reports evidences candidate competency at completion. Each report documents the pass rates of program completers for the required state licensure assessments by endorsement area.  The VDOE requires a minimum  80% pass rate for all program completers and non-completers. The EPP surpasses the minimum requirement.  

Title II Report AY 2023-24

Biennial Report  AY 2021-23

The EPP's Office of Field Experience tracks candidates through matriculation and completion of their program.  In order for an enrolled student to be recommended for licensure and deemed eligible for hire, they must complete their degree program, pass all licensure assessments and satisfy statutory requirements.  For AY 2023-24, 94% (81 out of 86) of students completed their program and were recommended for licensure.  

The EPP is a member of the Virginia Education Assessment Collaborative (VEAC), a statewide collaborative of 32 EPPs in Virginia designed to provide:

• EPP employer and completer satisfaction data about their initial licensure programs;
• EPPs benchmark data regarding how they compare to other statewide EPPs; and
• Our P-12 partners with a more efficient process for providing EPPs across the state feedback about their graduates. 

Radford University's Education Program Provider (EPP) participates in a statewide collaborative to collect employer satisfaction data based on CAEP RA1.1: Candidate Knowledge, Skills and Professional Dispositions.

As a participating VEAC institution, we use a common employer survey for our advanced programs administered annually to P-12 administrators across the Commonwealth. The survey is distributed in February-March. Survey results are posted the following fall. The survey is based upon the six skills/competencies listed in CAEP RA1.1: Candidate Knowledge, Skills and Professional Dispositions and RA3.4: Competency at Completion. Advanced completer program survey questions are program specific. The survey uses the following 4-point Likert scale: (4) exemplary, (3) proficient, (2) developing/needs improvement, or (1) unacceptable.

The Virginia Department of Education (VDOE) collects current employment and contact information on completers who are working in public schools within the Commonwealth. Additional contact information is collected by the EPP and submitted to VEAC to conduct the survey. The survey is distributed to P-12 school administrators to provide feedback on our completers who are within three years of completion of their advanced program. Overall, the EPP’s advanced program completers combined mean for each program specific survey questions were rated as proficient or exemplary.  The report reflects three cycles of data combined to protect the anonymity of our graduates as the n is less than 10 for certain cycles dependent on the program. 

Combined Three Cycles 2022, 2023, 2024 VEAC Advanced Program Employer Survey of Completers: Results

Each advanced program uses different methods to ensure their program completers demonstrate competency at program completion, as reported below for AY 2023-24:

Educational Leadership: 

• Students must pass the Praxis II: School Leaders Licensure Assessment 6990 and complete an internship experience prior to program completion.
• 100% of candidates who completed Praxis II: 6990 passed, with a group mean score of 159.07.
• 100% of candidates successfully completed the internship.

Literacy Education:  

• Students must pass the Praxis II: Reading for Virginia Educators (RVE)-Reading Specialist (5304) and complete a practicum prior to program completion.
• 100% of candidates who completed the RVE: Reading Specialist (5304) passed, with a group mean score of 172.86.
• 100% of candidates successfully completed the practicum.

School Counseling:

• Students must pass the School Counseling Program's Comprehensive Exam and complete an internship to be eligible for program completion.
• 100% of candidates who completed the Department of Counselor Education Comprehensive Exam passed, with a group mean score of 81%.  The Commonwealth of Virginia does not require a licensure assessment for School Counseling.  
• 100% of candidates successfully completed the final internship.

School Psychology: 

• Currently, the Commonwealth of Virginia does not require School Psychology candidates to pass the Praxis II: School Psychologist 5403. 
• Students must take and pass the Praxis II: School Psychologist  5403 test and complete a practicum in order to be eligible for program completion.
• 88% of candidates who completed the Praxis II: School Psychologist (5403) test passed, with a group mean score of 170.63.
• 100% of candidates successfully completed an internship and practicum and internship

The EPP tracks advanced candidates through matriculation and completion of their program.  In order for an enrolled student to be recommended for licensure and deemed eligible for hire, they must complete their degree program and pass all licensure assessments, if applicable. 

For AY 2023-24:

• Educational Leadership: 96% (22 out of 23) of students completed the program and passed the licensure assessment.
• Literacy Education: 100% (7) of students completed the program and passed the licensure assessment.
• School Counseling: 100% (3) of students completed the program. The Commonwealth of Virginia does not require a licensure assessment. 
• School Psychology: 100% (9) of students completed the program.  The Commonwealth of Virginia does not require a licensure assessment.