CAEP Annual Reporting Measures
CAEP Accountability Measures
UVA EHD is committed to preparing educators who positively impact the development and learning of their students. Positioned in a state which does not provide EPPs with data on completer impact, the Teacher Education program developed and monitors multiple measures to understand the degree to which completers are contributing to expected learning and growth in schools. This includes collection of teacher evaluation data from partner districts and division leader focus group data. These measures have been selected due to their alignment with the Virginia Uniform Performance Standards (VUPS) used to evaluate all Virginia teachers, as well as CAEP and InTASC competencies. In accordance with CAEP’s transition plan policy and due to the continued strain on P-12 partner divisions due to COVID-19, we have also developed and implemented a transition plan to scale up data collection over time to meet CAEP Initial Standard 4.1 and provide three cycles of representative data. This transition plan addresses our commitment to ensuring representativeness of licensure area sampling by scaling data collection from partner divisions over time, with the goal of sampling data which represent all degree programs offered by EHD across multiple data collection cycles. This plan also allows for sampling from schools and divisions representing the diversity of Virginia’s P-12 students to ensure completers positively impact the learning and growth of diverse students. The data presented below are indicative of our effort to collect data from P-12 partners whose schools reflect diversity in a number of areas. Focus group participants represent schools with diverse populations, grade levels, resources, and needs.
Teacher Evaluation Data: In 2012, VDOE established the VUPS, which measure performance in eight teaching domains: 1) Professional Knowledge; 2) Instructional Planning; 3) Instructional Delivery; 4) Assessment for Student Learning; 5) Learning Environment; 6) Culturally Responsive Teaching and Equitable Practices; 7) Professionalism; and 8) Student Academic Progress. Ratings on the VUPS are based on a building administrator's review of multiple pieces of evidence provided by the teacher, including evidence of students' academic progress and classroom observations by one or more building administrators. Each school system must document the validity and reliability of its rating process. Thus, completers' VUPS ratings provide evidence through structured and validated observation instruments to demonstrate completers effectively apply the professional knowledge, skills, and dispositions that the preparation experiences were designed to achieve. Each of the eight standards is closely aligned with the evidence needed for CAEP Standard 4. The Student Academic Progress standard specifically provides evidence that documents program completers' contribution to an expected level of student-learning growth. EHD reached out to six partner school divisions requesting help in a pilot study for gathering and utilizing EHD completers’ VUPS data. Divisions were asked to provide VUPS data for UVA EHD completers employed in the past three years. Two divisions provided data for the pilot. As evidenced in the data files, all completers earned either a proficient or exemplary rating for the Student Academic Progress standard, as well as all other VUPS standards. Following the transition plan, the EPP requested VUPS data for UVA EHD completers from five divisions during summer 2022. Two divisions provided data for an additional 20 completers for the 2021-22 academic year. As evidenced in the Summer 2022 report, all completers except two earned either a proficient or exemplary rating for the Student Academic Progress standard. Scores on all other VUPS standards were proficient or acceptable with the exception of two scores for one candidate. These data, while few, indicate that EHD graduates effectively contribute to P-12 student learning growth and are able to apply the skills and content knowledge gained from EHD programs.
Employer Focus Group: EHD recruited a panel of P-12 school leaders and developed a focus group protocol designed to elicit perspectives on the satisfaction of employers with EHD programs, graduates, and their impact on P-12 student learning and development. The Director of Assessment and Director of Teacher Education developed the focus group protocol, aligning items with the language of standard R4.1, CAEP workbook guiding questions, and questions of interest to the program for continuous improvement. The Director of Assessment then conducted the focus group via Zoom using the designed protocol, recording the focus group to transcribe participant responses and analyze responses for overarching themes. Participants were selected purposefully to reflect a diversity of school size, demographic and socio-economic makeup, and grade levels. In this way, EHD selected a representative sample of employers from whom to solicit program satisfaction feedback.
The purpose of the focus group was to establish evidence that the TED program provides effective preparation which results in completers’ instruction for P-12 student learning and development and that employers are satisfied with the relevance and effectiveness of the TED program. In addition, EHD aimed to establish evidence showing that our candidates effectively contribute to P-12 student learning growth and apply the professional knowledge, skills, and dispositions that our experiences were designed to achieve. Results suggest employers feel completers effectively contribute to P-12 student-learning growth and apply the appropriate knowledge, skills, and dispositions gained through the Teacher Education program. Completers demonstrate proficiency with content knowledge, identification of resources, applying effective and appropriate strategies for differentiation, and relationship-building with students, families, and peers. Participants indicated they perceive the TED program’s clinical experiences to be strong, providing a solid foundation for applying requisite knowledge and skills in the classroom as completers. Particularly, participants described completers’ strength around an asset-based approach to planning and instruction. They noted completers’ knowledge - as novice teachers - around initiatives focusing on equity, including culturally responsive teaching. The summary document linked above provides detailed focus group results.
EHD works closely with many internal and external partners to engage in program design, evaluation, and continuous improvement. Several partnerships are highlighted below.
Clinical Partners Groups: Teacher Education faculty and staff collaborate with P-12 teachers, building administrators, and central office personnel through Clinical Partners Groups (CPG). The purpose of these groups is to co-construct, review, and refine program materials, assessments, and clinical experiences to the mutual benefit of teacher candidates and school partners. Recent data dialogues have engaged partners in reviewing results from completer surveys, employer surveys, and clinical educator evaluations. Meeting minutes and materials also demonstrate ways in which partners have engaged in joint problem solving and curricular review. Outside of the CPG structure, EHD also solicits external stakeholder feedback through individual conversations and structured focus groups.
Virginia Education Assessment Collaborative (VEAC): VEAC represents another external partnership supporting EHD’s continuous improvement efforts. The purpose of VEAC is to provide a centralized assessment structure for Virginia EPPs. In addition to reducing the complexity of survey completion for administrators, the use of common instruments has allowed EHD to benchmark its data against the larger consortium of participating EPPs. The use of shared instruments also provides a common language for EPPs, which can prompt discussion around areas for collective improvement across Virginia. The VEAC steering committee - which includes EHD’s Director of Assessment and Director of Teacher Education - have facilitated a range of statewide virtual meetings to engage stakeholders in conversations about shared problems of practice. Recent meetings have focused on topics such as meaningful stakeholder engagement to inform program improvement and strategies for facilitating data dialogues across stakeholder groups. Through VEAC, EHD has positioned itself as a leader and collaborator in state-level assessment initiatives that support continuous improvement efforts.
Virginia Community College System: Recruitment is another area in which internal and external partnerships play a critical role in program design, implementation, and evaluation. The Teacher Education program maintains a strategic recruitment plan that details the many internal and external personnel across units with whom faculty and staff engage in recruitment efforts, including the dean’s office, central admissions, and the EHD Office of Communication and Marketing. The Teacher Education program also collaborates with partners from Virginia’s Community College System (VCCS) to build sustainable structures to support the recruitment and retention of transfer students.
Teacher Education Data Committee: The Teacher Education Data Committee was established in 2015 to provide oversight and support for all data collection efforts in the Teacher Education program. Committee members meet to review Teacher Education Participant Pool data collection processes and instruments. The committee includes faculty and staff with a range of expertise. The Director of Teacher Education convenes the committee, and the Director of Assessment and Curriculum, Instruction, and Special Education Data Manager serve as ex officio members. Faculty representatives include tenure-track and non-tenure track faculty from within the Teacher Education program as well as a faculty member from the Department of Leadership, Foundations, and Policy with expertise in education policy and teacher labor markets. By including a wide range of university personnel, the Data Committee ensures decisions balance considerations for research, programmatic improvement, and accreditation.
UVA K-12 Advisory Council: The UVA K12 Advisory Council was formed in 2001 as a way for the UVA School of Education and Human Development to establish coherent, sustained relationships with school districts that support the continuing professional development of the K12 education community. The model for these relationships incorporates job-embedded professional learning and examines the possibilities for co-creation of learning opportunities within the districts we serve.
The Employer Survey is distributed annually to principals in all schools in which graduates from the previous three years are currently teaching. The survey measures employers’ satisfaction with the preparation of School of Education and Human Development graduates for their assigned responsibilities in working with P-12 students. UVA participates in the Virginia Education Assessment Collaborative Employer Survey for Initial Licensure programs. The Virginia Education Assessment Collaborative utilizes a common, standardized survey of program completers and their employers using the language and concepts of the Commonwealth of Virginia’s Uniform Performance Standards. There are 36 Educator Preparation Programs in the Commonwealth of Virginia surveying program completers and their employers each year seeking to improve their programs and meet requirements of the Virginia Department of Education and the Council for the Accreditation of Educator Preparation. By standardizing survey recruitment, timelines, and process, VEAC hopes to reduce the complexity of surveys that principals and administrators complete each year to support EPPs. By benchmarking with the other VEAC participants, EPPs can better understand their strengths and areas for improvement. Additionally, VEAC provides a common language and prompts discussion for program improvement.
To provide additional data in support of CAEP R4.1 and R4.2, the Teacher Education program conducted an Employer Focus Group in fall 2021. Measure 1 includes a description of the focus group protocol and summary of findings. A copy of the protocol and results is available here: Employer Focus Group.
Administration and Supervision
The Employer Survey is distributed to leaders in all school divisions in which graduates from the previous three years are currently working. The survey measures employers’ satisfaction with the preparation of UVA’s School of Education and Human Development Administration and Supervision graduates for their assigned responsibilities in working with P-12 students and teachers. Data from the 2020 administration indicate that employers of UVA’s Administration and Supervision program graduates agree or strongly agree that our completers are prepared for the positions they enter upon program completion. The EPP began surveying employers of program graduates through VEAC in Spring 2023. Results will be shared in the 2024 CAEP Annual Report.
The Employer Survey is distributed to division and school leaders in schools in which graduates from the previous three years are currently working. The survey measures employers’ satisfaction with the preparation of UVA’s School of Education and Human Development Reading Specialist program graduates for their assigned responsibilities in working with P-12 students and teachers as Reading Specialists.
Data from the 2020-2021 survey administration indicate that, on the whole, employers are satisfied with preparation of our Reading Specialist graduates. Additionally, data collected from the pilot VEAC advanced employer survey (N>5) indicate that completers are prepared or fully prepared to meet the needs of their preK-12 learners.
As part of their ongoing CACREP accreditation, the Counselor Education Employer Survey is administered annually. Results demonstrate that, in general, employers of program graduates perceive these graduates as well-prepared for the positions they secure following program completion.
School and Clinical Psychology
The School and Clinical Psychology program has developed strong and ongoing relationships with employers across the Commonwealth in a variety of settings, including hospitals and acute-care clinics in the state. To gather information and feedback on the quality of preparation provided by the program for its graduates as they enter the field, the program has begun conducting interviews of employers who regularly hire program graduates. The feedback garnered from these interviews has been overwhelmingly positive. Summary remarks are shared below:
"They all come in with strong research backgrounds and clinical skills informed by current research and also are in the habit of consulting recent research to make treatment decisions and plans and are skillful at knowing how to do that."
"Overall I tend to be very, very pleased with their [graduates]...they’re really well-prepared, responsible, ethnically diverse which is wonderful."
"...the students from Curry are poised, professionally mature, have had a nice stance in terms of not knowing a lot about the job [yet] but comporting themselves in a way that positions them to be recognized as one of the doctors and that’s a tribute to the selection and the maturity of Curry students."
In addition, and as part of their ongoing APA accreditation activities, the School Psychology program tracks graduate employment milestones at five years post-graduation and two years post-graduation. All 7 alumni who graduated five years ago are employed in professional settings (n=1 at School District, n=1 at Correctional Facility, n=1 in Independent Practice, n=3 at Hospital/Medical Center, and n=1 in Other: Project Manager). The 5 alumni who graduated two years ago are also all fully employed (n=2 in Independent Practice; n=2 in Hospital/Medical Center, and n=1 in a mix of Independent Practice, Psychiatric Facility, and School District).
Teacher Education degree and course requirements are aligned to Virginia’s Regulations Governing the Review and Approval of Education Programs in Virginia, and the TED office submits program matrices demonstrating alignment with state regulations. Multiple sources of evidence are used to ensure candidates are prepared for certification at completion of the program. The TED office systematically monitors candidates’ progress from admission through program completion using a series of transition points. These transition points are aligned to competencies outlined in R1 and include measures used to measure candidates’ content knowledge, pedagogical knowledge, pedagogical skills, critical dispositions, professional responsibilities, and the ability to integrate technology effectively in their practice. At every transition point, candidates’ GPA and grades are monitored to ensure compliance with school and program benchmarks and to assess candidates’ acquisition of content and pedagogical knowledge in courses. Prior to the culminating internship, candidates submit passing scores on all licensure exams and evidence of completion of statutory requirements. At the end of each clinical experience, the Internship Evaluation is used to measure candidates’ professional dispositions and performance in the classroom. At the end of the program, the Portfolio is used to measure candidates’ growth and mastery of competencies across the trajectory of the candidate’s experience in the program.
Academic Requirements: Throughout the program, candidates must meet specified grade thresholds in courses applied to the state-approved program: B- for graduate candidates and C+ for undergraduate students. Candidates must pass all coursework with the minimum grades stated above to be eligible for degree completion and licensure. Across all programs, three cycles of GPA data indicate all cohort averages are above the CAEP threshold of 3.0 every semester, and 100% of completers meet the B- or C+ requirements for their respective programs.
Licensure Exams: Prior to the culminating internship, candidates must submit passing scores on all licensure exams, including the Virginia Communication and Literacy Assessment (all candidates), Praxis Specialty Test (all candidates except special education), and Reading for Virginia Educators (early childhood, elementary, and special education). Data from the last three Title II cycles indicate program pass rates for completers are 100%, and scores are consistently higher than the state average.
Statutory Requirements: Candidates must also provide evidence of completion of statutory requirements: child abuse and neglect training; training in emergency first aid, CPR, and the use of AED; dyslexia training; and behavior intervention and support training. Candidates are unable to complete the program until these requirements have been met, and therefore 100% of completers meet these requirements.
Internship Evaluation: Clinical coaches and mentor teachers evaluate candidates’ content knowledge, pedagogical knowledge, dispositions, and performance on the internship evaluation. Candidates also complete a self-evaluation. The internship evaluation measures candidates’ progression on competencies aligned to the VUPS and InTASC standards. The development, structure, and implementation of the evaluation are outlined in the internship evaluation instrument guide. The expectation is that candidates score a 3.0, the target rating of “Proficient,” on the final set of evaluations. Candidates participate in a final conference with their coach and mentor to review feedback and establish professional development goals for their first year of teaching. Data from this reporting cycle demonstrate candidates are meeting and exceeding this threshold.
Portfolio: To supplement current data collection efforts, the TED program has developed a new portfolio rubric that will be piloted in spring 2022. The portfolio measures candidates’ development on competencies aligned to the VUPS and InTASC standards. The portfolio is a summative assessment and occurs at the end of the candidates’ experience in the program, although candidates use artifacts from across their time in the program to demonstrate each competency. Information about the development, structure, and implementation of the portfolio is available in the portfolio instrument guide. The expectation is that candidates score a 3.0, the target rating of “Proficient,” on the portfolio rubric. Data from this reporting cycle demonstrate candidates are meeting and exceeding this threshold.
The School of Education and Human Development collects initial employment data of its program completers in various ways, including an employment survey distributed upon program completion, email communication between program completers and the Academic Programs and Student Affairs team, an alumni survey, and web-based searches. The table below provides the percentage of programs completers who were employed as a classroom teacher, employed in an education position other than that of a classroom teacher, employed outside of the education field, enrolled in higher education, or their employment status is unknown.
Administration and Supervision
Data from the INSPIRE Graduate Survey provide an indication of the types of positions program graduates take.
The reading program administers a survey to program graduates to elicit feedback on the program. As part of this survey, graduates are asked whether the degree and/or additional certification earned as part of the program enabled them to move into a new position. Data from the fall 2017 survey are not presented here, as N < 10. In future years, data will be compiled across administrations to meet or exceed a minimum N of 10.
The counselor education program surveys graduates at exit and annually to determine job placement and position type. Job placement rates for the counselor education program are consistently 100%.
School and Clinical Psychology
As part of accreditation requirements for APA, the school and clinical psychology program at UVA collects data on employment of its graduates. Please note that this program prepares both clinical and school psychologists, and the majority of program graduates enter the clinical psychology field. All 7 alumni who graduated five years ago are employed in professional settings (n=1 at School District, n=1 at Correctional Facility, n=1 in Independent Practice, n=3 at Hospital/Medical Center, and n=1 in Other: Project Manager). The 5 alumni who graduated two years ago are also all fully employed (n=2 in Independent Practice; n=2 in Hospital/Medical Center, and n=1 in a mix of Independent Practice, Psychiatric Facility, and School District).