Educator Preparation

Candidate Assessment of Performance (CAP) Office Hour Takeaways

ESE held a series of CAP Office Hours sessions in late-October and November to provide answers to your pressing questions about the CAP pilot and create opportunities for cross-organization sharing. These sessions were attended by over 280 individuals from 45 of our Sponsoring Organization, and the questions and discussions were engaging and solutions-driven. As always, the questions and comments from the field will help to inform our understanding of what is working and where additional supports may be helpful. Many of the commonly-asked questions across the Office Hours sessions have been captured in updated FAQ's posted on our website.

There were some consistent themes that emerged across the sessions that are worth noting:

• The choice to align CAP with the MA Educator Evaluation Framework was a good one. Participants at each session expressed enthusiastic support for this alignment and identified many potential opportunities for further strengthening the candidate experience by leveraging this connection. For instance, one provider has brought in an individual from their partner district to help support implementation based on that individual's experience implementing the Educator Evaluation Framework. While there was support overall, there were also some questions about why ESE was making this shift away from the existing tool. We plan to address this more fully in the "audio deep dives" that will serve as a companion to the Guidelines .

• CAP involves an increased expectation of dialogue between the assessors and candidate than the PPA had (for instance, needing to conduct a calibration meeting before debriefing an observation). Creating new structures to facilitate these conversations, particularly logistically, has been challenging. First, it is important to note that ESE encourages the use of technology to support these conversations when possible. In fact, ESE is building an online platform that will serve as a tool for interacting virtually throughout the 5-step cycle. Through this platform, assessors and candidates will be able to share feedback and ratings virtually, and discuss progress in real time. A few other concrete examples were provided during the sessions:

  • Scheduling post-conferences on the same day as the observation. The Guidelines require that program supervisors and supervising practitioners step away from the observation and conduct an analysis of the evidence prior to conferring with the candidate around feedback. Some organizations have figured out how to accomplish this all within a day. One program described how they wo uld schedule an observation period or two before lunch or free period so that the assessors had time to synthesize after the lesson and then hold a post-conference after school on the same day.
  • Place a cohort of candidates in one placement site and assign a single program supervisor to the building.
  
  

• Organizations, and in particular program supervisors and supervising practitioners, are adjusting to the new forms used in CAP. During the office hours (and again in the FAQ), ESE clarified which of the forms are required and which are optional, yet recommended. First and foremost, it is important to note that CAP is supposed to drive meaningful feedback and an accurate evaluation of candidate readiness, it is not an endeavor in paperwork. The online platform will help with this significantly as it facilitates the documentation process seamlessly alongside the 5-step Cycle. In the interim, the forms are in place to support the full process, even if only a few are actually required. Because the purpose of the pilot is to determine what works with CAP, and what should be improved, it is our hope that all of the forms will be used so that you will provide us feedback on all portions of the assessment. With that in mind, we have created a table in our updated FAQ¡¦s that more clearly establishes the required forms of CAP for the pilot. Additionally, to make access to these forms easier, we have built separate PDF files for each form that will be accessible on the website in the coming weeks.

• Organizations who expressed the most confidence in their implementation of CAP thus far have invested time and resources into developing and executing training for program supervisors and supervising practitioners specific to their organization. On the flipside, one provider described that just sending out the Guidelines, without additional context, as ineffective. It became evident throughout the sessions that managing communications and support for candidates and assessors was essenti al to a successful implementation.

• For many, CAP is a huge shift in responsibility. What had traditionally been the program supervisors role alone is now shared significantly with the supervising practitioner and where candidates may have spent the majority of effort collecting evidence towards standards, now that is expected in great part by the assessors. Throughout the Office hours, we heard frequently about the increase responsibilities, especially for supervising practitioners. First, it is important to note that these were intentional shifts made by ESE. Secondly, we want to work with the field to identify ways to support this shift. Specifically, we are looking into increasing the number of Professional Development Points that may be awarded for serving in the role and are doing additional outreach and communication with districts to encourage support for CAP.

Importantly, the themes that emerged reinforced that we are on the track in terms of implementing a new performance measure that is indicative of candidate readiness. Yet, there are clearly still important details to consider and that may or may not require revision in future iterations of the assessment.

Feedback on the Pilot:

As we work to ensure a successful CAP implementation in 2016-17, we want to collect data and feedback from all of the CAP Pilot participants to inform the final CAP. We are doing this through:

• Data collection of formative and summative assessment ratings (see below)
• Surveys: In early December we will releasing a link to a feedback survey through email for you to send on to pilot participants(including program supervisors, supervising practitioners and candidates) within your Organization. We will rely on your support in distributing the survey and helping us to collect a robust set of feedback.
• Focus groups: In January we will be holding a series of focus groups to collect qualitative feedback on the pilot from fall CAP participants. We will be reaching out in December to solicit participation.

CAP Rubric & Ratings Workshop:

We will be releasing another set of training aides for Sponsoring Organizations to use in the training of your Program Supervisors and Supervising Practitioners on the use of the CAP rubric at the beginning of December

Online Platform:

We are excited for our CAP Online Platform Pilot, which will be piloted in the spring semester. In December we will be launching the CAP Online Platform and releasing resources to support organizations in the pilot of this system for the spring semester. We appreciate all of the organizations that signed-up to pilot the new system, at this point we are at capacity for participation in the pilot. The features of the Online Platform that will support the implementation of the CAP include:

• Use of a free web-based platform that includes access to all CAP forms,
• Opportunity for completing, sharing, and accessing all forms and materials in real time
• A communication window feature that allows for communication between the Program Supervisor, Supervising Practitioner and Teacher Candidate
• Ability to extract assessment rating data—no additional step will be needed to send the ratings to ESE.