Views of Climate and Learning (VOCAL) Frequently Asked Questions

Below are answers to Frequently Asked Questions (FAQ) that are often asked about VOCAL student survey project.

What is VOCAL?
"Views of Climate and Learning" (VOCAL) is an annual student survey sponsored by the Massachusetts Department of Elementary and Secondary Education (DESE). Students are asked to share their views on three dimensions of school climate: engagement, safety, and environment.
Why participate in VOCAL?
Over the last three decades, research has demonstrated that positive school climate supports learning and positive youth development. The survey gives Massachusetts students a voice and an opportunity to provide feedback to schools and districts, which can use the data to strengthen students' social and emotional well-being, health, safety, and academic learning.
Who takes the survey, and how is it designed?
In 2022, VOCAL was administered to students in grades 4, 5, 8, and 10. All students who participate across the four grades answer a common set of items, in addition to items that are unique to each grade's survey.
What does the survey measure?
VOCAL is based on the conceptual framework of the U.S. Department of Education's School Climate Surveys, which focuses on measuring students' perception of three dimensions of school climate: engagement, safety, and environment. Each of these three dimensions is in turn composed of three topics. Engagement constitutes the topics of cultural and linguistic competence, relationships, and participation in class and school life. Safety is comprised of the topics of emotional safety, physical safety, and bullying/cyber-bullying. Environment incorporates the topics of instructional environment, discipline environment, and mental health environment.
When and how is the survey administered?
All surveys are administered online. The grade 5 and grade 8 surveys are administered online after students complete their MCAS Science and Technology/Engineering (STE) tests. The grade 4 and grade 10 surveys are administered online after students complete their MCAS Mathematics tests.
Must students, schools, or districts participate in the VOCAL survey?
No. Participation is voluntary.
How long does the survey take?
On average, the survey takes students approximately 20 minutes to complete.
Do schools and districts receive reports of their students' aggregate responses?
Yes. Districts receive a summary report of all their schools' survey data. To the extent data permits, each school receives an overall school climate index scaled score, an engagement index scaled score (plus a separate participation topic score), a safety index score (plus a separate bullying score), an environment index scaled score (plus a separate instructional environment score), and item-level frequency response data. For each type of score, school reports include comparative data to the district and to the state. Trend data is also provided.
How will DESE ensure school and district index scaled scores are reliable?
Over the years, DESE has refined the VOCAL survey to ensure that there are enough items measuring each dimension or topic in order to ensure the reliability of each index scaled score is greater or equal to 0.7 on a scale of 0 to 1 for districts and schools.
Will individual student identities be protected when reporting data to districts and schools?
Yes. No student identities are ever shared. In addition, to preserve respondent confidentiality in school and district aggregate reports, DESE suppresses data if fewer than 10 students responded or if all students responded identically on an item.
How will my school or district receive the 2022 VOCAL survey results?
New to 2022 is the VOCAL PowerBi/Edwin dashboard; this dashboard graphically displays all VOCAL data. This new dashboard includes all VOCAL data that is available from 2018 onwards (due to the Covid pandemic, no data is available for 2020). Districts can also view paginated district summary reports and school climate reports in Edwin (VC301 reports). These reports can be downloaded into Excel.
Does DESE offer training in how to analyze and interpret VOCAL data?
Yes. DESE will offer in-person/virtual trainings and one webinar each year, typically in the late fall.
Are the data received by schools and districts subject to public records requests?
Yes. The data reports could be requested by the media, parents, and/or other stakeholder groups.
My district/school participated in the survey last year, but we did not receive all our index scaled scores. Will we receive them this year?
DESE has made improvements to VOCAL that should ensure most schools receive a full complement of index scaled scores. New items have been added to each survey to help schools and districts receive all index scaled scores. Schools and districts can increase their chances of receiving all index scaled scores by ensuring all students participate in the survey.
If my district/school participated in 2018's and 2019's school climate surveys, will we be able to compare those index scaled scores to scores from 2022 onwards?
Yes. Index scaled scores can be used to compare trends in students' views over time between 2018, 2019, and 2022 (see Q16 for information on 2021 survey). The context for the survey administration in 2022 was similar to 2018 and 2019 with all districts and schools engaging in in-person learning throughout the 2022 school year. As long as the conditions for learning are similar to 2022, scaled scores can be compared to future years' scores.
If my district/school participated in 2021's school climate survey, will we be able to compare those index scaled scores to this year's 2022 scores and future scores?
No. Although index scaled scores from 2021 will be reported on the same scale as those from 2018, 2019 and 2022, the context in which students took the survey was different due to the Covid pandemic. For example, at the time of the 2021 survey, many students were still learning online or using a hybrid model; this change in context makes comparing 2021 scores to prior years' scores or scores for 2022 and future years problematic.
If my district/school participated in prior years' school climate surveys, will we be able to compare our item percent response data this year to prior years' scores?
No. Item response data is specific to the cohort responding and is not on the same scale as the data from the previous year. Item response data is transformed to make it equal-interval, and the resulting indices can be used to make accurate comparisons over time. Item response data provides rich contextual information about how a cohort perceives their school climate, but it is specific to the year of administration. Item response data can help educators explain what contributed to any index scaled score differences observed. You can look for patterns of item percent responses and compare patterns across any two cohorts. When item response data replicates across cohorts, this information provides contextual information about schools' strengths or areas for improvement identified in the index scaled score comparison.
Do we have to pay for students to participate in VOCAL?
No. DESE covers the entire cost of the school climate survey for schools and districts who chose to participate.
May we share our reports with staff, students and other stakeholders?
Yes. DESE encourages all stakeholders of the school community to have access to the reports. The school climate survey data are intended to be a conversation starter and used with other school data to inform and build a picture of schools' environments.
How will DESE use my data?
The data will be used by DESE's program offices to better identify resources and supports for Massachusetts educators to help improve local school climates. The surveys also help the state to meet related requirements included in Massachusetts' 2014 Act Relative to Bullying in Schools.
How did the Covid pandemic specifically impact the 2021 VOCAL results and their interpretation?
Student participation rates in 2021 were lower. Compared to 2019, the decline in participation rates ranged from 14% in grade 4 to 26% in grade 10. The lower participation rates may impact DESE's ability to provide schools and districts with student group data.

The context in which students took the survey was different. Almost a third of all students in grades 4 and 5 were either remote learning or learning using a hybrid model; in the upper grades, roughly half of all students were not participating in-person at the time of the survey administration. Many students were in the process of returning to school and a sense of normalcy. The different context in which students took the survey makes interpreting the results from 2021 problematic and any comparisons to prior years should be made with extreme caution.
Who do I contact if I have another question?
Please email DESE at vocalsurvey@doe.mass.edu .

Last Updated: November 17, 2022