The Massachusetts Board of Elementary and Secondary Education

Update on Next-Generation MCAS and Planning for the High School Competency Determination (CD) Standard

---

In November 2015, the Board of Elementary and Secondary Education (Board) voted to develop a next-generation MCAS program. The first new tests, in grades 3–8 English Language Arts (ELA) and mathematics, were administered in spring 2017. As we enter the third year of the program, we are continuing our transition to computer-based testing and preparing to give next-generation tests in high school ELA and mathematics for the first time in spring 2019, and in high school science and technology/engineering for the first time in spring 2020. This memorandum provides an update on the implementation of the next-generation tests and provides information about the upcoming transition to new high school tests and, consequently, the need to establish a new competency determination standard. In addition, I share below some thoughts about long-term goals for assessment in the Commonwealth.

Update on Implementation of Next-Generation Tests

Overall, the transition to next-generation, computer-based testing has been very smooth. We began by testing two grades and subjects online in 2016 (grades 4 and 8), and have added additional grades and subjects each year. This spring we expect to administer nearly all of our tests online, including grade 10 ELA and mathematics tests for the first time. In all, 89 percent of students in grades 3–8 took a computer-based test in spring 2018. We are scheduled to release official results from the spring 2018 tests on September 27.

I am pleased to report very high statewide participation rates in spring 2018 (over 99 percent). High participation rates are key to providing essential information about students' achievement and progress to schools, districts, parents, and students.

As we reported to you last fall, we continue to engage with the field through surveys, trainings, workgroups, and other projects, and make revisions and improvements based on feedback from stakeholders.

Also as we discussed last fall, Rhode Island administered the MCAS ELA and mathematics tests to approximately 70,000 students in grades 3 through 8 in spring 2018. Rhode Island will release the results from these tests around the same time that MCAS results are released.

Transition to New High School Tests in ELA and Mathematics

In spring 2019, students in grade 10 will participate in the first administration of the next-generation tests in ELA and mathematics, and standard setting will take place in summer 2019. Through the standard setting process, students' results will be scaled and reported according to the next-generation achievement level categories (Exceeding Expectations, Meeting Expectations, Partially Meeting Expectations, and Not Meeting Expectations). Even though the new reporting categories will be used, students in the first two classes to take the new tests (classes of 2021 and 2022) will be held to an interim passing standard for graduation, consistent with the Board's vote in February 2018. This interim standard is defined as a similar level of achievement to the required standard on the legacy tests: 240 (Proficient), or 220 (Needs Improvement) and the fulfillment of the requirements of an Educational Proficiency Plan. This approach allows students, parents, and educators to become familiar with the new assessments before any students are held to a new passing standard; it also gives the Board the opportunity to consider the results from the next-generation grade 10 tests in setting a new CD standard for the classes of 2023 and beyond.

The statistical analysis needed to set the interim passing standard will be conducted by the assessment experts here at the Department and at our testing contractor during the summer of 2019. To prevent any delay in the reporting of results, I plan to ask the Board in the spring to delegate to me the authority to approve those interim standards and allow results to be reported to schools, parents, and students in a timely manner.

Establishment of a New Graduation Standard for ELA and Mathematics

Following the standard setting, in the fall of 2019 the Board will begin discussing a new competency determination standard for the classes of 2023 and beyond. Students in the class of 2023 will be entering ninth grade in the fall of 2019, and we need to provide them with adequate notice of the expected graduation standard as they are beginning high school.

To support the Board in considering a new graduation standard, we are planning a number of activities leading up to the decision, including communications and outreach with stakeholder groups; ongoing analysis and research from over 20 years of assessment in the Commonwealth; and public hearings regarding the transition to a new standard. I am also recommending the formation of an external advisory committee, composed of representatives from relevant groups and organizations across the Commonwealth, to consider all the factors involved and to assist me in forming a recommendation to you in the fall of 2019.

A tentative timeline for our activities appears at the end of this memorandum, and staff will provide more details at the September meeting.

Transition to Next-Generation High School Science and Technology/Engineering (STE) Tests

Establishment of an interim standard

The next-generation high school STE tests are scheduled to be administered for the first time in the spring of 2020. We will not be able to set standards on these tests until after these students participate in the first administration of the next-generation high school STE tests in spring 2020.

I am recommending that the Board update the competency determination regulations this fall to establish a transition plan for STE similar to the plan for ELA and mathematics that the Board approved last year. The regulations would be updated to state that students in the classes of 2023 and 2024 will be held to an interim passing standard for high school STE, which will be defined as a similar level of achievement to the required standard on the legacy tests: 220 (Needs Improvement). The interim passing standard would be identified in the summer of 2020 - following the first administration of the next-generation high school tests in STE - through a two-step process: (1) identification of the standards through a statistical method, followed by (2) validation by a panel of expert educators.

Phasing out of the Chemistry and Technology/Engineering Tests, and Introduction of a Mid-Year Introductory Physics Test

During our planning for the next-generation high school STE tests, we have been considering another change to the program. Beginning in December 2016, this Board and the Board of Higher Education have discussed the possibility of phasing out the chemistry and technology/engineering tests and introducing a mid-year testing opportunity in introductory physics.

An overwhelming majority of students take the biology or introductory physics tests (96 percent in 2018). The number of students taking the chemistry and technology/engineering tests has always been low and has declined over the years; in particular, participation in the chemistry tests has declined significantly. In spring 2018, 460 (less than 1 percent) of students took the chemistry test, and 2,348 (3 percent) of students took the technology/engineering test. The low number of participants is not cost effective and creates challenges for test development and item release; as a result of the low participation rate, we have not been able to release items from these tests for the last several years.

Over the past year and a half, we have researched whether it makes sense to continue offering these two tests. Our findings from one recent analysis indicate that, in addition to low participation overall, few students in vocational/technical schools (about 10 percent) took the technology/engineering test, while about 80 percent took the biology test. In addition, only 20 – 35 percent of students taking the technology/engineering test actually took a technology/engineering course, and only about 50 percent of students taking the test were taught by a teacher licensed in technology/engineering.

We have also explored other options for including these subjects as part of our high school testing program (offering AP or SAT subject area tests, for example). However, we found more downsides than benefits to these alternatives, including issues related to cost, alignment to the standards, equivalence among the tests, and concerns for reporting and accountability.

Our investigation into options for high school science going forward shows that schools and districts would welcome the introduction of a mid-year test in introductory physics. Currently, only the biology test is offered twice annually, in February and June (all four tests are offered in June). Beginning in 2021, both the biology and introductory physics tests would be offered twice a year, allowing expanded testing opportunities for students on block schedules as well as students who need a retest.

Therefore, I am recommending that, when the Board considers revisions to the regulations to establish an interim standard for the high school STE tests, it also consider the phasing out of the chemistry and technology/engineering tests.

I plan to present my proposed amendments to the regulations for STE (including both the establishment of an interim standard and phasing out of the chemistry and technology/engineering tests) at the October 2018 meeting.

Goals for Our Assessment Program

We continue to work to improve our testing program and the experience of schools, parents, and students in taking the tests and receiving and using the results effectively. I am committed to investigating ways that we can reduce the amount of time that is spent on testing, while maintaining the quality of the tests and the validity of the results. I am also interested in ways that we can return results more quickly, especially to teachers. We had a successful pilot of early reporting this year (schools received preliminary results on many test items about 10 days after the end of the testing window), and we will continue to build on that work. We are also moving forward with researching other ways to reduce testing time and return results sooner, including the use of automated scoring, a topic we discussed with the Board last spring, and computer-adaptive testing.

Deputy Commissioner Jeff Wulfson, Associate Commissioner Michol Stapel, Director of MCAS Test Development Katie Bowler, and Lucy Wall of our legal staff will join us for the discussion at the September 18 Board meeting.

Tentative timeline of Board discussions and related activities, through winter 2020

Enclosures: