Scoring Student Answers to Constructed-Response Questions, Essays, and Open-Response Questions

What is Scored?

Over 6 million total student responses

• to constructed-response questions in English Language Arts, Mathematics, and Science and Technology/Engineering
• to English Language Arts essays (grades 3–10)
• to open-response questions in high school Biology, Chemistry, Introductory Physics, and Technology/Engineering

How is Scoring Done?

Use of Scoring Rubrics

• Constructed-response and open-response questions in Mathematics and Science and Technology/Engineering are scored using a rubric, scoring notes, and student responses. These scoring materials indicate what knowledge and skills students must demonstrate to earn each score point.
• English Language Arts constructed-response (short-response) questions for grades 3 and 4 are scored using a rubric, scoring notes, and student responses. Constructed-response rubrics are worth up to 3 score points per item. The three-point constructed-response question assesses reading comprehension by requiring students to construct an answer in a shorter format.
• Students' writing in grades 3–10 is scored using a rubric, scoring notes, and student responses. Students' writing in grades 3–10 is evaluated using two criteria:
  • Idea Development, based on a 1–4 score-point scale for grades 3–5 and a 1–5 score-point scale for grades 6–10
  • Standard English Conventions, based on a 1–3 score-point scale for grades 3–10

Scorer Training

Scorer training sessions require that scorers

• answer each test question to be scored;
• review the content covered by the question and discuss its scoring rubric and benchmarked student responses;
• score a set of responses (training pack);
• discuss training pack responses and scores assigned to them; and
• score another set of responses (qualifying pack).

A scorer has two opportunities to accurately score a set of qualifying responses. A scorer who is unable to consistently and accurately score the responses will not be allowed to score that item.

Scoring System

• Scorers view electronic copies of digitally scanned images of student responses on a computer monitor and assign scores electronically. Use of this computerized scoring system ensures that student responses are randomly assigned to scorers, second readings are truly "blind," and supervisors have immediate access to information about scorer accuracy.
• For high school, ELA writing questions (essays), constructed-response questions for Mathematics, and open-response questions for Science are scored twice by two separate scorers.

Who Does the Scoring?

Professional scorers are hired by the contractor to score all constructed-response questions, essays, and open-response items. The contractor actively seeks a diverse scoring pool and typically employs scorers with a broad range of backgrounds: teachers, business professionals, graduate school students, retired educators, and the like. To ensure this diversity, demographic information such as gender, race, educational background, etc., is collected.

All recruited scorers will meet, at a minimum, the following requirements:

• For grades 3–8, all scorers will have at least 48 college credits, with at least two classes related to the content area being scored.
• For high school, all scorers must have a 4-year college degree and a degree related to the content area being scored.

Last Updated: August 26, 2019