Official DESE Communications

Before identifying strategies and taking action, your school needs to ground itself in a thorough analysis of data that helps district and school leaders, staff, and stakeholders take stock of the school's strengths and challenges and develop a common understanding of what needs to change in order to achieve your school's vision for success.

Rather than leaping straight to developing solutions and strategies, taking time for a deep analysis of assets and challenges is likely to result in stronger outcomes for your school. In our experience, schools that have experienced the most success in improving student outcomes and sustaining them over time are those that:

1. took the time during the initial planning phase to use multiple data sources to engage in deep reflection about their assets and challenges;
2. used those analyses as a basis for developing common understandings of the key root causes their schools needed to address; and
3. selected and implemented a set of strategic objectives and initiatives that directly addressed the root causes they identified.

Without engaging in this deeper level of reflection, your school may end up addressing symptoms rather than actual causes and invest significant time, energy, and resources in implementing strategies that do not address your real challenges. Administrators and teachers from the Morningside Community School in Pittsfield, MA, share their experience in engaging in this process in one of the latest videos in DESE's Turnaround Practices in Achievement Gain Schools Video Series.

Overview of the Process

The initial process of self-reflection and analysis is a critically important aspect of the sustainable improvement planning process. Ideally, this happens in multiple stages, encompasses a wide range of data from multiple sources, incorporates the perspectives of multiple stakeholders, and takes place over several months. Be sure to build in opportunities to engage in analyses over multiple meetings, allowing for enough time to explore different angles on the same data and integrate additional data sources at each subsequent meeting.

The description below provides just one example of what the progression of analytic stages could look like. Keep in mind, however, that this progression is not completely linear. Data analysis is an iterative process and your planning team is likely to want or need to revisit data previously considered and gather additional data that will allow your school to analyze assets and challenges from different vantage points.

Getting the analysis process started

Since this guidance is grounded in the Turnaround Practices, a logical place for your leadership and planning teams and other stakeholders engaged in this process to begin is to become familiar with what the state has learned about what it takes for low performing schools to dramatically improve outcomes for their students. DESE's Turnaround Practices Digital Resource offers a useful introduction that integrates findings from multiple research studies with a practical field guide for schools that is based on Massachusetts schools' experiences.

As your school begins the data analysis process, it may be helpful for the sustainable improvement planning team (or ILT or other designated group) to conduct an initial set of analyses that explore state and local data that your school already collects (e.g., accountability data, state and local student assessment data, classroom observation data, student behavior data, data from routine surveys of teachers, students, and parents) before engaging the broader school community in analyzing and interpreting the data

At this stage it is also important to disaggregate data and look at student outcomes data by grade level and for historically marginalized student groups (e.g., students with disabilities, race/ethnicity, English learners, students from high poverty backgrounds) to gain a more detailed understanding of where there are gaps in outcomes that your school needs to address.

Taking stock of school and district systems, existing resources, policies, programs and initiatives, and practices. A critical next step involves using data to take thorough stock of existing school and district systems (e.g., teaming structures, schedules), existing resources (e.g., alignment of curriculum materials to the Massachusetts curriculum frameworks), policies (e.g., school discipline policy), programs and initiatives (e.g., Tier II instructional supports, PBIS), and practices (e.g., instructional practices, hiring practices) and begin reflecting upon the roles they may play in contributing to school gaps identified in your initial set of analyses - either as facilitators of or barriers to positive outcomes for all students. Ground your inquiry in a key set of guiding questions as you engage in this process. DESE's Guiding Questions Aligned to the Turnaround Practices resource offers an expansive set of questions connected to these practices that your school can utilize to help direct your own inquiry. The Additional Resources section, below, includes examples of self-assessments and surveys that your school may find useful at this stage in the process.

Conduct a Root Cause Analysis. The analyses conducted to this point lay the groundwork necessary to conduct an effective set of root cause analyses. As described in DESE's District Data Team Toolkit, a root cause is an underlying factor or condition that creates a problem and that, if addressed, would eliminate or dramatically alleviate an existing problem. (Note that the same approach can be applied when considering whether key assets might be leveraged or extended in ways that will support dramatic improvement in your school.) The Additional Resources section, below, includes examples root cause analysis protocols that your school may find useful as you conduct root cause analyses that your school may find useful at this stage in the process.

A root cause analysis can help a group with widely varying hypotheses narrow the field of contributing factors until it agrees on which one(s) will yield the largest return on investment if it acts on it. The process also involves an evidence review that examines the extent to which your data support or refute the various hypotheses set out in the protocol. Your school will complete the root cause analysis by selecting 2-5 key root causes to address in your sustainable improvement plan. Keep these root causes at the forefront of your planning as you begin the process of articulating strategic objectives and selecting specific strategies that directly address these root causes. (See guidance for "Strategic Objectives & Initiatives Aligned to the Turnaround Practices").

General Recommendations and Suggestions for Engaging in a Meaningful Analysis Process The following recommendations and suggestions are just that--ideas to support your school as you engage in this process.

Data to consider for analysis. Engage with a wide range of data, including both qualitative and quantitative data related to student achievement, social-emotional data, monitoring/targeted site visit data (MSV/TSV), student and staff demographics, educator evaluation data, stakeholder perceptions, and school processes (e.g., staff hiring, student referrals, curriculum review processes) as part of this process. See the Additional Resources section, below, for Specific Data Sources to Consider for Assessment of Assets and Challenges & Root Cause Analysis.

Make strategic decisions about whether your analysis requires additional data that will help your school get a fully rounded understanding of a particular challenge area. Focus groups, interviews, and surveys are relatively easy to implement mechanisms for gathering additional input that will add value to your school's self-reflection process. For instance, they can be used to help your school reflect upon school culture, climate, and practices from the perspectives of students, staff, parents, and other stakeholder groups. Your school may also find it useful to utilize self-assessment tools to gather data that will benefit the analysis process. See the Additional Resources section, below, for examples of self-assessments and surveys.

Use multiple data sources to explore a particular challenge, topic, or theme. Doing so will make it possible for your school to develop a more complete and rounded perspective on the issues addressed. For instance, a school could:

• Compare MCAS proficiency rates with course pass/fail data and look at the extent to which students passing coursework are demonstrating proficiency on MCAS;
• Survey or conduct focus groups with students, parents and teachers about similar topics (e.g., school climate and culture, rigor of the curriculum) and contrast their responses to those topics.
• Reflect upon the extent to which MSV/TSV data reflect your own classroom observation data.

For additional ideas about data sources your school might consider, see the Additional Resources listed at the send of this section.

Consider using a neutral facilitator for the process. An experienced facilitator can skillfully guide members of the planning team and stakeholder groups beyond their usual level of comfort and ensure a deep reflection of the data. This is an approach that can help participants to see data in new ways and pave the way to hold honest conversations about what is or is not working well at the school.

Make use of protocols at every stage in the analysis process. Data and root cause analysis protocols offer structured approaches to guiding conversations and may be particularly helpful to members of stakeholder groups who may not be accustomed to engaging in conversations about data. There are numerous data protocols available that:

• guide participants through the process of objectively examining data and then progressively interpreting and discussing the implications of data;
• build participants' awareness and understanding of their viewpoints, beliefs, and assumptions about data while suspending judgments;
• help participants to replace hunches and feelings with data-based facts, examine patterns and trends of performance indicators, and generate "root-cause" discussions that move from identifying symptoms to possible causes of student performance.

Examples of some of the resources and protocols that SSoS Regional Assistance Teams frequently use with districts and schools they support appear in the Additional Resources at the end of this section.

Flexible Format Submission

• Describe the analysis process the school engaged in, incorporating information about:
• the teams/roles that led and guided the process;
• the steps in the analysis process;
• the ways in which other stakeholders were involved in data analysis, data interpretation, and root cause analyses;
• the types of data utilized in the analyses;
• the ways in which analyses focused on equitable outcomes for student groups (e.g., racial/ethnic, students with disabilities, English learners)
• Articulate the results of the analyses, incorporating information about:
• The key set of assets and/or challenges the school identified for additional root cause analyses and the data that led the school to focus on them
• The specific root causes related to those assets and challenges that your school singled out as the ones most important to address as the school set its strategic objectives and selected initiatives (i.e., those that, if addressed, the school hypothesizes will provide the biggest boost in terms of improvement)

Additional Resources

While not required, these resources may be helpful as you engage in this step of the sustainable improvement planning process:

Related data resources:

• Specific Data Sources to Consider for Assessment of Assets and Challenges & Root Cause Analysis

• Suggested Local Data Sources About Students

• Suggested Local Data Sources Aligned to the Turnaround

• DESE's Accountability Lists, Materials, and Tools Website

Assessing Systems, Policies, and Practices

• School Turnaround Leaders: Competencies for Success and (INSERT PDF"Self-Assessment School Leader") Self-Assessment: School Turnaround Leader Competencies (Public Impact)

• Instructional Leadership Team Self-Assessment (Adapted from NISL)

• PLC Survey/Self Assessment

Curriculum Resources

• Ensuring Curricular Coherence

• Assessing Your Curriculum Landscape

• Aligning Curriculum to Massachusetts Standards

• Massachusetts CURATE (Curriculum Ratings by Teachers) Resources

• MTSS Self Assessment and Resources

• Equity, Inclusion, and Opportunity: Addressing Success Gaps, Indicators of Success Rubric (IDEA Data Center)

• WIDA School Improvement System (school-wide leadership assessment and feedback system to support teaching English learners)

• Conditions for Learning Surveys

• School Climate Survey Suite

Data Inquiry and Root Cause Analysis Resources and Protocols

• Atlas: Looking at Data Protocol

• Data Driven Dialogue

• Equity Inquiry Protocol Directions FAQ

• DESE's Planning for Success Analyzing Data for Root Causes resource

• The Five Why's – Jefferson Memorial Video

• Root Cause Analysis Protocol: Fishbone

• The Opportunity Atlas — Map of Economic Mobility Data

• Chronic Absence across the United States, 2015-16 School Year — Heat Map

Related data resources specific to middle and high schools:

• The ABC's of High School Success

• Early Warning Indicator System (EWIS)

• Analysis of Dropout Data

• Success After High School DART