It is not the quantity of the data that counts, but how the information is used to help improve instruction and learning
If interim assessments are high quality, the school’s own data is far more powerful and useful than any external tests, to modify instruction
Assessments can be a vital component in our efforts to improve education. But as long as we use them only as a means to rank schools and students, we will miss their most powerful benefits
There is evidence that effective use of data in schools can lead to dramatic improvements in instruction and learning. The challenges however are two fold; One – setting a consistent process; Two – developing targetted modifications in teaching
We’ve often heard the notion that schools have a lot of data on student learning but don’t effectively use it to improve teaching and learning. There is a clear factual basis for it, yet the statement implicitly assumes that it is ignorance and probably incompetence in schools that lead to this. The facts are otherwise and there are several barriers to effectively using data to improve student outcomes. Based on our experience of supporting schools in Dubai and Sharjah in becoming more data driven, we have listed key steps that school leaders can take to embed this practice across schools.
Start with Assessment philosophy, not data
Everything done in a school needs to start with pedagogy and use of data is no different. Start by asking yourself the most fundamental set of questions to ensure consistency and clarify on the WHY of using data to improve instruction
- What is the purpose of assessments? What different kinds of assessments (work scrutiny, formative, summative, group based etc) are necessary for us to have all the information we need?
- What is assessment in the classroom? How do we know we are assessing well and what data should we collect?
- What are the key student achievement metrics we need to track? Do we understand how they are measured ?
- Which external/ standardized benchmarking tests do we need or are required to do? What is the purpose of each of these and how does it fit into our assessment philosophy?
Answering these will help clarify to the team why we do what we do and this should be captured clearly in the school’s assessment policy document, which should then be shared with teachers, students and parents. Good assessment policies usually go much beyond the types of and number of formative, summative assessments the school will do and how it will grade and promote. It will specify what is being measured, how it fits into curriculum objectives or learning standards. It will also specify how to interpret reports and where students, teachers and parents can find more information to fully understand the student’s progress, how to set targets and plan next steps.
Plan Professional Development – Separately for Leaders and Teachers
Leaders are expected use data to track progress across groups, benchmark results and support teachers in making better use of insights. Teachers on the other hand have to collect high quality data as a first step. Then they need to learn how to use the data to improve instructions for groups as well as individuals. It is therefore important to allow your teachers own their professional development process, particularly in creating their own interim assessments (see steps three and four). Part of the professional development training is learning the process. The other part is looking at case studies of specific insights that they need to look out for and what to do with those insights to achieve significant student growth through the use of data-driven instruction. An expert organization, individual and another school which has faced similar challenges and is slightly ahead in their learning curve will be ideal partners in helping develop the relevant professional development materials on use of data for effective teaching and learning. See one such example here
Create High Quality Assessments
This is the most important step from a practitioner’s point of view – for it determines the quality of data. This involves both practice as well as understanding on how to design high quality assessments to effectively garner actionable insights. This involves setting clear assessment objectives that allow to measure both spread of learning across targeted learning standards (which come from the curriculum’s learning objectives and standards) and depth of learning (or mastery) in the topic and skills (possibly structured around Bloom’s taxonomy).
Create a team of teacher leaders to provide initial trainings on creating effective interim assessments (for about two hours). Once assessments are written, ensure a rigorous “moderation” process to meet clearly laid out criteria:
- Do the assessments cover the right objectives, standards and strands?
- Are they at the right level of rigor to lead to success on end-goal assessments. (For this calibration, have teachers look at practice papers of TIMSS, PISA, ACER or GL, PARCC , ACT or SAT based on your curriculum)
- Do the internal assessments mimic the format that students would encounter on those end-goal assessments?
Have teachers borrow questions from high-quality assessments — from the same practice tests they referenced to assess rigor. Also refer to resources like the Vanderbilt Center for Teaching — along with trial and error and collaboration with colleagues — to develop their own expertise in creating high-quality questions that pinpoint what students know and don’t know.
It would also be useful for the school to create their own online assessment and maintain these high quality questions in digital format for use as formatives. The effort to create new questions and align it to standards is high effort and expensive. It maybe useful to have an online assessment platform (like Koan Assessments, Literatu, Socrative) which allows tagging questions by standards and by Bloom’s taxonomy level.
Use an effective IT system for capturing data consistently
Without detailed data reports, data-driven instruction does not work. It is important to put all of your tests on a flexible student information system (like SchoolZen, Blackboard) so that the data can be exported or analyzed. Have your IT vendor or your data manager create insightful reports (or use the format templates from here). The Excel spreadsheets (or any other data dashboards) should be designed to provide detailed information by student, question, and standard. This process is quite technical and time consuming, so significant planning and resources must be devoted to getting it right. (Please reach out to us if you think we can help).
Develop a process for quick and reliable data analysis
An IT system can help generate reports quickly after an assessment (in about 48 hours or so), after which it is important for structured discussions to gleam insights from them. Making time for this analysis, and for the meetings to discuss observations, immediately after the assessment is essential. For the week after the assessment we recommend cancelling faculty meetings to allow teachers time for preparing their analysis.
We then recommend you to pair each teacher with a fellow teacher leader or your data manager for a 30-minute data-analysis meeting. This provides an opportunity for rich discussion of what the data shows — and what needs to be done next. We recommend to embed professional development in the process, as colleagues brainstorm instructional strategies that might be used for reteaching standards that were not yet mastered.
Re-teach for essential standards
Merely repeating the instruction or giving more home-work or scaffolding materials may not serve the purpose. To quote from ASCD’s Educational Leadership:Using Data to Improve Student Achievement:How Classroom Assessments Improve Learning
High-quality, corrective instruction is not the same as reteaching, which often consists simply of restating the original explanations louder and more slowly. Instead, the teacher must use approaches that accommodate differences in students’ learning styles and intelligences (Sternberg, 1994). Although teachers generally try to incorporate different teaching approaches when they initially plan their lessons, corrective instruction involves extending and strengthening that work. In addition, those students who have few or no learning errors to correct should receive enrichment activities to help broaden and expand their learning. Materials designed for gifted and talented students provide an excellent resource for such activities.
It is also important to follow up with your teachers on their reteaching. During collaborative teacher meetings and meetings with administrators, ask each other:
- How did it work?
- Will you use that approach from the outset next time?
- What do your students still not get?
Involve students and when required parents
To quote again from ASCD’s article:
To become an integral part of the instructional process, assessments cannot be a one-shot, do-or-die experience for students. Instead, assessments must be part of an ongoing effort to help students learn. And if teachers follow assessments with helpful corrective instruction, then students should have a second chance to demonstrate their new level of competence and understanding. This second chance helps determine the effectiveness of the corrective instruction and offers students another opportunity to experience success in learning.
For e.g. English teachers have long recognized the many benefits of a second chance in writing assessments. They know that students rarely write well on an initial attempt. Teachers build into the writing process several opportunities for students to gain feedback on early drafts and then to use that feedback to revise and improve their writing. Teachers of other subjects frequently balk at the idea, however—mostly because it differs from their personal learning experiences.
Reflect and iterate practice to review effect over time
Using assessments as sources of information, following assessments with corrective instruction, and giving students a second chance are steps in a process that all teachers use naturally when they tutor individual students. If the student makes a mistake, the teacher stops and points out the mistake. The teacher then explains that concept in a different way. Finally, the teacher asks another question or poses a similar problem to ensure the student’s understanding before going on. The challenge for teachers is to use their classroom assessments in similar ways to provide all students with this sort of individualized assistance.