Among education circles, the phrase “data-driven instruction” has been frequently invoked and often practiced. However, for the general public the word ‘data’ is often associated with the dismal results from standardized exams published in local news. People are concerned about the state of education in urban communities and sometimes wonder what schools are doing. There is indeed cause for concern regarding the state of education in urban areas (Irwin et al., 2023), and the general public, as taxpayers are stakeholders in public education. While data-driven instruction should be part of the solution, exploring what we mean by it might provide a helpful perspective for everyone.
How Did the Concept of Data-Driven Instruction Emerge?
In 1965, the Elementary and Secondary Education Act (ESEA) was established and signed into law by President Lyndon B. Johnson. The primary objective of the ESEA was to promote equity in access to educational opportunities. Then, in 2001, President George W. Bush introduced the No Child Left Behind Act (NCLB), which brought a significant emphasis on accountability. NCLB mandated that states implement assessments in literacy and mathematics for students in grades 3 through 8, in order to qualify for federal funding. This led educators to humorously dub the act “No Child Left Untested.”
In 2009, President Barack Obama introduced Race to the Top, which emphasized competitive grants and federal prescriptions, intensifying the focus on data use in education. Currently, we operate under the Every Student Succeeds Act (ESSA), also implemented in 2015 by President Obama, which aims to return control to state and local districts by relaxing some of the prescriptive conditions (U.S. Department of Education, 2016). Despite this shift, the ESSA did not lessen the emphasis on data and benchmarks; in fact, many educators believe it further intensified the reliance on data in education.
So What is Missing?
The concept of using data as a tool to guide instruction has been echoing within the community of math education for decades. The call for data-driven instruction has only intensified, due to the learning loss caused by the pandemic and the subsequent recovery efforts implemented by districts nationwide. In my experience, math teachers generally welcome the practice of using data to inform their instructional decisions; however, they often bristle and have reservations at the idea of data being weaponized for accountability purposes, which may feel punitive. Thus, any work towards using data must acknowledge the real concerns which many have about its being misused, or even worse, misinterpreted.
The remarkable thing is that when we shift the conversation, from accountability to the learning experiences which we want our students to have, such reservations melt away. Moreover, teachers feel that data generated by a variety of assessments has always been a part of math instruction, and that they have always utilized the data produced by the many interactions between teacher and learner.
So what is missing? I believe that what is lacking is a shared understanding of the different types of assessment and their utility (as well as their limitations). Perhaps we are also missing a common perspective between those who create policy, and those who create the classroom conditions for learning, every day. These are difficult issues, and interpreting data is not as straightforward as some may think. As it turns out, moving from understanding data to harnessing it is not a simple process. Rather, it is a complex activity that requires a great deal of effort, support and professional learning. (Bill & Melinda Gates Foundation, 2015)
Insights Gained from a Variety of Assessments: Data
The public is likely familiar with state exams administered toward the end of each school year, as every state has its own version. However, the public may not be aware that the data generated by these exams is not particularly useful for making instructional decisions. Rather, their utility is limited to measuring the effectiveness of policy, but not necessarily the effectiveness of teaching and learning. How students perform on state assessments depends on too many confounding factors. From parental involvement, socioeconomic status, and test anxiety to motivation, engagement, curriculum alignment, and access to resources, a student’s performance on a state assessment does not reflect learning and teaching alone. Therefore, other tools are necessary to assess the learning that occurs as a result of teaching activities (Hamilton et al., 2009).
The public is also likely to be familiar with other common assessments, often constructed by teachers in schools, such as departmental exams, mid-term exams, and final exams. Of course, most parents are familiar with special assignments that can generate a lot of data, such as science projects, reports, and essays. All of these fall within a wider category known as “summative” assessment, derived from the Latin root “summa,” meaning “sum” or “total.” This root reflects the concept of bringing together or summing up information. And precisely because the type of data these assessments provide is a ’summary’, the data they generate is not very useful for making instructional decisions. Again, if we are going to engage with the concept of “data-driven instruction,” then we need other tools.
Certainly, we have additional tools at our disposal. Among the most popular are screeners, diagnostics, and progress monitors. Together, these tools are considered key drivers of a multi-tier system of support (MTSS), which is designed to deliver data-informed support. A significant body of evidence suggests that analyzing the data generated by these assessments to provide tiered support is highly effective. Given its importance, MTSS deserves a separate discussion, which we will address in a future paper.
To Each Their Own: Formative vs Summative Assessment
The public might not be as familiar with the concept of ‘formative’ assessment. Derived from the Latin root “formare,” which means “to form” or “to shape,” formative assessment is intended for the sole purpose of informing instruction. While ‘summative’ assessment encompasses a wide scope of skills and topics, ‘formative’ assessment reveals specific information about a narrow scope of learning. Summative assessments are more concerned with general processes and ‘correct answers,’ whereas formative assessment focuses on detailed evidence of student work, ways of thinking, and making sense of the mathematics at hand, where only the ‘correct answer’ is of lesser importance.
Examples of formative assessment include diagnostics and certain performance assessments, which can be open-ended and provide insight into how students are making sense of concepts. Progress monitors are almost in their own class; however, because they are not intended for grading but rather to assess the effectiveness of a particular intervention, they can also be viewed as ‘formative’. In general, we can say that any ongoing assessment, as needed to calibrate instruction, would be considered formative. In contrast, summative assessments tend to occur at the end of the instructional process, and are used to evaluate student achievement and to certify achievements -not to inform instruction. Summative assessments are much less frequent and have utility to evaluate policy for sharing with stakeholders.
What Are Teachers Expected to Do?
Teachers need support to move from understanding data to harnessing data. Fortunately, many resources are available, including the powerful idea of re-conceptualizing formative assessment as Assessment for Learning. The clearest distinction between formative and summative assessment is captured in the description by Black & Wiliam, 1998, who referred to the former as assessment for learning, contrasting it with the latter, assessment of learning.
At a minimum, we understand that the enactment of Assessment for Learning is not about administering yet another test to be graded. That is already happening in most math classrooms, and more of it is not what is needed. Rather, a new vision is required—one that blurs the line between instruction and assessment. A better understanding involves enacting a vision where everything a student does is assessment, from peer-to-peer conversations and think-alone time, to answering and asking questions. If a student is quiet and looks confused, that too is a source of information for understanding where they are and what they know. However, the best enactment of Assessment for Learning occurs when this information is used to make adjustments to instruction minute-by-minute and day-by-day. (Thompson & Wiliam, 2008, 5).
In Summary
Learning and teaching are very complex activities, where humanity plays a role in a complex dance in which the teacher takes a leap of faith when planning to create the necessary conditions for learning, and deferring observation and experimentation (i.e., data) to a later time. However, this deferral should not be for “too much later” (i.e., summative); rather a dialogue with the data is needed (i.e., formative) to confirm that the hopes of creating the conditions for learning were, in fact, successful. A dialogue with the data makes any risk worth taking, and allows for responding to the learner’s needs, proving that what they are doing matters. In the absence of data, how does one determine which interactions are a “dead end” and which are fruitful? Data collected in the moment, when the learning is actually happening, can inform teachers’ efforts in real time. In contrast, data collected at the end of a process can reveal whether the efforts were productive or not, but too late to make any difference.
References
Bill & Melinda Gates Foundation. (2015). Teachers Know Best: Making Data Work For Teachers and Students. ERIC. Retrieved October 18, 2024, from https://eric.ed.gov/?id=ED557084
Black, P., & Wiliam, D. (1998, July 28). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. https://doi.org/10.1080/0969595980050102
Hamilton, L., Halverson, R., Jackson, S. S., Mandinach, E., Supovitz, J. A., & Wayman, J. C. (2009, September). Using Student Achievement Data to Support Instructional Decision Making. Institute of Education Sciences. Retrieved October 18, 2024, from https://ies.ed.gov/ncee/
Irwin, V., Wang, K., Tezil, T., Zhang, J., Filbey, A., Jung, J., Mann, F., Dilig, R., & Parker, S. (2023, May). Report on the Condition of Education 2023 Report on the Condition of Education 2023. Institute of Education Science.
Thompson, M., & Wiliam, D. (2008). Tight but Loose: A Conceptual Framework for Scaling Up School Reforms. In Tight but Loose: Scaling Up Teacher Professional Development in Diverse Contexts. Educational Testing Service.
U.S. Department of Education. (2016, May 31). Elementary and Secondary Education Act of 1965, As Amended by the Every Student Succeeds Act-Accountability and State Plans. Federal Register. Retrieved October 18, 2024, from https://www.federalregister.gov/documents/2016/05/31/2016-12451/elementary-and-secondary-education-act-of-1965-as-amended-by-the-every-student-succeeds