What data do districts need
to determine progress?
|
What data do districts need to determine progress? |
|
|
|
|
Superintendents and school boards need good data to inform their decisions for improving student achievement. Though data-driven decision making is a critical and well-accepted strategy, what data districts use makes all the difference. Most districts have mounds of data; determining which data are relevant is key. To meet their AYP target, districts need two sets of data. Their AYP data — state assessment data, participation rate, and other academic indicator — will tell them in what areas they met or failed to meet the AYP target. Their accountability data are distributed by the state often electronically. New NCLB guidelines for state reporting require data be disaggregated by the eight subgroups. The second set of data the district needs is not as easy to come by and may not have even been identified or defined. The district needs ongoing data about where their students and subgroups of students within schools are performing in relation to their attainment of the content standards assessed on the state tests. School districts need to monitor the progress that their schools are making by determining where their students are in relation to the content standards on an ongoing basis. They need to determine whether schools have in place the necessary monitoring system, collaborative examinations of student work, and professional development to support teachers in understanding where their students are.
How do we analyze our AYP data? How should we monitor ongoing student achievement data? How do you communicate the data? How should we collect and store all this data? Resources [top] How do we analyze our AYP data? It is only after the school district goes through a thorough data analysis and problem clarification process that school boards and superintendents have the information they need to inform district priorities and resource allocation. Understanding what the state assessments measure and how they are scored will help districts better interpret and use the results. Data presented in a clear format usually raise questions rather than answer them. The work for school districts, as it is for schools, is to interpret the state assessment data, identify low performing areas, clarify the problem in those areas, identify an action plan to improve student achievement, monitor progress and revise the plan when progress is not sufficient. School boards need to focus on the questions they need to ask to ensure the district has thoroughly analyzed the state assessment data, clarified the problem, and developed a monitoring system. After the district has analyzed their AYP data, they will need to clarify the district’s problem in low performing testing areas. Why do the data look like they do? What are the contributing factors and root causes that are impeding student achievement in that area? What problem does the district need to address to get better results? There are five processes that need to be in place to hit an instructional target and could serve as a basis for the clarification of your problem. These key processes include: Understanding the TargetOnly after the district has completed the problem clarification process are they ready to identify strategies to address the problem and allocate appropriate resources. Clarifying why the data are what they are will increase the odds that the strategies a district selects to improve student performance will produce the desired results. Districts who have participated in thorough problem clarification should be able to answer the following questions:
[top] How should we monitor ongoing student achievement data? Both research and common sense support the notion that we need to monitor student performance on an ongoing basis. If we are ever to know where we are, we will need to know where our students are. But that is exactly what we don’t know. A major "aha" for many schools and districts is that they do not have sufficient information about where their students are currently performing on the content standards to make informed decisions about what interventions program changes or professional development would be useful. The data needed to monitor student progress toward attainment of the content standard indicators used on the state assessment data must be directly aligned to those same indicators. Although much of the current emphasis on using assessment data began with data from high-stakes tests, schools and districts that are the most effective users of assessment data have begun to recognize and capitalize on the power of classroom assessment. Districts have spent much more time analyzing annual state and district data than they have spent on looking at ongoing school and classroom data. A state assessment program is designed to assess the relative success or failure of schools and school systems, not to monitor more frequent progress. School system assessments are also snap shots of student performance at a single point in time. They are not designed to monitor progress in real time. In addition, by the time test results are available, schools have moved forward instructionally. The data may be an accurate historical view of a school’s performance but they are not very useful in understanding where you are now and what you need to do to keep on track. Many systems use norm-referenced tests that are not specifically designed to measure the state’s content standards. Some districts use grades to monitor progress, but rarely do grades show any relationship to the standards students are asked to achieve. We seem to have the worst of both worlds: tests and grades misaligned with the state’s content standards and information that is so dated that it is of little relevance to the classroom. What is needed is a monitoring system that is aligned with the state content standards and that yields timely and meaningful results for classroom teachers and for district level decision making and interventions. The challenge becomes how to develop such a monitoring system. Thomas R. Guskey argues that classroom assessment data are superior to the once-a-year test data that rank students. "Looking carefully at how many students know certain items on a given test can help a teacher decide to give corrective instruction, to present the concept in new ways, and to pay attention to trouble spots in a timely fashion. Using data to promote mastery learning yields far more valuable information than using data to measure this year's class against last year's." In a Phi Delta Kappan article entitled, "Inside the Black Box: Raising Standards Through Classroom Assessment," Black and Wiliam argue that standards can be raised only by changes in classrooms. "There is a body of firm evidence that formative assessment is an essential component of classroom work and that its development can raise standards of achievement. We know of no other way of raising standards for which such a strong prima facie case can be made." In The Intersection of Instruction and Assessment: The Classroom, Niyogi argues, "Assessment should be used not simply to judge how much kids know but to illuminate the nature of their knowledge and understandings in order to help kids learn….Common sense tells us that on-going, classroom-based assessment can serve this purpose. Teachers interacting with students will observe the nuances of their cognitive growth and development over time, their individual strengths and weaknesses in ways that would be extremely difficult, if not impossible, to capture through standardized or conventional testing alone." The evidence of where students are is the data superintendents and boards need to examine. Unless a district already has in place a series of formative assessments aligned with the state content standards, they will need to collect data from the school. It may seem easier to analyze related data that the system has available to them, but we would argue that the only data that will ensure districts and schools improve student achievement on state content standards are data that are directly related to student performance on the standards. As districts attempt to develop monitoring systems that provide information on where their students are in relation to the standards, they will need to understand both what they want students to know and do and how to collect evidence of learning. If your state or district hasn’t defined proficiency (and many have not) on the standards, then your schools will need to define proficiency and you will probably note a lack of consistency in school’s expectations and assessments. To obtain this information teachers must understand what makes a good opportunity for students to demonstrate what they know and can do and then to provide students with ample opportunities to do so. As Stiggins reminds us, there are "two conditions essential to schools’ efforts to integrate assessment into the teaching and learning process:
[top] How do you communicate the data? All stakeholders are critical players in the reform process. Therefore, as your student achievement data becomes available, you should inform school staff, parents, board members, and other stakeholders. Sharing successes often produces additional support for your programs. Reporting failures may have the opposite effect. However, not reporting bad news may be more damaging. Keeping bad news from others results in a lack of trust and unchecked rumors that are sometimes worst than the truth. It is much better to share the facts and identify areas of concern with a game plan for further analysis or input for a solution Of critical importance is the fact that the results are well publicized through the print and electronic media. Most educators detest the idea that scores of any test are publicized, usually in a straight-ranking format, thereby comparing one school’s scores to that of others. Many people do misuse such information, making incorrect inferences that schools with higher overall test scores are better schools. We know that this is not the case. In many instances, schools that have high test scores are actually adding less value to the educational experience than schools with lower test scores. It is not the test score itself that is so important; rather, it is the progress that the school/district has made and the value the school/district is adding to student learning. Thus, a school that started with low achievement may have done much better than a school with high achievement, even though the low-achieving school may be considerably below the score of the high-achieving school. Nonetheless, unless scores are well publicized, there is no pressure on schools to increase student achievement and, once high, to maintain the achievement. Once again, states that have high-stakes assessments normally release the results in a manner that leads to direct comparison, and the media is only too happy to follow suit by publishing the results. If a state does not publish such information, it is important for the district itself to publish the data it has on its own schools. Otherwise, there is little impetus for change that would lead to increased student achievement. As Linda Dawson describes in an AASA School Administrator article, "The savvy superintendent now realizes that good data, presented in the context of board-adopted and district-wide goals for student achievement, can move a board to focus on core performance issues. Those include supporting administrative means to improve achievement while advocating for moral and fiscal support of district efforts throughout the community." Dawson shares four guiding principles for how superintendents should present data to their boards.
What we really need to create is a culture where the data are analyzed to raise questions that often require additional data and to provide information to help determine next steps. Data-driven dialogue assists boards in making meaning of data, in exploring all participants’ perspectives, and in distinguishing intuition from reality. AYP gives all of us a fresh opportunity to use data in this manner and start to align our data discussion with our student achievement goals and needs to provide a more coherent game plan. Achievement Now! How to Assure No Child is Left Behind by Dr. Donald Fielder [top] How should we collect and store all this data? One reason that districts have over-emphasized state and district assessment data is that it is readily available — provided by the state in the first case and possibly by the testing contractor in the second. There is no question that the collection of classroom level data is much more complicated and not in place in most districts. For that matter, it’s not in place in most schools. This is a huge shift in the way schools have traditionally done business. If school districts recognize the critical importance of schools knowing where each of their students are in relation to attaining the content standards, then districts need to consider how to ensure that schools have this in place. Accountability data are stored in large databases that contain student or school assessment results and other indicators and background variables. The challenge is to make these data available and useful for several interrelated purposes: monitoring schools and districts, making policy decisions, supporting school improvement, and helping teachers do a better job of diagnosing learning difficulties and placing students. It is important that schools collect data on an ongoing basis. Some schools collect data on a quarterly basis using a common assessment. Although quarterly assessments provide information about student performance, this strategy has a number of drawbacks: 1) it often results in teachers viewing the monitoring piece as external to their instructional program, 2) it is not frequent enough to inform instruction, and 3) it does not move staff to the ultimate goal. The ultimate goal is to have teachers instructing and assessing the indicators they are responsible for teaching on an ongoing basis so that they will know where their students are at any given time and will use the information to direct their instruction. In a Technology and Learning article, "The Administrators Guide to Data-Driven Decision Making," Todd McIntire describes what he found when he conducted an evaluation of the data that was being captured, where it was stored, and how well it was maintained in his district. His investigation revealed "a lack of clear and comprehensive data management standards had allowed us to devolve into a mess of incompatible parallel systems that gradually lost sync with each other. As a result, our ability to use data to make well-informed decisions was severely compromised." Many if not most districts are faced with the same challenge. Defining the standards for information collection and documentation are critical actions for districts as they develop their student data-collection systems. NCES has described nine key elements of an adequate statewide student data-collection system in a report entitled, "Essential Elements of Statewide Data-Collection Systems." Most of these are premised on the ability of the district to provide the information. Therefore, it would be advantageous for a district to develop a data collection system that also has most of these elements including
[top] Resources for What Data do Districts Need to Determine Progress?  "How Classroom Assessments Improve Learning" by Thomas R. Guskey in the February 2003 issue of Educational Leadership argues that the assessments most likely to improve student achievement are those that teachers create. "Improving Teaching and Learning with Data-Based Decisions: Asking the Right Questions and Acting on the Answers" by Nancy Protheroe in the Summer 2001issue of ERS Spectrum. "Inside the Black Box: Raising Standards Through Classroom Assessment" by Paul Black and Dylan Wiliam in the Phi Delta Kappan argues that there is firm evidence showing that formative assessment is an essential component of classroom work and that its development can raise standards of achievement. Cawelti, Gordon, and Nancy Protheroe. 2001. High Student Achievement: How Six School Districts Changed into High-Performance Systems. Arlington, VA: Educational Research Service. Niyogi, Nivedita S. 1995. The Intersection of Instruction and Assessment: The Classroom. Princeton, NJ: Educational Testing Service. Schafer, William D., et al. Undated. Study of Higher-Success and Lower-Success Elementary Schools. Online: www.mdk12.org/practices. Stiggins, Richard J. 2001. "The Principal’s Leadership Role in Assessment." NASSP Bulletin (January 2001): 13-26. Dawson, Linda J. "Providing Data to Your Board." The School Administrator Web Edition (December 2002). Lorraine Keeney. "Using Data for School Improvement." Annenberg Institute for School Reform (1998). Based on proceedings from a 1998 conference, this report describes how Annenberg Challenge sites have used data for school improvement and accountability. Yeagley, Raymond. "A Forum for Becoming Data Savvy." AASA The School Administrator Web Edition (December 2002). Using Data to Improve Schools: What's Working, AASA, June 15, 2002 "The Administrator's Guide to Data-Driven Decision Making"By Todd McIntire from T&L Magazine. A step-by-step guide to setting up information management systems for decision support. Results, The Key to Continuous School Improvement by Michael Schmoker, Association for Supervision and Curriculum Development, Alexandria, Va Building an Automated Student Record System: A Step-by-Step Guide for Local and State Education Agencies. Publication of the National Forum on Education Statistics Additional Resources Improving School Board Decision-Making: The Data Connection Web site. Inside you'll find PowerPoint presentations, quizzes, hands-on training tools and other helpful materials you won't see in the book. These materials are for school board members who want to know more about how to use data to make good decisions for children in public schools. Trainers who work with school board members also can use these materials at state and national conferences or in local training sessions. While covering key topics such as standards, professional development, and informing stakeholders, this guide helps schools see clear relationships between what they are doing in each topic and its impact on their school wide improvement effort. |
||||||||||||||
|
||||||||||||||
 |
||||||||||||||
| How Have States Implemented AYP? |
| How Have States Designed Their Accountability Systems? |
| How Do States Report Progress? |
| How Do States Provide Technical Assistance to Districts and Schools? |
 |
