How Do States
Report Progress?
|
How Do States Report Progress? |
|
|
|
|
Presenting a blizzard of data is not much help to anyone. Performance reports need to simplify the presentation and emphasize what is important for the reader to attend to. States need to consider presenting their accountability data is a way that is useful and makes it less likely that the reader will misinterpret or over-interpret the information. All information should be available through an interactive web based system. By all means, continue to publish hard copies for those who need them but create them as PDFs from an interactive web based system. The advantages of investing in a web based system far out weigh the cost. NCLB requires states receiving Title I funding to prepare and disseminate annual report cards that, among other items, must include:
Reporting features to consider State Electronic Report Cards Examples of graphic displays Linking results to test questions and examples of student performance [top] Reporting features to consider There are a few rules to follow. Avoid meaningless rankings and comparisons that play to the 'horse race' mentality. While it is the public's right to know where a subgroup, school, and school system are in relation to the school district and the state, comparing the performance of all students is not a fair comparison. Comparisons are most meaningful when comparison groups are closely matched. For example, comparing the performance of the same racial/ethnic group among districts without also matching on socioeconomic levels is likely to result in the reader coming to an inappropriate conclusion. Scatter plots are an excellent way to show the relationship between performance and selected demographic variables such as socioeconomic status. They quickly allow the reader to compare performance of schools with similar demographic characteristics. The so called 'value added' models which attempt to identify the effectiveness of teachers and/or schools, in the absence of actual levels of student performance are prone to gross misinterpretation. They often mask the simple fact that students are not achieving. Keep it simple. Only the statisticians in the audience will have the interest and patience to plow through complex tables to glean meaning. For the rest of us, simple graphic presentations fit the bill. However, an important corollary is to practice full disclosure of the data behind summary reports. To those who want to examine the data at finer levels of detail, keeping it from them raises red flags. And, after all, they can get most of the data through the freedom of information act. When ever possible show trends. Nothing is more salient than a graph that summarizes changes over time. Graphically presented trends allow the reader to see where you started, where you are now, and the path followed. An important feature of displaying trends is that they illustrate the year-to-year fluctuations seen in most performance data. Do not underestimate the intelligence of your audience. Include in the displays standard errors of measurement, confidence intervals, error bars, etc. These statistical tools are invaluable in helping the audience understand when changes are meaningful and when they are not. Too often, the reader is left to guess whether year-to-year improvements or declines are significant or meaningful. Provide links to content standards, assessments, and performance descriptions. Percentages of students at the proficient level has meaning only if the reader understands what students were expected to know (content standards), how they were asked to demonstrate what they know (assessments) and what an advanced, proficient, and basic student performance looks like. Too often the reader is allowed to jump to their own interpretation of what the results tell them about student achievement. Sample student responses to selected test questions at the three proficiency levels are very helpful. [top] State Electronic Report Cards There are a variety of sources for locating state report cards. NEA Foundation for the Improvement of Education is the best pick for quickly locating state report cards. They do so without comment on the quality or comprehensiveness of the reports. The Council of Chief State School Officers web site is another comprehensive source. The National Center for Education Statistics web site includes an evaluation of what they deem the quality of and comprehensiveness of state report cards. [top] Examples of graphic displays NAEP trend graph mathematics Trend data with confidence intervals  Box and Whisker plots that show the distribution of student performance  Stacked bar charts showing performance at each proficiency level  Scatter plot showing the relationship between performance and socioeconomic status 
[top] Linking results to test questions and examples of student performance http://nces.ed.gov/nceskids/eyk/index.asp?flash=true TIMMS sample test can get to international data from the practice tests http://nces.ed.gov/nationsreportcard/reading/itemmapgr4.asp NAEP Item Maps Scale scores linked to sample questions http://nces.ed.gov/nationsreportcard/itmrls/srtab.asp Sample student responses to essay questions http://nces.ed.gov/nationsreportcard/itmrls/pickone.asp NAEP question search tool http://mdk12.org/mspp/high_school/look_like/english/intro.html MD High School Assessment English intro  |
||||||||||||||
|
||||||||||||||
 |
||||||||||||||
| How Have States Implemented AYP? |
| How Have States Designed Their Accountability Systems? |
| How Do States Report Progress? |
| How Do States Provide Technical Assistance to Districts and Schools? |
 |
