Concerns About Interpreting CAASPP Data Included in the SARC

The California Department of Education (CDE) provided the results of the California Assessment of Student Performance and Progress (CAASPP) to all public schools in the state for inclusion in this year’s SARCs. We have reported these test results in the district’s SARCs in a format that conforms with the CDE’s SARC template.

We believe that the CAASPP results as presented are confusing and run the risk of causing readers to question the validity of all the data included in the document, thus undermining a primary intention of the SARC: to provide a reliable way to measure and compare schools’ performances and conditions.

We see two problems with this year’s CAASPP results as presented in the SARC:

  1. Usefulness of the percentage-tested data. CAASPP results are reported by grade level for both English language arts and mathematics portions of the test. The grade-level results are also reported for specified subgroups of students:

    2. Unfortunately, the number the CDE provided for the Total Enrollment column was the same for all student groups. This means that the figures in the fourth column, Percentage Tested, were calculated using the total number students enrolled at that grade level. For example:

    While not mathematically incorrect, this calculation does not provide the reader with much useful information.

    More helpful (but not possible using the CDE’s data) would be to calculate the percentage of each student group that took the test. For example:

  2. Confusion regarding achievement-level percentages. The CDE’s SARC template includes columns for percentages of students whose CAASPP results fall into each of four defined achievement levels. These four numbers should add up to 100 percent of all test takers. In many cases, however, they do not. For example:

    In this case, the achievement-level percentages in the first row add up to 85 percent, leaving the reader to wonder what happened to the other 15 percent of test takers and perhaps to suspect the data are inaccurate.

    Unfortunately, the CDE did not include an entire category of test results, Invalid Test Scores, in the data it provided. There are a number of reasons why a student who takes the CAASPP might not receive a valid score. In the example above, taken from an actual school site, there were 74 test takers who did not receive valid scores. A more complete table might look like this:

We are exploring the possibility of recalculating CAASPP results to address both of these problems.