a PISA what?

A study attempting to characterize the level of student achievement by country was released recently. It also concludes that continued lagging in math performance could have a negative impact on the future of our GDP. Although the report does not attempt to prescribe policy to address the issue, some concerns did arise for me regarding the methodology.

This study required a mapping of NAEP scores to PISA scores. Both tests are only representational (not taken by all students). One test was given to a subset of students while they were in 8th grade, the other was given to a different set of students (from the same graduating class) while they were in 10th grade. A couple points to make about that: first off, the difference in proficiency between the 8th and 10th grade can be enormous, even for the same kids, let alone from different representational samples. Second, about 30% of kids entering high school dont graduate with their peers. About a third of those (ie 10% overall, drop out before 10th grade). In theory that should have a positive impact on 10th grade scores, however, the flip side is that two-thirds of students who will eventually drop out are still there to be tested in 10th grade. It seems clear that would have a net depressant effect on scores. That of course does not mean that those dropouts dont exist as part of our student demographic, but the point is that even if they have some skills, the attitude associated with dropping out will likely lead them to underachieve on any test, which would misrepresent actual skill.
I also find it concerning that there is an statement in the report that 'even the richest' states dont do well, and some examples of the 'richest' are New York, California and Florida. These states have some of the highest levels of minority and poverty concentrations around and have increasing charter populations, which is leading to increased segregation and poverty/minority concentrations. The report also uses white scores as a way to try to dispel the claim that some US subgroups score well. Despite the fact that whites are correlated to high income in many areas, there are many others where the exact opposite is true. I question why the Asian subgroup was not examined and I question why the word 'poverty' does not even show up once in this report or why that metric was not addressed instead of the racial one.
Regardless, the report is descriptive more than prescriptive, so at least it is trying to identify the cost associated with some reality. Many of these same scholars have written studies that blame virtually all of this on teachers. I think it would be interesting to have a study of the impact of poverty on our education system, and perhaps thus our GDP. Having a good grasp of the input to our education system is necessary if we are going to try to assess why the output of that system is not as we would like.