Friday, February 11, 2011

The "impact" of Teacher Unionization and Dollars spent on NAEP proficiency scores

There has been a lot of talk recently about whether money matters in education and whether union and/or tenure are good or bad. So recently I went to find some data that might give us at least a general characterization. I found a report released by the NAEP with state scores for 4th and 8th grade reading and math proficiency in 1992 and 2005 (NAEP scores, although only representative, have the advantage of being consistent across states. It is of course debatable how much value they should be given because of their representative nature, but people do use them as 'evidence' on both sides of the debate). Here is the data in excel format along with dollars spent per student from

I sorted the states by percent proficient in 2005 and then looked at two things for each state: their rate of unionization (based on data from and how much they spend per student. If unionization is bad for kids (as the argument goes, since it places teachers' needs above kids' needs) then we'd expect to see performance to drop as the percent of unionization increases. Similarly, if the amount of money spent did not matter for education, then we'd expect to see no correlation between the performance and dollars spent per student. As it turns out, performance not only doesn't decline but it even increases with an increase in unionization. Similarly, an increase in kids' performance accompanies an increase in per-student dollars spent (actually, this should not be too surprising given that its likely the rate of unionization will generally correlate with dollars spent).

Because my goal is not to be an ideologue, I will readily admit that correlation does not equal causation. In fact if I were to attack this data, I'd use that as one point, the other being the representative nature of NAEP scores. Here is a good discussion on why NAEP state score comparisons should be done with caution. Regardless, correlation is a useful indicator in and of itself, and if the graphs were not consistently indicative of these correlations I'd be much more likely to question whether NAEP scores mis-represent the population by virtue of their representative nature.

A couple clarifications. Because it is difficult to 'see' a slope when the rate of unionization data is graphed directly on a state by state basis (variation between states that are 100% unionized vs ones that are 0% unionized dont provide much value) I did a running average of 10 states to get a more generalized trend to the data. In other words, the first rate of unionization value is an average of the lowest 10 state's rates (1-10), the next is an average of the next 10 (2-11) etc.

NB: the reason there is a dip near the right hand side of the math scores is that there are two states with low single-digit unionization (Texas and South Carolina) that score at about the 75th percentile among a number of states that are almost 100% unionized.

Please feel free to provide an alternative interpretation.


sorted by % proficient 4th reading 2005 NAEP

sorted by % proficient 4th math 2005 NAEP

sorted by % proficient 8th reading 2005 NAEP

sorted by % proficient 8th math 2005 NAEP


PoliticAli said...

You did it. I have more stuff to read now. Look forward to following your blog.

PoliticAli said...

I believe that the conservatives are using this economic downturn to eliminate the last remnants of union representation, which is in the public sector. The conservative governors are shouting crisis and pointing to unions as the cause and their elimination as the solution. Last Sunday I saw Fareed Zakaria on his GPS show on CNN interview Jamie Diamond, the CEO of Merrill Lynch- Chase, who, when asked about the financial crisis in municipalities, mentioned that one needs to separate the state government from the municipalities. He pointed out that the states have options and cited California as an example saying the California’s deficit is only 1% of the State’s GDP and that a 1% increase in taxes would solve the problem and not be unreasonable. As a Californian, how do you feel about this tax increase as opposed to cutting jobs?

navigio said...

Thanks PoliticAli. Yes, I took the plunge.. now I hope I have enough time to contribute to my blog.. and comment on yours. :-) (I saw your recent post on education and meant to follow up..)

Yes, I agree that much of the union attacks (today was quite interesting in Wisconsin) are politically motivated. I even heard Chris Matthews say something like, "remove the unions today, hurt he democrats in 2012." (Unions are big donors and generally politically powerful)

I have also been having some thoughts about the fact that without unions, all state workers would be subject to firing as they neared retirement to avoid things like paying pension benefits. Though, once unions go, pensions would probably not be far behind. I need to go back and read some of the context surrounding why public employee unions came about in the first place (there is usually a good reason for everything when viewed in context). I think people underestimate the need for them.

Yes, the question of raising taxes is actually quite a tricky one. We recently had a parcel tax election in our city that would have imposed an additional slightly more than $100/year on homeowners specifically intended for overcoming part of the budget deficit for our local school district. It was defeated. Note that based on the average home value in this area, yearly property taxes for homeowners is nearing the $10,000 mark. It's kind of insane that someone who pays that much would have such a problem with paying a bit over $100.

I am not a good person to ask on taxes, because I personally have no problem paying them. I understand that as a society we need to provide for all of our members and this costs money. Obviously no one wants waste, but I believe dealing with waste can be independent of the question of taxation itself. I even understand that sometimes in such a complex system, you will have people who defraud the system (the public system does not have a monopoly on fraud, I might add) and accept that some level of that will likely always exist. I could never buy the argument that we should remove welfare, for example, for the people who need it because there are some who defraud it.

That is the purely fiscal side of the question though. I know there is also the question of whether subsidies impact behavior, e.g., someone may be more willing to remain unemployed when there is a financial benefit to it. I have even heard some more libertarian-minded people claim that entitlements are a violation of the receivers freedoms because of the impact it can have on their behavior. Thats an interesting point of view.

Our Governor is going to put to the voters the question of extending some recent tax increases that are set to expire. We generally have a history of not approving additional taxation though and its rarely because of fiscal beliefs, rather philosophical ones imho. The next few months will be interesting.

PoliticAli said...

Hi Navigio;
One of the things people do not take into consideration is that the larger and more complex a society becomes the higher the cost of running it and the relationship of cost to complexity is not linear.
For some reason that is yet not clear to me, the conservatives get real up tight about non-entreponeurial individuals leading a good life.
I need to organize my thoughts a bit first but I want to post a pice on the "market place" and the disadvantage people have selling their labor to the public or private sector if they do it strictly individually.
Keep up the good work. I like the fact that unlike me, you back your stuff up with references and statistics.

PoliticAli said...

Should I give up or are you going to find time to put more thoughts on your blog?

navigio said...

Hi PoliticAli, I apologize for not having the time to post on my blog, and even more so for not having the time to comment on yours. I did read your assault on unions post and wanted to comment on it but have not had the chance. Unfortunately my job sometimes leaves me more time than at others and this is one of those lacking times. I will see what I can do to force myself to budget some time. Btw, consider this a compliment: your blog entries are quite intricate and thought provoking. They thus require much more thought that most of the other things I can respond to in a minute or two (eg most studentsfirst stuff). It is my nature to attempt to make my responses of equal caliber and this sometimes means putting them off until I get a larger, quieter block of time.. :-)

PoliticAli said...

Thanks. I understand. Priorities. I am retired and have some time on my hands though may look into something more meaningful to do than rant and rave.