Filed under: Education
“Data analysis is so trendy these days that Brad Pitt is getting millions of people to sit through a movie about quantitative methodology.” – Andrew Rotherham
Rotherham argues that quantitative education analysis fails for three reasons: poor data quality, lack of common definitions, and little respect for evidence. While his article mostly applies to using quantitative methods to evaluate teachers, his arguments can also be applied to the larger questions of education policy and reform.
First, poor data quality. Rotherham notes that the data quality in education is poor because of the limited use of education data, the differing standards for the collection of data across the states, and the lack of transparency. Yes, the educational system is not very transparent – or at least it wasn’t before NCLB was enacted. Yes, there is a lack of data on how to correctly evaluate teachers. And yes, data is used very sparing in the educational policy sector (kind of). Still, it is wrong to argue that the data quality overall is poor. In fact ,one of the US Department of Education’s main functions in the collection of data. This data collection – at the state, national, and district levels – expanded drastically since the implementation of NCLB. Heck, AYP is all about data. I don’t think that the educational system lacks data – it’s certainly out there, the issue is that we don’t have enough of it. The other issue is that, like many social science fields, it’s often difficult to quantify the variables that we’re dealing with. That gets to Rotherham’s second point – lack of common definitions.
Without common definitions of “good teacher,” “proficient,” or “graduation standard” (hell, even 4-year graduation rate), it’s extremely difficult to quantify variables. But that doesn’t mean that it couldn’t be done and that certainly doesn’t mean that it shouldn’t be done.
Finally, Rotherham points to little respect for evidence. Education policy is inherently political. When the big wigs in DC, at the state house, or in the school committee don’t listen to the results of your quantitative study, they’re not just doing it to disregard the methodology, they’re doing it because they have their own political agenda. When all of the evidence points to how extended learning time helps students learn more in school, how do you think the teachers’ unions can disregard this without completely disregarding the evidence. Yet, as political science and science more generally have shown, people can disregard your results, but that shouldn’t stop the research.
So break out STATA and R everyone. Let’s quantify education policy!
Leave a Comment so far
Leave a comment
