Wednesday, February 18, 2009

SAT Correlations

In my other post on the SAT, I had said that, as an indicator of college success, "Familial wealth is first, HS grades and SATs come in close together at 2nd and 3rd respectively. Most of the rest of the admissions packet is busy-work."

rightwingprof added some fuel to the the fire when he said ...
"I taught applied math in a business school. We did an internal study (it took us years to get permission from the university to collect the data), and with a sample space of over 7,000, found that the correlation coefficient between high school GPA and performance in our class was 0.03, while there was a 0.71 r between the SAT math score and performance."
Nothing like some hard data. Part of my statement was based on an article in the NYTimes by Peter Salins,
If we look merely at studies that statistically correlate SAT scores and high school grades with graduation rates, we find that, indeed, the two standards are roughly equivalent, meaning that the better that applicants do on either of these indicators the more likely they are to graduate from college. However, since students with high SAT scores tend to have better high school grade-point averages, this data doesn’t tell us which of the indicators — independent of the other — is a better predictor of college success.
It would seem ( R=0.03 ) that HS GPA is not the one and that SAT takes the lions share. Salins comes to similar conclusions later. I'm not, however, entirely convinced yet. HS GPA contains a broad range of courses and grades. SAT Math is fairly targeted to a subset. I find it perfectly reasonable that the overall HighSchool GPA vs. College Math correlation is R=0.03 while the SAT Math v College Math correlation is 0.7; I think HS math class GPA is the metric we should look at here. Anyone have such numbers?

I'm thinking of this year's graduates. Many have excellent GPA overall but that includes a lot of non-math curriculum; some are admittedly math-phobic. They may have been the beneficiaries of effort-based grade inflation but generally students' grades are relative - there should be noticeable differences based on math ability.

Exploration of this is definitely warranted but I'm figuring that SAT scores and "pertinent course GPA" would probably have similar prediction capabilities. After all, college is the start of specialization so why would we use a generalized average such as overall High School GPA to predict performance in a specialty such as math in a business school?


Just for informational purposes, let me add to this discussion a chart I constructed from the ETS's aggregate data on SATs in my state.
 

This trend repeats itself in every state, in every year I've looked at. Income is correlated with SAT scores. This is such an improbable cause-effect that I feel we must have a confounding factor. A simple question shows this easily, "Could I give your family $40,000 and raise your scores by 60 points?" "No, that's absurd."

The fact that SAT scores are also correlated with college success almost closes the circle for me.

I'm tempted to think that the underlying effect is that, by and large, parents are well-educated because they are motivated and intelligent, and have the money for an education or were motivated enough to scrape together the money. 

These characteristics, along with a little luck or athletic ability are what also lead to wealth in this country. Children of such families are likely to have the same characteristics as their parents and thus also be successful at college, an enterprise that takes wealth, motivation and intelligence.

Wealth, while not a cause, is nonetheless correlated and is the most easily measured indicator.

I have no controlled evidence or properly-conducted study to back this up. Any of my readers have some more fuel for this fire, one way or the other?

3 comments:

  1. As part of NCLB, states adopted tests to measure student progress, and in order to receive federal funds students had to show improvement on these tests. I found it strange that states were allowed to design and administer their own tests, rather than meet a uniform national standard (since it was federal funding that was involved).

    Our state created the hated WASL (Washington Assessment of Student Learning), which takes the most amazing amount of student prep and (last I heard) costs about $75/student to administer. I always wondered why the fed didn't just set the SAT as the standard since it is universally available, has a track history, is less expensive to run, and is required for most college admissions anyway.

    Thanks, your article added weight to my argument. Our state has just got a new superintendant of public schools, and the WASL is about to be scrapped. I haven't heard what they are planning to put in its place, but may I send your article along with my letter to OSPI? Not that they listen to parents, but maybe with the tighter budget they would consider it ...

    ReplyDelete
  2. You're welcome to send it but I'm not sure it really promotes your cause.

    I can hear it now: "SATs are classist and therefore racist. We don't do that sort of thing in the Soggy State we're in." You also get to try and answer why the state should mandate a test from a private, for-profit company. Third, the state wants to find out how much the students have accomplished towards (insert sound of thunder) achieving the State Standards (end thunder) and the SATs are not written to the State Standards. The fact that many colleges are ignoring the test now won't help much either.

    Not trying to dissuade you, understand, just laying out what I think are the most likely arguments so you can prepare ahead of time. This is one of those one-shot deals and you need to make the best go in that one take.

    Good Luck

    ReplyDelete
  3. Thanks, I concede those points. The SAT is "classist" had occurred to me, but I hadn't realized that so many colleges were scrapping the SAT.

    ReplyDelete