Consider this: You can miss two questions and still score perfectly on some SAT test dates. While on others, missing two questions drops you far down the point scale. What gives? Is it really true that some SAT test dates are just easier than others? Today, we’re going to examine the SAT scale and settle the debate once and for all.
The easiest test date: is it a myth or reality?
We know some tests are harder than others.
Try as they might, the test makers cannot produce two identically challenging exams.
Because of this, and because the pool of test takers changes with every exam, some tests have more perfect scorers than others.
If not for an SAT scale, what makes for a hard SAT test?
Is it just the luck of the draw, or is it a function of whom you take the tests with? Is it the behind-the-scenes magic of the SAT scale, or is it the time-of-year?
The old logic goes like this:
Testing with more seniors must make for a more competitive base. Thus students will face more difficulty achieving a top score. So the trick must be to test with the most inexperienced group. That must happen during spring, after the seniors are done testing.
Right?
The short answer is no.
According to our research and a meta-study of analyses: there’s no such thing as an ‘easy’ test date for the SAT or ACT.
The long answer is a little more complicated.
These tests are scaled, and the methods used are opaque. This has led to speculation over ways to game the system by choosing the optimal factors like time and cohort.
While those factors do not appear to impact test difficulty, there are variations that are worth noting.
Understanding those requires knowledge of exactly what happens in the process of translating a a raw score to a scaled score.
How the SAT Scale was Made
When most parents and students think of a grading scale, they think of a curve. In other words, a student’s raw score is mapped to their final score based on their performance compared to the other members of their class.
This is NOT how score scaling works on the ACT or SAT.
Students are not being compared to the other students who took the test during their same test date. Or even their same testing year for that matter. If the SAT scale used the usual curve, the proportion of students who get a great, average, or lackluster score would be the same for every testing day.
But we do not see this.
Instead there are some testing days when more students than expected get a perfect score and others when fewer do.
ACT score scaling data
The ACT compares students to data based on the Academic Skills Study conducted in 1988. This data was weighted and distributed into two categories, all examinees and college-bound examinees, and the outliers were removed.
The college-bound data was then used to create the initial score scale. Minor adjustments were then made to this initial scale to ensure that:
- each score fell within a reasonable margin of error
- that there were no gaps in the score scale
- and that there were not too many raw scores which converted to a single scaled score
The ACT conducted another study in 1995 to ensure that the introduction of calculators (which had previously been prohibited) to the test did not change the meaning of the 1988 scale. It was found that there was not a significant difference in meaning, and the scale based on the 1988 data remains in use to this day.
Let’s look at the SAT scale and score data
The SAT’s score scaling data is based on the 2014 Scaling Study conducted by the College Board. Because the SAT’s process for screening and distributing its data was fairly similar to the one described for the ACT, and for the sake of brevity, I won’t go into further detail.
For a more in depth look at the SAT scale and scaling process, I recommend reading:
- Chapter 9 of the ACT Technical Manual
- Chapter 6 of the SAT Suite of Assessments Technical Manual
Worried about the SAT scale? Don’t worry, seniors will not break the curve
All of this is to say that with whom a student takes their test will have absolutely no bearing on their final score.
Students are being compared to college-bound students from 2014 or 1988, not their peers.
Therefore, there is absolutely no advantage to avoiding test dates which have a high proportion of Seniors (who usually test in September and October).
Seniors will not “break the SAT curve”: their high scores will have no impact on the scores of other students.
But Some Tests are Still “Harder” than Others
Even with the misconception that other students can “break the curve” out of the way, many students will still say that certain ACT or SAT test dates felt harder than others.
This makes sense:
The logistics of creating and administering the SAT and ACT make it almost impossible for each test to be the exact same level of difficulty as all the others.
The makers of the SAT and ACT know this. But before getting into what steps the test makers have taken to reduce the effect of this reality on final scores, let’s define what it means for a test to be “hard.”
What makes some ACT/SAT tests harder than others?
What students tell their parents that a test was hard, what they really mean is that the test “felt hard”. Unfortunately, this is not a very useful way to think about difficulty.
First, everyone’s personal metric for what feels difficult is different. Additionally, something “feeling easy” is too vague to create a codified definition for what makes a test “easy” or hard”.
Here’s a more useful way to think about difficulty:
Think in terms of “how many questions can I miss and still achieve my target score”.
If a student can miss more questions than they usually get wrong in practice and achieve their target score, then that is what makes the test easy.
Similarly, if a student needs to miss fewer questions than they usually get wrong to reach their target, then that makes the test hard.
Now, recall that test makers have a system for making sure small differences in difficulty between tests do not affect final scaled scores.
This system is called Equating.
Ironically, because of it, tests which feel hard for students are actually easy when using the above definition for difficulty.
Similarly, tests which feel easy are actually quite difficult.
Equating is what makes the SAT Scale
Equating is the process that each new official test form must through to make sure that its final converted score has the same meaning as all the other test forms which came before it.
New test forms are equated using a carefully selected sample of national examinees.
They are given a set of tests which contain the new test forms as well as one anchor form which has already been equated to previous forms.
The new forms are then compared to the anchor form using equipercentile equating, outliers are removed, and the result becomes the raw-to-scaled conversion chart for that specific test form.
The charts which come out of this system are designed so that when a test is easier, more questions need to be answered correctly, and when a test is more difficult, fewer questions need to be answered correctly to achieve the same reported score.
In this way, a certain score demonstrates the same amount of skill on the part of the student, regardless of what version of the test they are taking.
For an example of this in practice, take a look at the two raw-to-scaled conversion charts below:
Test A: 2017-2018 ACT Practice Test Conversion Chart
Test B: 2008-2009 ACT Practice Test Conversion Chart
Focusing on the English Section, we see that in order to receive a minimum score of 35, a student can miss 3 questions on Test B but may only miss 2 questions on Test A.
This means that if a particular student took both versions of the test and they missed 2 questions on Test A, that student would be expected to miss 3 questions on Test B.
The scale of Test B is easier, because the test itself was harder.
Now, think back to our new definition of what “easy” and “difficult” mean. Because a student can miss more questions on the English Section of Test B and still receive a certain score, that makes that section of Test B easier, even though the actual material written in that section is more difficult.
So when is the easiest test date then?
The easiest test date is when a student is most prepared.
If a student is performing consistently and can avoid making simple errors such as misreading questions or typing things into their calculator incorrectly, then the exact scale of the test they are taking will not affect their final score.
Furthermore, there is no way to predict what the scale of a test date will be before getting the results back. To do so would require knowing what specific questions would be asked on the test, which is problematic for obvious reasons.
Therefore, students should focus on taking the test when the timing works for them and when they have reached a point in their preparation where they can achieve their target score.
For a full analysis on the optimal timeframe, and for help on deciding when to take the SAT and ACT, please review our articles:
Don’t stress the SAT Scale, focus on your test prep instead
The best way for student’s to get a good SAT score is for them to not worry about which test date will be “easier”.
Rather, the best use of a student’s time is to figure out which test is best for them and create a test prep schedule far enough in advance that they can improve their score.
