1. Big improvements? We're exaggerating.
It's common for test-prep programs to tout big score gains on college admissions tests like the SAT and the ACT. But critics say these coaching companies have little hard evidence to back up their claims. A 2009 report by the National Association for College Admission Counseling found that while many test-prep providers advertise average score increases of 100 points or more on the SAT, average gains were closer to 30 points, out of a possible total score of 1,600 (the research predated the addition of the writing section of the SAT in 2005). For the ACT, the average gain was less than one point out of a possible 36.
Such criticism has pushed at least one big test-prep company to dial back its marketing messages. Last May, Princeton Review stopped claiming an average gain of 255 points for students who took its SAT Ultimate Classroom course after the National Advertising Division of the Better Business Bureau, which examines accuracy in advertising, because the data was based on two different tests -- a diagostic test issued by the company and the students' self-reported SAT scores, according to the NAD. (Princeton Review declined to comment on that ad campaign.) Other test-prep firms are expected to follow suit, says Dave Berry, senior adviser for College Confidential, a forum for students and parents applying for college.
2. The test may be over, but we're sticking around.
Once you get on the mailing list for some test-preparation companies, good luck getting off. Only about 60 consumer complaints have been filed to the Federal Trade Commission regarding Princeton Review and Kaplan since 2006, but the majority is from students or former students complaining that they're still being contacted by the companies long after repeated requests to be removed from their contact lists. One former student said 15 requests to unsubscribe to Princeton Review's mailing list over a six-month period went ignored.
While Princeton Review declined to comment, a Kaplan spokesman Russell Schaffer says his company's policy is to remove people from their mailing and calling lists if requested, but that the company has received reports of unauthorized individuals making calls and claiming to be from Kaplan.
3. Well, we think this works.
From the number of students who use test-prep services to the size of the providers to the effectiveness of the classes, there's little reliable data on the industry, in part because there's no official government agency supervision. "There's no outside check as to whether what they say is true, close to true or snake oil," says Robert Schaeffer, public education director for the National Center for Fair and Open Testing. As a result, many students find themselves caught up in a test-prep "arms race," simply because everyone else is, he says.
There have also been relatively few studies on the effectiveness of test coaching services and the ones that have been are not exactly conclusive. For example, most of the research conducted on SAT prep programs since the 1950s involved studies of small groups of students and not "necessarily representative of the national population," according to the 2009 report by the National Association for College Admission Counseling. Even less research has been done on prep courses for the ACT and other admissions tests. Kaplan, Princeton Review, PrepMe and other companies all say they conduct regular surveys and studies to monitor what's working (and what's not) when it comes to their classes and products.
4. You can get our services for free.
Many consumers may not be aware that school districts often buy their services and make them available to students for free or at a discount. Many coaching companies offer their classes at a discount, or even free to lower-income individuals. Kaplan is offering its SAT or ACT test prep courses for $45, instead of the full $499, to high school juniors in Texas registered for the May or June 2011 exams, says Jeff Olson, vice president of research for Kaplan Test Prep. Both Princeton Review and Kaplan say their services are often subsidized by schools.
Before they fork over all that cash, students may want to check their local school districts and non-profit organizations: The savings can be substantial. Companies typically charge $1,100 for a class and $100 to $200 an hour for individual tutoring, according to the NACAC. It's estimated some 1.5 million students spend about $530 million a year on test preparation and tutoring for the SAT alone, according to Eduventures, a Boston research and consulting firm.
5. Or you can prep yourself.
Counselors and college admissions coaches say students who take advantage of free practice tests, booklets and online services provided through their schools or the test administrators can perform just as well as those who pay full price for a course. "Most of the improvement comes from just having some practice," says Steve Schneider, a counselor at Sheboygan South High School in Sheboygan, Wis., and the secondary level vice president of the American School Counselor Association. Students using the free test-prep site Number2.com posted score gains in line with those of students who used more services that cost as much as $400, according to Consumer Reports.
For do-it-yourselfers, Schneider recommends taking the actual test, or a timed practice exam under test-like conditions, to get a baseline score and to learn what areas you need improvement on. The College Board, which administers the SAT, offers one free practice exam online each year, along with a free question-of-the-day feature on CollegeBoard.com. ACT also posts free practice questions for various sections of the exam at Actstudent.org. And many high schools purchases test preparation workbooks to loan out to students.
6. What guaranteed refund?
Vinh Chuon, a mother of two in Union City, Calif., says she paid $1,000 for her oldest son to take an SAT prep course in the summer of 2007 from Princeton Review, which guaranteed a score increase on the test or a full cash refund. But after taking the class, her son's test score dropped by about 40 points, she says. Her calls about getting her money back were ignored, she claims, until the Better Business Bureau contacted Princeton on her behalf. In the end, she got a 50% refund; the rest of the money came as a voucher to take the SAT course again. A spokeswoman for Princeton Review also said there is no situation in which a customer would receive a 50% refund under their current policy, but she declined to comment specifically on Chuon's case. According to the Better Business Bureau, the case was resolved, meaning Princeton Review responded to Chuon in a "manner that the consumer, at the time, found acceptable."
Kaplan, Princeton Review, Boston Test Prep and PrepMe all guarantee a refund if students don't increase their scores. But the fine print reveals it may not be so simple. "If you don't get the score gain they just let you take the coaching again, and there's a lot of red tape involved," says Derek Briggs, a professor at University of Colorado in Boulder and the author of the NACAC report on exam preparation programs. For instance, Princeton Review's 150 Point Money Back Guarantee for the SAT Ultimate Classroom Course says students must retake a course and then retake the official SAT, which costs $47. If their score still doesn't improve, they may be eligible for a full cash refund. But that guarantee offer only applies to students who score between 801 and 2100 on their preliminary practice test (out of 2400). The company's other guarantees allow students to retake a course if they aren't satisfied but charge a $200 administrative fee.
Princeton Review and PrepMe say the majority of students qualify for their guarantee program. Boston Test Prep did not respond to requests for comment.
7. Our online materials and software are buggy.
Online test-preparation products account for about $50 million of the $530 million market nationally for SAT prep and growing, according to Eduventures. But Consumer Reports found errors on practice tests for 6 out of 10 online test-prep services, including grammatical problems, questions without answers and missing sections of text. Services offered by Barron's Test Prep, Peterson's Online SAT Course and PrepMe were "particularly problematic," according to Consumer Reports. And only one of the companies tested, Boston Test Prep, provided a system for reporting errors, the report said. PrepMe, which was tested, said it has always provided a system for reporting errors.
"That was kind of wake up call to us," says Bob O'Sullivan, publisher of the test prep division for Barron's Educational Series, adding that the company has also discovered typos in its books after the study was released and has worked to correct errors in its materials. "We really have been putting a huge amount of effort into the fact that we do not want any error to appear in our books." Peterson's says it has also upgraded its software to correct errors. Boston Test Prep did not respond to requests for comment.
8. We aren't the biggest influence on test scores.
High school counselors and test administrators say the biggest influence on a student's test score is their school course load, not the coaching they receive prior to the test. Some students even see their scores drop. "There were a few cases I found with the ACT where on average people who took coaching did worse than they did if they hadn't take the coaching," says Briggs. His study also found that students who had previously taken the PSAT and then used a computer program saw their math SAT scores drop by about seven points on average.
Even the test administrators themselves downplay the impact of test-prep services. For example, research from the ACT found that taking math courses, such as geometry, trigonometry and calculus, in addition to the core classes recommended for college prep can increase your ACT score by up to 5.8 points, compared to the average 1.5 point gain due to courses, workshops and computer software used prior to the test.
9. This is going to be stressful.
Some of the most dedicated students have trouble fitting test-preparation courses into their schedules, which may already be crammed with advanced placement courses, sports and honor society meetings, says Linda Brannan, a high school counselor in Cary, N.C. The overbooking can add more pressure to an already stressful situation, says Berry. "They're killing themselves," he says. "The net result is they're getting three to four hours of sleep per night and they're propping themselves up with energy drinks and Starbucks."
Kaplan and Princeton Review dispute that characterization, saying test-preparation classes and programs can actually ease anxiety about the exam and eliminate all that last-second cramming. That said, there's no getting out of hard work, they say: Students need to attend all the prep classes, take the practice tests and do all of their homework to get the most out their packages.
10. Test scores aren't really that important anyway.
A growing number of schools are choosing not to consider SAT or ACT scores at all. Already about 850 colleges and universities, including DePaul University in Chicago and Bard College in upstate New York, have "test optional" policies, according to FairTest, a nonprofit advocacy organization which looks to prevent the misuse of standardized tests.
Applicants who do not submit test scores will be evaluated on their school performance and essays which it says were always the biggest factors in the admissions process anyway, says Jon Boekenstedt, associate vice president for enrollment management for DePaul University. This kind of change in admissions policy is good news for those students with high grades and low test scores. The new policy also levels the playing field for lower-income students, Boekenstedt says, who often perform less well on standardized tests than students from middle- or high-income families. "And I don't believe that's because of lack of talent and intellect."
Corrections and clarifications: A previous version of the story said that Kaplan was offering its discounted SAT and ACT prep courses to Texas juniors in partnership with the Texas Education Association. It is in fact an independent Kaplan initiative. Also, the story was updated to include a comment from PrepMe regarding its online program.