Do High-Stakes Placement Exams Predict College Success?
Issue/Topic: Postsecondary Success--Developmental/Remediation
Author(s): Scott-Clayton, Judith
Organization(s):
Publication: Community College Research Center
Published On: 2/15/2012
Background:
Community colleges are typically assumed to be nonselective, open-access institutions, yet access to college-level courses at such institutions is far from guaranteed. Instead, many students' first stop on campus will be to an assessment center where they will take exams in math, reading and/or writing. However, the remedial "treatment" that is assigned on the basis of these assessments is not obviously improving outcomes.
Purpose:
To analyze the predictive validity of one of the most commonly used assessments and to examine whether other measures of preparedness, such as high school background and prior college-preparatory units completed, might be equally or even more predictive of college success.
Findings/Results:
Using placement exams as a screen actually results in substantially lower accuracy rates than using nothing at all; for example, the number of qualified students who are prevented from accessing college-level courses with the exams outweighs the decrease in the number of unqualified students who are admitted into college-level courses.
- Placement test scores are better at predicting who is likely to do well in the college-level course than predicting who is likely to fail.
- Placement test score have much more predictive power in math than in English.
- While assigning all students to college-level math lowers the success rate of unprepared students, it dramatically increases the percentage of students who are predicted to succeed at the college level in their first term.
Using high school background and test scores plus the number of years since high school and whether the student graduated from a local high school is the most accurate in determining placement, yet even this strategy comes far from eliminating severe placement mistakes.
- Allowing students to test out of remediation based on the best of either their placement scores or high school achievement could substantially lower remediation rates.
Policy Implications/Recommendations:
In both math and English, high school background measures may be more useful predictors of success in a wide range of settings because they capture both a wider range of cognitive skills than can be evaluated on a brief placement exam, and because they also incorporate non-cognitive factors such as student motivation. However,
- There are limitations to relying on high school grades, such as social promotion and the variability of grading across institutions.
- Placement exams may be the only option for some students who do not have a high school transcript.
Weighing the costs and benefits:
- Policymakers must consider the prevalence and consequences of all types of placement errors – not just overplacements, but the less visible underplacements as well.
- Placement mistakes can occur under any system and policymakers should choose the policy that minimizes the severity of those errors.
- Policymakers have options beyond simply using or not using placement exams. Multiple measures can enable a system to reduce severe placement errors and improve college-level success rates, while keeping the remediation rate unchanged.
- Allowing students to test out of remediation based on the best of either their placement scores or high school achievement could substantially lower remediation rates without compromising success rates in college-level coursework.
- Lowering the cutoffs by just a few points would enable more of the “marginal” students to pass a college-level course in the first semester.
Even if an exam has high predictive validity, evaluations of the impact of remediation are ultimately needed to determine the overall validity of a placement testing system.
Research Design:
Quantitative
Population/Participants/Subjects:
42,000 first time entrants to a Large Urban Community College System (LUCCS)
Year data is from:
Fall 2004 - Fall 2007
Setting:
District
Data Collection and Analysis:
Analyzed data from 42,000 student assessments using standards statistical measures of predictive power, other decision-theoretical measures, including absolute and incremental placement and accuracy rates, and a new measure called the 'severe error rate.'
Disclaimer:
Reference in this Web site to any specific commercial products, processes or services, or the use of any trade, firm or corporation name is for the information and convenience of the public, and does not constitute endorsement or recommendation by the Education Commission of the States. Please contact Jennifer Thomsen at 303-299-3633 or jthomsen@ecs.org for further information regarding the information posting standards and user policies of the Education Commission of the States.
© 2020 by the Education Commission of the States (ECS). All rights reserved. ECS is the only nationwide, nonpartisan interstate compact devoted to education. 700 Broadway #810, Denver, CO 80203-3442
To request permission to excerpt part of this publication, either in print or electronically, please contact the Education Commission of the States’ Communications Department at 303.299.3636 or mzatynski@ecs.org.
Your Education Policy Team www.ecs.org