Issue Briefs
College Scorecard
The College Scorecard is an interactive federal tool, created during the Obama Administration, that lets students and families compare institutions of higher education across a broad array of standardized metrics, such as net price, graduation rates, student body diversity, median graduate debt, loan repayment, and post-graduation earnings. NAICU views it as a far superior alternative to the federal postsecondary ratings system originally proposed. However, the Scorecard still contains flaws that undermine its overall effectiveness and must continue to evolve to be helpful to families and fair to all institutions, particularly those serving a high proportion of low-income students.
About
The Scorecard was released on September 12, 2015, after growing out of the Obama Administration's failed attempt to create a federal postsecondary ratings system. Rather than rate institutions, the Scorecard compiles existing federal data into institutional profiles. A central NAICU concern is the Scorecard's focus on limited variables, particularly monetary measures of value: reducing an institution's worth to two or three numeric factors does not present a full or accurate representation of its mission and character, and the tool contains no qualitative information, limiting its ability to guide a student toward a best-fit college.
Nevertheless, in NAICU’s view, the tool is important and has real value for students and families trying to make informed decisions about pursuing higher education. Additions to the original tool, such as the introduction of a customized search function letting students build their own comparison groups, which was a longtime NAICU policy goal, and the 2018 removal of national comparison data that could give a misleading impression of an institution's value, were welcome.
Other changes, such as when the Biden Administration restored that national-median comparison feature in 2022, compelled NAICU to reiterate that, given the vast diversity among institutions in size, type, mission, selectivity, and populations served, national comparisons can paint an inaccurate picture of an individual institution's quality.
NAICU has also expressed concern that the publication of program-level debt and earnings data (added in 2019–2020) could amplify calls in Congress to limit Title IV eligibility based on a student's major – a concern that has since materialized in the earnings-based accountability framework established in the One Big Beautiful Bill Act.
Technical Concerns with the Data
Beyond questions of which variables to display, the underlying data carry significant methodological limitations that affect how the Scorecard's figures should be read, most of which students and families may not be aware of when using the tool.
The most consequential is that earnings and debt metrics cover only students who receive Title IV federal financial aid because the Higher Education Act prohibits a comprehensive federal unit-record data system, leaving the Department able to use only the aid data it already collects. Nationally, only about 54–55% of undergraduates received federal aid in recent cohorts, and that share varies widely by sector. Therefore, the figures may not be representative of schools with a low proportion of Title IV recipients, which can underrepresent institutions with even marginally-wealthier student bodies. Research comparing the Scorecard's Title IV-only population with the Census Bureau's all-graduate Post-Secondary Employment Outcomes data found that short-run earnings are similar across the two samples, but longer-run outcomes (about ten years out) are significantly lower for the Title IV population, with the gap widening over time.
Several other limitations compound this. Data are suppressed for programs with small numbers of students (generally fewer than 30 in the denominator) for privacy reasons, which means fewer than one-fifth of all programs display debt or earnings information. Earnings reflect only those not enrolled in further education during the measurement window, and some metrics historically captured earnings early in the workforce, which understates lifetime trajectories. The Department's own documentation cautions that earnings variation across programs within a single institution may be even greater than the variation across institutions, so an institution-level number can mask wide program-level differences.
The Scorecard's 2017 publication of inaccurate loan repayment rates for most colleges, later attributed to a coding error and corrected. further illustrates the data-quality risks.
History
The Scorecard emerged in 2015 from an eventually abandoned proposal to create federal ratings for colleges and universities. The Trump Administration added customized comparisons (2017), removed national comparison data (2018), and added program-level data (2019), with median earnings two years after graduation added in a 2020 update.
The Biden Administration maintained and expanded the tool, including restoring the national-median comparison feature in 2022, and the Scorecard has arguably served as partial inspiration for subsequent institutional accountability frameworks.
Review how your institution appears on the Scorecard and be prepared to contextualize earnings and debt figures, noting they reflect only Title IV recipients.
Communicate the qualitative dimensions of your institution’s value that the Scorecard cannot capture.
Engage in NAICU’s advocacy as program-level data is increasingly tied to federal accountability and Title IV eligibility.
Comparing Earnings Outcome Differences Between All Graduates and Title IV Graduates - Andrew Foote, ScienceDirect
College Scorecard and Data Documentation - Department of Education
Justin Monk, Justin@NAICU.edu
In the News
-
NAICU Washington Update (2/11/22)Updated College Scorecard Restores National Comparison Data