Policymakers are increasingly including early-career earnings data in consumer-facing college search tools to help students and families make more informed post-secondary education decisions. We offer new evidence on the degree to which existing college-specific earnings data equips consumers with useful information by documenting the level of selection bias in the earnings metrics reported in the U.S. Department of Education’s College Scorecard. Given growing interest in reporting earnings by college and major, we focus on the degree to which earnings differences across four-year colleges and universities can be explained by differences in major composition across institutions. We estimate that more than three-quarters of the variation in median earnings across institutions is explained by observable factors, and accounting for differences in major composition explains over 30 percent of the residual variation in earnings after controlling for institutional selectivity, student composition, and local cost of living differences. We also identify large variations in the distribution of earnings within colleges; as a result, comparisons of early-career earnings can be extremely sensitive to whether the median, 25th, or 75th percentiles are presented. Taken together, our findings indicate that consumers can easily draw misleading conclusions about institutional quality when using publicly available earnings data to compare institutions.