Late this spring or early this summer, the Supreme Court is expected to bar colleges from using affirmative action in the college admissions process. It’s a result many white conservatives have demanded for more than 20 years — but one that may have significant unintended consequences for folks who have argued that affirmative action is unfair to whites.

Bluntly, if you are white and believe that affirmative action is unfair, know that if it is eliminated, it’s quite likely that it will be your kids who suffer.

Let me explain.

While I am someone who is a proponent of affirmative action, I recognize that — at least if framed in moral terms — reasonable people can disagree, and the research on the academic benefits are mixed. But there is a different way to look at it. Since most students go to college to get jobs, we should look at not what happens in college but what happens when students graduate — specifically on their earnings. We also should look at all the students, not just the students admitted because of affirmative action.

Some years ago, I did just that, exploring whether a college’s student body has any impact on the amount its graduates earn, taking into account other things like college selectivity and a student’s GPA and major. You may discover the findings surprising.

It turns out that on average students who go to slightly more diverse colleges make more than students who don’t. This seemed to be true among the “Ivies” as well as among community colleges — across the whole spectrum of higher education, and controlling for other things that impact salaries like major and GPA.

But what was most surprising was that if one controlled for the race of the student, it suggested that being Latino or Black and going to a more diverse college didn’t help at all; instead, all the gains seemed to be to white students.

Why might this be the case? Perhaps there is something that students who attend these more diverse colleges pick up outside the classroom — it doesn’t show up in grades, but employers recognize the value and pay for it. Since Blacks and Latinos already have learned how to work with people who are different, they only get the bumps for grade, major and college selectivity but don’t get an extra bump but diversity — but for the white students who haven’t been exposed to kids who are different, they learn a skill valued by employers.

My work is both limited and dated (now more than 10 years old), but if anything, businesses seem to value these traits even more nowadays. Regardless of the limitations, the research suggests that if we limit affirmative action because it is “unfair to whites,” the great irony is that we may be depriving all those white college students of increased future earnings — because in the absence of affirmative action they don’t learn how to work with people different from them.

Some proponents of affirmative action might find my argument problematic in that it focuses on the benefits to white students; I am merely pointing out that there is evidence that folks who are against affirmative action on fairness grounds might actually be robbing themselves of the very thing they are fighting for.

Finally, given that we pride ourselves on a market-driven college system, we are actually limiting colleges’ ability to make themselves better. The power of our system (the best in the world) is that colleges compete in the marketplace for students and send their alumni out in the world to be productive. So not allowing colleges to admit who they want in order to build the best set of graduates they can, hurts the graduates, the college and the employers. Indeed, if one follows the logic, one could imagine actually awarding merit-based aid to people of color simply to make the rest of us better off.

Doug Lynch is senior fellow at the University of Southern California Rossier School of Education.