If you listen to the experts, they’ll tell you that, despite the rising cost of a degree, college still pays off.

And the statistics bear this argument out. A college graduate will on average make $1 million more than a worker with just a high school degree over the course of his lifetime, making even a six-figure upfront investment well worth it in the end. Yet anecdotal evidence abounds of students who either flunked out of college or graduated with large debt loads and are still unable to land jobs that will make the investment in a degree pay off.

How do we reconcile the statistics with these stories? This chart from the Federal Reserve Bank of New York shows that we can miss a lot when looking at just the average experience:

college.wage.

The bottom quarter of earners with a college degree don’t make more money than the average high school graduate. And this hasn’t really changed much in 40 years. In fact, this graphic shows that a college degree has become more valuable even for the bottom quarter of earners, likely as a result of the evaporation of high-paying blue collar jobs, like those in the manufacturing industries. Of course, over the past 40 years, the cost of a degree has increased 12-fold, while a degree holder isn’t making more money at all, when accounting for inflation.

So, it’s quite likely that, given the huge upfront investment for a college degree, many more workers today would have been better off not going to college at all. There are a few caveats that should be mentioned, however. First, we don’t know for sure how much money this bottom quarter of degree-holding earners would have made without their college education. Furthermore, much of this could boil down to career choice: there are many jobs that require a degree but don’t pay very well. If someone earns a degree for reasons beyond making more money, it could be that the upfront investment is worthwhile regardless.

That being said, the above graph is certainly evidence that both the public and private sector need to make serious changes to provide higher education more efficiently. Unfortunately, this bit of information isn’t really helpful for the millions of Americans who need to decide how and where to send their kids to college.

There is a well-established principle in behavioral economics that people systematically overestimate their abilities in general, and we can see this in the exponential increase in college costs. Up to 25% of college grads would probably be better off not pursuing a degree, yet nobody actually thinks they’re going to be the ones for whom the investment doesn’t pay off.

Further compounding the problem is the fact that students usually don’t have a great idea of how they are going to fare in college, or what sort of degree or career they will pursue until after they’ve already made the decision to attend.

One could argue that all this uncertainty is evidence that we should be investing more in public support of higher education. There’s plenty of evidence that there are what economists call “spillover effects” from students educating themselves, that society as a whole benefits from higher education. So there is justification for paying for higher education with tax dollars. And since students don’t know ahead of time what degree they will personally benefit from, it’s possible that students will just begin to forego college altogether, hurting society in general. There’s evidence that this is beginning to happen.

In the meantime, students who are unsure of what they want to study or do are probably best advised to be very cost-conscious when choosing a college, and to be unafraid to wait until they are sure how they will use their degree before they start to pursue one.