FORTUNE — Let’s get this straight: Successful people tend to be more confident, but only because they are usually more aware of their competence. In reality, successful people do not differ much in their confidence levels from their less successful peers.
Consider the following fact: The correlation between career success and any mea- sure of career confidence is .30 at most, which suggests that if we measured someone’s confidence to estimate how successful he may be, we would be only 15% more accurate than if we just guessed. And that is the largest correlation reported by any credible independent study.
Moreover, the modest overlap between career confidence and competence is mostly accounted for by the effects of competence on confidence rather than vice versa. In other words, career success boosts career (and generic) confidence, but no form of confidence has been found to have concrete, observable, or meaningful positive effects on career competence.
The Scottish philosopher David Hume noted in the 18th century what psychologists observed only 300 years later, namely that it is not possible for us to directly observe any form of cause and effect in the real world. All we can do is observe covariations: When X happens so does Y; when someone is successful, she is confident, etc. The key covariation regarding the role of confidence in career success is that “confident people seem more successful”; ergo, you may be inclined to think, “If I sort out my confidence problems, I will be more successful in my career”—but you shouldn’t.
MORE: GM’s Mary Barra to staff: ‘No more crappy cars.’
First, the confidence-career success correlation is small — many successful people are not that confident, and there are even more unsuccessful people who are very confident (I am sure you can think of examples). Second, when people are both successful and confident, their confidence is more often a product of their success than vice versa. And yet, unless you live in total isolation from the rest of the world, you have probably been brainwashed into believing that high self-belief is the most important single cause of career success; that if you think you can do something, you most certainly will.
When scientific studies measure not just current levels of confidence and career success but also previous competence (e.g., talent, skill, or potential), the already small correlation between confidence and career success disappears. For example, one of our studies tested thousands of school pupils on initial competence (their school performance), subsequent career confidence, and later academic performance. The kids who were more confident at age nine tended to do a bit better in their studies at age twelve. However, when we took into account how they had performed until age nine, it became clear that the only reason for their higher confidence was their previous higher competence — that they had done well in the first place. The path is quite simple and intuitive: Kids who do well feel confident because they did well; kids who feel confident despite not having done well don’t end up doing any better. Competence leads to confidence, but not vice versa.
My team and I have replicated these findings with college students. In many studies involving thousands of universities from all over the world (literally; we looked at data from five continents), students who displayed higher levels of confidence tended to have better grades — but it was their previous grades that led to higher levels of confidence; confidence did not cause any competence gains. There was only one exception to this rule: males. Indeed, when we broke down the results by sex we noticed that although male students tended to display higher levels of confidence than did females, males’ grades were generally not higher, but lower. Furthermore, analyzing the data for male students only, those who displayed higher levels of confidence were often performing worse academically than those who displayed lower confidence levels. This shows that male confidence is delusional, and that the more overconfident males are, the more incompetent they tend to be.
Looking at the combined data for both sexes, we found that males almost always exhibited more confidence than females did, despite the fact that they were being systematically outperformed by them. So what do these findings mean? Men are cocky and it doesn’t pay off. Women are modest and it doesn’t harm them. And that’s not the end of the story: Women are less delusional than men when it comes to assessing their academic career potential, and that does pay off. In fact, in almost every country around the globe women’s academic performance has been rising, often to the point of outperforming men (this is certainly the case in the United States), yet men remain more confident in their career success than women do.
MORE: Why Macy’s relented to an early Thanksgiving opening
What about those beyond college? Good question. Psychological research is often based just on college students, who are hardly representative of the overall population, though one day they will hopefully become adults. As it turns out, when it comes to the relationship between confidence and adult career success (competence postcollege), the findings from our unrepresentative high school and college students are replicated almost perfectly with grown-ups. And as with students, the modest positive association found between adult confidence and career success (the .30 correlation) is not indicative of the effects of confidence on career success; rather, it is indicative of the fact that more successful people tend to be more confident about their career success.
In other words, being more talented makes you more competent, which in turn makes you more confident. Given that competent people tend to come across as confident, and that individuals who lack confidence tend to be aware of their incompetence, the gap between career confidence and competence is not always easily observable. Still, most confident people are not as competent as they think, and most competent people are confident only as a result of being competent, which they did not achieve by being confident.
The following adapted excerpt is from Confidence: Overcoming Low Self-Esteem, Insecurity, and Self-Doubt by Tomas Chamorro-Premuzic, PhD. Reprinted by arrangement with Hudson Street Press, a member of Penguin Group (USA) LLC, A Penguin Random House Company. Copyright © Tomas Chamorro-Premuzic, 2013.
