For more than half a century, the share of American men in the prime of their lives who are employed has fallen relatively steadily, taking deeper dives during recession years. A new report from the White House’s Council of Economic Advisers tries to answer the tangled question of why.

The CEA report finds that participation in the labor force among “prime-aged” men—males between 25 and 54 years old—peaked at 98% in 1954 and began to slide sharply in 1965 to a nadir of 87.9% in October 2013 after years of recession following the financial crisis, before stabilizing at 88% today.

The report identifies a few key causes for the decline. First of all, the demand for less skilled labor in the U.S. has fallen sharply since the 1950s. In other words, there are fewer jobs for more unskilled workers. The steep rise in incarceration in the U.S. seems to be another culprit. By some estimates as of 2008 between 6 and 7% of prime-aged males in the U.S. had been incarcerated at some point in their lives. A stint in prison makes it significantly harder to find a job. Since incarceration affects black men more than other groups, this also helps explain why workforce participation is lower in that cohort.

Read More: 12 signs America is on the decline

Why Are So Many White Americans So Unhappy?

The persistent, long-term decline in employment among prime-aged men is particularly significant for society as a whole because men ages 25 to 54 are in their most productive years, and their absence from the labor force “has outsized implications for individual well-being as well as for broader economic growth,” the CEA report says. A large body of evidence links joblessness with lower overall well-being, less happiness, higher mortality and negative consequences for families and communities.

The report identifies a few key trends within the data.

• The less educated a man is, the more likely he is to be unemployed.
• Joblessness affects men of all races and ethnicities but is particularly severe among black men.
• Men are not simply choosing not to work. Less than a quarter of unemployed prime-age men have a working spouse, and about a third of them live in poverty. The analysis says that “at most” 0.5% of the decline in prime-age male participation in the labor force can be explained by the availability of Social Security Disability Insurance.

Lastly, the report notes that the U.S. has one of the lowest rates of prime-age male labor force participation compared to similarly advanced economies in places like Canada, Japan and Western Europe, which have less “flexible” labor markets—meaning its harder to hire and fire workers—and more support systems like job training programs.