Some see this as a positive trend, because it adds to the economy. But others rightfully view it with trepidation, because there’s the distinct possibility that many of these elderly people just can’t afford to retire. Whether their nest eggs were wiped out in the housing crash, or they just didn’t save enough, or whether their kids don’t make enough money to support them, the decline of retirement seems like an ominous development.

The pressures on older Americans to work will likely only become greater in the coming years. This is because the young, working population needed to support retirees will see slower growth, and possibly outright shrinkage.

As recently as 10 years ago, the U.S. had unusually high fertility rates for a developed nation. The total fertility rate — the number of children a woman can be expected to have over her lifetime — was about 2.1 children per woman, which is the level required for long-term population stability. But since then, the rate has fallen to 1.8 in 2016, implying long-term population shrinkage:

Much of this is due to a fall in fertility among Hispanics, whose birth rates are converging with those of other groups. The Great Recession was undoubtedly a trigger as well; permanently lower expectations of income and wealth made child-rearing seem like a more financially daunting prospect.

Fewer kids means, eventually, fewer young workers to support an increasing population of retirees. This will result in less money being paid into the Social Security and Medicare systems, requiring either cuts in benefits, a higher retirement age or ever-ballooning deficits. Past experience suggests that Americans will be asked to work longer.

The U.S. bounced back from falling fertility once before, in the late 1980s. But as economist Lyman Stone has written, there are reasons why history may not repeat itself. High and increasing costs of housing, child care and education show no sign of reversing. The need for ever-higher levels of education in order to thrive in the U.S. job market is causing families to delay childbirth, which results in fewer children. Stone projects that U.S. fertility rates could fall as low as 1.5 or 1.4 — the levels that prevail in Japan and some European countries.

There is one more source of population growth that the U.S. has traditionally depended on — immigration. Low-skilled immigrants make it easier to raise kids by providing cheap child-care services. High-skilled immigrants earn more and pay a lot of taxes, while using few government services themselves, meaning that their fiscal contribution is enormously positive:

But low-skilled immigration to the U.S. has declined, meaning that more expensive childcare is on the horizon. And high-skilled immigration may soon taper off, as President Donald Trump’s policies and rhetoric make the country less hospitable for the world’s best and brightest.

In other words, the U.S. may soon find itself without its two big long-term population boosters, and wind up as a graying, shrinking nation, with young people burdened with supporting ever-more old people, and the elderly themselves forced to work long into what used to be the golden years.

In this, the U.S. will be following in the footsteps of Japan. Japan has had low fertility for much longer than the U.S., and until recently had little immigration. Even now, although immigration has been increased, it’s mostly of the low-skilled variety — with uncompetitive low salaries and an ossified corporate culture, Japan has had great difficulty attracting high-skilled foreigners.

As a result, the country’s social security system has come under great strain. As the country grew older and older, Japan repeatedly raised the retirement age, cut benefits and raised taxes on the working population. It also created a wage subsidy for elderly employment, and allowed older people to claim higher total benefits if they delayed their retirements.

This combination of carrots and sticks successfully pushed older Japanese people to stay in the labor force. Both men and women aged 60 through 64 began working much more than before, while employment levels for people aged 65 through 69 recovered to the high levels that had prevailed before Japan’s lost decade in the 1990s. People over 65 are estimated to have raised the country’s employment levels by about 2 million since the economic recovery began in 2012.

This is being hailed as a miracle or good news in some quarters. And it’s true that putting armies of old people to work has helped Japan arrest its economic stagnation despite its lack of skilled immigration or young people. But is this really the solution that the U.S. should want? Is putting grandma and grandpa behind a store counter or in an office into their late 60s and 70s a victory, or a grim necessity?

If the U.S. wants to avoid Japan’s fate, it needs to take steps to avoid Japan’s population structure. Urgent measures to make housing cheaper, such as those suggested by Senator Elizabeth Warren, should be paired with generous child-tax credits and universal pre-K education, in order to bring down the costs of having children. And the U.S. should emphatically reverse course on Trump’s policies toward skilled immigrants, dramatically expanding the number of employment-based green cards and implementing a Canadian-style points-based system for letting in large numbers of skilled foreigners.

Keeping a healthy, stable population of productive young workers is better than working until we drop.