Much of that divide can be attributed to long-standing issues like the gender pay gap and the so-called motherhood penalty, where women with children generally earn lower salaries than their male counterparts and coworkers (both male and female) without kids.

But the retirement discrepancy also stems from more subtle attitudes around long-term planning. In fact, nearly one in five mothers who have considered stepping out of the workforce to stay home with their children say they didn’t even consider the potential hit to their retirement savings when weighing the decision, according to a recent survey of about 1,500 mothers conducted on behalf of TIAA. 

Just over a third of women with a college degree reported putting a lot of thought into how staying at home would affect their retirement savings, versus a full 50% who put a lot of thought into childcare costs.

Read more: Childcare crisis: How men and employers can combat ‘time poverty’ for working mothers

“When you make a choice about returning to labor force or not, it’s actually a big enough decision financially, that it’s one where you should really dive in,” says Emily Oster, an economist and professor at Brown University, as well as the author of a twice-weekly newsletter, ParentData, focused on on pregnancy and parenting. 

So often, mothers say they run the numbers and ask themselves what’s the point of working when all of their after-tax income is going to childcare, Oster says. And that may be true. “But I think what that misses is that a retained attachment to the labor force means more income later,” she says. Childcare costs do go down over time, yet that extra time in the workforce, for many people, means you’re accruing more retirement savings.

When mothers weigh the choice to leave the workforce, childcare costs are the immediate concern, Oster says. That makes sense, but it doesn’t mean parents considering a break from the workforce shouldn’t also consider longer-term factors as well. 

When it comes to long-term financial security, like having a robust retirement savings balance, taking small steps over time adds up. Take the example of a mother who makes $60,000 and contributes just 3% of her income to her retirement. If she takes off two years, it could result in $38,000 less in retirement savings by the time she hits 65, according to a recent joint white paper published by Oster and TIAA. Five years out of the workforce? That would result in nearly $100,000 less in retirement savings for that same mother, assuming a 7% rate of return. 

Or take the fact that 54% of women surveyed reported they’d rather stay at their current job than switch to a job with higher pay, but also higher childcare costs. Running the numbers shows that taking the higher salary will ultimately be the better deal—despite the potential for short-term pain.

“All these things sort of stack up,” Oster says. Retirement is so far off that it doesn’t come up for a lot of parents weighing their financial options after having a baby, Oster says, but adds then she will hear people say later, “I wish I knew that before, or I wish I had considered that.”

“Take a step back, look at the big picture—it may well be more financially beneficial to stay in the labor force,” Oster says. It doesn’t mean that everybody should always keep working for this reason, but only focusing on short-term gains may leave some parents with regrets.