We hear about the superior tech savvy of people born after 1980 so often that we tend to assume it must be true. But is it?

Researchers at Princeton-based Educational Testing Service (ETS) expected it to be when they administered a test called the Program for the International Assessment of Adult Competencies (PIAAC). Sponsored by the OECD, the test was designed to measure the job skills of adults, aged 16 to 65, in 23 countries.

When the results were analyzed by age group and nationality, ETS got a shock. It turns out, says a new report, that Millennials in the U.S. fall short when it comes to the skills employers want most: literacy (including the ability to follow simple instructions), practical math, and — hold on to your hat — a category called “problem-solving in technology-rich environments.”

Not only do Gen Y Americans lag far behind their overseas peers by every measure, but they even score lower than other age groups of Americans.

Take literacy, for instance. American Millennials scored lower than their counterparts in every country that participated except Spain and Italy. (Japan is No. 1.) In numeracy, meaning the ability to apply basic math to everyday situations, Gen Yers in the U.S. ranked dead last.

Okay, but what about making smart use of technology, where Millennials are said to shine? Again, America scored at the bottom of the heap, in a four-way tie for last place with the Slovak Republic, Ireland, and Poland.

Even the best-educated Millennials stateside couldn’t compete with their counterparts in Japan, Finland, South Korea, Belgium, Sweden, or elsewhere. With a master’s degree, for example, Americans scored higher in numeracy than peers in just three countries: Ireland, Poland, and Spain. Altogether, the top U.S. Gen Yers, in the 90th percentile, “scored lower than their counterparts in 15 countries,” the report notes, “and only scored higher than their peers in Spain.”

“We really thought [U.S.] Millennials would do better than the general adult population, either compared to older coworkers in the U.S. or to the same age group in other countries,” says Madeline Goodman, an ETS researcher who worked on the study. “But they didn’t. In fact, their scores were abysmal.”

What does that mean for U.S. employers hiring people born since 1980? Goodman notes that hiring managers shouldn’t overestimate the practical value of a four-year degree. True, U.S. Millennials with college credentials did score higher on the PIAAC than Americans with only a high school diploma (albeit less well than college grads in most other countries).

“But a degree may not be enough,” Goodman says, to prove that someone is adept with basic English, can do what she calls “workaday math,” or has the ability to use technology in a job. Curious about how the PIAAC measures those skills, or how you’d score yourself? Check out a few sample math questions, or take the whole test.