The landmark rule, issued by the Centers for Medicare and Medicaid Services (CMS), requires hospitals to post online a “machine-readable file” of charges for the services it offers, including all the different rates it charges insurers, plus a consumer-friendly list of charges for the hospital’s 300 most “shoppable services,” meaning services that can be scheduled in advance. The goal, CMS said, is “empowering patients with the necessary information to make informed health care decisions.”

It sounds great, and the rule has been lauded in the media. But so far it isn’t working out as planned. Among the problems:

Only 9% of U.S. adults know about the rule. More than twice as many, 22%, think hospitals are not required to post the prices of treatments and procedures. The rest aren’t sure. The findings are in a new survey by the Peterson Center on Health Care and the Kaiser Family Foundation.

Few hospitals comply fully with the rule. The most recent research, published in JAMA Internal Medicine in mid-June, finds that among 100 randomly chosen hospitals, 83% fail to comply with at least one of the rule’s requirements. About half the hospitals offer the required consumer-friendly cost estimator; only 33% provide payer-specific negotiated rates.

CMS says it began sending warnings to noncompliant hospitals in April, giving them 90 days to comply. If they don’t, CMS may send a second warning letter or require a “corrective action plan” from the hospital. Its ultimate penalty is imposing a fine of up to $300 a day. When the rule was first proposed, many hospitals objected vehemently to disclosing their negotiated rates with different payers, saying the requirement would destroy their negotiating power. For many hospitals, keeping those rates secret may be well worth $300 a day.

Third-party comparison apps using the new data aren’t appearing. The machine-readable files of payer-specific charges aren’t much use to consumers. Massachusetts General Hospital’s file, for example, is a 1.7 gigabyte mammoth that would crash many home computers—the website warns that “these files are not intended for patients”—and would baffle nonexperts. The idea is that independent developers would use those massive files to create convenient comparison apps for consumers.

But with so few hospitals posting files, and others omitting some of the required data, developers face little incentive to create the envisioned empowering apps. In addition, files can be made machine-readable in many ways, potentially requiring developers to decipher each one individually.

Some hospitals don’t want you to see their price transparency data. Hundreds of hospitals used coding on their websites to prevent Google and other search engines from finding the required price information, the Wall Street Journal reported in March. Those hospitals could tell CMS they were complying with the transparency rule while thwarting the rule’s purpose. CMS has since told hospitals they can’t use search-blocking code, but those hospitals have shown their intent.

CMS has issued similar price transparency rules for insurers. From 2022 through 2024, they’ll have to post data files for potential use by comparison app developers and create online shopping tools enabling patients to compare costs, learn what their insurer is paying providers, and know out-of-pocket costs.

How implementation will play out remains to be seen, but health care cost transparency has broad bipartisan support in Washington. Experience with hospitals so far suggests achieving it could be much harder than anticipated.