That’s exactly what happened when the David Agus, the influential physician and CBS correspondent, polled an audience of several dozen in the medical field at Fortune’s Brainstorm Health conference on Tuesday. But ask them what they want to do about it, and it gets a little trickier.

“I hear this a lot about modernizing HIPAA, but no one actually gives a recommendation for what they would do,” said Valerie Montgomery Rice, president and dean of the Morehouse School of Medicine. “If we had a magical wand, what would we do tomorrow?”

Indeed. There are many challenges involved in fixing America’s broken healthcare system, but certainly one of the knottiest is privacy—and the many questions that surround the exploding amounts of personal healthcare data.

That data is as incredibly personal to patients as it is valuable to medical providers who need the information to understand their patients and provide the best care, and to researchers who, by aggregating and analyzing it, can make important, life-saving and life-extending medical discovery. Then there are of course those interested in the gold mine of health data for more nefarious reasons like fraud. Eric Topol, director of the Scripps Translational Science Institute, pointed out that personal medical data goes for five times the amount that financial or other data does on the dark web.

Topol and most of his colleagues agree that patients should own their data. But how that works and can be managed given its value and the various demands for it, leads to more questions than answers.

Hal Wolf, CEO of the Healthcare Information and Management Systems Society, said it’s helpful to think about the issue of data and its access in terms of three buckets: ownership, access, and usage.

“It’s a delicate balance,” Wolf said. “From a physician standpoint, you cannot do your job unless you have access to as much of the data as is available….anything that limits that access literally puts patients at risk.” Plus, he said, “We have to be very mindful of aggregated information and access to it in order to support the research side. If we fail to do that we will not continue the advancements that are happening right now at lightning speed.”

Not everyone was totally comfortable with the idea of patients owning their data. Some raised worries that patients aren’t ready to own their data—both in terms of being medically literate enough to correctly interpret it, or to fully understand the permissions and consent issues that compel people to share research-essential data.

Others also fretted over the potential that patients would only selectively share their data with physicians, hiding parts of their relevant medical history from doctors. (For example, a patient who once suffered from an eating disorder keeping that information from an orthopedist.)

Regardless of who owns it, Wolf added that those who interact with healthcare data, given its highly personal nature, cannot afford to have a Facebook-like moment in which they lose control of it.

Many in the audience believed that blockchain, a security technology using cryptography, will play a larger and larger role in securing health data in the future.

Others pointed out that there are some basic solutions, beyond updating HIPAA, that could improve the current landscape. Topol pointed out that currently there’s a ton of “information blocking” going on by health providers that don’t’ want to share data with others for fear of losing their patients (i.e. business). Mona Siddiqui, chief data officer at the U.S. Department of Health and Human Services commented that in tackling the issue the government was still struggling to get various agencies (mostly their lawyers)—FDA, CMS, CDC—to unify around definitions of basic terms like “research.”

“There’s a tremendous gulf between government and the real world,” remarked Topol.