Some medical professionals are using A.I. chatbots like OpenAI’s ChatGPT, introduced in November, to relate to patients. 

“I wish I would have had this when I was in training,” Dr. Gregory Moore, former corporate vice president at Microsoft Health and Life Sciences, told the New York Times, referring to ChatGPT, in whose parent Microsoft is an investor.  Dr. Moore specialized in radiology and neurology as a physician, and he used the tool when trying to tell his friend, who had cancer, that there were few options to effectively treat the condition.

According to the Times, When Dr. Moore explained the situation via prompt to ChatGPT, the A.I. tool returned this response in one instance: “I know this is a lot of information to process and that you may feel disappointed or frustrated by the lack of options…I wish there were more and better treatments…and I hope that in the future there will be.”

Dr. Moore was surprised by how good the tool was. It’s unclear if he ever communicated the chatbot’s responses to his friend.  

“I have never seen or had a coach like this,” he said.

Chatbots like ChatGPT are being integrated into the day-to-day routine in all sorts of industries—whether banking or online retail. The tool is being adopted faster than the pace at which regulations are being drafted, leading to rallying cries from A.I. industry experts warning about the tech’s potential dangers if it’s misused or developed too fast. Within the medical profession, some doctors are questioning whether using chatbots is ethical. 

“I know physicians are using this,” Dr. Dev Dash, an emergency medicine physician and member of Stanford Health Care’s data science team, told the Times. “I’ve heard of residents using it to guide clinical decision making. I don’t think it’s appropriate.”

OpenAI did not immediately return Fortune’s request for comment.

Many others in the medical field have also been trying out the chatbot despite the concerns. Dr. Harlan Krumholz, the director of Center for Outcomes Research and Evaluation at Yale School of Medicine, thinks that even though it “would be admitting you don’t know how to talk to patients,” there’s value in testing the tool.

“You’d be crazy not to give it a try and learn more about what it can do,” Dr. Krumholz told the Times.

There’s clearly a need for empathy in how doctors approach their patients—a trait that a study found to be lacking among many physicians. Authors of the paper reviewed the audio recordings of physician-family conferences that addressed end-of-life decisions, and concluded that in one-third of the cases, doctors made no empathetic statements. Empathy has also been linked to better outcomes among patients. So, if ChatGPT is helping doctors with communicating more effectively with their patients, some say, why not? 

“Most doctors are pretty cognitively focused, treating the patient’s medical issues as a series of problems to be solved,” Dr. Douglas White, who is the director at University of Pittsburgh’s Program on Ethics and Decision Making in Critical Illness, told the Times. He said that could sometimes lead them to overlook the emotional aspect of treating illnesses.

For its part, ChatGPT has shown some talent for medicine by passing a U.S. Medical Licensing Exam, a requirement to practice medicine in the U.S. Additionally, in an April study by JAMA Internal Medicine, one of the journals published by the American Medical Association, authors fed questions from a Reddit forum where people ask about illnesses and health conditions into ChatGPT. Health care professionals preferred the chatbot’s response in nearly 80% of medical evaluations to those by physicians. 

“There is now research showing that this is going to help—well, let’s see if we can translate this into practice,” Christopher Longhurst, the chief digital officer and chief medical officer at University of California at San Diego Health, told the Wall Street Journal earlier this year.