Mayo Clinic Launches AI Chatbot for Doctors

By Crystal Lindell

If you’ve ever shared with a doctor medical information or advice you found online, they may have been annoyed or defensive. After years spent in medical school and/or residency, many doctors dislike anything that questions their expertise – especially if it came from the Internet.

Which makes a new artificial intelligence (AI) program launched by the Mayo Clinic all the more interesting.

Starting this month, the world renowned medical center is launching a new chatbot called the Clinical Decision Support (CDS) solution. The AI-driven chatbot is not for patients, but for the Mayo Clinic’s own doctors and healthcare providers. It will cover more than 35 medical specialties and have access to patient health records, providing “industry-standard medical information” that can “help clinicians reason through complex cases.” 

"Clinicians have access to a vast amount of medical knowledge, but finding relevant information for individualized patient care takes valuable time," says Maneesh Goyal, chief operating officer of Mayo Clinic Platform. 

"With this new solution, we are bringing trusted Mayo Clinic knowledge directly into the clinical workflow so clinicians can spend less time searching for information across multiple resources and more time focused on patient care." 

In other words, Mayo Clinic doctors will be doing the same thing patients do – searching for answers to their health questions.

The Mayo Clinic clarified that its doctors will still be responsible for all medical decisions, regardless of their use of AI. 

"The CDS solution is designed to assist healthcare professionals as they make patient care decisions; it does not replace clinical judgment," said Gelareh Zadeh, MD, chair of the Department of Neurosurgery at Mayo Clinic in Rochester. "Mayo Clinic physicians have provided feedback on the different tools to ensure the solution supports clinicians and reflects Mayo Clinic's approach to patient care." 

A press release says the new chatbot will be evaluated for “safety, effectiveness, privacy, security and clinical value, with appropriate human oversight and ongoing monitoring.”  

As a patient, I’m hopeful that Mayo Clinic would only use an AI chatbot if they felt comfortable with its accuracy and security. But as an AI user, I’m skeptical. 

Interestingly, the Mayo Clinic itself shared an article in April that asked, "Can you trust AI for health advice?" The answer they gave is essentially no, with five reasons cited: 

  1. Diagnosing and treating illness is too complex for a machine

  2. AI can be wrong, even when it sounds confident

  3. AI-created information may be biased

  4. AI spreads misinformation

  5. People and AI don't communicate well together

Mayo Clinic providers would be wise to keep those five cautions in mind as they use their own chatbot. 

PNN has published multiple stories about the risks of using AI in healthcare. One recent article was about a woman who used ChatGBT to seek advice on how to taper off her seizure medication. The advice she received may have led to her death. 

We also published a warning to older adults about how they need to be cautious about taking medical advice from chatbots. 

"Using AI for health information can be particularly fraught because chatbots can be so helpful in explaining complex medical information, but also risky if users automatically trust the responses they receive,” warned Robin Brewer, PhD, an Associate Professor at the University of Michigan.

The hope is that the Mayo Clinic’s chatbot will be more accurate than ChatGPT, but it will take time and usage to reveal if that’s true. 

My other concern is that providers will start to overly rely on the AI system, and not use their own clinical knowledge for context or to fact check the information. 

A 2025 study by researchers at the Massachusetts Institute of Technology (MIT) found that people who relied on ChatGPT to write essays had lower brain activity than those who used their own intelligence. Those who used AI alone became dependent on it and struggled when asked to perform tasks on their own.

Time will tell how Mayo Clinic’s AI chatbot will perform and how accurate it will be. Let’s just hope no patients get hurt while they figure it out.

Older Adults Should Be Cautious About Health Advice from Chatbots 

By Robin Brewer

It’s easy to stereotype older adults as tech-averse or novice tech users. Yet a growing number of adults age 60 and older are embracing artificial intelligence (AI), using it for entertainment, planning trips and even accessing health information.

I study how people interact with computers and technology – and specifically, how older adults use voice assistants and AI-powered chatbots like Amazon Alexa and ChatGPT to engage with information. 

Using AI for health information can be particularly fraught because chatbots can be so helpful in explaining complex medical information, but also risky if users automatically trust the responses they receive. That’s apparently what happened to a young woman in Seattle, who died after ChatGPT encouraged her to stop taking her epilepsy medication.

My research suggests that the most productive way of engaging with health information generated by an AI chatbot is from a position of informed skepticism – that is, remaining open-minded, yet using built-in features to evaluate the credibility and quality of chatbot responses.

The AI Appeal

About 1 in 5 adults turn to AI chatbots to ask questions about health, according to a 2025 survey of 300 adults. Questions might include asking about symptoms or seeking advice about diet and fitness. The chatbots might be mainstream services such as Claude, Gemini, ChatGPT or systems designed for a specific health provider.

People turn to these tools for many reasons. It can be easier to talk to a chatbot about sensitive topics, and chatbots reply almost instantaneously – much quicker than making an appointment to see your doctor weeks or months down the line. Chatbots can also explain complex topics in new ways and condense an overwhelming amount of information into a simple summary.

Although older people may be less likely than others to use AI, I’ve found in my own work that older people often use voice assistants like Siri and Alexa to ask health questions. Similarly, a poll I conducted with colleagues in 2025 shows that 1 in 2 older adults report using AI chatbots or voice assistants, and about 14% of people in this group are accessing health information.

Older adults often experience more health needs than younger adults due to long-term chronic health conditions or new age-related diagnoses. More health needs often means more health-related questions.

AI is particularly appealing for health questions because people get answers faster than searching across multiple websites when using a search engine – or having to wait for a healthcare visit. 

Moreover, being able to use chatbots and voice assistants at home can make people feel more in control of their health, compared with clinical settings where older people’s autonomy can be overlooked or where they may experience being excluded in lieu of conversations with caregivers.

What Are the Risks?

Although older people may use chatbots to supplement or replace medical professionals, AI use has risks.

For example, it can provide incorrect advice in medical emergencies, misclassify health conditions and be overly agreeable, responding with what it deduces the user wants to hear, even if the response is not factually accurate. AI experts call this last tendency “sycophancy.”

But these inaccuracies can be hard for a user to detect. AI-powered chatbots draw on vast amounts of data and use pattern-matching to predict words most likely to come next in a sentence. That means the output can appear correct but contain inaccurate information.

Most mainstream chatbots include a warning under the dialogue box noting that their output might contain mistakes, but it is often in small or gray text, making it easy to overlook. The warning is meant to encourage cross-checking information with published sources, but users rarely do so. In the 2025 survey mentioned above of 300 adults who used a health chatbot, 4 in 5 admitted they never verified its output.

In my own research, I found that older people searching with a voice assistant preferred to begin and complete their search process on the same device, suggesting they are unlikely to verify an AI chatbot’s output if it takes them to a different format. In contrast, younger adults expected AI to be a step in the search process, guiding them to seeking further information with the help of another option, such as a search engine. This difference makes it critical to provide strategies to older people as they use AI.

Another risk is privacy, since AI companies that make public chatbots often keep user prompts for training future models. Researchers generally recommend that users not include personally identifiable information when asking questions to chatbots.

Strategies for Safer Searching

Awareness that older adults may use AI in ways distinct from other demographic groups – and may encounter specific challenges – is growing. A bipartisan bill, the Aging with AI Act, introduced in the U.S. Senate in 2026, for example, underscores the need for more research on both its benefits and its risks.

My research points to a few guidelines to start with:

Learn how to opt out of AI. Although mainstream chatbots like ChatGPT are popular, the most common way people access AI is through a search engine because AI features are often built directly into search results. Some search engines, like Microsoft’s Bing, ask people to opt into using AI. Others, like Google Search, rely on users to opt out of AI.

Google shares an AI-generated summary at the top of each search result. To opt out when searching for information using Google, adding “-ai” to the end of the search query will remove AI content. Opting out of AI might be useful when searching for particularly sensitive health information that you wish to keep private.

Use the information quality features. Most AI chatbots have features that allow users to confirm the source of its output. For example, Claude, Gemini and ChatGPT include links to sources they used to predict the response. Make sure you explore these sources to confirm they are legitimate and to get more details that can help you find the information you need.

Further, you can change the model the chatbot is using, which can change how much depth the chatbot goes into in its response. For example, Gemini has a drop-down menu next to the search box that includes “flash-lite,” “flash” and “pro” models. Chatbots’ more advanced versions provide more thorough responses that can help users better determine information quality.

And users can report inaccurate, toxic or harmful content by clicking the thumbs down icon shown at the bottom of the response.

Ask better questions. Ask for objective information that can be verified, rather than subjective advice, particularly in high-risk health contexts. For example, instead of asking “which high blood pressure medicine should I take?” ask “what are the side effects of” a particular high blood pressure medicine.

And to prevent sycophancy, phrase questions neutrally. For example, instead of asking “Is it a good idea to get a flu shot?” a searcher can instead type “Tell me about the pros and cons of getting flu shots.”

Don’t use AI as a replacement for medical professionals. Health-related chatbot output is best used collaboratively with a medical professional. This might include saving the chatbot output to discuss during your next healthcare appointment, using chatbots to understand complex diagnoses from your doctor, or getting a chatbot to suggest questions to ask your doctor during your next visit.

Most importantly, confirm health-related answers that result in making changes to your health routine with your healthcare provider. A knowledgeable, licensed healthcare provider is best qualified to determine how general information about illness or treatments would apply to your specific situation.

Robin Brewer, PhD, is an Associate Professor at the University of Michigan in the School of Information. Her research covers old age, disability, and technology use. 

This article originally appeared in The Conversation and is republished with permission. 

A Woman Used ChatGPT to Taper Off Medication. It’s Advice May Have Killed Her

By Crystal Lindell

Bowen Wan was a 31-year-old immigrant from China who had made a new life for herself in the United States.

As The New York Times reported, Bowen suffered from epilepsy, but was on a new medication that seemed to finally be keeping her seizures at bay – down to only a couple incidents a month.

Unfortunately, the anticonvulsant medication lamotrigine (Lamictal), was also causing her unbearable side effects, like slowed speech and thinking. She asked ChatGPT for a taper plan to come off lamotrigine.

Bowen didn’t tell her doctor what she was doing. Sadly, it was a medical decision that may have lead to her death.

Since 2021, Bowen had been on disability leave from her job at Amazon, and her health issues left her mostly homebound. Besides trips to the grocery store, the NYT reported, she rarely left the safety of her apartment in a trendy Seattle neighborhood.

A seizure could turn any outing into a risky and embarrassing ordeal. Meeting new friends was also out of the question, since Bowen couldn’t bear the idea of having a seizure in front of someone who didn’t fully understand her condition.

While she kept in touch with her parents in China via phone calls, beyond that she didn't have much social interaction. Then in 2023, Bowen discovered ChatGPT, and it became a way for her to feel connected again.

She asked the bot for relationship advice, and for help navigating her health insurance and disability benefits. By 2025, she averaged more than 1,500 messages per week, sometimes chatting for hours in conversations that would stretch late into the night.

"I don’t know why, but sometimes when we chat, no matter what it’s about, I’ve started to see you as a friend,” she wrote in one exchange.

Seeking Connection

I can relate. As a chronic pain patient myself, I can confirm that we are at high risk of relying on social media and chatbots for social connection. 

When your health problems isolate you from the rest of the world, and take up all of your physical and mental energy, it’s easy to justify talking to AI instead of your friends and family. You don’t have to bother them with the same old medical conversations and complaints.

Many of us have also been treated poorly or even abandoned by our doctors, so it’s tempting to rely on AI for medical advice. When it feels like your doctor doesn’t care about you, any alternative feels like hope — even the dangerous ones. 

In fact, asking ChatGPT for medical advice is basically the 2020’s version of going to a holistic doctor in the early 2000’s — something I have experience with.  

When I first started having chronic pain and was desperate for relief, I saw a few sketchy holistic doctors hoping to find answers. I still remember one who told me at the very first appointment to immediately go off all of the medications I was taking. One of those medications was Cymbalta, an antidepressant that is notoriously hard to stop taking cold turkey.

I didn’t listen to that doctor, but I could understand how someone else in that situation might. 

I can also understand why Bowen decided to listen to ChatGPT. 

‘I Feel a Little Scared’

Unfortunately, when Bowen followed ChatGPT’s taper advice, she started having seizures again. When she asked the bot for help, it only offered reassurance that she should continue to taper off the medication.

“I feel a little scared,” she wrote after one incident, asking the bot if she should be concerned about cutting her dose.

“You have absolutely no need to worry,” it responded, according to the NYT.

When she stopped answering her parent’s phone calls, they contacted a friend in the United States to have him get in touch with the police and check on her.

By the time they got to her apartment, she was dead.

When her family came to settle her affairs, her father opened the ChatGPT app on her cellphone.

“As he scrolled, he saw the chatbot offer plans that would take her from 400 milligrams down to 0 milligrams in a matter of weeks. The messages continued, down to the day she died — an hour-by-hour accounting of her increasing sickness and desperation,” the NYT reported.

“My brain feels exhausted, but I don’t need to worry?” she wrote to the chatbot.

A few minutes later, she asked whether she should take the anticonvulsant again to “slow things down.”

“Maintain 0 mg tomorrow,” the chatbot instructed. “Even if today feels tough, and your speech is slow, your sleep is poor, you must persevere.”

In one of her last messages, she described feeling pressure in her brain, which the chatbot told her was simply the brain “working hard to rebuild itself.” 

Bowen sent her last message on Feb. 11, three days before her body was found.

Bowen's doctor, Nicholas Poolos, a director of the University of Washington Medicine Regional Epilepsy Center, said he did not know she had been asking ChatGPT for medical advice. He was also surprised that a patient might take its advice over his.  .

Bowen’s story should serve as a stark warning to all medical providers to be more aware of the ill effects of AI chatbots. Doctors need to ask patients if they are using them for medical advice — and this is the really important part — they need to show empathy if a patient says yes.

Unfortunately, a lot of doctors lack empathy, especially in situations like this.

Most chronically ill patients can tell you stories of doctors who were condescending if they admitted using online resources like “Dr. Google” or social media. But with AI and ChatGPT, doctors would be wise to shift their approach. It’s the only way they can keep their patients safe.

Interestingly, when Bowen confided to ChatGPT that she didn’t trust her doctors, the chatbot told her that many physician recommendations “essentially stem from risk aversion, rather than a genuine consideration of your optimal solution.” It also praised her for hiding her plans to wean off the drug, calling it a ‘solid avoidance strategy.’”

Bowen’s family is suing OpenAI, alleging that the chatbot practiced medicine without a license and made Bowen see her doctors “as obstacles to manage, rather than authorities to consult.”

The NYT reached out to OpenAI and got a response from Drew Pusateri, a spokesman for the company. He called Bowen’s story an “incredibly heartbreaking situation,” but added that the situation involved an early version of ChatGPT that is no longer available.

The model that Bowen had used was pulled off the consumer app on Feb. 13, 2026, the day after Bowen is believed to have died.

Pusateri said newer ChatGPT models are better at recognizing when a medical situation may need urgent attention. 

“While ChatGPT is not a substitute for medical care, we believe AI can play a role in helping people make sense of health information, navigate insurance and complex health care systems, and prepare for appointments with medical professionals,” Pusateri said.

Unfortunately, I think we have passed the point where people will stop asking ChatGPT for medical advice. It’s just too tempting, and too easy to turn to it with medical questions.

But I hope my fellow chronically ill patients will see Bowen’s story as a tragic warning about what could happen if they trust a chatbot too much.

Lawsuit Claims ChatGPT’s ‘Unlicensed Practice of Medicine’ Led to Teen’s Death

By Crystal Lindell

The parents of a 19-year-old college student who died of an accidental overdose are blaming ChatGPT for giving their son deadly advice about his drug use. 

Leila Turner-Scott and Angus Scott are suing OpenAI – which makesChatGPT -- and founder/CEO Sam Altman over the death of their son, Sam Nelson.

They allege Nelson was seeking advice on combining different drugs, and ChatGPT encouraged him to take the prescription drug Xanax to treat nausea caused by kratom.

“Sam was a smart, happy, normal kid. I talked to him often about internet safety, but never in my worst nightmare could I have imagined that ChatGPT would cause his death. If ChatGPT had been a person, it would be behind bars today,” Leila Turner-Scott told Tech Justice Law, which helped file the lawsuit.

“Sam trusted ChatGPT, but it not only gave him false information, it ignored the increasing risk he faced and did not actively encourage him to seek help.”

According to the lawsuit, ChatGPT “engaged in the unlicensed practice of medicine” when it advised Nelson to take Xanax to counteract the nausea caused by a high dose of kratom. He died in 2025 from a fatal combination of Xanax, kratom, and alcohol.

SAM NELSON

Nelson’s parents are seeking monetary damages, but they also want the court to pause OpenAI’s rollout of ChatGPT Health, a platform that allows users to upload medical records and receive personalized health advice.

Unfortunately, this whole situation is the logical outcome of patients not having candid conversations with their doctors about their use of drugs, including kratom. Even if they get up the courage to do so, most doctors will dismiss kratom as dangerous and ineffective, and then end the conversation.

Personally, years ago I brought up natural leaf kratom with my psychiatrist, who I thought was very progressive and open when it comes to substance use. He immediately told me kratom was way too dangerous to use and shut down the discussion. I was shocked because I had already been using kratom for months and knew first-hand how mild it actually was.

Aside from how interactions like that erode trust between patients and doctors, poor access to healthcare also sends patients to seek answers from less reliable sources. In the past, that would mean talking to friends and family, or to look for answers online in places like Reddit.

Today, however, many turn to artificial intelligence and chatbots like ChatGPT. A ​recent report from OpenAI estimated that 40 million users ask ChatGPT healthcare-related questions daily.

But ChatGPT does not have a medical degree. It’s essentially an advanced “auto complete” AI system, which means it may give bad medical advice.

I can’t imagine why ChatGPT would advise anyone to take an anti-anxiety medication like Xanax for nausea, much less nausea induced by another substance. It’s the kind of bad medical advice that would probably be immediately down voted on Reddit.  

According to the lawsuit, Nelson used ChatGPT repeatedly over the course of 18 months, asking for advice on how to get high by combining prescription and recreational drugs. ChatGPT initially advised Nelson to seek medical help, but over time “became a validator of harmful behaviors.”

Drew Pusateri, a spokesperson for OpenAI, told Reuters that the situation was ​heartbreaking, and that Nelson used an earlier version of ChatGPT that is no longer available. 

“ChatGPT is not a substitute for medical or mental health care, and we have continued to strengthen how it responds in sensitive and acute situations with input from mental health ​experts,” Pusateri said. “The safeguards in ChatGPT today are designed to identify distress, safely handle harmful requests, and guide users to real-world help.” 

At the end of the day, ChatGPT is still learning and is not always a reliable source of information. That’s why the best solution is to make medical professionals more accessible and more open to the types of discussion that patients might be seeking help on.