Researchers Say Opioid Risk Tool Has ‘Too Many False Alarms’

By Pat Anson

The use of artificial intelligence (AI) continues to grow in healthcare, with patient health data and behavior increasingly being used to assess whether a patient is at risk of an illness or chronic health condition.

NarxCare and Epic, for example, scan electronic health records and prescription drug databases to create Opioid Risk Scores (ORS) for patients, which are then shared with healthcare providers to flag patients at risk of opioid misuse or an overdose. Patients deemed to be at high risk may not be able to get a prescription for opioids or they may be abandoned as “too risky.”

But a new study – the first of its kind – suggests that using opioid risk scores to predict patient outcomes is flawed, with unacceptably high rates of false positives.   

The study, recently published in the Journal of General Internal Medicine, looked at Epic’s opioid risk scores for over 700,000 U.S. patients being treated by primary care providers. The vast majority of patients (99.6%) were classified as low risk, with only 0.4% considered at high risk of an overdose or OUD.

It’s reassuring to see so many patients deemed low risk. But how accurate is the risk score in predicting patient outcomes? 

Of the 702,099 patients deemed low risk, only 2,177 went on to have an overdose or OUD diagnosis within the next 12 months. That means the system correctly predicted outcomes 99.7% of the time.

Conversely, of the 2,665 patients deemed high risk, only 185 later had an overdose or OUD diagnosis. That means Epic’s scoring system correctly predicted outcomes only about 7% of the time. 

Researchers say the false positive rate of 92.2% in the high risk category means that Epic’s ORS “produces too many false alarms” and is of little value to providers.  

“In this study, most high-risk patients were false positives, and most who developed OUD or overdosed were false negatives. Because these outcomes are rare, achieving adequate PPV (the proportion of cases that are accurate) is challenging. The ORS’s misclassification could undermine its external validity, leading to misallocated resources and missed interventions,” wrote lead author Stephanie Hooker, PhD, a Research Investigator at HealthPartners Institute.

“Missed interventions” in this case could mean a patient being denied opioid medication or being referred to addiction treatment, when neither move is justified.

On the flip side, Epic’s 99.7% success rate in identifying low risk patients also isn’t foolproof. 

Of the 2,362 patients who experienced an overdose or OUD diagnosis, Epic’s system flagged only 185 of them as high risk — which suggests that over two thousand were incorrectly labeled as “low risk.”

Maybe the lesson here is that “low risk” doesn’t mean no risk, and “high risk” doesn’t provide any certainty either.  

Pain management expert Dr. Lynn Webster says no opioid risk score — whether Epic’s or NarxCare’s – should be viewed as authoritative by doctors and pharmacists in making clinical decisions.     

“Both tools can be harmful if used punitively. The NarxCare scores have shown that overestimated risk may lead to forced tapering, abandonment, or other punitive responses, which could paradoxically increase overdose risk. With Epic, the harm is a bit different: the score can both stigmatize flagged patients and falsely reassure clinicians about the much larger group labeled low risk,” said Webster, a Senior Fellow at the Center for U.S. Policy (CUSP).

In 2023, CUSP petitioned the FDA to take Narxcare’s ORS software off the market as an unproven and misbranded medical device. The FDA rejected the petition on procedural grounds. 

In the case of Epic’s ORS, Webster says it is a mistake to count OUDs and overdoses in the same prediction model because they are distinct events. Someone can overdose without having OUD, while someone can have OUD without ever experiencing an overdose.   

“Opioid risk tools will always struggle to predict overdose death risk because overdoses can occur in patients who have no opioid use disorder and no aberrant drug-related behavior,” Webster told PNN. “Some patients overdose even when they take their medications exactly as prescribed. Overdose can also occur because of comorbid medical conditions or other factors unrelated to OUD.”

As flawed as they might be, Epic and NarxCare are already embedded in the U.S. healthcare system. Data on over 190 million patients has been collected by Epic’s MyChart software, while NarxCare is used by Walmart, Rite Aid, CVS and other major pharmacy chains to analyze patient risk.

“Whether the score comes from NarxCare or Epic, the core danger is the same: once a proprietary risk label is embedded in the chart, it can take on a false authority that changes how patients are treated,” says Webster.

FDA Approves Genetic Test for Opioid Addiction Risk

By Pat Anson, PNN Editor

The U.S. Food and Drug Administration has approved a controversial genetic test that uses a patient’s DNA to assess whether they are at risk of developing opioid use disorder (OUD).  

Although the test is only intended for patients with short-term acute pain who have not used opioids before, there is concern about the test’s accuracy and whether it will be used “off-label” to assess addiction risk in chronic pain patients – who could potentially lose access to opioids as a result.

In approving the AvertD test, the FDA stipulated that it only be available by prescription to patients who consent to its use and have no prior history of using an oral opioid for pain relief.

The test is administered by a provider swabbing the cheek of a patient to collect a DNA sample, which will then be tested in a laboratory to see if the patient has 15 genetic markers that puts them at elevated risk of OUD.

According to the FDA, the test will help patients “make better informed decisions” about opioids, such as a patient facing surgery who wants to know what analgesic to use for acute post-operative pain.

AUTOGENOMICS IMAGE

"AvertD may help patients who are concerned about being treated with an opioid for acute pain,” Jeff Shuren, MD, director of the FDA's Center for Devices and Radiological Health, said in a statement. “The test is not intended to be used in patients being treated for chronic pain.”

But given the history of opioid guidelines being mistakenly applied to all kinds of patients, regardless of their condition, some worry the test will be misused.

“I’m sure it would be used for anyone who may be considered for opioid therapy,” says Lynn Webster, MD, a pain management expert and Senior Fellow at the Center for U.S. Policy. “I am all for gathering more data to help clinicians make better decisions, but we must exercise caution with such tests. Otherwise, the test may be over-read or misinterpreted. Some patients may be deprived of access to an opioid if they test positive or there can be a false sense of harmlessness from opioids if the test is negative. 

“I am most concerned that providers will see the results as binary. Either a patient will or won’t develop OUD, depending on the result. That would be a big mistake. Any such device or test must be used along with other clinical and personal information to help mitigate harm from using, or being denied, opioids.”

80% Accuracy

As part of its approval order, the FDA is requiring AutoGenomics – the company that makes AvertD – to provide training to healthcare providers on the proper use of the test and to conduct a post-market study of its performance and accuracy.

In 2022, an FDA advisory committee voted 11-2 against recommending an earlier version of AvertD, primarily because of concerns about false-negative and false-positive results. An observational study found the test was about 80% accurate in detecting genes associated with OUD.

"I believe 100% of the risk associated with this test is with false positives and false negatives -- both people being untreated or poorly treated because somehow it came back as a positive result, or being given inappropriate treatment because it said negative," said Timothy Ness, MD, an anesthesiologist and Professor Emeritus at the University of Alabama at Birmingham, an advisory panel member who voted no.

After the advisory committee vote, the FDA worked with AutoGenomics to modify the test and improve its accuracy. The company then submitted a premarket approval application for the modified test, which the FDA granted without going back to the advisory committee for further review.

“The FDA recognizes that in premarket decision-making for devices, there generally exists some uncertainty around benefits and risks. Given the totality of available evidence and the urgent need for medical devices that can make a positive impact on the overdose crisis, and specifically devices that can help assess the risk of developing OUD, the FDA determined that there is a reasonable assurance of AvertD's safety and effectiveness,” said Dr. Shuren.

But no test is foolproof in either its accuracy or implementation, as Dr. Webster learned when he developed a questionnaire that assesses addiction risk by asking patients about their family history and other potential risk factors. Webster was disappointed to learn his questionnaire was “weaponized” by some providers to deny opioid therapy to patients, particularly women with a history being sexually abused.

Webster says the risk of OUD can’t be measured by a genetic test alone.

“We should not think it is a diagnostic tool or a crystal ball. Having an increased risk due to genetics does not mean that, if exposed to an opioid, an individual necessarily will develop an opioid addiction,” Webster told PNN. 

“We have known for a long time that about fifty percent of the risk of developing an opioid addiction is due to genetics. The other fifty percent is due to environmental factors and life’s experience. Furthermore, people can develop OUD without genetic risks. OUD risk is dynamic, meaning it changes over time with adverse events in life and often co-morbid conditions. For example, there was a surge in all forms of drug abuse, including OUD, during the pandemic because of isolation and loneliness. This is not detected by a genetic test.”

Although the risk of a surgery patient misusing opioids or becoming addicted is low – less than one percent -- the parent company of AutoGenomics has a more stark assessment, calling surgery “a gateway to addiction” that puts another 7 million Americans at risk every year.

Large Study Debunks Myths About Rx Opioid Addiction

By Pat Anson, PNN Editor

A large new study in Australia is debunking myths about the addictive nature of opioid pain medication and how a single prescription can lead to long term use.

In an analysis of nearly 3.5 million Australians who were prescribed opioids for the first time, researchers found that 92% never progressed beyond low opioid use and only 3% became persistent users or needed higher doses. The “sustained use” of opioids occurred mostly in seniors (65 and older) suffering from cancer, depression, anxiety and other chronic health problems.

“Results of this cohort study suggest that most individuals commencing treatment with prescription opioids had relatively low and time-limited exposure to opioids over a 5-year period,” wrote lead author Natasa Gisev, PhD, a clinical pharmacist at National Drug and Alcohol Research Centre, UNSW Sydney.

“The small proportion of individuals with sustained or increasing use was older with more comorbidities and use of psychotropic and other analgesic drugs, likely reflecting a higher prevalence of pain and treatment needs in these individuals.”

Previous research estimated the risk of long-term opioid use as high as 57 percent, but Gisev and her colleagues say those studies are flawed because they excluded people with cancer and opioid use disorder, or used various definitions of long-term use. The Australian researchers took a different approach in tracking opioid prescriptions, developing five “trajectories of opioid use.”

The vast majority of patients had very low (75%) or low use (17%) of opioids. Others started with moderate opioid use and decreased to low use (3%), or began with low use and increased to moderate use (3%). Only 3% became sustained users or took higher doses.

Five-Year Trajectories of Opioid Use

JAMA NETWORK OPEN

“Overall, these findings suggest that most people who initiate an opioid prescription are likely to have low, time-limited exposure to opioids with little indication of ongoing use. This possibility is an important consideration for policymakers and stakeholders considering population-level prescribing of high-risk drugs,” researchers reported in JAMA Network Open.

“Opioids are essential drugs for acute and cancer pain, and many people with CNCP (chronic non-cancer pain) benefit from opioids. Continued focus and policy responses based on findings from a small group of people with increased risk of harms run the risk of limiting access to people who safely derive objective benefits from opioids.”  

That’s exactly what happened to millions of pain patients in the US, who lost access to opioids due to steep cuts in prescribing, chronic shortages of opioid medication, and a relentless drumbeat in the media about people becoming addicted or overdosing after one pill.  

“One prescription can be all it takes to lose everything,” the CDC said in a 2017 advertising campaign that emphasized the risks of prescription opioids, while ignoring the far more serious problem of illicit fentanyl and other street drugs.

Just how common is opioid addiction? Estimates are all over the map, ranging as low as 1% to as high as 40% for long-term opioid users.

“The problem with all of these studies is that they are retrospective in nature, limited to a particular patient population, and constrained by the diagnostic criteria in use at the time,” my late colleague Roger Chriss pointed out in a 2019 column on opioid addiction.  

Roger was prescient about the imprecision of diagnostic criteria. A little known 2021 study of U.S. veterans who were diagnosed with opioid use disorder (OUD) challenged how many actually had it. Researchers found that nearly 20% of veterans with OUD used prescription opioids without any signs of abuse and evidence was lacking in another 7% of cases. Less than 60% of the veterans had a “high likelihood” of being correctly diagnosed with OUD.

The Rx Opioid Most Likely To Be Misused May Surprise You

By Pat Anson, PNN Editor

For well over a decade, addiction treatment providers and public health officials have been touting the benefits of buprenorphine, an opioid that can treat both pain and addiction. When combined with naloxone in drugs like Suboxone that treat opioid use disorder (OUD), buprenorphine reduces cravings for opioids and lowers the risk of abuse.    

But a new study published in JAMA Network Open suggests that someone is far more likely to misuse buprenorphine than other opioids. In fact, the misuse rate for buprenorphine is over two times higher than misuse rates for hydrocodone, oxycodone and other opioid pain medications.

Researchers at the National Institute on Drug Abuse and the Centers for Disease Control and Prevention conducted the study, looking at data from nearly 215,000 people who participated in the National Surveys on Drug Use and Health from 2015 to 2019.

Respondents were asked if they misused prescription opioids “in any way that a doctor did not direct you to use them.” If they used someone else’s prescription or took opioids in greater amounts or more often than they were told by a doctor, that was considered “misuse.”

Researchers crunched the numbers and found that the vast majority of people do not misuse opioid pain medication and take it as directed. Oxycodone, for example, was misused by 12.7% of respondents who took it, followed by hydromorphone (11.8%), hydrocodone (11.6%), and prescription fentanyl (11.5%). Tramadol (7.8%) was misused the least.  

Addiction treatment drugs were misused the most. Buprenorphine was misused by 29.2% of the people who took it, followed by methadone at 22.2 percent. It’s not uncommon for someone getting OUD treatment to have relapses, so perhaps that finding is not altogether surprising.

% MISUSE RATES FOR PRESCRIPTION OPIOIDS

SOURCE: JAMA NETWORK OPEN

Although buprenorphine is misused at a rate over two times higher than other opioids, researchers chose to focus on the positive: a recent downward trend in buprenorphine misuse, despite increases in the number of patients receiving buprenorphine treatment.

“In 2019, nearly three-fourths of US adults reporting past-year buprenorphine use did not misuse their prescribed buprenorphine, and most who misused reported using prescription opioids without having their own prescriptions. These findings underscore the need to pursue actions that expand access to buprenorphine-based OUD treatment, to develop strategies to monitor and reduce buprenorphine misuse,” researchers concluded.

What Is Misuse?

Every study has its flaws, and this one is no exception. Findings based on self-reported survey results are subject to poor memories, recall bias and concerns about stigma. The researchers’ broad definition of “misuse” could also result in a diagnosis of OUD when none actually exists, according to a pain management expert.

“It is not really clear what any of the data means clinically because of the very broad definition of the word misuse,” said Lynn Webster, MD, Senior Fellow at the Center for U.S. Policy (CUSP) and Chief Medical Officer of PainScript. “Behavior of taking an extra pill to control pain, despite it not being specifically directed by the prescribing provider, could be described as ‘misuse.’ This is not necessarily harmful, even if it is inappropriate. The implication is that simply taking an additional pill is an indication of OUD behavior. That would not be an appropriate characterization of the behavior. 

“In fact, the authors report the most common reason to misuse medication is to relieve pain in the OUD and non-OUD groups. This may imply that most people who are misusing their medications are experiencing undertreated pain.” 

The JAMA study is not the first to report a high rate of buprenorphine misuse. The DEA’s 2020 National Drug Threat Assessment reported that buprenorphine is misused more often than methadone or hydrocodone, and that it was poised to replace oxycodone as the most commonly misused prescription opioid. Unlike the JAMA study, the DEA said the misuse of buprenorphine was increasing, not declining.