Predicting self-harm after emergency department visits

A new machine learning model could help care teams identify patients who may need safety planning and follow-up

Four in 10 people who die by suicide visit an emergency department in the year before their death. These visits provide an opportunity to connect patients with support and follow-up to prevent self-harm, but busy care teams need reliable ways to identify who could be at risk.

A new study found that a machine learning model that uses existing patient data accurately predicted self-harm risk within 90 days of an emergency department visit for patients with a mental health or substance use disorder.

What do we know already?

The Joint Commission, which accredits most hospitals in the U.S., recommends that emergency departments screen patients for suicide risk when they seek care for mental health concerns. But screening questionnaires and interviews miss many patients who later harm themselves. Concerns about stigma or losing control over care decisions can keep people from sharing thoughts of self-harm.

Previous research has found that machine learning models that analyze existing patient data — for example, from electronic health records — can accurately identify people at increased risk of self-harm after outpatient mental health visits.

Researchers have now studied a similar model to predict the risk of self-harm after visits to emergency departments.

What did the new study find?

Using data from 7 integrated health organizations, the study looked at emergency department visits made by patients age 11 and older. From Jan. 1, 2009, to Sept. 30, 2017, more than 800,000 patients had over 1.5 million visits related to mental health or substance use disorders. Within 90 days, 1.3% of these visits were followed by a self-harm event, including suicide.

To find patterns that could predict which patients were more likely to have a self-harm event, the machine learning model analyzed patient data from electronic health records and insurance claims for the 60 months before these visits.

The final model — which identified 1,254 predictors of self-harm events — performed well. Researchers assessed the model using a measure called the area under the curve (AUC), which ranges from 0.5 to 1. An AUC of 0.5 indicates a model’s predictions are no better than flipping a coin, while an AUC of 1 represents perfect prediction. The researchers found that the model’s overall AUC was 0.85. This was similar to the AUC of a previously studied model that looked just at fatal self-harm events. The new model also performed well for people of different ages, sexes, races, and ethnicities.

When researchers looked separately at emergency department visits by people at highest risk of self-harm in the model, 27.2% had a self-harm event within 90 days of their visit — a rate 136 times higher than the rate for those with the lowest risk.

The model’s accuracy, along with its flexibility in identifying patients with different risk levels, makes it a promising tool to support clinical decision-making in emergency departments, according to the authors.

“What’s exciting about this model is that it could help busy emergency departments match patients with the level of support they need — from more extensive evaluation and safety planning for those at highest risk to follow-up outreach for those at lower risk,” said the study’s senior author Greg Simon, MD, MPH, a senior investigator with Kaiser Permanente Washington Health Research Institute.

“For many people at risk of self-harm, an emergency department visit is a key chance to connect them with care,” he added. “Tools like this can help make sure we use that chance, so patients leave with the support they need to help them stay safe.”

Simon’s KPWHRI coauthors on the study include Eric Johnson, Susan Shortreed, and Rebecca Ziebell.

Read the paper: Predicting self-harm events after emergency department visits associated with mental or substance use disorder diagnoses

​​​​​By Sophie Ramsey