Implementing financial incentives and penalties to clinics based on patient retention benchmarks demonstrates little benefit to substance use disorder patients in Maine

Many patients with substance use disorder do not remain in treatment for the recommended duration, prompting efforts to implement systems-level interventions to incentive or penalize clinics on the basis of reaching patient retention and engagement benchmarks. This study evaluated the impact of one such state-wide incentive-based contracting intervention in the state of Maine on patient retention and engagement.

WHAT PROBLEM DOES THIS STUDY ADDRESS?

Participation in substance use disorder treatment drops considerably after the first few sessions resulting in poorer treatment response and higher likelihood of costly readmissions. One strategy to improve patient engagement has been to intervene at the policy level to financially incentivize or penalize programs for meeting established performance benchmarks. The idea here is that such strategies might trickle down and positively impact patient engagement and retention in care. Evaluation of the efficacy of these approaches remains limited and the data that are available show mixed results, with some research showing improved wait times and lengths of stay in outpatient treatment, and also research documenting clinics “cherry picking” less severe patients to improve on performance measures. In this study, the research team examined differences in patient engagement and treatment retention before and after a state-wide incentive-based contracting intervention went into effect for publicly funded treatment agencies in Maine, relative to non-publicly funded agencies for which the incentive-based intervention did not apply.

HOW WAS THIS STUDY CONDUCTED?

Using state-wide administrative claims data from years 2005-2011, the researchers conducted a quasi-experimental study in which they evaluated differences in treatment engagement and retention between patients in programs receiving incentives for meeting performance benchmarks vs. those that did not receive these incentives, and before and after the statewide incentive program went into effect.

For this study, only publicly funded treatment centers were subject to these benchmarks and exposed to these incentives. The incentive-based intervention, in which publicly funded programs could earn or lose up to 9% of their base support for reaching or failing to reach performance benchmarks, went into effect for publicly funded programs in 2008. Thus, the researchers were able to evaluate differences in treatment engagement and retention for all patients receiving publicly funded and non-publicly funded treatment during this time, and specifically evaluate differences in both groups before and after the incentive-based policy took effect.

For this analysis, the authors used a “difference in difference” analytic approach, wherein they examined changes in patient engagement and retention outcomes within the incentivized program group relative to changes among patients in the non-incentivized group. Specifically, they tested whether the two groups changed differently over time answering this question: “Did programs subject to the performance benchmarks and performance-based incentives do a better job engaging and retaining patients than programs over time than programs that were not subject to the performance benchmarks?”

For this study, the authors excluded private practice providers and clinics with fewer than 30 patients per year. Since the researchers were not able to employ true randomization of programs to receive or not receive incentivization, they instead used a statistical technique to ensure that programs and patients in the incentivized program group were being compared to similar programs and patients in the non-incentivized program group. After finding suitable matches for members of each group, the final matched sample included 18,375 outpatients and 5,986 intensive outpatients. The researchers defined different attendance benchmarks for outpatient and intensive outpatient programs, consistent with benchmarks set for each level of care. For their study outcomes, the researchers defined treatment engagement as attendance in 4 or more treatment sessions for outpatients and 4 or more days for intensive outpatients. The researchers defined treatment retention as attending treatment for 90 days or more for outpatients and treatment completion for intensive outpatients.

The average age of patients at treatment admission was approximately 35. The majority of patients were men, unemployed (two-thirds), and on Medicaid (over half). Approximately half of patients were referred by criminal justice settings. Primary substances reported by patients were alcohol, then opioids. Approximately 70% of patients had been in treatment before and most also had a co-occurring psychiatric disorder.

WHAT DID THIS STUDY FIND?

Impacts of Performance-Based Incentives.

Counter to their expectation, the researchers found there were no significant differences in treatment engagement (4+ sessions) among outpatients, and patients in non-incentivized programs were actually more likely to reach the 90-day retention threshold. For intensive outpatient programs, the researchers observed a higher percentage of patients who engaged in 4+ days of intensive outpatient care among incentivized clinics, but no differences in the percentages of patients who completed intensive outpatient programs.

Figure 1.

Patient Factors Associated with Treatment Engagement and Retention.

That said, researchers found a number of patient-level factors that were significantly associated with treatment engagement and retention. Among outpatients, younger (18-24) patients relative to all older age groups, women, single or separated/divorced/widowed (relative to married), and patients with more severe substance use disorder and with less than a high school or equivalent education (compared to those with less than a high school education or equivalency) were less likely to attend 4 or more outpatient sessions. Patients on Medicaid, criminal justice referred, or who received wraparound services were more likely to complete 4 or more sessions. A similar pattern was observed regarding retention in treatment beyond 90 days, with patients reporting opioids as their primary substance being more likely to be retained.

Among intensive outpatients, older patients (relative to 18-24-year-olds) were generally more likely to engage and be retained in treatment. Being married, having at least a high school or equivalent education were associated with higher engagement and retention. Patients referred by criminal justice settings were also more likely to engage past 4 sessions or complete intensive outpatient treatment. Receipt of wraparound services during treatment was associated with a higher probability of completing intensive outpatient treatment. However, having a more severe substance use disorder, a co-occurring psychiatric disorder, or being on Medicaid were associated with a lower likelihood of 4+ session attendance and intensive outpatient treatment completion.

WHAT ARE THE IMPLICATIONS OF THE STUDY FINDINGS?

In this study, researchers conducted a naturalistic experiment to evaluate the potential impact of a state-wide program that incentivized or penalized publicly-funded outpatient and intensive outpatient substance use disorder treatment clinics for meeting patient engagement and retention benchmarks.

They found insufficient evidence that incentivizing was associated with higher rates of early intensive outpatient treatment engagement (4+ days), as this finding was due to stable rates of engagement over time among incentivized programs and a decline in non-incentivized programs. The researchers also found that outpatients in the incentivized programs were less likely to reach the 90-day treatment threshold set for outpatient programs relative to non-incentivized clinics. They did not find significant differences regarding rates of early outpatient treatment engagement (4+ sessions) or intensive outpatient completion rates. These mixed findings are consistent with other reports and potentially speak to the disconnect between policy-level interventions and real-world implementation. The researchers matched comparison groups on clinic and patient characteristics, but it was not possible to examine the clinic-level factors that might relate to the degree to which this incentive program was implemented with fidelity.

Given the large literature on the benefits of providing motivational incentives for patients (“contingency management”), findings here are counterintuitive. The timing of delivery of incentives may be key to the non-findings here – in classic contingency management studies, for example, patients are typically incentivized very close in time/immediately following demonstrating a certain behavior, whereas the clinics may have experienced a longer time lag between their performance and the reward making the reward less directly motivating. Also, while the clinic or program at large may obtain or lose a financial incentive, the providers themselves may be too disconnected from the incentive to be motivated to act differently and make a difference to patients’ engagement. Qualitative exploration of why, how, and where in the causal chain of events this incentivization program did not work will help elucidate areas for future treatment program improvement.

The findings of this study might also speak to the considerable impact of patient-level factors, such as age, gender, substance use disorder and psychiatric complexity, which were just a few of the variables found to predict treatment engagement and retention. Even if implemented with complete fidelity, incentivizing clinics for reaching service benchmarks still does not have a direct impact on patient-level barriers that undermine, or patient-level facilitators that help mobilize, treatment adherence and retention. It remains to be seen if a comprehensive, coordinated approach that bridges systems-level, provider-level, and patient-level interventions can increase treatment engagement and retention. At the same time, it is important to build on what is known about effective reinforcement and prioritize incentivizing program goals that are within the control of the program/providers (e.g., incentivizing making outreach calls to encourage patient treatment engagement or re-engagement). However, caution is warranted in light of findings that incentivizing providers has been shown to produce worse patient outcomes.

LIMITATIONS

1. This was a natural quasi-experimental study and as such, clinics and patients were not randomized to groups but instead compared after matching on key clinic and patient-level factors to account for possible differences between publicly and non-publicly funded substance use treatment clinics.
2. The clinic and patient-level matching strategy used is effective in overcoming between group differences that might otherwise bias results of the comparisons, but there is always risk that the groups differ on unmeasured factors that may still influence study results.
3. The incentive program evaluated was one based on achieving set service benchmarks and were not based on whether programs improved relative to their own baseline performance. Future research might also examine the latter to see whether improvements in the absence of achieving set benchmarks are related to improved patient engagement and retention outcomes.
4. The results of this study were based on data gathered between 2005 and 2011 and might be outdated as a result.
5. The researchers were unable to account for the degree to which publicly funded programs attempted or were successful in implementing the incentive-based policy, or the potential impact that not meeting benchmarks might have negatively impacted service delivery and potentially patient engagement and retention.

BOTTOM LINE

Researchers in this study examined rates of treatment engagement and retention among patients in outpatient and intensive outpatient programs following the implementation of a state-wide program that incentivized publicly-funded programs for achieving patient engagement benchmarks. The researchers did not find the incentives program to be associated with improved patient engagement or retention overall, and that patient-level characteristics (e.g., age, gender, education, disorder severity, Medicaid status) were most associated with engagement and retention.

CITATIONS

Reif, S., Stewart, M.T., Torres, M.E., Davis, M.T., Dana, B.M., & Ritter, G.A. (2021). Effectiveness of value-based purchasing for substance use treatment engagement and retention. Journal of Substance Abuse Treatment, 122. Doi: 10.1016/j.jsat.2020.108217