Does adding online coaching improve outcomes of a digital intervention for problematic drinking?

Online interventions for problematic drinking can, in theory, reach and support a much larger population of individuals than face-to-face interventions. How much guidance and support should be provided by peers, counselors, or other clinicians to individuals using such online interventions to improve their drinking remains an open question. This study tested whether online coaching with certified mental health workers via real-time chat messages would improve outcomes of an otherwise self-guided online program.

WHAT PROBLEM DOES THIS STUDY ADDRESS?

Large proportions of the populations in most middle- and high-income countries around the world, including the U.S. and Australia, consume alcohol at a rate that is deemed as placing individuals “at-risk” by national guidelines for adverse health effects. A small subset of such individuals also have alcohol use disorder. Among those who have alcohol use disorder, only a small subset seeks or receives treatment.

Providing online interventions, in theory, can help reach and support a much larger population of individuals seeking to overcome problematic drinking because people can typically access these interventions at any time from home. A concern in providing such an online resource is that treatment needs may be higher for help-seekers than can be provided by an entirely self-guided intervention. This need became apparent within the context of an Australian social media campaign and platform. Initially, this social media platform, called “Hello Sunday Morning,” asked participants to publicly set a personal goal to stop drinking or reduce their consumption, for a set period of time, and to record their reflections and progress on blogs and social networks. Over time, it became evident that a large number of people who used this platform were demonstrating signs of moderate to severe alcohol use disorder. In response, the “Daybreak” program was developed (and replaced “Hello Sunday Morning”), which offers online health coaching with a trained mental health clinician in an effort to provide increased support. Tait and colleagues tested if the addition of this online health coaching improved drinking outcomes.

HOW WAS THIS STUDY CONDUCTED?

This study is a quasi-experimental randomized controlled trial that involved 793 Australian residents who signed up for “Daybreak” and indicated harmful or hazardous drinking (i.e., a score greater than 7 on the AUDIT-10), 398 (50%) of whom were randomized to receive online coaching, and 395 (50%) to have access to “Daybreak” without the online coaching. Unfortunately, a programming error occurred, which allowed at least some (if not all) of the participants in the control group to access the online coaching (n=48 (12%) did so).

Study participants were reassessed on their degree of risky alcohol use one and three months after randomization. A six-month assessment was originally planned as well, but the study was ended early due to the programming error. Participants completed surveys online via a texted survey link and received up to three phone calls to remind them to complete the survey. Survey completion rates were 42% and 37%, respectively, for surveys one and three months after randomization. No monetary incentive was provided to study participants, though they were entered into a drawing to win an iPad2. The investigators identified factors related to survey completion. These were less alcohol use at baseline, higher quality of sleep, older age, not being single, and more engaged with the Daybreak program. The team controlled for them statistically in the analyses to help isolate the effects attributable to the Daybreak intervention alone.

The primary outcome of interest was the score on the first three items of the AUDIT-10, which focus on alcohol consumption. These three items form a subscale, called the AUDIT-C (the “C” stands for consumption), which distinguishes low-risk drinking (scores of 0-7), hazardous or harmful alcohol use (scores of 8-19), and probable alcohol dependence (scores of 20-40; “alcohol dependence” is equivalent to DSM-5 severe alcohol use disorder). Also of interest were overall level of alcohol consumption (in standard drinks based on Australian guidelines of 10g of ethanol for each standard drink, lower than the U.S. guideline of 14g ethanol per standard drink), mental distress, days spent out of role (i.e., the number of days either wholly or partially out of work roles due to alcohol consumption during the last 30 days), and quality of life. Additionally, the research team measured health services use, sleep quality, physical exercise, and adverse events due to alcohol (e.g., missed class, trouble with police).

Study participants were largely female (71%), on average 40 years of age, and many (69%) were classified as “probably dependent” on alcohol based on their AUDIT-C score.

The “Daybreak” program can be accessed via app or desktop, and consists of four components:

Weekly check-ins (i.e., self-reported questionnaires to encourage self-reflection).
Peer-support (i.e., using a blog function to connect with other users of the program).
Behavioral experiments (i.e., self-guided experiments and associated learnings on targeting mindfulness, connectedness, resilience, situational strategies, and health).
Online health coaching (i.e., live-chat via chat messaging with certified mental health workers to assist program users with goal setting and goal reaching).

The critical difference between the intervention and control groups was access to online coaching, which only intervention participants were meant to receive.

WHAT DID THIS STUDY FIND?

Utilization of “Daybreak” tools:

As planned, the treatment group utilized online coaching via live chat more commonly than the control group. Namely, 17% of participants in the treatment group ever used the live-chat option to engage in one-on-one coaching, compared to 10% in the control group. Other functions of the program were used more commonly, with 66% of participants ever making a blog post, 58% ever commenting on the blog post someone else made, and 35% ever using a self-experiment to engage with the presented materials and concepts more deeply. Note, however, that repeated use after the first time trying out a feature was limited (median uses of 0-2 times per person for these listed activities). The two groups did not differ in their use of these other tools.

Figure 1.

Effect on primary outcome:

Instead of the originally planned analyses that would have compared the two groups on outcomes, the investigators tested if the study participants in general changed over time. They found significant changes on their primary outcome of interest, where study participants reported lower AUDIT-C scores (i.e., less hazardous drinking) one month and three months after starting their engagement with the “Daybreak” program. On average, AUDIT-C scores decreased from 9 (within “hazardous or harmful alcohol use”) at baseline to 6 (within “low-risk drinking”) at months one and three. Recall, however, that surveys were more likely to be completed by participants who were less severe at baseline (i.e., used less alcohol), and who engaged more with the “Daybreak” program, suggesting these outcomes may be better than average if everyone was included.

Effect on secondary outcomes:

Beneficial changes were also found for all other outcomes of interest, except for exercise, with fewer days out of work role, less alcohol use, less distress, higher quality of life, and improved sleep quality.

Reductions in alcohol consumption were significant both for participants who indicated “hazardous or harmful” drinking at baseline, and those classified as “probably dependent” (i.e., 40.8 down to 20.1 drinks for “probably dependent” and 22.9 down to 11.9 drinks for hazardous or harmful”). The retention rates in these two groups were comparable.

Factors related to changes in alcohol consumption:

The investigators then tested which factors were associated with changes in alcohol-related variables. Use of online coaching was not among these factors. Demographic variables were (i.e., better outcomes for females, younger participants, partnered participants, higher education), as were blog posts, but only for one-month outcomes. Similarly, improvements in distress and quality of life were associated with demographic variables, but not with use of online coaching or other “Daybreak” engagement variables.

WHAT ARE THE IMPLICATIONS OF THE STUDY FINDINGS?

While flawed in its execution, this study indicates that adding online coaching did not improve drinking or quality of life related outcomes for participants of the “Daybreak” program. Instead, the Daybreak program either with or without the clinician support appeared to produce substantial positive changes in risky alcohol use.

This finding is in line with the results of a relatively recent meta-analysis of guided and unguided low-intensity internet interventions for problematic alcohol use, which concluded that effect sizes were similar for guided and unguided interventions for internet-delivered alcohol interventions. It is in contrast to a more recent meta-analysis of 19 randomized controlled trials of internet-based interventions for problem drinking that concluded that human-supported interventions were superior to fully automated ones on both outcome measures. Thus, while the answer is not clear cut, at least within the context of “Daybreak” it appears that offering the cost-intensive resource of real-time online coaching does not improve treatment outcomes.

It is also noteworthy that overall engagement in the program was low. These rates of engagement are similar to other online, publicly available programs to support persons recovering from problematic drinking. For example, a study examining the online social network site “InTheRooms” showed that only 14-16% of the surveyed InTheRooms users ever made posts to discussion boards. More InTheRooms users ‘consumed’ offered information (e.g., 41% read discussion board; 66-74% engaged in a daily meditation) than contributed information themselves (e.g., 23-36% made a status update). Thus, while not unexpected, overall low engagement rates with these digital health resources raise the question of what the optimal level of engagement should be and for whom. Clearly only content is offered in these digital programs that is believed to be beneficial, thereby suggesting that engaging in more content must be better. On the other hand, these resources are oftentimes used “as-needed,” and perhaps lower engagement reflects lowered needs. At present, the optimal dose – both in terms of frequency of use, and length of user over time – for digital health technologies is not well understood.

LIMITATIONS

Randomization did not work – unfortunately, the programming error that allowed participants in the control group to access the online coaching substantially weakens the strength of the contribution of this evidence to our understanding in this area. While the presented analysis still provides some insight into the underlying research question, these data are now confounded with self-selection (i.e., choosing to engage in online coaching rather than being given access vs. not).
Retention in the study was low and was less likely for participants with worse drinking profiles at the start of the study. While the investigators reported on and adjusted for missing data (i.e., participants choosing not to complete follow-up surveys), retention in this study was low (<43%), and lower than in comparable studies. It is possible that outcomes were less favorable in those who did not complete surveys; participants might also have chosen not to complete surveys because they were dissatisfied with the “Daybreak” program. Inclusion/exclusion of such data could fundamentally alter the conclusions of this study.

BOTTOM LINE

For individuals and families seeking recovery: This large quasi-experimental study on a publicly accessible Australian online platform to support recovery from problematic drinking found that many individuals sought out help using this platform, many tried out features of it, but adding live-chat coaching with certified mental health workers did not improve outcomes. Importantly, the “Daybreak” program is not the only digital support of its kind. Increasingly, online and other digital interventions are becoming available that can help support individuals in reducing heavy drinking. These interventions fill an important niche, as they can be interacted with from home and on-the-go, as they are most needed. It may be helpful to try out one or more of these types of resources, ideally those that have some evidence supporting their effectiveness.
For treatment professionals and treatment systems: This large quasi-experimental study on a publicly accessible Australian online platform to support recovery from problematic drinking found that many individuals sought out help using this platform, many tried out features of it, but adding live-chat coaching with certified mental health workers did not improve outcomes. Credible online interventions for problematic drinking have been developed and tested and are continuing to be developed. Accessing such interventions, however, can be more difficult, especially for patients who may be less familiar with the rigors and standards that are expected by different developers for these types of interventions. Providing patients links to vetted online interventions would help guide them to the resources most likely to work for them. It may also be useful to refer patients to ongoing studies of such interventions as, through these studies, patients may gain access to state-of-the-art treatments free of cost, while helping ensure that these treatments are being developed with their input. At present, little is known about the optimal ‘dose’ of engagement with these technologies. Thus, it might also be helpful to review use of these resources with patients.
For scientists: This large quasi-experimental study on a publicly accessible Australian online platform to support recovery from problematic drinking found that many individuals sought out help using this platform, many tried out features of it, but adding live-chat coaching with certified mental health workers did not improve outcomes. More studies are needed to help us understand the factors that impact the engagement with online drinking interventions, and that document more rigorously their impact on drinking outcomes. In fact, understanding factors impacting engagement with such programs is an important area of research – not just for substance use digital resources, but for mental health digital resources in general, as outlined by a recent call from the National Institute of Mental Health for more research in this area. Optimally leveraging digital support opportunities is an important area of growth given their potential reach in an affected population.
For policy makers: This large quasi-experimental study on a publicly accessible Australian online platform to support recovery from problematic drinking found that many individuals sought out help using this platform, many tried out features of it, but adding live-chat coaching with certified mental health workers did not improve outcomes. While online interventions for problematic drinking have been developed and evaluated in randomized controlled trials, very few of them are available to the public. Indeed, of the eight alcohol interventions with functioning websites recently identified in a review, only four were publicly available, and the only one offered in English no longer has a functioning link. Thus, while these resources are being developed, they fail to reach the populations they seek to support. Hosting and maintaining such online resources costs money – much less so than providing human-delivered interventions, but still requires money nevertheless. In order to capitalize on the potential these resources could provide, proper funding and support are necessary to ensure that they reach their intended target audiences. Also important is funding research that can inform best-practice guidelines for creating, hosting, and maintaining these digital resources.