Talk:Effectiveness of Alcoholics Anonymous

Issues about "The Sober Truth"

I have concerns about the treatment of the book "The Sober Truth" in this article and I'd like to make two suggestions for how to fix it.

1) As a psychologist with a specialty in addictions, I have found "The Sober Truth" to be a serious treatment of much of the outcome literature on AA. I'm not sure if previous editors have read the book itself, but it spells out in great detail the experimental biases and scientific errors in some of the prior outcome literature that this Wikipedia article specifically references positively, and as such is able to effectively debunk some of those studies.

As a result, I'd recommend adding information about each of these studies explaining exactly how they have been shown to be debunked. (I'm happy to provide that information if necessary.)

2) I'm aware that "The Sober Truth" is controversial among AA adherents, but it is far from the only major publication to argue against inflated statistical claims of AA's success. As such, I'm perplexed why it has been singled out for attack in its own section. That is has been set apart in this fashion suggests a bias against it.

As a result, I'd recommend removing this special section. MLNYPhD (talk) 23:07, 17 February 2020 (UTC) — Preceding unsigned comment added by MLNYPhD (talk • contribs) 23:02, 17 February 2020 (UTC)

@MLNYPhD: The section as it stands now looks like a mix of reviews, some are positive and some are critical. I'd recommend we make an article just on the book The Sober Truth is it seems like there's more to say about it and it's out of scope for this article but passes WP:NBOOK. As far as using it as a source in this article, I believe it passed WP:RS but will also want to give it WP:DUE weight. - Scarpy (talk) 00:16, 18 February 2020 (UTC)

I thought The Sober Truth would redlink, but looks like it redirects here. - Scarpy (talk) 01:11, 18 February 2020 (UTC)

The reason why we have a section on The Sober Truth is because it has had multiple mentions in reliable third party sources.

In terms of putting Dodes’s criticisms about individual studies in the sections about the studies, I don’t think that’s a good idea as per WP:MEDPOP. If you can find a criticism of, say, Moos and Moos 2006 from a peer-reviewed journal, include it, but The Sober Truth is not peer-reviewed and, quite bluntly, I have read his criticism of Moos and Moos 2006 and it’s not very well argued; I don’t think Dodes’s complaints would pass the muster of peer review. Peer-reviewed papers which cite Moos and Moos 2006 agree with Moos and Moos 2006’s findings; as does a book published last year; if that Google Books link goes bad, try this: Miller, William. Treating Addiction, Second Edition: A Guide for Professionals. very frequent AA attendance predicts higher rates of abstinence [...] 5 years or longer after treatment (Gossop, Stewart, & Marsden 2007; Moos & Moos 2006; Pagano et. al. 2013)

Since “inflated numbers” have been mentioned here, the numbers from scientific studies show that 67-75% of people who reguarly go to AA meetings get and stay sober. That’s not inflated numbers; it’s scientific fact, replicated multiple times in multiple studies.

I am also worried that this editor, with all of two edits, is the same editor who got in an edit war a year ago over whether this article was neutral, they have the same writing style. SkylabField (talk) 09:20, 2 March 2020 (UTC)

I have added a footnote in the Moos and Moos 2006 section noting one of Dodes’s objections to this study’s results (that, for the 72% success rate numbers, we’re only looking at the 17% of people who 1. Did enough follow-up interviews to be counted and 2. Were heavily involved in AA in their second or third year of alcoholism treatment). I have not seen any coverage in reliable third party sources about Dodes’s objection to this survey (or any of the other surveys), so putting it in the main article text violates WP:UNDUE since those objections, as far as I know, have not been published in any peer reviewed journals, have not been mentioned much less supported by any other expert in the treatment industry, and, indeed, have not been mentioned at all in any third party reliable source.

I should also mention that Lance Dodes’s book inaccurately looks at the numbers in Moos and Moos 2006. Dodes claims that “the total number of people who remained through the sixteen-year follow-up and also stayed in AA for longer than six months -- that is, the group on which the authors’ major findings are based -- was just 107, or 17 percent of the original sample”. Dodes is flat out wrong here: 115 people stayed in AA for longer than six months during their first year of treatment. 107 people had six or more months of AA treatment in their second and third year of treatment. 129 people had 26 weeks or more of AA participation at the 4-8 year marks; and 115 people (with 26 weeks or more of first-year AA participation) were available to give information at the 16 year mark. Point being, Dodes did not accurately count the number of subjects who showed a high success rate for AA in Moos and Moos 2006; it appears he just chose the lowest number to make Moos and Moos look as bad as possible. SkylabField (talk) 18:05, 5 March 2020 (UTC)

The Sober Truth is published by Beacon Press not a medical or scientific publisher, and since this book has not been peer reviewed, the book is not a medically reliable source, and can not be used to make biomedical claims about AA's efficacy. See Talk:Effectiveness_of_Alcoholics_Anonymous#RfC:_Is_AA_Efficacy_a_medical_claim_which_should_only_be_backed_by_WP:MEDRS? for discussion. SkylabField (talk) 21:53, 16 September 2020 (UTC)

The 2020 Cochrane Review is out

The 2020 Cochrane review is out. Considering how important Cochrane reviews are in the medical field, I have made the following changes:

• I have added a short summary of the new Cochrane study.
• I am retaining the summary from the older study, noting that the new Cochrane review uses a lot more studies (as per the NYT article on the study)
• I have removed all of the now-outdated discussion about the 2006 Cochrane review, because of WP:MEDDATE

I have a copy of the review and I hope @NatGertler: and @Scarpy: have access to the review so that we can, together, summarize and discuss the new Cochrane review in more depth. I know Nat has access to the NYT summary, but I also am quoting the actual review in a manner which is hopefully neutral. SkylabField (talk) 04:01, 13 March 2020 (UTC)

For people without access to the full journal, here is a free summary; my quotes come only from the summary, so that other editors can better determine if my edits are neutral. My sense reading the full review is that a lot of studies saying "everything looks just as effective", and the only thing they see AA having an edge with a high level of confidence is the number of participants completely abstinent from alcohol (see page 4 for the summary, page 28 for more details, and pages 88-89 for the raw numbers used) The page 4 summary says, that at the 12-month follow up period, 41.8% of people undergoing AA/TSF are abstinent, compared to 34.5% receiving alternate non-AA treatment. The New York Times summary says it’s 15-25% of people who remain abstinent for non-AA treatment, compared to 22-37% for people undergoing AA treatment; I’m not sure where they are getting those numbers from in the actual Cochrane Study. SkylabField (talk) 06:59, 13 March 2020 (UTC)

I think that the part you're taking from the key conclusions of the summary, "clinically-delivered TSF interventions designed to increase AA participation usually lead to better outcomes over the subsequent months to years in terms of producing higher rates of continuous abstinence," isn't really addressing the central subject here. It's not saying AA is better or worse than other methods; it's discussing ""clinically-delivered TSF interventions" as a way of increasing how much AA someone does, and that more AA is better than less AA (the next sentence: "This effect is achieved largely by fostering increased AA participation beyond the end of the TSF intervention".) "Manualized AA/TSF interventions usually produced higher rates of continuous abstinence than the other established treatments investigated. Non-manualized AA/TSF performed as well as other established treatments." seems to be more key.... but it's meaning confounded for me by not knowing what "manualized" means in this context. --Nat Gertler (talk) 15:04, 13 March 2020 (UTC)

Page three of the open access version explains “manualized” briefly near the top of the page: “whether treatments were delivered according to standardized procedures (i.e. manualized)”.

Here is, for everyone to review, the entire “Key results” section on page three of the open access version:

Key findings in the 2020 Cochrane Review of Alcoholics Anonymous efficacy

Manualized AA/TSF interventions usually produced higher rates of continuous abstinence than the other established treatments investigated. Non-manualized AA/TSF performed as well as other established treatments. AA/TSF may be superior to other treatments for increasing the percentage of days of abstinence, particularly in the longer-term. AA/TSF probably performs as well as other treatments for reducing the intensity of drinking (of alcohol). AA/TSF probably performs as well as other treatments for alcohol-related consequences and addiction severity. Four of the five economics studies found substantial cost-saving benefits for AA/TSF, which indicate that AA/TSF interventions probably reduce healthcare costs substantially. In conclusion, clinically-delivered TSF interventions designed to increase AA participation usually lead to better outcomes over the subsequent months to years in terms of producing higher rates of continuous abstinence. This effect is achieved largely by fostering increased AA participation beyond the end of the TSF intervention. AA/TSF will probably produce substantial healthcare cost savings while simultaneously improving alcohol abstinence.

The reason why I feel “clinically-delivered TSF interventions designed to increase AA participation usually lead to better outcomes” is a reasonable summary is because those are the words which come right after “In conclusion” in the actual report. Also, “Manualized AA/TSF interventions usually produced higher rates of continuous abstinence than the other established treatments investigated.” says pretty much the same thing; I would word that sentence like this: “If we use a standardized treatment which gets someone to go to AA more often, alcoholics will be more likely to stay sober than using another treatment not geared to get an alcoholic in to the rooms of AA.”

Note also that “TSF” here is Twelve-step facilitation, therapy where the therapist works with the patient to get them to reguarly attend AA meetings. SkylabField (talk) 21:32, 13 March 2020 (UTC)

The question isn't whether it's a reasonable summary of their findings, it's whether it's the important information for this article, which is more generally on AA's effectiveness, rather than on which is the method that will make AA most effective. A summary is meant to address their study's needs, not ours. --Nat Gertler (talk) 22:57, 13 March 2020 (UTC)

The 2020 Cochrane review on Alcoholics Anonymous has the same goal this article has: To answer the question “Does Alcoholics Anonymous Work”. From the abstract (page 1 of the open access version of the review), we have this quote:

Objectives To evaluate whether peer-led AA and professionally-delivered treatments that facilitate AA involvement (Twelve-Step Facilitation (TSF) interventions) achieve important outcomes, specifically: abstinence, reduced drinking intensity, reduced alcohol-related consequences, alcohol addiction severity, and healthcare cost offsets.

When trying to come up with a properly randomized trial with a randomized condition, it’s a lot easier to compare therapy A which encourages patients to go to more AA meetings to therapy B which doesn’t, which is why a lot of studies compare TSF treatments to non-TSF treatments. There just isn’t, short of engaging in unethical behavior, any way to randomize people to go to or not go to AA meetings. From the New York Times article: “Even randomized trials can succumb to bias from self-selection if people assigned to A.A. don’t attend, and if people assigned to the control group do. (It may go without saying, but we’ll say it: It would be unethical to prevent people in a control group from attending Alcoholics Anonymous if they wanted to.)” SkylabField (talk) 01:24, 14 March 2020 (UTC)

I have combined Nat’s summary of the review and my own summary. In addition, I define what “TSF” and “manualized” are. SkylabField (talk) 17:21, 14 March 2020 (UTC)

The 2020 Cochrane review specifically defines AA success

@Scarpy: We have had a lot of discussion over the years about what constitutes “success” when looking at AA efficacy. Since the new 2020 Cochrane review of AA has a specific definition for success, “abstinence, reduced drinking intensity, reduced alcohol-related consequences, alcohol addiction severity, and healthcare cost offsets”, I have added their definition at the beginning of the “overview” section. SkylabField (talk) 01:36, 14 March 2020 (UTC)

Dr. Peele’s response to the 2020 Cochrane Review

I have observed that an editor has added Dr. Stanton Peele’s response to the 2020 Cochrane Report. The correct thing to do, as per Wikipedia policy, would be to completely remove Dr. Peele’s statements, since they have not been reliably published anywhere; the “Filter Mag” website where his article was published is not a notable or a reliable publication. It is particularly telling that Peele, who has previously been published in Reason (a reliable source), was unable to get his response to Cochrane 2020 published in a notable source.

I will not do that, since I feel that since Stanton Peele is notable enough to have a Wikipedia article, his comments on matters are notable enough to be in articles across the Wikipedia.

That said, putting his comments about Cochrane 2020 in a separate paragraph, when we put the reliably published statements of other doctors together in a single paragraph, violates WP:UNDUE, so I will move it. SkylabField (talk) 21:56, 1 September 2020 (UTC)

I also added a "better source" tag. Stanton Peele himself is a "reliable source." But I agree that ideally it should also be published in a peer-reviewed journal. I'm also not clear on exactly what his criticism is. - Scarpy (talk) 23:19, 2 September 2020 (UTC)

I have read the article, and it’s really really bad. First, Peele claims there were not any “reliable” studies showing AA effectiveness in the mid-2000s (we had Moos and Moos 2006 and Fiorentine 1999 which showed a strong correlation between abstinence and regular AA attendance, not to mention Project MATCH which should AA was just as effective as psychotherapy). He then claims it was a “self selecting” population, but all of the studies used randomization of subjects. He then continues to build up a straw man about Cochrane 2020: He claims they only used two studies to show manualized AA-based TSF treatment has better abstinent outcomes, but pages 88 and 89 of the actual Cochrane review show six different studies, all showing better results for therapies which get people in the rooms of AA. Then he claims the Cochrane review shows higher drinking intensity, but page 92 shows that it’s only a single study (McCrady 1996) which saw a higher percentage of days heavy drinking, while two other studies show either the same number of days of heavy drinking, or a slight improvement for AA-based treatments. Peele then links to studies which did not pass the muster to be part of Cochrane 2020 (I can do that too, Mr. Peele: Moos and Moos 2006, which shows a 67% 16-year success rate for Alcoholics Anonymous). He then complains that the studies only pick good subjects, which isn’t true: The studies pick subjects who only have alcoholism because, you know, AA is not a program for treating mental illness; it’s a program for treating alcoholism. He then goes on about how he was blacklisted for supporting the long discredited Sobell 1973 study, never mind that we knew back then and know now that most alcoholics who try to moderately drink fail miserably.

Point being, there’s a reason why this screed from Peele has not been reliably published. It’s garbage, he clearly didn’t actually fully read and understand the Cochrane 2020 study. Something like this would never pass the muster of peer review, or even be published in something reasonable like Psychology Today or Reason.

I am hoping @Apeximius: or @NatGertler: can provide their opinion, but I would be OK with removing the Peele screed altogether from the article space. SkylabField (talk) 00:18, 3 September 2020 (UTC)

I think that Filter does not meet our applicable WP:MEDRS guidelines. I think that's also true for a number of other sources used in the article, including the New York Times -- a fine source for non-medical items, but this is a medical article. --Nat Gertler (talk) 13:36, 6 September 2020 (UTC)

The reason we have non-medical sources here discussing AA’s effectiveness is because of consensus from four years ago: Talk:Alcoholics_Anonymous/Archive_9#Some_recent_edits and Talk:Effectiveness_of_Alcoholics_Anonymous/Archive_1#Dodes_quote. Since that consensus came out, over at WP:RS/P#The_New_York_Times, there are now concerns about how well The New York Times can make medical claims. Interestingly, if we only quote WP:MEDRS sources, we will have to make this article a lot more negative towards Lance Dodes’s claims that AA has a 5% success rate: Anything with a DOI I have found, when discussing Dodes’s figures, is quite skeptical: DOI:10.1080/07347324.2016.1217713 and DOI:10.1177/0003065114565235. I will remove the filtermag reference and clean up statements based solely on claims made in The New York Times SkylabField (talk) 17:38, 6 September 2020 (UTC)

I'm not seeing sufficient consensus in those discussions to override much of anything. --Nat Gertler (talk) 17:51, 6 September 2020 (UTC)

In terms of removing anything not a high-bar WP:MEDRS, I think we will need to hammer out the consensus on it, as per WP:CCC; we had at least two editors saying not every source on AA’s efficacy has to be a high-bar MEDRS back in 2016, using the argument one does not need to be a MD to count how many people continue to go to meetings and get sober (and trying to add Lance Dodes’s figures without taking in to account the number of non-MEDRS articles critical of Dodes’s figures for AA effectiveness). This article will have to be rewritten if we remove any claim which can not be backed up by a DOI (even if we go that far, it makes a lot of sense to also use press articles which summarize the DOI article, since DOI articles are often times behind a paywall). In the mean time, I have removed the dubious claim only published by that “FilterMag” website, and have removed a claim which is only backed up by The New York Times. If we go this route, let’s not forget WP:MEDDATE — most medically reliable peer-reviewed articles posted since 2000 show either that 1) AA is as effective as other treatments across the board (Abstinence, Drinking Consequences, Percentage days of heavy drinking, etc.) or 2) AA is more effective than other treatments getting alcoholics abstinent. SkylabField (talk) 18:09, 6 September 2020 (UTC)

Thinking it through, my primary argument against Filter Mag is that it’s an openly biased source advocating harm-reduction. My argument against strictly applying WP:MEDRS here is that this is different than discussing the effectiveness of a drug or some other kind of treatment. There’s more philosophical, social, and spiritual dimensions that are relevant here. Meaning it’s possible to have a philosophical or spiritual objection to, or reasoned endorsement of, AA in a way that’s different from a philosophical or spiritual objection to statins. It also feels a bit like “medicalization” to ignore the other dimensions. I suppose I’d be fine with a content fork where we MEDRS this article and move other notable but not strictly medical material to say History of Alcoholics Anonymous create something like Commentary on Alcoholics Anonymous that would be for notable moral, theoretical and constructive commentary/criticism following then definitions in Varieties of criticism. - Scarpy (talk) 16:38, 8 September 2020 (UTC)

RfC: Is AA Efficacy a medical claim which should only be backed by WP:MEDRS?

RFC: Should we consider any and all claims about the effectiveness of Alcoholics Anonymous be considered biomedical information which needs to only use sources considered medically reliable? Here is a ping for users who have discussed this topic over the years: @Apeximius, NatGertler, Scarpy, WhatamIdoing, Ozzie10aaaa, and Desoto10: SkylabField (talk) 14:01, 7 September 2020 (UTC)

• Yes Claims about the Effectiveness of Alcoholics Anonymous are medical claims, so this article should only use articles from the popular press when they echo medical claims (e.g. using articles from The New York Times, Vox, and other generally reliable sources) and not use anything which is not a medically reliable source to make any claims about AA’s efficacy. SkylabField (talk) 14:10, 7 September 2020 (UTC)

  How much of an article could we realistically write if we required that only MEDRS's "ideal" sources be used? That means that you could use sources such as the Cochrane review, this claim that half the people who get sober did so through AA and similar programs (but if only a few people ever get sober, that's still not many...), this review that says we don't know if it works for Native Americans or other Indigenous people, or this review that says AA plus medical treatment is more effective at producing abstinence than medical treatment alone for dual diagnosis patients (but that same problem: What's the effect size? What's the Number needed to treat to produce a year of continuous alcohol abstinence with AA?). You could also use recent textbooks, especially ones intended for use in medical school, and position statements from the key professional bodies. (National Institute for Health and Care Excellence, for example, recommends that primary-care providers encourage patients to attend self-help groups; if they published a statement that said something about the efficacy, then you could cite them.) WhatamIdoing (talk) 22:50, 7 September 2020 (UTC)

• Yes if we clearly define through consensus what we mean by efficacy see also my above comments in Talk:Effectiveness of Alcoholics Anonymous#Dr. Peele’s response to the 2020 Cochrane Review. There are relevant ways to comment on the usefulness of AA that are notable and have encyclopedic value that would need to be moved elsewhere depending on the consensus definition on efficacy. But I’m okay with that so long as we’re all on the same page. - Scarpy (talk) 16:49, 8 September 2020 (UTC)

  How about this for efficacy: “abstinence, reduced drinking intensity, reduced alcohol-related consequences, alcohol addiction severity, and healthcare cost offsets”. That’s the wording the Cochrane 2020 report on AA uses, so it’s reliably sourced, and it seems reasonable to me. SkylabField (talk) 17:42, 8 September 2020 (UTC)

I'm okay with that. I would say in the lede something to the effect of "this article relates to efficacy of AA as measured by one or more of five metrics (abstinence, reduced drinking intensity, reduced alcohol-related consequences, alcohol addiction severity, and healthcare cost offsets)." And somewhere provide short definitions of those were it would not be obvious to the read (e.g. drinking intensity, severity). Anything that's notable and encyclopedic that we isn't related to one or more of those we can try to find a new home for. - Scarpy (talk) 18:08, 8 September 2020 (UTC)

Those five metrics could be the main sections of the article. That is, write ==Abstinence==, and then summarize the recent papers and high-quality independent secondary sources. I believe a summary for that section would say something like 'AA fails less at producing long-term total abstinence than most other conventional medical treatments, but most people who try AA (or anything else) don't stop drinking'. Then go on to write a ==Reduced drinking intensity== section, etc. The current organization focuses more on individual studies than about the subject of the article. WhatamIdoing (talk) 22:36, 8 September 2020 (UTC)

Since the best source for making an article organized like that is the currently paywalled Cochrane 2020 report on Alcoholics Anonymous, I don’t feel comfortable making that change until the study becomes open access in 2021, so that more editors have access to the source material and can verify that any changes I make are neutral. SkylabField (talk) 15:34, 16 September 2020 (UTC)

I believe one our mutual friend has some tips that solve this problem. - Scarpy (talk) 19:50, 16 September 2020 (UTC)

We may also want to add a FAQ on the talk page (the most recent example of one I've seen is here) just to have a clear summary of some of the decisions we've gotten consensus on throughout the years. - Scarpy (talk) 18:11, 8 September 2020 (UTC)

@Scarpy: I have added a FAQ. SkylabField (talk) 17:38, 16 September 2020 (UTC)

• Yes, but we should understand that there may be reasons to have non-WP:MEDRS sources making non-biomedical claims. I could see, say, in the introduction having something like "deisagreement and concerns about the efficacy of AA have had an impact on public policy", which would speak to the import of the topic without making a statement about the efficacy; something like that could be sourced to the NY Times. --Nat Gertler (talk) 18:36, 8 September 2020 (UTC)

There is rough consensus that claims about AA’s efficacy are biomedical claims, so we shouldn’t be making claims about efficacy which can not be backed up by a medically reliable sources. This in mind, I have removed claims from the article which comes only from non-medical sources. Because of WP:MEDDATE, I have removed summaries of older studies from the last century from the article. SkylabField (talk) 15:34, 16 September 2020 (UTC)

"42% success rate after one year" is an unhinged claim

Just.. no way. I've been to rehab and that's some bullshit. That number has to be extremely cherry picked, like "among people who remain compliant with the program for 11 months" or something (if not outright fabricated).

Tomato7331 (talk) 04:52, 7 September 2021 (UTC)

The claim is referenced using medically reliable medical sources (no footnotes in lead, as per Wikipedia style, but the claim is footnoted in sections below the lead). See the following:

• Kelly, John F.; Humphreys, Keith; Ferri, Marica (2020). "Alcoholics Anonymous and other 12-step programs for alcohol use disorder". Cochrane Database of Systematic Reviews. 3: CD012880. doi:10.1002/14651858.CD012880.pub2. PMC 7065341. PMID 32159228.
• For a lay summary: Becker, Deborah. "New Review Finds Alcoholics Anonymous Is Effective, But Not For Everyone". NPR.

If reading the actual paper, look for “Alcoholics Anonymous (AA)/Twelve‐Step Facilitation (TSF) (manualized) compared to other clinical interventions for alcohol use disorder (1A)”, then look for “Proportion of participants (%) completely abstinent Follow‐up: 12 months”, and see this: “345 per 1000” (non-AA based treatments) then “418 per 1000” (AA based treatments). NPR agrees with the numbers. SkylabField (talk) 22:07, 9 September 2021 (UTC)

The reason we’re looking at the 12-month follow-up is because that finding has a very low P-value, because N (the number of people in the studies) is high enough to make the standard deviation small enough that we can say with a high degree of competence that AA-based treatments are helping alcoholics, and that it’s not just random chance that people who went to AA-based treatments did better. Or, as the study itself words it using fairly opaque language, and I will wikilink this to help make it easier to read: “RCTs comparing manualized AA/TSF to other clinical interventions (e.g. CBT), showed AA/TSF improves rates of continuous abstinence at 12 months (risk ratio (RR) 1.21, 95% confidence interval (CI) 1.03 to 1.42; 2 studies, 1936 participants; high‐certainty evidence). This effect remained consistent at both 24 and 36 months.” (As per the article: a "manualized" treatment is one where a standard procedure was used; A TSF treatment is a "twelve-step facilitation" treatment: A treatment which encourages a patient to attend Alcoholics Anonymous.) We’re seeing similar rates of abstinence with other studies at other periods of time after treatment, but it was the 1990s Project MATCH study which gave us enough subjects to decrease the p-value enough to give out this finding with high confidence; that MATCH study only looked at results once a year. SkylabField (talk) 03:52, 10 September 2021 (UTC)