Tuesday, 7 August 2007

How the misreporting of a student dissertation wrecked bereavement counselling's reputation

It has become the received wisdom in psychological circles that bereavement counselling is at best ineffective and at worst harmful, especially when offered to people experiencing 'normal' grief (see here for a recent example; p61).

Why the dire reputation? According to counselling psychologists Dale Larson and William Hoyt, it's thanks largely to inappropriate reporting of an unpublished student dissertation by Barry Fortner, in which it was claimed 38 per cent of bereaved clients would have fared better if, instead of receiving counselling, they had been in the no-treatment control group.

The trouble, Larson and Hoyt argue, is that Fortner's 1999 dissertation has only been cited once, by his colleague Robert Neimeyer in 2000. Since then, over 14 studies have reported the 38 per cent figure, but each time they have cited Neimeyer's published paper (a summary of past research), not Fortner, thus giving the misleading impression that the result came from a piece of quality, peer-reviewed empirical research.

Worse still, like a game of Chinese Whispers (or Telephone if you're American), recent papers discussing the 38 per cent figure have cited not only Neimeyer, but also subsequent papers citing Neimeyer, thus giving the impression that the 38 per cent figure has been corroborated by later investigations!

But now Larson and Hoyt have hit back. In a journal article and technical analysis (the latter freely available on-line), they claim Fortner's methodology that led to the 38 per cent figure is flawed. Moreover, they asked the APA Publisher Gary VandenBos to submit Fortner's dissertation to a post hoc peer review. And according to Larson and Hoyt, “The experts conclusively agreed that [Fortner's methodology] is seriously flawed and that there is no valid basis for the claim that 38 per cent of grief counselling clients suffered deterioration.”

Apart from Fortner's 38 per cent statistic, the reputation of bereavement counselling has also suffered from the reported outcomes of three key meta-analyses (where the outcome data from lots of studies is lumped together), one of which is in Fortner's dissertation.

For example, the most extensive of the meta-analyses, published by Allumbaugh and Hoyt in 1999, is often reported as having found poor efficacy for bereavement counselling. But according to Larson and Hoyt, the efficacy rates in the 35 assessed studies varied hugely, due mainly to differences in whether clients had referred themselves and how long they had been bereaved. If the analysis was confined to the recently bereaved, and to those who had chosen to receive counselling, then compared to no-treatment control, counselling showed the kind of benefits typically found for other types of psychological therapy for other conditions.

Larson and Hoyt acknowledge the need for more research and conclude: “...findings to date indicate that cautious optimism, rather than the recently fashionable dire pessimism, is the attitude most congruent with empirical findings on grief counselling outcomes.”

Larson, D.G. & Hoyt, W.T. (2007). What has become of grief counselling? An evaluation of the empirical foundations of the new pessimism. Professional Psychology: Research and Practice, 38, 347-355.

Post written by Christian Jarrett (@psych_writer) for the BPS Research Digest.

Image credit: Wellcome Library, London. A woman whose face expresses sadness. Etching in the crayon manner by W. Hebert, c. 1770, after C. Le Brun.


Anonymous said...

Generally, when one writes an article, you'd typically look at the orignial article. I've lost count of the times I've checked one persons citation and the original persons article is completely different. If you can't get hold of a dissertation because it is unpublished, surely it shouldn't be cited, especially as fact, and more importantly, when other research contradicts it!
Very sloppy work all round for those writers!

Chris said...

Interesting, certainly, that the citation practices in psychology are so shithouse, but let's not open the champagne for counselling just yet. With the amount of money being poured into the field, 35 ambiguous studies seems hardly enough evidence to justify our budgeting; and if it's able to be broken down into some justifiable and some unjustifiable client groups, the obvious issue is whether we're currently selecting the good ones or the bad ones. After all, much grief counselling now operates geographically - if someone dies at a school, all the other kids get counselling - which does not imply much contribution from self-referral. Really, the money shouldn't come out of the purse until the evidence clearly shows it works, not just when the evidence showing it doesn't work is shown to be dicey, and I'd question whether we were there yet.

Anonymous said...

Is this article deliberately ironic?
What reputation is being wrecked by who?
Who is perpetuating ridiculous ideas?

From reading the article and the links,
I read that 2 guys, Larson and Hoyt, are pushing a paper that one of them wrote 10 years ago and they're using that as the basis to attack the reputation of another couple of guys, both of whom are successful, and one of whom (Neimeyer) has written loads of books and articles, helped plenty of grieving people, taught a whole bunch of therapists how to help others, and who is still on the cutting-edge of research and practice in grief counselling. Are Larson and Hoyt just jealous?
Probably...and they have good reason to be. I don't see them packing much.

I clicked on the links provided and got a very different picture to the one in the article:

Neimeyer seems to be
-a well-respected therapist (respected by the APA enough to be invited to record training videos)
-he's the editor of 2 respected journals
-he's traveling the world speaking to therapists about how they can help people suffering from complicated grief (seems an optimist to me)
-he's at the cutting edge of research into bereavement counselling (written over 21 books, over 300 articles - a lot more than the puny 14 articles referred to as problematic in this post)

When I checked the link provided in the article it was clearly in contradiction with the view presented by Larson, Hoyt and the author.
Neimeyer clearly says that,
therapy has been found to be useful for people who are traumatically bereaved, and not so useful (and maybe even harmful) to those who are just suffering normal grief.
That seems perfectly reasonable and very helpful information!

I can't find the idea in here that bereavement counselling is not helpful...rather, I read here that Neimeyer is telling us who it's helpful to. That is the point of research isn't it? To find out who can be helped and how and tell others about it.

Has bereavement counselling's reputation really been wrecked?
I can't see that idea in the article or links coming from anyone other than Larson and Hoyt...and now perpetuated by this blog post.

(oops..Hope I haven't upset them...worried now they might make a career out of pursuing me for the next 10 years after I finish my dissertation. Yikes!)

I'd say, looking at everything provided in this article, that Larson and Hoyt probably ought to get over it, move on and accept that their article and argument is old, their view is warped, and maybe think about saying sorry to us all for wasting our time (I just spent the whole day looking into this -when I should have been studying- to find nothing of any value, except, I guess, that now I want to read a lot more of Neimeyer).

Maybe Hoyt and Larson could look at doing something more useful for the field of therapy than wasting readers' time trying to convince us all that some hard-working, scientist-practitioner-teacher is somehow less than optimistic about the field he obviously is passionate about, helpful in, and, from what I can read, optimistic about.

Neimeyer wins this smackdown, I'd say!

Post a Comment

Note: only a member of this blog may post a comment.