Thursday, 14 February 2013

Working memory training does not live up to the hype

According to CogMed, one of the larger providers of computerised working memory training, the benefits of such training is "comprehensive" and includes "being able to stay focused, resist distractions, plan activities, complete tasks, and follow and contribute to complex discussions." Similar claims are made by other providers such as Jungle Memory and Cognifit, which is endorsed by neuroscientist Susan Greenfield.

Working memory describes our ability to hold relevant information in mind for use in mental tasks, while ignoring irrelevant information. If it were possible to improve our working memory capacity and discipline through training, it makes sense that this would have widespread benefits. But that's a big if.

A new meta-analysis by Monica Melby-Lervåg and Charles Hulme has just been published in the February issue of the respected APA journal Developmental Psychology, which combined the results from 23 studies of working memory training completed up to 2011 (PDF is freely available). To be included, studies had to compare outcomes for a working memory training treatment group against outcomes in a control group. Most of the studies available are on healthy adults or children, with just a few involving children with developmental conditions such as ADHD.

The results were absolutely clear. Working memory training leads to short-term gains on working memory performance on tests that are the same as, or similar to, those used in the training. "However," Melby-Lervåg and Hulme write, "there is no evidence that working memory training produces generalisable gains to the other skills that have been investigated (verbal ability, word decoding, arithmetic), even when assessments take place immediately after training."

There was a modest, short-term benefit of the training on non-verbal intelligence but this disappeared when only considering the studies with a robust design (i.e. those that randomised participants across conditions and which enrolled control participants in some kind of activity). Similarly, there was a modest benefit of the training on a test of attentional control, but this disappeared at follow-up.

All of this suggests that working memory training isn't increasing people's working memory capacity in such a way that they benefit whenever they engage in any kind of task that leans on working memory. Rather, people who complete the training simply seem to have improved at the specific kinds of exercises used in the training, or possibly even just at computer tasks - effects which, anyway, wear off over time.

Overall, Melby-Lervåg and Hulme note that the studies that have looked at the benefits of working memory training have been poor in design. In particular, they tend not to bother enrolling the control group in any kind of intervention, which means any observed benefits of the working memory training could be related simply to the fun and expectations of being in a training programme, never mind the specifics of what that entails. Related to that, some dubious studies reported far-reaching benefits of the working memory training, without finding any improvements in working memory, thus supporting the notion that these benefits had to do with participant expectations and motivation.

A problem with all meta-analyses, this one included, is that they tend to rely on published studies, which means any unpublished results stuck in a filing cabinet get neglected. But of course, it's usually negative results that get left in the drawer, so if anything, the current meta-analysis presents an overly rosy view of the benefits of working memory training.

Melby-Lervåg and Hulme's ultimate conclusion was stark: "there is no evidence that these programmes are suitable as methods of treatment for children with developmental cognitive disorders or as ways of effecting general improvements in adults' or children's cognitive skills or scholastic achievements."

_________________________________ ResearchBlogging.org

Melby-Lervåg M, and Hulme C (2013). Is working memory training effective? A meta-analytic review. Developmental psychology, 49 (2), 270-91 PMID: 22612437 Free, full PDF of the study.

--NB.--
This meta-analysis only took in reviews published up to 2011. If you know of any quality studies into the effects of working memory training published since that time, please do share the relevant links via comments. 

--Further reading--
Brain training games don't work.
Brain training for babies actually works (short term, at least)

Post written by Christian Jarrett (@psych_writer) for the BPS Research Digest.

22 comments:

  1. What about this study Christian?

    Training Can Improve Memory and Increase Brain Activity in Mild Cognitive Impairment
    Mar. 1, 2012

    http://www.sciencedaily.com/releases/2012/03/120301180950.htm

    ReplyDelete
    Replies
    1. hi, that's v interesting, thanks for sharing. It's a bit different from the claims around working memory training since CogMed and others are claiming that working memory training has all round benefits, beyond straight-forward benefits for memory.

      Delete
    2. Anonymous2:26 pm

      Does this change in brain activity translate to relevant changes in behavior? If not, then training remains worthless.

      Delete
  2. Excellent article which we agree with - we recently had a patient who had spent 3 months improving their score on game based test which was a computerised version of a proven neuro-psych battery. When we tested them on paper they went from top decile to bottom decile on the same test. So they don't even transfer between media, let alone generalise across different tasks

    ReplyDelete
  3. This might be useful. Recently published:

    http://www.sciencedirect.com/science/article/pii/S0306452213000201

    ReplyDelete
    Replies
    1. hi Sam, thanks, that's an interesting study with a promising finding. From a quick look, it doesn't seem it was fully blinded though. So possible experimenter effects could be at play.

      Delete
    2. Anonymous8:24 pm

      Another one:

      research from T. Alloway, Bibile and Lau published in may 2013.
      See article: Computerized working memory training: can it lead to gains in cognitive skils in students?

      tracyalloway.com/journal-articles/

      Delete
  4. Anonymous4:19 pm

    May studies prove the contrary and the effect of brain training on memory and other number of cognitive skills have been well established in the scientific literature. What is important is to distinguish the programs which have a real scientific background and peer-reviewed publications to the one which are merely brain games.

    The question regarding the transfer to real-life activities also depends how you measure the abilities in question and how you can track the progress in adults which already have developed mental health conditions which continue to worsen. More studies are needed but keeping your brain constantly challenged is a good idea.

    ReplyDelete
  5. Anonymous11:58 pm

    Does any of this apply to the claims made by the Arrowsmith Program, where learning disabilities are claimed to be cured by using specific brain exercises?

    If so, what is the difference between these type of exercises and the ones used in the Arrowsmith Program?

    ReplyDelete
  6. I just found this RCT from last year "No Evidence of Intelligence Improvement After Working Memory Training: A Randomized, Placebo-Controlled Study." http://psycnet.apa.org/psycinfo/2012-16236-001/

    ReplyDelete
  7. Anonymous9:31 pm

    * Relating to Cogmed (as the paper focusses on this programme)- I think people are missing the point here -

    Cogmed are not claiming all round benefits! - see

    http://www.cogmed.com/faq

    Cogmed markets (or indeed it really should do and let me know if you have seen otherwise) itself as a working memory and attention intervention, - and this what this meta analysis and associated peer reviewed papers also show.

    It does appear that there is limited transfer from working memory training to other skills - but please bear in mind that these papers all look at skills transfer over a very short time period - we can't rule out the possible cumulative effect of increased working memory function over say a 5 year period.

    Another thing to bear in mind is that many of these studies did not use what Cogmed call 'extension training', so yes it is likely that without that these additional booster exercises, training did not transfer to other skills - i.e. like many skills without regular
    practise you may not see the benefits.

    Indeed up until recently, researchers thought that working memory capacity was more or less fixed, whereas now there is a great wealth of research to show that working memory can be improved with a relatively short intervention - in my book this is pretty remarkable!

    People should also release that the research as quoted in this meta-analysis, is that of a very high quality (and indeed made the grade into the Meta-analysis in the first place and so I really don't think people should in any way critise the research itself (which incidently is always independently run and funded from Cogmed itself).

    If indeed some staff from Cogmed are using this research in the wrong way in their marketing or other promotional material - yes this should be addressed.

    However, the evidence speaks for itself, Cogmed improves working memory skills, both on the programme itself and on objective measure of working memory ability. Whether or not this transfers to other skills - at the moment the evidence would suggest not - but we absolutely should not rule this out over a longer term period. All this shows to me is that the research dissemination should be very carefully managed.

    *I am a researcher who has evaluated Cogmed and seen first hand how it has benefitted children with working memory impairment.

    ReplyDelete
  8. Reply from Cogmed regarding the article:
    http://www.cogmed.com/commentary-working-memory-training-effective-metaanalytic-review

    ReplyDelete
  9. My understanding is that improved Working Memory leads to improved Executive Function.

    I've never heard anyone argue that improved Working Memory leads to increased “verbal ability, word decoding, arithmetic”.

    This is kind of like arguing that Aspirin is useless because it has not anti-bacterial properties.

    ReplyDelete
    Replies
    1. http://www.readspeak.co.nz2:05 am

      Yes . I remediate working memory disabilities and have an assessment The Prosodic Assessment One of the first signals of progress is usually through the more observant members noticing improved communication..I won't say speech because working memory is needed for remembering the language and expressions shared. Not being able to do this on a day to day basis is shattering. Additionally reading comprehension is dependent upon working memory ..Dyslexia is red herring really , some clever ruse by the former Department of Retardation, I suppose. Because dyslexia is not about mixing-up the words and the letters.. It's about working memory ...

      Delete
  10. Anonymous6:17 pm

    I have been using Cogmed with students for nearly 2 years now, and have found that when tailored interventions are employed alongside and after the training to help children catch up with concepts they found difficult before training, good progress is made with reading, spelling and maths. As Cogmed doesn't teach any content, but strengthens processes, it would be unreasonable to think that improvements to working memory alone would suddenly result in better test scores. As a teacher I have seen much better retention and use of taught strategies following Cogmed, which performance on standardised tests is not always able to capture. I agree that children need time to learn to use improved working memory capabilities and perhaps it might be useful to wait and see how children progress over a longer period before rushing to make a judgement about the training.

    ReplyDelete
  11. Anonymous4:05 am

    Can anyone recommnd a professional to evaluate working memory related learning disability?

    ReplyDelete
  12. Anonymous8:27 pm

    Okay, now I'm really confused. As a parent of a child w/ low working memory (13%) and Mixed Receptive-Expressive Language Disorder - would it hurt (or be a complete waste of $42) to have him try the Jungle Memory Program over the summer? If you say don't use it, please let me know if you have any other suggestions. Thanks!

    ReplyDelete
  13. I'm the mother of a six-year-old who has just finished first grade. He has some learning dissabilities which I would like to work on over the summer. He's working with a speech therapist on a weekly basis, but I would like to do something that improves his reasoning skills so that he can understand what he reads, what he is told as well as math concepts. What do you think would be a good program to help him over the summer? All suggestions will be appreciated!

    ReplyDelete
  14. Anonymous2:59 pm

    If he has working memory difficulties then whatever you teach will need to be repeated and rehearsed until stored in long term memory. To improve reading comprehension etc an intensive program that explicitly teaches reading and writing skills which incorporates a multisensory approach is the only way to go as this has evidence to support its effectiveness.

    ReplyDelete
  15. Anonymous6:24 pm

    I used Cogmed for my high school sophmore son with very good results. He found his ability to concentrate and pay attention in class much easier. He studied and did projects on his own. He graduated HS and is now a freshman in college. He struggled his younger school years. I also did other interventions with him and avoided meds. My 7th grade daughter is also struggling in school. I am going to have her start Jungle Memory. My view is that it can't hurt. It may help her a little or it may help a lot. Every person is different. With additional support protocols, I view these types of programs as one piece in the overall puzzle to help.

    ReplyDelete
  16. I struggle with everyday life. I'm a caregiver for a living. I do get things done on time but it's such a struggle for my brain. Simple care-giving mentally exhausts me. There are many other things I want to get done (start a business, get a girlfriend, presentation on organic food at community library) but they are so mentally exhausting. I feel ashamed for not protecting the environment and doing other things because of my mental exhaustion. I go out more than most city dwellers (I spend most time outside exercising). I eat mostly healthy. My bad working memory is also affecting my friendships. I find it hard to make friends because I get mentally exhausted around people.

    Plz, do not chew raw garlic. I did chew raw garlic, and I damaged my working memory.

    My long term memory is fine. I can still write about things I learned in college.
    I'm sad about my working memory. I need it for my engineering career. Perhaps, I'll reincarnate into a healthier person. Until then, I need a non-judgmental and stress-free support group. I live in San Jose, CA. Any suggestions?

    ReplyDelete
  17. Anonymous12:42 am

    How does chewing raw garlic affect working memory?

    ReplyDelete