How valid is Lumosity's Brain Performance Index and what normative information is available?

  • Background

    There is a test called Lumosity's Brain Performance Index. A sample profile of scores might be:

    Speed           680
    Memory          707
    Attention       277
    Flexibility     304
    Problem Solving 900
    Overall BPI     573
    

    I'm interested in how to interpret a profile of scores.

    Questions

    • What validity information is available for the BPI?
    • What normative information is available for BPI? In other words, how could the sample profile of scores be interpreted both absolutely and relative to other groups of people?

    Lumosity's *Brain Perfomance Index* is not a scientifically-valid measure of mental performance. If you want a valid test, find a testing center and perform an IQ test: http://en.wikipedia.org/wiki/Intelligence_quotient. From the Wikipedia article on Mental Retardation: "Mental Retardation has historically been defined as an Intelligence Quotient of under 70." http://en.wikipedia.org/wiki/Mental_retardation

    So Lumosity.com isn't a valid service? They state "Lumosity.com is advised by leading experts in neuroscience and cognitive psychology from Stanford, Berkeley, UCSF and other prestigious institutions"

    Ah, my apologies, I didn't intend to imply that (I should have said it isn't a valid measure of *general* mental performance). Lumosity is valid in that they score your progress in such a way that improvement can be tracked. However, from my knowledge, they track performance for specific mental abilities/tasks, rather than specifically testing for general intelligence. Either way, their scores do not equate with IQ scores, although I suspect they may measure similar qualities.

    I'm going to be fairly short in my reply to "How valid is Luminosity's Brain Performance Index and what normative information is available?" (although it is lumosity...i only know this because i searched for luminosity.com as well) and just say although it can assist with sharpening ones skills and ability to use a computer, it's not necessarily going to truly make one smarter...which after stated i realized they never really say it does that. Regardless, if someone is inept in using a pc yet continues to use lumosity.com of course their scores are going to increase....they're getting more use

    I would posit the only assurance you have when participating in this type of commercial partnership; you have exchanged your coin for the opportunity to have your data mined by yet another ad agency masquerading as pseudo science.

  • General thoughts on brain training: Lumosity is a commercial tool that aims to improve brain functioning. In general, I am sceptical of the potential for "brain training programs" to improve cognitive functioning in a generalised way (e.g., see this Nature discussion). Practice is powerful, but tends to be domain specific. So if you want to become skilled in brain training tasks, then do brain training exercises, but if you want to become skilled in a particular domain of life, then study and get experience in that domain. While I can see the value in researching the topic, I can also see the potential for sellers of "brain training" programs to exploit people's concerns about their mental shortcomings or about fears of mental deterioration with ageing.

    Normative data: From what I can gather from this review, Lumosity has lots of normative data based on existing users which you can use to compare your performance. However, I imagine this is proprietary information that they would be reluctant to publish in a useful form.

    There was an unpublished report I found by Cruz et al based on a small sample of young adults. The following summarises Table 2:

    Number of days        4.75 ± 3.18
    Number of sessions    10.79 ± 9.31
    Median (IQ R )        9.00 (2.25‐17.75)
    Minimum‐Maximum       0 ‐ 34
    Overall BPI           676.93 ± 323.94
    Lumosity points     173.43 ± 158.89
    

    Validity of the BPI: The website writes:

    Your Overall BPI is your average BPI across each of the four cognitive areas: attention, memory, processing speed and cognitive control.

    It also states

    "We then evaluate your game scores and use a proprietary algorithm to derive your BPI"

    Performance on almost all cognitive tasks are intercorrelated to some extent, and if you take the average, or preferably the first principal component, of a battery of cognitive tasks, it will tend to have a general factor, which will have some loading on g. Of course, if the tests are of a particular type, this might lead to different outcomes.

    I could not find any validity data on the website whereby BPI scores are correlated with other measures. Thus, it is unclear what relationship BPI has with other ability measures, or IQ for example.

    In general, practice effects on cognitive tests are seen as a source of error variance. What a test measures after people have taken the test many times may be very different. In particular, the degree to which the test correlates with a domain general quality of interest may be reduced. Instead, the test may start to reflect a domain specific adaptation. Also, if people differ in the amount of practice, this would compound the measure of a domain general ability.

    Overall thoughts: In general, the target market for Lumosity's product seems to be consumers. In contrast, tests like the WAIS or the CANTAB have a target market which is researching and applied psychologists. In the consumer market, it is typical for companies to use "proprietary algorithms" and for there not to be a test manual with extensive validity data. While this might make commercial sense, it limits the scientific value of such instruments.

    References

    • Cruz, A., Canelas, A., Machado, F., Pedreira, J., Barroso, J., Beir\~ao, L., Freitas, L., Belo, L., Seabra, M., Portero, M. & others (). Brain Fitness--Clinical Trial to Measure the Effect of Cognitive Training in Brain’s Capabilities. PDF

License under CC-BY-SA with attribution


Content dated before 7/24/2021 11:53 AM