Please read Dr. Zorowitz’s article in Nature Human Behaviour titled, “Inattentive responding can induce spurious associations between task behaviour and symptom measures.”
Although online samples have many advantages for psychiatric research, some potential pitfalls of this approach are not widely understood. Here we detail circumstances in which spurious correlations may arise between task behaviour and symptom scores. The problem arises because many psychiatric symptom surveys have asymmetric score distributions in the general population, meaning that careless responders on these surveys will show apparently elevated symptom levels. If these participants are similarly careless in their task performance, this may result in a spurious association between symptom scores and task behaviour. We demonstrate this pattern of results in two samples of participants recruited online (total N = 779) who performed one of two common cognitive tasks. False-positive rates for these spurious correlations increase with sample size, contrary to common assumptions. Excluding participants flagged for careless responding on surveys abolished the spurious correlations, but exclusion based on task performance alone was less effective. To read the full article.
Inattentive responding can induce spurious associations between task behaviour and symptom measures. Samuel Zorowitz, Johanne Solis, Yael Niv, Daniel Bennett. Nat Hum Behavior. 2023 Jul 6. doi: 10.1038/s41562-023-01640-7. Online ahead of print.