Is there really no Theory of Mind deficit in autism? Part II: the validity of the standard Theory of Mind measures
In my previous post on Gernsbacher and Yergeau’s 2019 paper, Empirical Failures of the Claim That Autistic People Lack a Theory of Mind, I discussed problems with the authors’ arguments that the original studies that showed Theory of Mind deficits in autism have failed to replicate and been overturned by later studies. As the article continues, the authors embark on a second line of argumentation—this one concerning the inherent validity of the various ToM tests.
Morton A. Gernsbacher, Professor of Psychology, University of Wisconsin
The various ToM tests, Gernsbacher and Yergeau argue, fail to “converge” (that is, agree in their conclusions) on anything that meaningfully distinguishes autism. The ToM tests they consider are:
The Strange Stories Test, which measures the ability to deduce the social/emotional reasons for characters’ behaviors in a narrative sequence
The Animated Triangles Test, which measures the ability to ascribe emotions to self-propelled, interacting shapes in a short animation
The Eyes Test, which measures the ability to recognize emotions from facial information that is restricted to a rectangular region around the eyes
The Faux Pas Test, which measures the ability to detect social blunders
The false-belief tests, which measures the ability to make inferences about the beliefs held by (or about) individuals who are missing key pieces of information
The Sally-Anne “unexpected location change” test, in which Sally doesn’t witness Anne changing the location of her marble
The Smarties “unexpected contents” test, in which a candy box contains a pencil rather than candy
This “lack of convergent validity among theory-of-mind tasks,” they conclude, “undermines the core construct validity of theory of mind.”
Gernsbacher and Yergeau cite, in particular, the repeated failure of the Strange Stories test to converge with the Eyes test or the Animated Triangles test; and the repeated failure of the Eyes test to converge with the Faux Pas test. These test pairs, however, measure quite different social phenomena: detecting emotion from eyes; making sense of social behavior in a narrative; mentalizing about animated shapes; and detecting social blunders. Thus, none of these convergence failures is particularly surprising. The most related of these two phenomena, the Strange Stories and the Faux Pas tests, are one pair whose degree of convergence Gernsbacher and Yergeau do not mention. The lack of convergence of the other pairs may only mean that other variables, like individual variation within the autism spectrum in frequency of eye contact or in verbal skills, also play a role. This doesn’t rule out that each of these tests tap into an underlying social factor that is atypical in autism.
But Gernsbacher and Yergeau go on to make an even more surprising claim: that even within the false-belief tests there is lack of convergence, with different false-belief tasks failing to correlate significantly with one another. The only specific examples of non-convergent false-belief tasks in the studies they cite, however, are (1) first- vs. second-order false-belief tasks and (2) one study in which the Smarties task fails to correlate with the Sally-Anne test. As far as first vs. second-order false-belief tests go, we’ve already seen their non-convergence: first-order false-belief tests correlate with autistic traits (and language development); second-order false-belief tests correlate with working memory/processing abilities. But that does not undermine the validity of first-order false-belief tests in singling out autism.
As far as the Smarties vs. the Sally-Anne test, one study, Hughes (1998), found that performance on the Smarties test was not correlated with the Sally-Anne test. Nonetheless, each of these tests correlated with a different ToM task. The Sally-Anne test, which has to do with the location of hidden objects, correlated with a penny-hiding task, while the Smarties test, which has to do with a deceptive container, correlated with a deception task. Furthermore, as Lind and Bowler (2009) point out, the Sally-Anne task involves a narrative and a relatively simple test question (“Where will Sally look for her marble?”), while the Smarties task does not involve a narrative but does involve a more complex test prompt (“Billy hasn’t seen the box. When he comes in I’ll show him the box just like this and ask what’s in there. What will he say?”). Different skillsets (comprehension of hiding vs. deception; following a narrative vs. parsing a complex question), therefore, may be involved in passing each one, even if each also taps into the same underlying ToM skills. Beyond this one study, studies in general (e.g., Ozonoff, 1991, mentioned *in my earlier post*, for Smarties; or Kerr and Durkin, 2004, mentioned below, for Sally-Anne) have found both of these tests to be disproportionately difficult for individuals with autism.
Gernsbacher and Yergeau’s other two references for lack of convergence among false-belief tests do not involve autistic subjects. Charman & Campbell (1997) addresses individuals with learning disabilities; Duval et al. (2011), individuals with age-related cognitive decline.
In short, to the extent that there is a lack of convergence between ToM tests as applied to autistic individuals, this lack of convergence does not undermine the tests’ validity as measures that tap into the various aspects of the ToM deficits in autism.
REFERENCES:
Charman, T., & Campbell, A. (1997). Reliability of theory of mind task performance by individuals with a learning disability: A research note. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 38, 725–730. https://doi.org/10.1111/j.1469-7610.1997.tb01699.x
Duval, C., Piolino, P., Bejanin, A., Eustache, F., & Desgranges, B. (2011). Age effects on different components of theory of mind. Consciousness and Cognition, 20, 627–642. https://doi.org/10.1016/j.concog.2010.10.025
Gernsbacher, M. A., & Yergeau, M. (2019). Empirical Failures of the Claim That Autistic People Lack a Theory of Mind. Archives of scientific psychology, 7(1), 102–118. https://doi.org/10.1037/arc0000067
Hughes, C. (1998). Executive function in preschoolers: Links with theory of mind and verbal ability. British Journal of Developmental Psychology, 16, 233–253. https://doi.org/10.1111/j.2044-835X.1998.tb00921.x
Kerr, S., & Durkin, K. (2004). Understanding of thought bubbles as mental representations in children with autism: Implications for theory of mind. Journal of Autism and Developmental Disorders, 34, 637–648. https://doi.org/10.1007/s10803-004-5285-z
Lind, S. E., & Bowler, D. M. (2009). Language and theory of mind in autism spectrum disorder: the relationship between complement syntax and false belief task performance. Journal of autism and developmental disorders, 39(6), 929–937. https://doi.org/10.1007/s10803-009-0702-y
Ozonoff, S., Rogers, S. J., & Pennington, B. F. (1991). Asperger’s syndrome: Evidence of an empirical distinction from high-functioning autism. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 32, 1107–1122. https://doi.org/10.1111/j.1469-7610.1991.tb00352.x
