Citation: Fernando Ferreira-Santos. Facial Emotion Processing in the Laboratory (and elsewhere): Tradeoffs between Stimulus Control and Ecological Validity[J]. AIMS Neuroscience, 2015, 2(4): 236-239. doi: 10.3934/Neuroscience.2015.4.236
[1] | Frances A. Maratos, Matthew Garner, Alexandra M. Hogan, Anke Karl . When is a Face a Face? Schematic Faces, Emotion, Attention and the N170. AIMS Neuroscience, 2015, 2(3): 172-182. doi: 10.3934/Neuroscience.2015.3.172 |
[2] | Justyna Gerłowska, Krzysztof Dmitruk, Konrad Rejdak . Facial emotion mimicry in older adults with and without cognitive impairments due to Alzheimer's disease. AIMS Neuroscience, 2021, 8(2): 226-238. doi: 10.3934/Neuroscience.2021012 |
[3] | Mariana FP de Araújo, Wagner A de Castro, Hiroshi Nishimaru, Susumu Urakawa, Taketoshi Ono, Hisao Nishijo . Performance in a gaze-cueing task is associated with autistic traits. AIMS Neuroscience, 2021, 8(1): 148-160. doi: 10.3934/Neuroscience.2021007 |
[4] | Pauline M. Insch, Gillian Slessor, Louise H. Phillips, Anthony Atkinson, Jill Warrington . The Impact of Aging and Alzheimers Disease on Decoding Emotion Cues from Bodily Motion. AIMS Neuroscience, 2015, 2(3): 139-152. doi: 10.3934/Neuroscience.2015.3.139 |
[5] | Charles R Larson, Donald A Robin . Sensory Processing: Advances in Understanding Structure and Function of Pitch-Shifted Auditory Feedback in Voice Control. AIMS Neuroscience, 2016, 3(1): 22-39. doi: 10.3934/Neuroscience.2016.1.22 |
[6] | Brion Woroch, Alex Konkel, Brian D. Gonsalves . Activation of stimulus-specific processing regions at retrieval tracks the strength of relational memory. AIMS Neuroscience, 2019, 6(4): 250-265. doi: 10.3934/Neuroscience.2019.4.250 |
[7] | Kenneth R. Paap, Hunter Myuz, Regina Anders-Jefferson, Lauren Mason, Brandon Zimiga . On the ambiguity regarding the relationship between sequential congruency effects, bilingual advantages in cognitive control, and the disengagement of attention. AIMS Neuroscience, 2019, 6(4): 282-298. doi: 10.3934/Neuroscience.2019.4.282 |
[8] | Mohamad Arif Awang Nawi, Nor Farid Mohd Noor, Ramizu Shaari, Ameera Kamal Khaleel, Muhamamd Amirul Mat Lazin, Ibrahim Mohammed Sulaiman, Mustafa Mamat . The patterns of facial fractures in traumatic brain injury (TBI) patients using ordinal regression: a retrospective study of five years. AIMS Neuroscience, 2022, 9(3): 345-357. doi: 10.3934/Neuroscience.2022019 |
[9] | Marie Vandekerckhove, Yu-lin Wang . Emotion, emotion regulation and sleep: An intimate relationship. AIMS Neuroscience, 2018, 5(1): 1-17. doi: 10.3934/Neuroscience.2018.1.1 |
[10] | Shahzaib Maqbool, Arham Ihtesham, Muhammad Nadeem Langove, Sara Jamal, Tabdar Jamal, Hafiz Abu Safian . Neuro-dermatological association between psoriasis and depression: an immune-mediated inflammatory process validating skin-brain axis theory. AIMS Neuroscience, 2021, 8(3): 340-354. doi: 10.3934/Neuroscience.2021018 |
The study by Maratos, Garner, Hogan, and Karl [1] demonstrates that simple schematic faces evoke neural responses comparable to those evoked by photographic face stimuli. The authors argue that this shows that schematic visual stimuli may still validly represent face processing, although it has been argued that such non-photographic faces lack many of the visual properties of the human face. In this commentary I would like to elaborate on this issue.
Indeed similar claims of lack of ecological validity have been made to photographic face stimuli as well. It is true that viewing faces and facial expressions of emotion in natural contexts is quite different from the viewing conditions usually present in face perception experiments in the laboratory. But this is not a problem that is specific to face processing studies and stimuli. Indeed any laboratory study is faced with issues of ecological validity. Ultimately it is a matter of the tradeoff between ecological validity and control of experimental conditions and extraneous variables. For face processing in particular, many different kinds of stimuli have been used in the literature, with varying degrees of proximity to natural viewing conditions. Simplified stimuli like schematic faces, as those used by Maratos et al [1] have featured in some studies on face processing (e.g., [2]). Photographic faces, on the other hand, retain most of the actual visual features of faces and have been the most widely used stimuli in face perception research. Photographic stimuli have been used under many different visual conditions: for instance, color vs. greyscale, with or without occlusion of facial characteristics, controlling for spatial frequency contents (for details on spatial frequency manipulations, see [3]).
Still, critics of the static photographic facial expression stimuli have suggested the use of dynamic facial stimuli in the form of small video clips in which the facial expression of emotion emerges from a neutral to an emotional expression (e.g., [4]). But perhaps the closest to an ecologically valid situation that is still possible to conduct in an ERP laboratory setting was done in a study by Pönkänen et al [5], in which facial stimuli were dynamically displayed by actors standing in front of the research participants. A system of blinds that were opened only when the stimulus was to be presented and that were synchronized to the EEG recorder was in place. So, these facial stimuli were live facial expressions being shown by an actual person (actor) standing in front of the participant.
Of course that as one makes the facial expression stimuli more ecologically valid, one is introducing more sources of potentially uncontrolled variance in the study. Photographic stimuli are much more visually complex than schematic stimuli; colored photographs have additional perceptual dimensions when compared to greyscale photographs; if stimuli are dynamic video clips, then there is an additional temporal dimension; if an actual actor in a live situation is the source of the facial stimulus, there is no guarantee that every instance of a given expression is the same, within and across different participants. Again this is the tradeoff between control and ecological validity.
Indeed, I agree with Maratos et al [1] that there is a place for schematic face stimuli in face perception research. In this context, schematic faces are likely the most artificial and simplified depictions of a face (i.e., they show minimal ecological validity), but such schematic faces are also the stimuli that may be controlled more carefully in terms of their visual characteristics. For instance, it is possible to control what is the exact angle of the eyebrows (and to relate it to perceptions of different intensities of happy/angry expressions), what is the curvature of the mouth, or the size of the eyes. By quantifying these characteristics it is possible to systematically relate them to behavioral and brain responses. As stated before, a more common alternative in face perception research has been to use photographic quality stimuli, and more recent studies have used additional kinds of face stimuli (videos, live actors).
To finalize my commentary I would like to systematize the different kinds of face stimuli in terms of a complementary continuum. As one moves along this continuum one will lose control over the physical and visual properties of the images, but the stimuli will become more ecologically valid. Figure 1 is a graphical representation of the tradeoffs I have been considering. On one end there are schematic faces that are the most easily controlled face stimuli, but show the worst ecological validity. Photographic faces follow by allowing a lower but still reasonable degree of control (these stimuli may be graphically edited for changes in color/luminance/contrast, spatially filtered, morphed, etc.), while being better in terms of ecological validity. Dynamic videos of facial motion are then closer to real face perception contexts, but less susceptible to control, and live acting even more so—indeed the expressions portrayed by live actors will never be exactly the same across multiple repetitions.
By considering the different kinds of studies, with different kinds of stimuli, along this continuum then it is possible to grade the contribution that these studies offer to the literature, as gains in ecological validity will be offset by decreased control over stimuli and thus a less precise understanding of the exact mechanisms involved in face processing. It is likely that the adjustment between experimental control and ecological validity will depend on the research question. If one is interested in determining low-level perceptual mechanisms, then simplified artificial faces will not only be sufficient, but will actually be better suited to answer the question given the possibility of manipulating their visual features. If, on the other hand, one is curious about the role of face perception on social behavior, then pictures or even videos of faces may be insufficient, and viewing faces in naturalistic contexts would be the necessary alternative. Indeed it is not impossible to take neurobehavioral research out of the laboratory with recent efforts in mobile brain/body imaging, although studying the human brain “in the wild” is still a formidable endeavor [6,7].
The author is supported by a grant (242/14) from the BIAL Foundation.
The author declares to have no conflict of interest.
[1] | Maratos FA, Garner M, Hogan AM, et al. (2015) When is a Face a Face? Schematic Faces, Emotion, Attention and the N170. AIMS Neurosci 2: 172-182. |
[2] |
Krombholz A, Schaefer F, Boucsein W (2007) Modification of N170 by different emotional expression of schematic faces. Biol Psychol 76: 156-162. doi:10.1016/j.biopsycho.2007.07.004 doi: 10.1016/j.biopsycho.2007.07.004
![]() |
[3] |
Almeida PR, Ferreira-Santos F, Vieira JB, et al. (2014) Dissociable effects of psychopathic traits on cortical and subcortical visual pathways during facial emotion processing: An ERP study on the N170. Psychophysiology 51: 645-657. doi:10.1111/psyp.12209 doi: 10.1111/psyp.12209
![]() |
[4] |
Sato W, Kochiyamab T, Yoshikawab S, Naitoa E, et al. (2004) Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. Brain Res Cogn Brain Res 20: 81-91. doi:10.1016/j.cogbrainres.2004.01.008 doi: 10.1016/j.cogbrainres.2004.01.008
![]() |
[5] |
Pönkänen LM, Alhoniemi A, Leppaänen JM, et al. (2011) Does it make a difference if I have an eye contact with you or with your picture? An ERP study. Soc Cogn Affect Neurosci 6: 486-494. doi:10.1093/scan/nsq068 doi: 10.1093/scan/nsq068
![]() |
[6] |
Gramann K, Ferris DP, Gwin J, et al. (2014) Imaging natural cognition in action. Int J Psychophysiol 91: 22-29. doi:10.1016/j.ijpsycho.2013.09.003 doi: 10.1016/j.ijpsycho.2013.09.003
![]() |
[7] | Makeig S, Gramann K, Jung T-P, et al. (2009) Linking brain, mind and behavior: The promise of mobile brain/body imaging (MoBI). Int J Psychophysiol 73: 985-100. |
1. | Harry Farmer, Raqeeb Mahmood, Samantha E.A. Gregory, Polina Tishina, Antonia F. de C. Hamilton, Dynamic emotional expressions do not modulate responses to gestures, 2021, 212, 00016918, 103226, 10.1016/j.actpsy.2020.103226 | |
2. | Yağmur Damla Şentürk, Ebru Ecem Tavacioglu, İlker Duymaz, Bilge Sayim, Nihan Alp, The Sabancı University Dynamic Face Database (SUDFace): Development and validation of an audiovisual stimulus set of recited and free speeches with neutral facial expressions, 2022, 1554-3528, 10.3758/s13428-022-01951-z | |
3. | Wataru Sato, Sakiko Yoshikawa, Influence of stimulus manipulation on conscious awareness of emotional facial expressions in the match-to-sample paradigm, 2023, 13, 2045-2322, 10.1038/s41598-023-47995-9 | |
4. | A.L. Abreu, L. Fernández-Aguilar, F. Ferreira-Santos, C. Fernandes, Increased N250 elicited by facial familiarity: An ERP study including the face inversion effect and facial emotion processing, 2023, 188, 00283932, 108623, 10.1016/j.neuropsychologia.2023.108623 |