The costs of changing modality in visuo-haptic recognition of scenes

Magdalena Szubielska; Katarzyna Jaroszek; Bartłomiej Kiljanek

Magdalena Szubielska Institute of Psychology, The John Paul II Catholic University of Lublin
Katarzyna Jaroszek
Bartłomiej Kiljanek

Keywords: cross-modal recognition; vision; touch

Abstract

The experiment is aimed at investigating the factors that may modulate the costs of cross-modal visuo-haptic recognition of scenes. Participants learned a scene either visually or by touch (in the latter case they were blindfolded); then, following a delay, they identified the scene using the same or changed modality. The level of difficulty was adjusted by introducing two or three changes in the placement of scene elements at the recognition stage. It has been demonstrated that the costs of modality change, related to both decreased accuracy of recognition and extended time for making decision, occur only in a situation when a significant burden is imposed on working memory, i.e., with tactile learning of a scene and a high level of difficulty of the recognition task.

References

Amedi, A., Malach, R., Hendler, T., Peled, S., & Zohary, E. (2001). Visuo-haptic object-related activation in the ventral visual pathway. Nature Neuroscience, 4(3), 324-330.
Casey, S. J., & Newell, F. N. (2007). Are representations of faces independent of encoding modality? Neuropsychologia, 45(3), 506-513.
Easton, R. D., Greene, A. J., & Srinivas, K. (1997). Transfer between vision and haptics: Memory for 2-D patterns and 3-D objects. Psychonomic Bulletin & Review, 4(3), 403-410.
Easton, R. D., Srinivas, K., & Greene, A. J. (1997). Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(1), 153-163.
Ernst, M. O., Lange, C., & Newell, F. N. (2007). Multisensory recognition of actively explored objects. Canadian Journal of Experimental Psychology, 61(3), 242-253.
Lacey, S., Flueckiger, P., Stilla, R., Lava, M., & Sathian, K. (2010). Object familiarity modulates the relationship between visual object imagery and haptic shape perception. NeuroImage, 49(3), 1977-1990.
Loomis, J. M., Klatzky, R. L., McHugh, B., & Giudice, N. A. (2012). Spatial working memory for locations specified by vision and audition: Testing the amodality hypothesis. Attention, Perception, & Psychophysics, 74(6), 1260-1267.
Newell, F. N., Ernst, M. O., Tjan B. S., & Bülthoff, H. H. (2001). Viewpoint dependence in visual and haptic object recognition. Psychological Science, 12(1), 37-42.
Newell, F. N., Woods, A. T., Mernagh, M., & Bülthoff, H. H. (2005). Visual, haptic and crossmodal recognition of scenes. Experimental Brain Research, 161, 233-242.
Newport, R., Rabb, B., & Jackson, S. R. (2002). Noninformative vision improves haptic spatial perception. Current Biology, 12(19), 1661-1664.
Pasqualotto, A., & Proulx, M. J. (2012). The role of visual experience for the neural basis of spatial cognition. Neuroscience and Biobehavioral Reviews, 36(4), 1179-1187.
Pathak, K., & Pring, L. (1989). Tactual picture recognition in congenitally blind and sighted children. Applied Cognitive Psychology, 3(4), 337-350.
Pensky, A. E. C., Johnson, K. A., Haag, S., & Homa, D. (2008). Delayed memory for visual-haptic exploration of familiar objects. Psychonomic Bulletin & Review, 15(3), 574-580.
Reales, J. M., & Ballesteros, S. (1999). Implicit and explicit memory for visual and haptic objects: Cross-modal priming depends on structural descriptions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25(3), 644-663.
Szubielska, M. (2009). The role of visual experience on the strategies of blind and sighted individuals in mental imagery. Poster presented on XI European Congress of Psychology. Oslo, Norway, July 7-10, 2009.
Zuidhoek, S., Kappers, A. M. L., van der Lubbe, R. H. J., & Postma, A. (2003). Delay improves performance on a haptic spatial matching task. Experimental Brain Research, 149(3), 320-330.