), 张明1() 1苏州大学心理学系, 心理与行为科学研究中心, 苏州 215123
2辽宁师范大学心理学院, 儿童青少年健康人格评定与培养协同创新中心, 大连 116029
收稿日期:2019-11-20出版日期:2020-10-15发布日期:2020-08-24通讯作者:唐晓雨,张明E-mail:tangyu-2006@163.com;psyzm@suda.edu.cn基金资助:* 国家自然科学基金(31700939);国家自然科学基金(31871092);国家自然科学基金(31600882);江苏省基础研究计划(BK20170333)Sound-induced flash illusion in multisensory integration
WANG Aijun1, HUANG Jie1, LU Feifei1, HE Jiaying1, TANG Xiaoyu2(), ZHANG Ming1() 1Department of Psychology, Soochow University, Research Center for Psychology and Behavioral Sciences, Suzhou 215123, China
2School of Psychology, Liaoning Normal University, Liaoning Collaborative Innovation Center of Children and Adolescents Healthy Personality Assessment and Cultivation, Dalian 116029, China
Received:2019-11-20Online:2020-10-15Published:2020-08-24Contact:TANG Xiaoyu,ZHANG Ming E-mail:tangyu-2006@163.com;psyzm@suda.edu.cn摘要/Abstract
摘要： 声音诱发闪光错觉效应是典型的视听整合错觉现象, 是指当视觉闪光刺激与间隔100 ms内的听觉声音刺激不等数量呈现时, 被试知觉视觉闪光的数量与听觉声音的数量相等。声音诱发闪光错觉的影响因素既包括自下而上和自上而下的被试内差异因素, 也包括视听刺激依赖程度、视听整合的发展和视听刺激知觉敏感性等被试间差异因素。该效应的产生在时程上主要体现在早期加工阶段, 在脑区上主要涉及多处皮层及皮层下相关脑区。未来研究应进一步考察注意、奖赏和视听整合方式等认知加工对声音诱发闪光错觉的影响, 同时也应该关注声音诱发闪光错觉对记忆和学习的影响以及结合计算模型和神经科学的手段进一步探讨其认知神经机制。
图/表 1
表1声音诱发闪光错觉效应的认知神经机制研究
 |
表1声音诱发闪光错觉效应的认知神经机制研究
| |
参考文献 80
| [1] | 罗霄骁, 康冠兰, 周晓林. (2018). McGurk效应的影响因素与神经基础. 心理科学进展, 26(11), 1935-1951. |
| [2] | 文小辉, 李国强, 刘强. (2011). 视听整合加工及其神经机制. 心理科学进展, 19(7), 976-982. |
| [3] | 于薇, 王爱君, 张明. (2017). 集中和分散注意对多感觉整合中听觉主导效应的影响. 心理学报, 49(2), 164-173. doi: 10.3724/SP.J.1041.2017.00164URL |
| [4] | 张明, 唐晓雨, 于薇, 宁波, 王智楠, 王爱君. (2018). 基于通道的内源性注意对声音诱发闪光错觉的影响. 心理学报, 50(11), 20-29. |
| [5] | Abadi, R. V., & Murphy, J. S. (2014). Phenomenology of the sound-induced flash illusion. Experimental Brain Research, 232(7), 2207-2220. doi: 10.1007/s00221-014-3912-2URLpmid: 24691756 |
| [6] | Andersen, T. S., Tiippana, K., & Sams, M. (2004). Factors influencing audiovisual fission and fusion illusions. Cognitive Brain Research, 21(3), 301-308. doi: 10.1016/j.cogbrainres.2004.06.004URLpmid: 15511646 |
| [7] | Balz, J., Keil, J., Romero, Y. R., Mekle, R., Schubert, F., Aydin, S., ... Senkowski, D. (2016). GABA concentration in superior temporal sulcus predicts gamma power and perception in the sound-induced flash illusion. NeuroImage, 125, 724-730. URLpmid: 26546865 |
| [8] | Balz, J., Roa Romero, Y., Keil, J., Krebber, M., Niedeggen, M., Gallinat, J., & Senkowski, D. (2016). Beta/Gamma oscillations and event-related potentials indicate aberrant multisensory processing in schizophrenia. Frontiers in Psychology, 7, 1896. URLpmid: 27999553 |
| [9] | Bao, V. A., Doobay, V., Mottron, L., Collignon, O., & Bertone, A. (2017). Multisensory integration of low-level information in autism spectrum disorder: Measuring susceptibility to the flash-beep illusion. Journal of Autism and Developmental Disorders, 47(8), 2535-2543. doi: 10.1007/s10803-017-3172-7URL |
| [10] | Barutchu, A., Sahu, A., Humphreys, G. W., & Spence, C. (2019). Multisensory processing in event-based prospective memory. Acta Psychologica, 192, 23-30. URLpmid: 30391627 |
| [11] | Bertelson, P., & Ascherslebenc, G. (2003). Temporal ventriloquism: Crossmodal interaction on the time dimension: Evidence from auditory-visual temporal order judgment. International Journal of Psychophysiology, 50(1-2), 147-155. doi: 10.1016/s0167-8760(03)00130-2URLpmid: 14511842 |
| [12] | Bidelman, G. M. (2016). Musicians have enhanced audiovisual multisensory binding: Experience-dependent effects in the double-flash illusion. Experimental Brain Research, 234(10), 3037-3047. doi: 10.1007/s00221-016-4705-6URLpmid: 27334887 |
| [13] | Bolognini, N., Rossetti, A., Casati, C., Mancini, F., & Vallar, G. (2011). Neuromodulation of multisensory perception: A tDCS study of the sound-induced flash illusion. Neuropsychologia, 49(2), 231-237. URLpmid: 21094177 |
| [14] | Brighina, F., Bolognini, N., Cosentino, G., Maccora, S., Paladino, P., Baschi, R., ... Fierro, B. (2015). Visual cortex hyperexcitability in migraine in response to sound-induced flash illusions. Neurology, 84(20), 2057-2061. URLpmid: 25888559 |
| [15] | Brunetti, R., Indraccolo, A., Mastroberardino, S., Spence, C., & Santangelo, V. (2017). The impact of cross-modal correspondences on working memory performance. Journal of Experimental Psychology: Human Perception and Performance, 43(4), 819-831. doi: 10.1037/xhp0000348URLpmid: 28345948 |
| [16] | Cecere, R., Rees, G., & Romei, V. (2015). Individual differences in alpha frequency drive crossmodal illusory perception. Current Biology, 25(2), 231-235. doi: 10.1016/j.cub.2014.11.034URLpmid: 25544613 |
| [17] | Chan, J. S., Connolly, S. K., & Setti, A. (2018). The number of stimulus-onset asynchronies affects the perception of the sound-induced flash illusion in young and older adults. Multisensory Research, 31(3-4), 175-190. doi: 10.1163/22134808-00002605URLpmid: 31264623 |
| [18] | Chan, J. S., Kaiser, J., Brandl, M., Matura, S., Prvulovic, D., Hogan, M., & Naumer, M. J. (2015). Expanded temporal binding windows in people with mild cognitive impairment. Current Alzheimer Research, 12(1), 61-68. URLpmid: 25523426 |
| [19] | Chan, J. S., Wibral, M., Wollstadt, P., Stawowsky, C., Brandl, M., Helbling, S., ... Kaiser, J. (2017). Predictive coding over the lifespan:Increased reliance on perceptual priors in older adults — A magnetoencephalography and dynamic causal modelling study. BioRxiv, 178095. |
| [20] | Cuppini, C., Magosso, E., Bolognini, N., Vallar, G., & Ursino, M. (2014). A neurocomputational analysis of the sound- induced flash illusion. NeuroImage, 92, 248-266. doi: 10.1016/j.neuroimage.2014.02.001URLpmid: 24518261 |
| [21] | de Haas, B., Kanai, R., Jalkanen, L., & Rees, G. (2012). Grey matter volume in early human visual cortex predicts proneness to the sound-induced flash illusion. Proceedings of the Royal Society B: Biological Sciences, 279(1749), 4955-4961. URLpmid: 23097516 |
| [22] | DeLoss, D. J., & Andersen, G. J. (2015). Aging, spatial disparity, and the sound-induced flash illusion. PLoS ONE, 10(11), e0143773. doi: 10.1371/journal.pone.0143773URLpmid: 26619352 |
| [23] | DeLoss, D. J., Pierce, R. S., & Andersen, G. J. (2013). Multisensory integration, aging, and the sound-induced flash illusion. Psychology Aging, 28(3), 802-812. doi: 10.1037/a0033289URLpmid: 23978009 |
| [24] | Georgios, M., & Julian, K. (2018). High cognitive load enhances the susceptibility to non-speech audiovisual illusions. Scientific Reports, 8(1), 11530. doi: 10.1038/s41598-018-30007-6URLpmid: 30069059 |
| [25] | Gieseler, A., Tahden, M. A. S., Thiel, C. M., & Colonius, H. (2018). Does hearing aid use affect audiovisual integration in mild hearing impairment? Experimental Brain Research, 236, 1161-1179. URLpmid: 29453491 |
| [26] | Ha?, K., Sinke, C., Reese, T., Roy, M., Wiswede, D., Dillo, W., ... Szycik, G. R. (2017). Enlarged temporal integration window in schizophrenia indicated by the double-flash illusion. Cognitive Neuropsychiatry, 22(2), 145-158. doi: 10.1080/13546805.2017.1287693URLpmid: 28253091 |
| [27] | Hernández, B., Setti, A., Kenny, R. A., & Newell, F. N. (2019). Individual differences in ageing, cognitive status, and sex on susceptibility to the sound-induced flash illusion: A large-scale study. Psychology and Aging, 34(7), 978-990. doi: 10.1037/pag0000396URLpmid: 31621358 |
| [28] | Innes-Brown, H., & Crewther, D. (2009). The impact of spatial incongruence on an auditory-visual illusion. PLoS ONE, 4(7), e6450. URLpmid: 19649293 |
| [29] | Jiang, Y., & Han, S. H. (2007). Perceptual gain and perceptual loss: Distinct neural mechanisms of audiovisual interactions. International Journal of Magnetic Resonance Imaging, 1(1), 1749-8023. |
| [30] | Kaiser, M., Senkowski, D., Busch, N. A., Balz, J., & Keil, J. (2019). Single trial prestimulus oscillations predict perception of the sound-induced flash illusion. Scientific Reports, 9(1), 5983. doi: 10.1038/s41598-019-42380-xURLpmid: 30979927 |
| [31] | Kamke, M. R., Vieth, H. E., Cottrell, D., & Mattingley, J. B. (2012). Parietal disruption alters audiovisual binding in the sound-induced flash illusion. NeuroImage, 62(3), 1334-1341. doi: 10.1016/j.neuroimage.2012.05.063URLpmid: 22658974 |
| [32] | Kawabe, T. (2009). Audiovisual temporal capture underlies flash fusion. Experimental Brain Research, 198(2-3), 195-208. doi: 10.1007/s00221-009-1877-3URLpmid: 19521693 |
| [33] | Keil, J., Müller, N., Hartmann, T., & Weisz, N. (2013). Prestimulus beta power and phase synchrony influence the sound-induced flash illusion. Cerebral Cortex, 24(5), 1278-1288. URLpmid: 23300109 |
| [34] | Keil, J., & Senkowski, D. (2017). Individual alpha frequency relates to the sound-induced flash illusion. Multisensory Research, 30(6), 565-578. URLpmid: 31287091 |
| [35] | Kim, R., Peters, M. A. K., & Shams, L. (2011). 0 + 1 > 1: How adding noninformative sound improves performance on a visual task. Psychological Science, 23(1), 6-12. doi: 10.1177/0956797611420662URLpmid: 22127367 |
| [36] | Klatzky, R. L., Marston, J. R., Giudice, N. A., Golledge, R. G., & Loomis, J. M. (2006). Cognitive load of navigating without vision when guided by virtual sound versus spatial language. Journal of Experimental Psychology: Applied, 12(4), 223-232. doi: 10.1037/1076-898X.12.4.223URLpmid: 17154771 |
| [37] | Kostaki, M., & Vatakis, A. (2016). Crossmodal binding rivalry: A “race” for integration between unequal sensory inputs. Vision Research, 127, 165-176. URLpmid: 27591367 |
| [38] | Kumpik, D. P., Roberts, H. E., King, A. J., & Bizley, J. K. (2014). Visual sensitivity is a stronger determinant of illusory processes than auditory cue parameters in the sound-induced flash illusion. Journal of Vision, 14(7), 12. URLpmid: 24961249 |
| [39] | Laurienti, P. J., Burdette, J. H., Maldjian, J. A., & Wallace, M. T. (2006). Enhanced multisensory integration in older adults. Neurobiology of Aging, 27(8), 1155-1163. doi: 10.1016/j.neurobiolaging.2005.05.024URLpmid: 16039016 |
| [40] | Lewald, J., & Guski, R. (2003). Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. Cognitive Brain Research, 16(3), 468-478. doi: 10.1016/s0926-6410(03)00074-0URLpmid: 12706226 |
| [41] | Maccora, S., Giglia, G., Bolognini, N., Cosentino, G., Gangitano, M., Salemi, G., & Brighina, F. (2019). Cathodal occipital tDCS is unable to modulate the sound induced flash illusion in migraine. Frontiers in Human Neuroscience, 13, 247. doi: 10.3389/fnhum.2019.00247URLpmid: 31379542 |
| [42] | Magnotti, J. F., & Beauchamp, M. S. (2015). The noisy encoding of disparity model of the McGurk effect. Psychonomic Bulletin & Review, 22(3), 701-709. doi: 10.3758/s13423-014-0722-2URLpmid: 25245268 |
| [43] | Mahoney, J. R., Li, P. C. C., Oh-park, M., Verghese, J., & Holtzer, R. (2011). Multisensory integration across the senses in young and old adults. Brain Research, 1426(17), 43-53. doi: 10.1016/j.brainres.2011.09.017URL |
| [44] | Marques, L. M., Lapenta, O. M., Costa, T. L., & Boggio, P. S. (2016). Multisensory integration processes underlying speech perception as revealed by the McGurk illusion. Language, Cognition and Neuroscience, 31(9), 1115-1129. doi: 10.1080/23273798.2016.1190023URL |
| [45] | Máté, A., Anette, G., Verena, C., Masataka, W., & Uta, N. (2015). A spatially collocated sound thrusts a flash into awareness. Frontiers in Integrative Neuroscience, 9, 1-8. doi: 10.3389/fnint.2015.00001URLpmid: 25698945 |
| [46] | McCormick, D., & Mamassian, P. (2008). "What does the illusory-flash look like?". Vision Research, 48(1), 63-69. doi: 10.1016/j.visres.2007.10.010URLpmid: 18054372 |
| [47] | Mcgovern, D. P., Roudaia, E., Stapleton, J., McGinnity, T. M., & Newell, F. N. (2014). The sound-induced flash illusion reveals dissociable age-related effects in multisensory integration. Frontiers in Aging Neuroscience, 6, 250. doi: 10.3389/fnagi.2014.00250URLpmid: 25309430 |
| [48] | Merriman, N. A., Whyatt, C., Setti, A., Craig, C., & Newell, F. N. (2015). Successful balance training is associated with improved multisensory function in fall-prone older adults. Computers in Human Behavior, 45, 192-203. doi: 10.1016/j.chb.2014.12.017URL |
| [49] | Mishra, J., Martinez, A., & Hillyard, S. A. (2008). Cortical processes underlying sound-induced flash fusion. Brain Research, 1242(4), 102-115. doi: 10.1016/j.brainres.2008.05.023URL |
| [50] | Mishra, J., Martínez, A., & Hillyard, S. A. (2010). Effect of attention on early cortical processes associated with the sound-induced extra flash illusion. Journal of Cognitive Neuroscience, 22(8), 1714-1729. URLpmid: 19583464 |
| [51] | Mishra, J., Martinez, A., Sejnowski, T. J., & Hillyard, S. A. (2007). Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. Journal of Neuroscience, 27(15), 4120-4131. URLpmid: 17428990 |
| [52] | Moro, S. S., & Steeves, J. K. E. (2018). Normal temporal binding window but no sound-induced flash illusion in people with one eye. Experimental Brain Research, 236(6), 1825-1834. URLpmid: 29675714 |
| [53] | Narinesingh, C., Goltz, H. C., & Wong, A. M. F. (2017). Temporal binding window of the sound-induced flash illusion in amblyopia. Investigative Opthalmology & Visual Science, 58(3), 1442-1448. |
| [54] | O’Brien, J., Ottoboni, G., Tessari, A., & Setti, A. (2017). One bout of open skill exercise improves cross-modal perception and immediate memory in healthy older adults who habitually exercise. PLoS ONE, 12(6), e0178739. URLpmid: 28570704 |
| [55] | Olasagasti, I., Bouton, S., & Giraud, A. L. (2015). Prediction across sensory modalities: A neurocomputational model of the McGurk effect. Cortex, 68, 61-75. URLpmid: 26009260 |
| [56] | Quak, M., London, R. E., & Talsma, D. (2015). A multisensory perspective of working memory. Frontiers in Human Neuroscience, 9, 197. doi: 10.3389/fnhum.2015.00197URLpmid: 25954176 |
| [57] | Rosenthal, O., Shimojo, S., & Shams, L. (2009). Sound- induced flash illusion is resistant to feedback training. Brain Topography, 21(3-4), 185-192. URLpmid: 19340570 |
| [58] | Samuel, A. G. (2011). Speech perception. Annual Review of Psychology, 62(1), 49-72. doi: 10.1146/annurev.psych.121208.131643URL |
| [59] | Setti, A., & Chan, J. S. (2011). Familiarity of objects affects susceptibility to the sound-induced flash illusion. Neuroscience Letters, 492(1), 19-22. URLpmid: 21262319 |
| [60] | Shams, L., & Beierholm, U. R. (2010). Causal inference in perception. Trends in Cognitive Sciences, 14(9), 425-432. doi: 10.1016/j.tics.2010.07.001URL |
| [61] | Shams, L., Kamitani, Y., & Shimojo, S. (2000). What you see is what you hear. Nature, 408(6814), 788. URLpmid: 11130706 |
| [62] | Shams, L., Kamitani, Y., & Shimojo, S. (2002). Visual illusion induced by sound. Cognitive Brain Research, 14(1), 147-152. URLpmid: 12063138 |
| [63] | Shams, L., Kamitani, Y., Thompson, S. M., & Shimojo, S. (2001). Sound alters visual evoked potentials in humans. Neuroreport, 12(17), 3849-3852. doi: 10.1097/00001756-200112040-00049URLpmid: 11726807 |
| [64] | Shams, L., Ma, W. J., & Beierholm, U. (2005). Sound- induced flash illusion as an optimal percept. Neuroreport, 16(17), 1923-1927. doi: 10.1097/01.wnr.0000187634.68504.bbURLpmid: 16272880 |
| [65] | Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9(4), 255-266. doi: 10.1038/nrn2331URLpmid: 18354398 |
| [66] | Sterzer, P., Kleinschmidt, A., & Rees, G. (2009). The neural bases of multistable perception. Trends in Cognitive Sciences, 13(7), 310-318. doi: 10.1016/j.tics.2009.04.006URL |
| [67] | Sun, Y. W., Liu, X. L., Li, B. L., Sava-Segal, C., Wang, A., & Zhang, M. (2020). Effects of repetition suppression on sound induced flash illusion with aging. Frontiers in Psychology. 11, 216. doi: 10.3389/fpsyg.2010.00216URLpmid: 21833271 |
| [68] | Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14(9), 400-410. doi: 10.1016/j.tics.2010.06.008URL |
| [69] | Tang, X. Y., Wu, J. L., & Shen, Y. (2016). The interactions of multisensory integration with endogenous and exogenous attention. Neuroscience & Biobehavioral Reviews, 61, 208-224. doi: 10.1016/j.neubiorev.2015.11.002URLpmid: 26546734 |
| [70] | Wang, A. J., Sang, H. B., He, J. Y., Sava-Segal, C., Tang, X. Y., & Zhang, M. (2019). Effects of cognitive expectation on sound-induced flash illusion. Perception, 48(12), 1214-1234. doi: 10.1177/0301006619885796URLpmid: 31808371 |
| [71] | Wang, M., Arteaga, D., & He, B. J. (2013). Brain mechanisms for simple perception and bistable perception. Proceedings of the National Academy of Sciences of the United States of America, 110(35), E3350-3359. doi: 10.1073/pnas.1221945110URLpmid: 23942129 |
| [72] | Watkins, S., Shams, L., Josephs, O., & Rees, G. (2007). Activity in human V1 follows multisensory perception. NeuroImage, 37(2), 572-578. doi: 10.1016/j.neuroimage.2007.05.027URLpmid: 17604652 |
| [73] | Watkins, S., Shams, L., Tanaka, S., Haynes, J.-D., & Rees, G. (2006). Sound alters activity in human V1 in association with illusory visual perception. NeuroImage, 31(3), 1247-1256. doi: 10.1016/j.neuroimage.2006.01.016URLpmid: 16556505 |
| [74] | Whittingham, K. M., Mcdonald, J. S., & Clifford, C. W. G. (2014). Synesthetes show normal sound-induced flash fission and fusion illusions. Vision Research, 105, 1-9. URLpmid: 25173429 |
| [75] | van Erp, J. B. F., Philippi, T. G., & Werkhoven, P. (2013). Observers can reliably identify illusory flashes in the illusory flash paradigm. Experimental Brain Research, 226(1), 73-79. doi: 10.1007/s00221-013-3413-8URL |
| [76] | Vanes, L. D., White, T. P., Wigton, R. L., Joyce, D., Collier, T., & Shergill, S. S. (2016). Reduced susceptibility to the sound-induced flash fusion illusion in schizophrenia. Psychiatry Research, 245, 58-65. doi: 10.1016/j.psychres.2016.08.016URLpmid: 27526318 |
| [77] | Xie, Y. J., Xu, Y. Y., Bian, C., & Li, M. (2017). Semantic congruent audiovisual integration during the encoding stage of working memory: An ERP and sLORETA study. Scientific Reports, 7(1), 5112. doi: 10.1038/s41598-017-05471-1URLpmid: 28698594 |
| [78] | Yalachkov, Y., Bergmann, H. J., Soyda?, D., Buschenlange, C., Fadai Motlagh, L. Y., Naumer, M. J., ... Gehrig, J. (2019). Cognitive impairment in multiple sclerosis is reflected by increased susceptibility to the sound-induced flash illusion. Frontiers in Neurology, 10, 373. URLpmid: 31031699 |
| [79] | Zhang, N. Y., & Chen, W. (2006). A dynamic fMRI study of illusory double-flash effect on human visual cortex. Experimental Brain Research, 172(1), 57-66. doi: 10.1007/s00221-005-0304-7URLpmid: 16369788 |
| [80] | Zhou, W., Jiang, Y., He, S., & Chen, D. (2010). Olfaction modulates visual perception in binocular rivalry. Current Biology, 20(15), 1356-1358. URLpmid: 20598540 |
相关文章 6
| [1] | 杨伟平, 李胜楠, 李子默, 郭敖, 任艳娜. 老年人视听觉整合的影响因素及其神经机制[J]. 心理科学进展, 2020, 28(5): 790-799. |
| [2] | 赵佩琼, 陈巍, 张静, 平贤洁. 橡胶手错觉:拥有感研究的实验范式及其应用[J]. 心理科学进展, 2019, 27(1): 37-50. |
| [3] | 彭姓, 常若松, 任桂琴, 王爱君, 唐晓雨. 外源性注意与多感觉整合的交互关系[J]. 心理科学进展, 2018, 26(12): 2129-2140. |
| [4] | 罗霄骁, 康冠兰, 周晓林. McGurk效应的影响因素与神经基础[J]. 心理科学进展, 2018, 26(11): 1935-1951. |
| [5] | 周爱保;张彦驰;刘沛汝;尹玉龙;张 奋. 我是谁？——人际间多感觉刺激下的识脸错觉[J]. 心理科学进展, 2015, 23(2): 159-167. |
| [6] | 袁祥勇;黄希庭. 多感觉整合的时间再校准[J]. 心理科学进展, 2011, 19(5): 692-700. |
PDF全文下载地址:
http://journal.psych.ac.cn/xlkxjz/CN/article/downloadArticleFile.do?attachType=PDF&id=5188
