), 吴英楠1, 彭姓2, 王爱君3(), 李奇4 1 辽宁师范大学心理学院, 辽宁省儿童青少年健康人格评定与培养协同创新中心, 大连 116029
2 中国民用航空飞行学院飞行技术学院, 广汉 618307
3 苏州大学心理学系, 心理与行为科学研究中心, 苏州 215123
4 长春理工大学计算机科学技术学院, 长春 130022
收稿日期:2019-11-15出版日期:2020-07-25发布日期:2020-05-25通讯作者:唐晓雨,王爱君E-mail:2006@163.com;ajwang@suda.edu.cn基金资助:* 国家自然科学基金项目(31600882);国家自然科学基金项目(31700939);国家自然科学基金项目(61773076);辽宁省教育厅高水平创新团队国(境)外培养项目(2018LNGXGJWPY- YB015)The influence of endogenous spatial cue validity on audiovisual integration
TANG Xiaoyu1(), WU Yingnan1, PENG Xing2, WANG Aijun3(), LI Qi4 1 School of Psychology, Liaoning Collaborative Innovation Center of Children and Adolescents Healthy Personality Assessment and Cultivation, Liaoning Normal University, Dalian 116029, China
2 Flight Technology College, Civil Aviation Flight University of China, Guanghan 618307, China
3 Department of Psychology, Research Center for Psychology and Behavioral Sciences, Soochow University, Suzhou 215123, China
4 School of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, China
Received:2019-11-15Online:2020-07-25Published:2020-05-25Contact:TANG Xiaoyu,WANG Aijun E-mail:2006@163.com;ajwang@suda.edu.cn摘要/Abstract
摘要： 采用内源性线索-靶子范式, 操纵线索类型(有效线索、无效线索)和靶刺激通道类型(视觉刺激、听觉刺激、视听觉刺激)两个自变量, 通过两个实验, 分别设置50%和80%两种内源性空间线索有效性来考察不同空间线索有效性条件下内源性空间注意对视听觉整合的影响。结果发现, 当线索有效性为50%时(实验1), 有效线索位置和无效线索位置的视听觉整合效应没有显著差异; 当线索有效性为80%时(实验2), 有效线索位置的视听觉整合效应显著大于无效线索位置的视听觉整合效应。结果表明, 线索有效性不同时, 内源性空间注意对视听觉整合产生了不同的影响, 高线索有效性条件下内源性空间注意能够促进视听觉整合效应。
图/表 7
图1实验刺激示例图和实验流程图 (a)为靶刺激呈现位置的示意图; 图(b)为单个试次的流程图。图(b)中视觉线索(中央箭头)指向左侧, 靶刺激(视听觉)呈现在左侧(即, 有效线索位置), 要求被试对靶刺激进行既快又准的定位反应。其中, 靶刺激(V/A/AV)分别代表视觉(visual)、听觉(auditory)和视听觉(audiovisual)通道目标。ISI是指刺激间的时间间隔(inter-stimulus interval)。ITI是指试次间的时间间隔(inter-trial interval)。
图1实验刺激示例图和实验流程图 (a)为靶刺激呈现位置的示意图; 图(b)为单个试次的流程图。图(b)中视觉线索(中央箭头)指向左侧, 靶刺激(视听觉)呈现在左侧(即, 有效线索位置), 要求被试对靶刺激进行既快又准的定位反应。其中, 靶刺激(V/A/AV)分别代表视觉(visual)、听觉(auditory)和视听觉(audiovisual)通道目标。ISI是指刺激间的时间间隔(inter-stimulus interval)。ITI是指试次间的时间间隔(inter-trial interval)。表1每个实验中不同条件下的反应时(RT/ms)、正确率(ACC/%)(M ± SD)
| 靶刺激 通道类型 | 线索类型 | 实验1 | 实验2 | ||
|---|---|---|---|---|---|
| RT (ms) | ACC (%) | RT (ms) | ACC (%) | ||
| 视听觉 | 有效线索 | 302 ± 34 | 99.2 ± 1.2 | 289 ± 42 | 98.7 ± 1.2 |
| 无效线索 | 312 ± 36 | 99.2 ± 1.2 | 333 ± 51 | 95.8 ± 4.8 | |
| 听觉 | 有效线索 | 341 ± 39 | 97.1 ± 2.8 | 318 ± 51 | 97.1 ± 2.6 |
| 无效线索 | 353 ± 42 | 96.8 ± 2.7 | 363 ± 56 | 92.7 ± 4.5 | |
| 视觉 | 有效线索 | 334 ± 42 | 98.0 ± 2.3 | 324 ± 41 | 97.5 ± 2.3 |
| 无效线索 | 345 ± 47 | 97.7 ± 2.9 | 362 ± 46 | 95.4 ± 4.7 | |
表1每个实验中不同条件下的反应时(RT/ms)、正确率(ACC/%)(M ± SD)
| 靶刺激 通道类型 | 线索类型 | 实验1 | 实验2 | ||
|---|---|---|---|---|---|
| RT (ms) | ACC (%) | RT (ms) | ACC (%) | ||
| 视听觉 | 有效线索 | 302 ± 34 | 99.2 ± 1.2 | 289 ± 42 | 98.7 ± 1.2 |
| 无效线索 | 312 ± 36 | 99.2 ± 1.2 | 333 ± 51 | 95.8 ± 4.8 | |
| 听觉 | 有效线索 | 341 ± 39 | 97.1 ± 2.8 | 318 ± 51 | 97.1 ± 2.6 |
| 无效线索 | 353 ± 42 | 96.8 ± 2.7 | 363 ± 56 | 92.7 ± 4.5 | |
| 视觉 | 有效线索 | 334 ± 42 | 98.0 ± 2.3 | 324 ± 41 | 97.5 ± 2.3 |
| 无效线索 | 345 ± 47 | 97.7 ± 2.9 | 362 ± 46 | 95.4 ± 4.7 | |
图2实验1中不同条件下的反应时和relative MRE (rMRE) 注：图(a)为不同条件下的反应时, 反应时结果为平均中位数; 图(b)为不同条件下的rMRE。*** p < 0.001
图2实验1中不同条件下的反应时和relative MRE (rMRE) 注：图(a)为不同条件下的反应时, 反应时结果为平均中位数; 图(b)为不同条件下的rMRE。*** p < 0.001
图3实验1中不同线索类型的反应竞争模型分析和违反竞争模型的时间窗口 注：图(a)为不同线索类型的反应竞争模型分析, 图(a)中横轴表示显著违反竞争模型(实际AV累积概率(CPAV)显著大于竞争模型预测累积概率(CPRace model))的时间窗口; 图(b)为不同线索类型下违反竞争模型的时间窗口, 图(b)中*代表峰值, 箭头分别代表违反竞争模型时间窗口的起始时间、峰值发生时间。
图3实验1中不同线索类型的反应竞争模型分析和违反竞争模型的时间窗口 注：图(a)为不同线索类型的反应竞争模型分析, 图(a)中横轴表示显著违反竞争模型(实际AV累积概率(CPAV)显著大于竞争模型预测累积概率(CPRace model))的时间窗口; 图(b)为不同线索类型下违反竞争模型的时间窗口, 图(b)中*代表峰值, 箭头分别代表违反竞争模型时间窗口的起始时间、峰值发生时间。
图4实验2中不同条件下的反应时和relative MRE (rMRE) 注：图(a)为不同条件下的反应时, 反应时结果为平均中位数; 图(b)为不同条件下的rMRE。*** p < 0.001, * p < 0.05
图4实验2中不同条件下的反应时和relative MRE (rMRE) 注：图(a)为不同条件下的反应时, 反应时结果为平均中位数; 图(b)为不同条件下的rMRE。*** p < 0.001, * p < 0.05表2不同靶刺激通道类型下的Cueing effect结果对比
| 实验 | 条件 | M | 95% CI | t | p | ||
|---|---|---|---|---|---|---|---|
| 下限 | 上限 | ||||||
| 实验1 | Cueing effect | ||||||
| A | 12.23 | 5.97 | 18.49 | 3.97 | < 0.001 | ||
| AV | 9.44 | 5.48 | 13.41 | 4.84 | < 0.001 | ||
| V | 10.23 | 5.62 | 14.84 | 4.51 | < 0.001 | ||
| Cueing effect对比 | |||||||
| A vs. AV | 2.79 | -1.31 | 6.88 | 1.38 | 0.176 | ||
| A vs. V | 2.00 | -3.28 | 7.28 | 0.77 | 0.447 | ||
| AV vs. V | -0.78 | -4.37 | 2.80 | -0.45 | 0.659 | ||
| 实验2 | Cueing effect | ||||||
| A | 45.04 | 29.77 | 60.30 | 6.06 | < 0.001 | ||
| AV | 43.54 | 29.75 | 57.32 | 6.49 | < 0.001 | ||
| V | 38.70 | 26.00 | 51.41 | 6.26 | < 0.001 | ||
| Cueing effect对比 | |||||||
| A vs. AV | 1.50 | -3.14 | 6.14 | 0.67 | 0.512 | ||
| A vs. V | 6.33 | 0.21 | 12.46 | 2.13 | 0.043 | ||
| AV vs. V | 4.83 | -0.88 | 10.54 | 1.74 | 0.094 | ||
表2不同靶刺激通道类型下的Cueing effect结果对比
| 实验 | 条件 | M | 95% CI | t | p | ||
|---|---|---|---|---|---|---|---|
| 下限 | 上限 | ||||||
| 实验1 | Cueing effect | ||||||
| A | 12.23 | 5.97 | 18.49 | 3.97 | < 0.001 | ||
| AV | 9.44 | 5.48 | 13.41 | 4.84 | < 0.001 | ||
| V | 10.23 | 5.62 | 14.84 | 4.51 | < 0.001 | ||
| Cueing effect对比 | |||||||
| A vs. AV | 2.79 | -1.31 | 6.88 | 1.38 | 0.176 | ||
| A vs. V | 2.00 | -3.28 | 7.28 | 0.77 | 0.447 | ||
| AV vs. V | -0.78 | -4.37 | 2.80 | -0.45 | 0.659 | ||
| 实验2 | Cueing effect | ||||||
| A | 45.04 | 29.77 | 60.30 | 6.06 | < 0.001 | ||
| AV | 43.54 | 29.75 | 57.32 | 6.49 | < 0.001 | ||
| V | 38.70 | 26.00 | 51.41 | 6.26 | < 0.001 | ||
| Cueing effect对比 | |||||||
| A vs. AV | 1.50 | -3.14 | 6.14 | 0.67 | 0.512 | ||
| A vs. V | 6.33 | 0.21 | 12.46 | 2.13 | 0.043 | ||
| AV vs. V | 4.83 | -0.88 | 10.54 | 1.74 | 0.094 | ||
图5实验2中不同线索类型的反应竞争模型分析和违反竞争模型的时间窗口 注：图(a)为不同线索类型条件下的反应竞争模型分析, 图(a)中横轴表示显著违反竞争模型(实际AV累积概率(CPAV)显著大于竞争模型预测累积概率(CPRace model))的时间窗口; 图(b)为不同线索类型下违反竞争模型的时间窗口, 图(b)中*代表峰值, 箭头分别代表违反竞争模型时间窗口的起始时间、峰值发生时间。
图5实验2中不同线索类型的反应竞争模型分析和违反竞争模型的时间窗口 注：图(a)为不同线索类型条件下的反应竞争模型分析, 图(a)中横轴表示显著违反竞争模型(实际AV累积概率(CPAV)显著大于竞争模型预测累积概率(CPRace model))的时间窗口; 图(b)为不同线索类型下违反竞争模型的时间窗口, 图(b)中*代表峰值, 箭头分别代表违反竞争模型时间窗口的起始时间、峰值发生时间。参考文献 48
| [1] | Arjona, A., Escudero, M., & Gómez, C. M. ( 2016). Cue validity probability influences neural processing of targets. Biological Psychology, 119, 171-183. |
| [2] | Baluch, F., & Itti, L. ( 2011). Mechanisms of top-down attention. Trends in Neurosciences, 34( 4), 210-224. URLpmid: 21439656 |
| [3] | Bertelson, P., Vroomen, J., de Gelder, B., & Driver, J. ( 2000). The ventriloquist effect does not depend on the direction of deliberate visual attention. Perception & Psychophysics, 62( 2), 321-332. |
| [4] | Buschman, T. J., & Miller, E. K. ( 2007). Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science, 315( 5820), 1860-1862. URLpmid: 17395832 |
| [5] | Busse, L., Roberts, K. C., Crist, R. E., Weissman, D. H., & Woldorff, M. G. ( 2005). The spread of attention across modalities and space in a multisensory object. Proceedings of the National Academy of Sciences of the United States of America, 102( 51), 18751-18756. |
| [6] | Carrasco, I. ( 2014). Gender gap in innovation: An institutionalist explanation. Management Decision, 52( 2), 410-424. |
| [7] | Chica, A. B., & Lupiá?ez, J. ( 2009). Effects of endogenous and exogenous attention on visual processing: An Inhibition of Return study. Brain Research, 1278, 75-85. |
| [8] | Dombert, P. L., Kuhns, A., Mengotti, P., Fink, G. R., & Vossel, S. ( 2016). Functional mechanisms of probabilistic inference in feature- and space-based attentional systems. Neuroimage, 142, 553-564. URLpmid: 27523448 |
| [9] | Eimer, M. ( 1997). Uninformative symbolic cues may bias visual-spatial attention: Behavioral and electrophysiological evidence. Biological Psychology, 46( 1), 67-71. URLpmid: 9255432 |
| [10] | Fairhall, S. L., & Macaluso, E. ( 2009). Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. European Journal of Neuroscience, 29( 6), 1247-1257. |
| [11] | Giard, M. H., & Peronnet, F. ( 1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11( 5), 473-490. URLpmid: 10511637 |
| [12] | Jonides, J. ( 1983). Towards a model of the mind's eye's movement. Bulletin of the Psychonomic Society, 21( 4), 247-250. |
| [13] | Koelewijn, T., Bronkhorst, A., & Theeuwes, J. ( 2010). Attention and the multiple stages of multisensory integration: A review of audiovisual studies. Acta Psychologica, 134( 3), 372-384. doi: 10.1016/j.actpsy.2010.03.010URLpmid: 20427031 |
| [14] | Kuhns, A. B., Dombert, P. L., Mengotti, P., Fink, G. R., & Vossel, S. ( 2017). Spatial attention, motor intention, and Bayesian cue predictability in the human brain. Journal of Neuroscience, 37( 21), 5334-5344. doi: 10.1523/JNEUROSCI.3255-16.2017URLpmid: 28450541 |
| [15] | Laurienti, P. J., Burdette, J. H., Maldjian, J. A., & Wallace, M. T. ( 2006). Enhanced multisensory integration in older adults. Neurobiology of Aging, 27( 8), 1155-1163. doi: 10.1016/j.neurobiolaging.2005.05.024URLpmid: 16039016 |
| [16] | Lewald, J., & Guski, R. ( 2003). Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. Cognitive Brain Research, 16( 3), 468-478. |
| [17] | Li, Q., Wu, J. L., & Touge, T. ( 2010). Audiovisual interaction enhances auditory detection in late stage: an event-related potential study. Neuroreport, 21( 3), 173-178. |
| [18] | Lunn, J., Sjoblom, A., Ward, J., Soto-Faraco, S., & Forster, S. ( 2019). Multisensory enhancement of attention depends on whether you are already paying attention. Cognition, 187, 38-49. URLpmid: 30825813 |
| [19] | McCracken, H. S., Murphy, B. A., Glazebrook, C. M., Burkitt, J. J., Karellas, A. M., & Yielder, P. C. ( 2019). Audiovisual multisensory integration and evoked potentials in young adults with and without attention-deficit/hyperactivity disorder. Frontiers in Human Neuroscience, 13, 95. doi: 10.3389/fnhum.2019.00095URLpmid: 30941026 |
| [20] | Mengotti, P., Boers, F., Dombert, P. L., Fink, G. R., & Vossel, S. ( 2018). Integrating modality-specific expectancies for the deployment of spatial attention. Scientific Reports, 8( 1), 1210. doi: 10.1038/s41598-018-19593-7URLpmid: 29352145 |
| [21] | Meredith, M. A., Nemitz, J. W., & Stein, B. E. ( 1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. Journal of Neuropathology and Experimental Neurology, 7( 10), 3215-3229. |
| [22] | Meredith, M. A., & Stein, B. E. ( 1986a). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Research, 365( 2), 350-354. doi: 10.1016/0006-8993(86)91648-3URLpmid: 3947999 |
| [23] | Meredith, M. A., & Stein, B. E. ( 1986b). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56( 3), 640-662. |
| [24] | Meyer, K. N., Du, F., Parks, E., & Hopfinger, J. B. ( 2018). Exogenous vs. endogenous attention: Shifting the balance of fronto-parietal activity. Neuropsychologia, 111, 307-316. doi: 10.1016/j.neuropsychologia.2018.02.006URLpmid: 29425803 |
| [25] | Miller, J. ( 1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14( 2), 247-279. |
| [26] | Miller, J. ( 1986). Timecourse of coactivation in bimodal divided attention. Perception & Psychophysics, 40( 5), 331-343. |
| [27] | Peelen, M. V., & Kastner, S. ( 2014). Attention in the real world: Toward understanding its neural basis. Trends in Cognitive Sciences, 18( 5), 242-250. |
| [28] | Posner, M. I. ( 1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32( 1), 3-25. |
| [29] | Raab, D. H. ( 1962). Statistical facilitation of simple reaction times. Transactions of the New York Academy of Sciences, 24( 5), 574-590. |
| [30] | Riggio, L., & Kirsner, K. ( 1997). The relationship between central cues and peripheral cues in covert visual orientation. Perception & Psychophysics, 59( 6), 885-899. URLpmid: 9270363 |
| [31] | Santangelo, V., Fagioli, S., & Macaluso, E. ( 2010). The costs of monitoring simultaneously two sensory modalities decrease when dividing attention in space. Neuroimage, 49( 3), 2717-2727. URLpmid: 19878728 |
| [32] | Santangelo, V., Ho, C., & Spence, C. ( 2008). Capturing spatial attention with multisensory cues. Psychonomic Bulletin & Review, 15( 2), 398-403. doi: 10.3758/pbr.15.2.398URLpmid: 18488658 |
| [33] | Santangelo, V., & Spence, C. ( 2007). Multisensory cues capture spatial attention regardless of perceptual load. Journal of Experimental Psychology: Human Perception and Performance, 33( 6), 1311-1321. |
| [34] | Stein, B. E., Meredith, M. A., & Wallace, M. T. ( 1993). The visually responsive neuron and beyond: Multisensory integration in cat and monkey. Progress in Brain Research, 95, 79-90. |
| [35] | Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. ( 2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14( 9), 400-410. URLpmid: 20675182 |
| [36] | Talsma, D., & Woldorff, M. G. ( 2005). Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. Journal of Cognitive Neuroscience, 17( 7), 1098-1114. URLpmid: 16102239 |
| [37] | Tang, X. Y., Gao, Y. L., Yang, W. P., Ren, Y. N., Wu, J. L., Zhang, M., & Wu, Q. ( 2019). Bimodal-divided attention attenuates visually induced inhibition of return with audiovisual targets. Experimental Brain Research, 237( 4), 1093-1107. |
| [38] | Ulrich, R., Miller, J., & Schr?ter, H. ( 2007). Testing the race model inequality: An algorithm and computer programs. Behavioural Brain Research, 39( 2), 291-302. |
| [39] | van den Brink, R. L., Cohen, M. X., van der Burg, E., Talsma, D., Vissers, M. E., & Slagter, H. A. ( 2014). Subcortical, modality-specific pathways contribute to multisensory processing in humans. Cerebral Cortex, 24( 8), 2169-2177. |
| [40] | van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. ( 2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34( 5), 1053-1065. |
| [41] | van der Burg, E., Talsma, D., Olivers, C. N. L., Hickey, C., & Theeuwes, J. ( 2011). Early multisensory interactions affect the competition among multiple visual objects. Neuroimage, 55( 3), 1208-1218. URLpmid: 21195781 |
| [42] | van der Stigchel, S., Meeter, M., & Theeuwes, J. ( 2007). Top-down influences make saccades deviate away: The case of endogenous cues. Acta Psychologica, 125( 3), 279-290. URLpmid: 17022930 |
| [43] | van der Stoep, N., van der Stigchel, S., & Nijboer, T. C. W. ( 2015). Exogenous spatial attention decreases audiovisual integration. Attention Perception & Psychophysics, 77( 2), 464-482. |
| [44] | van der Stoep, N., van der Stigchel, S., Nijboer, T. C. W., & Spence, C. ( 2017). Visually induced inhibition of return affects the integration of auditory and visual information. Perception, 46( 1), 6-17. |
| [45] | Vossel, S., Mathys, C., Daunizeau, J., Bauer, M., Driver, J., Friston, K. J., & Stephan, K. E. ( 2014). Spatial attention, precision, and Bayesian inference: A study of saccadic response speed. Cerebral Cortex, 24( 6), 1436-1450. doi: 10.1093/cercor/bhs418URLpmid: 23322402 |
| [46] | Vossel, S., Mathys, C., Stephan, K. E., & Friston, K. J. ( 2015). Cortical coupling reflects Bayesian belief updating in the deployment of spatial attention. Journal of Neuroscience, 35( 33), 11532-11542. doi: 10.1523/JNEUROSCI.1382-15.2015URLpmid: 26290231 |
| [47] | Vossel, S., Thiel, C. M., & Fink, G. R. ( 2006). Cue validity modulates the neural correlates of covert endogenous orienting of attention in parietal and frontal cortex. Neuroimage, 32( 3), 1257-1264. |
| [48] | Zou, H., Müller, H. J., & Shi, Z. ( 2012). Non-spatial sounds regulate eye movements and enhance visual search. Journal of Vision, 12( 5), 2. URLpmid: 22562709 |
相关文章 4
| [1] | 唐晓雨, 孙佳影, 彭姓. 双通道分配性注意对视听觉返回抑制的影响[J]. 心理学报, 2020, 52(3): 257-268. |
| [2] | 彭姓,常若松,李奇,王爱君,唐晓雨. 不同SOA下视觉返回抑制对视听觉整合的调节作用[J]. 心理学报, 2019, 51(7): 759-771. |
| [3] | 孙远路,胡中华,张瑞玲,寻茫茫,刘强,张庆林. 多感觉整合测量范式中存在的影响因素探讨[J]. 心理学报, 2011, 43(11): 1239-1246. |
| [4] | 陈文锋, 焦书兰. 客体注意在等级模式加工中的作用[J]. 心理学报, 2003, 35(增刊): 51-63. |
PDF全文下载地址:
http://journal.psych.ac.cn/xlxb/CN/article/downloadArticleFile.do?attachType=PDF&id=4739
