), 肖焕禹3 1 上海工程技术大学体育教学部, 上海 201620
2 同济大学国际足球学院, 上海 200092
3 上海体育学院休闲学院, 上海 200438
收稿日期:2019-10-24出版日期:2020-08-15发布日期:2020-06-28通讯作者:王泽军E-mail:ddbbt@126.com基金资助:* 国家社科基金青年项目(17CTY019)Dual system theory of physical activity: A reinforcement learning perspective
CHU Xin-Yu1,3, WANG Ze-Jun2(), XIAO Huan-Yu3 1 Shanghai University of Engineering Science, Department of Physical Education, Shanghai 201620, China
2 Tongji University, International College of Football, Shanghai 200092, China
3 Shanghai University of Sport, School of Leisure Sport, Shanghai 200438, China
Received:2019-10-24Online:2020-08-15Published:2020-06-28Contact:WANG Ze-Jun E-mail:ddbbt@126.com摘要/Abstract
摘要： 以理性决策为基础的锻炼行为理论被认为是理解身体活动的主导体系, 它提供了与身体活动相关的认知构念作为有价值的信息。基于社会生态模型设计的行为干预措施, 因表现出了更好的效果而备受关注。近期研究表明, 积极的运动认知和当前体育环境都没能很好地促进个人锻炼习惯的养成, 因此有必要探索新的理论体系来阐明个人锻炼习惯的形成机制。解释身体活动的最新体系是双系统理论, 由于其考虑了身体活动的无意识和快乐决定因素, 有望提供一个更广泛的动机视角。一方面, 多个有代表性的身体活动双系统模型, 从简单的自发路径, 到情境线索与锻炼习惯, 再到突出自动情感评价作用的复杂概念模型, 阐明了系统1的构建, 结合锻炼行为理论所关注的系统2, 为模型的构建提供了依据。另一方面, 通过对双系统的竞争、协调和层级控制原则的分析, 为模型的控制提供了建议。经典的强化学习框架解释了双系统模型的构建与控制原则:在模型的构建方面, 无模型与基于模型的强化学习分别表示系统1和系统2。在模型的控制方面, Dyna协作架构与分层强化学习, 为身体活动可能是一种相互协作、分层执行的复杂行动组合提供了合理解释。最后提出强化学习视角下锻炼者-体育环境的互动模式, 试图从一个全新的角度探讨锻炼行为。
图/表 7
图1突出自动情感评价作用的身体活动双系统概念模型 资料来源:Conroy和Berry (2017)
图1突出自动情感评价作用的身体活动双系统概念模型 资料来源:Conroy和Berry (2017)
图2身体活动与习惯养成模型 资料来源:Aarts et al. (1997)
图2身体活动与习惯养成模型 资料来源:Aarts et al. (1997)
图3熟虑与习惯促进身体活动之间的建议转换 资料来源:Rhodes和Rebar (2018)
图3熟虑与习惯促进身体活动之间的建议转换 资料来源:Rhodes和Rebar (2018)
图4强化学习中智能体-环境的互动 资料来源:Sutton和Barto (2018)
图4强化学习中智能体-环境的互动 资料来源:Sutton和Barto (2018)
图5Dyna协作架构 资料来源:Gershman et al. (2014)
图5Dyna协作架构 资料来源:Gershman et al. (2014)
图6基于模型的搜索与无模型的目标选择 资料来源:Daw (2015)
图6基于模型的搜索与无模型的目标选择 资料来源:Daw (2015)
图7强化学习视角下锻炼者-体育环境的互动
图7强化学习视角下锻炼者-体育环境的互动参考文献 80
| [1] | 艾炎, 胡竹菁. (2018). 推理判断中双重加工过程的协作与转换机制. 心理科学进展, 26(10), 1794-1806. |
| [2] | 段文婷, 江光荣. (2008). 计划行为理论述评. 心理科学进展, 16(2), 315-320. |
| [3] | 冯玉娟, 毛志雄. (2014). 高中生身体活动意向和行为的促进策略:自我决定动机对TPB的贡献. 体育科学, 34(8), 64-69. |
| [4] | 郭强, 汪晓赞, 蒋健保. (2017). 我国儿童青少年身体活动与久坐行为模式特征的研究. 体育科学, 37(7), 17-21. |
| [5] | 韩慧, 郑家鲲. (2016). 西方国家青少年体力活动相关研究述评——基于社会生态学视角的分析. 体育科学, 36(5), 62-70. |
| [6] | 沈梦英, 毛志雄, 张一民. (2010). 中国成年人锻炼行为的影响因素——HAPA与TPB两个理论模型的整合. 体育科学, 30(12), 48-54. |
| [7] | 司琦. (2007). 身体活动的行为科学理论综述. 体育科学, 27(9), 72-80. |
| [8] | 许昭, 毛志雄. (2015). 身体活动熟虑-冲动双系统模型的构建与检验. 体育科学, 35(8), 16-23. |
| [9] | 叶娜, 佐斌. (2007). 联想-命题评价模型——态度改变的新解释. 心理科学进展, 15(5), 834-839. |
| [10] | 张加林, 唐炎, 胡月英. (2017). 我国儿童青少年体育环境特征与存在问题研究. 体育科学, 37(3), 21-34. |
| [11] | 张业安. (2018). 青少年体质健康促进的媒介责任: 概念、目标及机制. 体育科学, 38(6), 14-26. |
| [12] | 张展嘉, 王正珍, 于洪军, 陈蔚云. (2018). 第65届美国运动医学会年会关于身体活动促进的研究热点与进展综述. 北京体育大学学报, 41(8), 72-76. |
| [13] | 周文吉, 俞扬. (2017). 分层强化学习综述. 智能系统学报, 12(5), 590-594. |
| [14] | 朱为模. (2009). 从进化论、社会-生态学角度谈环境、步行与健康. 体育科研, 30(5), 12-16. |
| [15] | Aarts, H., Paulussen, T., & Schaalma, H. (1997). Physical exercise habit: On the conceptualization and formation of habitual health behaviours. Health Education Research, 12(3), 363-374. URLpmid: 10174219 |
| [16] | Aarts, H., Verplanken, B., & van Knippenberg, A. (1997). Habit and information use in travel mode choices. Acta Psychologica, 96(1-2), 1-14. |
| [17] | Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior & Human Decision Processes, 50(2), 179-211. |
| [18] | Ajzen, I. (2011). The theory of planned behaviour: Reactions and reflections. Psychology & Health, 26(9), 1113-1127. doi: 10.1080/08870446.2011.613995URLpmid: 21929476 |
| [19] | Althoff, T., Sosi?, R., Hicks, J. L., King, A. C., Delp, S. L., & Leskovec, J. (2017). Large-scale physical activity data reveal worldwide activity inequality. Nature, 547(7663), 336-339. doi: 10.1038/nature23018URLpmid: 28693034 |
| [20] | Antoniewicz, F., & Brand, R. (2014). Automatic evaluations and exercise setting preference in frequent exercisers. Journal of Sport & Exercise Psychology, 36(6), 631-636. doi: 10.1123/jsep.2014-0033URLpmid: 25602145 |
| [21] | Antoniewicz, F., & Brand, R. (2016a). Learning to like exercising: Evaluative conditioning changes automatic evaluations of exercising and influences subsequent exercising behavior. Journal of Sport & Exercise Psychology, 38(2), 138-148. doi: 10.1123/jsep.2015-0125URLpmid: 27385674 |
| [22] | Antoniewicz, F., & Brand, R. (2016b). Dropping out or keeping up? Early-dropouts, late-dropouts, and maintainers differ in their automatic evaluations of exercise already before a 14-week exercise course. Frontiers in Psychology, 7, 838. URLpmid: 27313559 |
| [23] | Aral, S., & Nicolaides, C. (2017). Exercise contagion in a global social network. Nature Communication, 8, 14753. |
| [24] | Betsch, T., Haberstroh, S., Glockner, A., Haar, T., & Fiedler, K. (2001). The effects of routine strength on adaptation and information search in recurrent decision making. Organizational Behavior and Human Decision Processes, 84(1), 23-53. |
| [25] | Biddle, S. J. H., Hagger, M. S., Chatzisarantis, N. L. D., & Lippke, S. (2007). Theoretical frameworks in exercise psychology. In G. Tenenbaum & R. C. Eklund (Eds.). Handbook of sport psychology (3rd ed., pp. 537-559). Hoboken, NJ: Wiley. |
| [26] | Bluemke, M., Brand, R., Schweizer, G., & Kahlert, D. (2010). Exercise might be good for me, but I don’t feel good about it: Do automatic associations predict exercise behavior? Journal of Sport & Exercise Psychology, 32(2), 137-153. doi: 10.1123/jsep.32.2.137URLpmid: 20479475 |
| [27] | Botvinick, M. M., Niv, Y., & Barto, A. G. (2009). Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition, 113(3), 262-280. doi: 10.1016/j.cognition.2008.08.011URLpmid: 18926527 |
| [28] | Botvinick, M., & Weinstein, A. (2014). Model-based hierarchical reinforcement learning and human action control. Philosophical Transactions of the Royal Society B, 369(1655), 20130480. |
| [29] | Brand, R., & Antoniewicz, F. (2016). Affective evaluations of exercising: The role of automatic-reflective evaluation discrepancy. Journal of Sport & Exercise Psychology, 38(6), 631-638. doi: 10.1123/jsep.2016-0171URLpmid: 28033023 |
| [30] | Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, Massachusetts: Harvard University Press. |
| [31] | Calitri, R., Lowe, R., Eves, F. F., & Bennett, P. (2009). Associations between visual attention, implicit and explicit attitude and behaviour for physical activity. Psychology & Health, 24(9), 1105-1123. doi: 10.1080/08870440802245306URLpmid: 20205048 |
| [32] | Chevance, G., Caudroit, J., Romain, A. J., & Boiche, J. (2017). The adoption of physical activity and eating behaviors among persons with obesity and in the general population: The role of implicit attitudes within the Theory of Planned Behavior. Psychology, Health & Medicine, 22(3), 319-324. doi: 10.1080/13548506.2016.1159705URLpmid: 26952746 |
| [33] | Conroy, D. E., & Berry, T. R. (2017). Automatic affective evaluations of physical activity. Exercise & Sport Sciences Reviews, 45(4), 230-237. URLpmid: 28704217 |
| [34] | Cooper, R. P., & Shallice, T. (2006). Hierarchical schemas and goals in the control of sequential behavior. Psychological Review, 113(4), 887-916. URLpmid: 17014307 |
| [35] | Cushman, F., & Morris, A. (2015). Habitual control of goal selection in humans. Proceedings of the National Academy of Science of the United States of America, 112(45), 13817-13822. |
| [36] | Daw, N. D. (2015). Of goals and habits. Proceedings of the National Academy of Science of the United States of America, 112(45), 13749-13750. |
| [37] | Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69(6), 1204-1215. doi: 10.1016/j.neuron.2011.02.027URLpmid: 21435563 |
| [38] | Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704-1711. doi: 10.1038/nn1560URL |
| [39] | Dezfouli, A., & Balleine, B. W. (2012). Habits, action sequences and reinforcement learning. European Journal of Neuroscience, 35(7), 1036-1051. doi: 10.1111/j.1460-9568.2012.08050.xURL |
| [40] | Dezfouli, A., & Balleine, B. W. (2013). Actions, action sequences and habits: Evidence that goal-directed and habitual action control are hierarchically organized. PLoS Computational Biology, 9(12), e1003364. doi: 10.1371/journal.pcbi.1003364URLpmid: 24339762 |
| [41] | Dolan, R. J., & Dayan, P. (2013). Goals and habits in the brain. Neuron, 80(2), 312-325. URLpmid: 24139036 |
| [42] | Ekkekakis, P. (2017). People have feelings! Exercise psychology in paradigmatic transition. Current Opinion in Psychology, 16, 84-88. doi: 10.1016/j.copsyc.2017.03.018URLpmid: 28813362 |
| [43] | Ekkekakis, P., Hargreaves, E. A., & Parfitt, G. (2013). Invited Guest Editorial: Envisioning the next fifty years of research on the exercise-affect relationship. Psychology of Sport & Exercise, 14(5), 751-758. |
| [44] | Endrighi, R., Basen-Engquist, K., Szeto, E., Perkins, H., Baum, G., Cox-Martin, M., … Waters, A. J. (2016). Self-reported and automatic cognitions are associated with exercise behavior in cancer survivors. Health Psychology, 35(8), 824-828. URLpmid: 27505202 |
| [45] | Evans, J., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8(3), 223-241. doi: 10.1177/1745691612460685URLpmid: 26172965 |
| [46] | Gardner, B., de Bruijn, G.-J., & Lally, P. (2011). A systematic review and meta-analysis of applications of the self-report habit index to nutrition and physical activity behaviours. Annals of Behavioral Medicine, 42(2), 174-187. URLpmid: 21626256 |
| [47] | Gardner, B., Phillips, L. A., & Judah, G. (2016). Habitual instigation and habitual execution: Definition, measurement, and effects on behaviour frequency. British Journal of Health Psychology, 21(3), 613-630. doi: 10.1111/bjhp.12189URLpmid: 26991427 |
| [48] | Gawronski, B., & Bodenhausen, G. V. (2006). Associative and propositional processes in evaluation: An integrative review of implicit and explicit attitude change. Psychological Bulletin, 132(5), 692-731. URLpmid: 16910748 |
| [49] | Gershman, S. J., Markman, A. B., & Otto, A. R. (2014). Retrospective revaluation in sequential decision making: A tale of two systems. Journal of Experimental Psychology: General, 143(1), 182-194. |
| [50] | Graybiel, A. M. (2008). Habits, rituals, and the evaluative brain. Annual Review of Neuroscience, 31, 359-387. doi: 10.1146/annurev.neuro.29.051605.112851URLpmid: 18558860 |
| [51] | Hagger, M. S., & Chatzisarantis, N. L. (2014). An integrated behavior change model for physical activity. Exercise & Sport Sciences Reviews, 42(2), 62-69. URLpmid: 24508739 |
| [52] | Kaushal, N., Rhodes, R. E., Meldrum, J. T., & Spence, J. C. (2017). The role of habit in different phases of exercise. British Journal of Health Psychology, 22(3), 429-448. doi: 10.1111/bjhp.12237URLpmid: 28374463 |
| [53] | Keatley, D., Clarke, D. D., & Hagger, M. S. (2012). Investigating the predictive validity of implicit and explicit measures of motivation on condom use, physical activity, and healthy eating. Psychology & Health, 27(5), 550-569. URLpmid: 21895458 |
| [54] | Keramati, M., Dezfouli, A., & Piray, P. (2011). Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Computational Biology, 7(5), e1002055. URLpmid: 21637741 |
| [55] | Kiviniemi, M. T., Voss-Humke, A. M., & Seifert, A. L. (2007). How do I feel about the behavior? The interplay of affective associations with behaviors and cognitive beliefs as influences on physical activity behavior. Health Psychology, 26(2), 152-158. URLpmid: 17385966 |
| [56] | Kurdi, B., Gershman, S. J., & Banaji, M. R. (2019). Model-free and model-based learning processes in the updating of explicit and implicit evaluations. Proceedings of the National Academy of Science of the United States of America, 116(13), 6035-6044. |
| [57] | Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014). Neural computations underlying arbitration between model-based and model-free learning. Neuron, 81(3), 687-699. doi: 10.1016/j.neuron.2013.11.028URLpmid: 24507199 |
| [58] | Maher, J. P., & Conroy, D. E. (2015). Habit strength moderates the effects of daily action planning prompts on physical activity but not sedentary behavior. Journal of Sport & Exercise Psychology, 37(1), 97-107. doi: 10.1123/jsep.2014-0258URLpmid: 25730895 |
| [59] | Markland, D., Hall, C. R., Duncan, L. R., & Simatovic, J. (2015). The effects of an imagery intervention on implicit and explicit exercise attitudes. Psychology of Sport and Exercise, 17, 24-31. doi: 10.1016/j.psychsport.2014.11.007URL |
| [60] | Marteau, T. M., Hollands, G. J., & Fletcher, P. C. (2012). Changing human behavior to prevent disease: The importance of targeting automatic processes. Science, 337(6101), 1492-1495. doi: 10.1126/science.1226918URLpmid: 22997327 |
| [61] | McEachan, R. R. C., Conner, M., Taylor, N. J., & Lawton, R. J. (2011). Prospective prediction of health-related behaviours with the theory of planned behaviour: A meta-analysis. Health Psychology Review, 5(2), 97-144. doi: 10.1080/17437199.2010.521684URL |
| [62] | Neal, D. T., Wood, W., Labrecque, J. S., & Phillippa, L. (2012). How do habits guide behavior? Perceived and actual triggers of habits in daily life. Journal of Experimental Social Psychology, 48(2), 492-498. doi: 10.1016/j.jesp.2011.10.011URL |
| [63] | Phillips, L. A., & Gardner, B. (2016). Habitual exercise instigation (vs. execution) predicts healthy adults’ exercise frequency. Health Psychology, 35(1), 69-77. |
| [64] | Rebar, A. L., Dimmock, J. A., Jackson, B., Rhodes, R. E., Kates, A., Starling, J., & Vandelanotte, C. (2016). A systematic review of the effects of non-conscious regulatory processes in physical activity. Health Psychology Review, 10(4), 395-407. URLpmid: 27118430 |
| [65] | Rebar, A. L., Elavsky, S., Maher, J. P., Doerksen, S. E., & Conroy, D. E. (2014). Habits predict physical activity on days when intentions are weak. Journal of Sport & Exercise Psychology, 36(2), 157-165. URLpmid: 24686952 |
| [66] | Rhodes, R. E. (2014). Adding depth to the next generation of physical activity models. Exercise & Sport Sciences Reviews, 42(2), 43-44. doi: 10.1249/JES.0000000000000014URLpmid: 24657940 |
| [67] | Rhodes, R. E., Fiala, B., & Conner, M. (2009). A review and meta-analysis of affective judgments and physical activity in adult populations. Annals of Behavioral Medicine, 38(3), 180-204. doi: 10.1007/s12160-009-9147-yURLpmid: 20082164 |
| [68] | Rhodes, R. E., Janssen, I., Bredin, S. S. D., Warburton, D. E. R., & Bauman, A. (2017). Physical activity: Health impact, prevalence, correlates and interventions. Psychology & Health, 32(8), 942-975. doi: 10.1080/08870446.2017.1325486URLpmid: 28554222 |
| [69] | Rhodes, R. E., & Rebar, A. L. (2018). Physical activity habit: Complexities and controversies. In Verplanken, B. (Eds.), The psychology of habit: Theory, mechanisms, change, and contexts |
| [70] | Rhodes, R. E., Saelens, B. E., & Sauvage-Mar, C. (2018). Understanding physical activity through interactions between the built environment and social cognition: A systematic review. Sports Medicine, 48(8), 1893-1912. |
| [71] | Sala, M., Baldwin, A. S., & Williams, D. M. (2016). Affective and cognitive predictors of affective response to exercise: Examining unique and overlapping variance. Psychology of Sport & Exercise, 27, 1-8. |
| [72] | Sniehotta, F. F., Presseau, J., & Araujo-Soares, V. (2014). Time to retire the theory of planned behaviour. Health Psychology Review, 8(1), 1-7. URLpmid: 25053004 |
| [73] | Strack, F., & Deutsch, R. (2004). Reflective and impulsive determinants of social behavior. Personality and Social Psychology Review, 8(3), 220-247. URLpmid: 15454347 |
| [74] | Sutton, R. S., & Barto, A. G. (2018) . Reinforcement learning: An introduction Cambridge, Massachusetts: Massachusetts Institute of Technology Press An introduction. Cambridge, Massachusetts: Massachusetts Institute of Technology Press. |
| [75] | Sutton, R. S., Precup, D., & Singh, S. (1999). Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1-2), 181-211. |
| [76] | Tappe, K. A., & Glanz, K. (2013). Measurement of exercise habits and prediction of leisure-time activity in established exercise. Psychology, Health & Medicine, 18(5), 601-611. |
| [77] | Verplanken, B., Aarts, H., & van Knippenberg, A. (1997). Habit, information acquisition, and the process of making travel mode choices. European Journal of Social Psychology, 27(5), 539-560. |
| [78] | Verplanken, B., & Melkevik, O. (2008). Predicting habit: The case of physical exercise. Psychology of Sport & Exercise, 9(1), 15-26. |
| [79] | Williams, D. M., & Evans, D. R. (2014). Current emotion research in health behavior science. Emotion Review, 6(3), 277-287. |
| [80] | Wood, W., & Runger, D. (2016). Psychology of habit. Annual Reviews of Psychology, 67(1), 289-314. |
相关文章 15
| [1] | 张玮玮, 朱莉琪. 同伴对青少年冒险行为的影响及其作用机制[J]. 心理科学进展, 2021, 29(8): 1462-1471. |
| [2] | 黎穗卿, 陈新玲, 翟瑜竹, 张怡洁, 章植鑫, 封春亮. 人际互动中社会学习的计算神经机制[J]. 心理科学进展, 2021, 29(4): 677-696. |
| [3] | 刘毅, 王君起, 邬辛佳. 双系统模型视角下的罪犯自我控制[J]. 心理科学进展, 2020, 28(8): 1379-1391. |
| [4] | 陈乐乐, 黄蓉, 贾世伟. 反馈相关负波与成瘾[J]. 心理科学进展, 2020, 28(6): 959-968. |
| [5] | 周浩, 盛欣怡. 管理者征求建言的内在机制[J]. 心理科学进展, 2019, 27(12): 1980-1987. |
| [6] | 李丹阳, 李鹏, 李红. 反馈负波及其近10年理论解释[J]. 心理科学进展, 2018, 26(9): 1642-1650. |
| [7] | 王建明, 吴龙昌. 亲环境行为研究中情感的类别、维度及其作用机理[J]. 心理科学进展, 2015, 23(12): 2153-2166. |
| [8] | 赵春黎. 社会从众的生物学基础[J]. 心理科学进展, 2015, 23(11): 1956-1965. |
| [9] | 王良秋;孙婷婷;董妍;贾燕飞;安然. 道路交通违法行为研究：基于计划行为理论的视角[J]. 心理科学进展, 2015, 23(11): 2009-2019. |
| [10] | 蒋长好;陈婷婷. 身体活动对情绪的影响及其脑机制[J]. 心理科学进展, 2014, 22(12): 1889-1898. |
| [11] | 黄桢炜;刘力;谭旭运;郑雯雯;张冀琦. 集体主义与贿赂[J]. 心理科学进展, 2014, 22(12): 1944-1952. |
| [12] | 周正;辛自强. 数学能力与决策的关系：个体差异的视角[J]. 心理科学进展, 2012, 20(4): 542-551. |
| [13] | 白蓉;范会勇;张进辅. 身体活动对老年认知功能的影响[J]. 心理科学进展, 2011, 19(12): 1777-1787. |
| [14] | 沈杰;王詠. 品牌社区的形成与发展：社会认同和计划行为理论的视角[J]. 心理科学进展, 2010, 18(6): 1018-1024. |
| [15] | 简丹丹; 段锦云; 朱月龙. 创业意向的构思测量、影响因素及理论模型[J]. 心理科学进展, 2010, 18(1): 162-169. |
PDF全文下载地址:
http://journal.psych.ac.cn/xlkxjz/CN/article/downloadArticleFile.do?attachType=PDF&id=5134
