), 何明霜1, 何洁1 1四川师范大学教育科学学院, 成都 610066
2德阳市东汽小学, 四川 德阳 618000
收稿日期:2020-02-06出版日期:2020-12-15发布日期:2020-10-26通讯作者:毛秀珍E-mail:maomao_wanli@163.com基金资助:国家自然科学基金青年项目(31400897)Item selection methods for cognitive diagnostic computerized adaptive testing
TANG Qian1,2, MAO Xiuzhen1(), HE Mingshuang1, HE Jie1 1Institute of Educational Science, Sichuan Normal University, Chengdu 610066, China
2Dongqi Primary School, Deyang, 618000, China
Received:2020-02-06Online:2020-12-15Published:2020-10-26Contact:MAO Xiuzhen E-mail:maomao_wanli@163.com摘要/Abstract
摘要： 随着认知诊断计算机化自适应测验(cognitive diagnostic computerized adaptive testing, CD-CAT)理论与实践的发展, 兼顾知识状态与能力的双目标CD-CAT逐渐受到重视。选题策略是CAT的核心, 通过梳理传统CD-CAT和双目标CD-CAT选题策略的研究, 并对它们的特点、关系及表现进行介绍和评析。最后, 基于认知诊断模型与CAT实践发展指出未来应加强一般化认知模型、复杂测验条件认知诊断模型下选题策略的研究; 应开发双目标诊断测验的项目和测验特征指标; 还应加强非参数选题方法和CD-CAT的实践应用研究。
图/表 2
表1传统CD-CAT选题策略汇总表
| 分类标准 | 特点 | 具体方法 | 适用情景 |
|---|---|---|---|
| 基础选题指标 | 反应分布信息量指标 | KL、PWKL、HKL、MPWKL | 提高测量精度 |
| KS后验分布信息量指标 | SHE、MI | ||
| 基于项目、被试特征选题 | HA、GDI、PWCDI、PWADI | ||
| 加权选题方法 | 基于区分度、KS后验概率加权 | CDIPWKL、ADIPWKL、PPWKL、PHKL | |
| 优先指标加权：MPI及变式$MP{{I}_{i}}(i=1,2,3,4)$ | 对信息量(KL、PWKL、MPWKL、MI)加权; $MP{{I}_{\text{1}}}\cdot CDI$、$MP{{I}_{\text{2}}}\cdot CDI$ | 平衡属性测量次数 | |
| 属性偏差指标加权 | WDKL、SWDKL | ||
| 结合多种思路 | 运用多个步骤或方法 | RT、RPG、分层方法、优先指标法、P-SWDKL | 平衡项目曝光率 |
表1传统CD-CAT选题策略汇总表
| 分类标准 | 特点 | 具体方法 | 适用情景 |
|---|---|---|---|
| 基础选题指标 | 反应分布信息量指标 | KL、PWKL、HKL、MPWKL | 提高测量精度 |
| KS后验分布信息量指标 | SHE、MI | ||
| 基于项目、被试特征选题 | HA、GDI、PWCDI、PWADI | ||
| 加权选题方法 | 基于区分度、KS后验概率加权 | CDIPWKL、ADIPWKL、PPWKL、PHKL | |
| 优先指标加权：MPI及变式$MP{{I}_{i}}(i=1,2,3,4)$ | 对信息量(KL、PWKL、MPWKL、MI)加权; $MP{{I}_{\text{1}}}\cdot CDI$、$MP{{I}_{\text{2}}}\cdot CDI$ | 平衡属性测量次数 | |
| 属性偏差指标加权 | WDKL、SWDKL | ||
| 结合多种思路 | 运用多个步骤或方法 | RT、RPG、分层方法、优先指标法、P-SWDKL | 平衡项目曝光率 |
表2双目标CD-CAT选题策略汇总表
| 两阶段选题 | 信息量加权平均 | 约束加权信息量 |
|---|---|---|
| 两步法 | 直接加权平均：DI | Q矩阵、KL信息控制指标加权：${{P}_{\text{1}}}\cdot FI(\hat{\theta }){{P}_{\text{2}}}\cdot FI(\hat{\theta }){{P}_{\text{3}}}\cdot FI(\hat{\theta })$ |
| 影子测验方法 | 统一量纲：ASI、ARI、MASI、MARI、DWI | 信息量乘积：IPA |
| 分布反应加权：JS |
表2双目标CD-CAT选题策略汇总表
| 两阶段选题 | 信息量加权平均 | 约束加权信息量 |
|---|---|---|
| 两步法 | 直接加权平均：DI | Q矩阵、KL信息控制指标加权：${{P}_{\text{1}}}\cdot FI(\hat{\theta }){{P}_{\text{2}}}\cdot FI(\hat{\theta }){{P}_{\text{3}}}\cdot FI(\hat{\theta })$ |
| 影子测验方法 | 统一量纲：ASI、ARI、MASI、MARI、DWI | 信息量乘积：IPA |
| 分布反应加权：JS |
参考文献 31
| [1] | 杜宣宣. (2010). 具有认知诊断功能的计算机化自适应测验的选题策略研究 (硕士学位论文). 江西师范大学, 南昌. |
| [2] | 郭磊, 郑蝉金, 边玉芳, 宋乃庆, 夏凌翔. (2016). 认知诊断计算机化自适应测验中新的选题策略:结合项目区分度指标. 心理学报, 48(7), 903-914. doi: 10.3724/SP.J.1041.2016.00903URL |
| [3] | 刘舒畅, 涂冬波, 蔡艳, 赵洋. (2018). 四种新的基于属性平衡的CD-CAT选题策略开发研究. 心理科学, 41(4), 976-981. |
| [4] | 罗照盛, 喻晓锋, 高椿雷, 李喻骏, 彭亚风, 王睿, 王钰彤. (2015). 基于属性掌握概率的认知诊断计算机化自适应测验选题策略. 心理学报, 47(5), 679-688. doi: 10.3724/SP.J.1041.2015.00679URL |
| [5] | 毛秀珍, 辛涛. (2013). 认知诊断CAT中项目曝光控制方法的比较. 心理学报, 45(6), 694-703. doi: 10.3724/SP.J.1041.2013.00694URL |
| [6] | 孙小坚, 王钰彤, 张世夷, 辛涛. (2019). 认知诊断计算机自适应测验中平衡属性收敛的新方法. 心理科学, 42(5), 1236-1244. |
| [7] | 汪文义, 丁树良, 宋丽红. (2014). 兼顾测验效率和题库使用率的CD-CAT选题策略. 心理科学, 37(1), 212-216. |
| [8] | 余丹, 潘奕娆, 丁树良, 杨庆红. (2011). 计算机化自适应诊断测验新的选题策略. 江西师范大学学报(自然科学版), 35(5), 548-550. |
| [9] | Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74(4), 619-632. doi: 10.1007/s11336-009-9123-2URL |
| [10] | Cheng, Y. (2010). Improving cognitive diagnostic computerized adaptive testing by balancing attribute coverage: The modified maximum global discrimination index method. Educational and Psychological Measurement, 70(6), 902-913. doi: 10.1177/0013164410366693URL |
| [11] | Cheng, Y. (2007). The dual information method for item selection in cognitive diagnostic computerized adaptive testing (Master’s thesis). University of Illinois at Urbana-Champaign. |
| [12] | Dai, B. Y., Zhang, M. Q., & Li, G. M. (2016). Exploration of item selection in dual-purpose cognitive diagnostic computerized adaptive testing: Based on the RRUM. Applied Psychological Measurement, 40(8), 625-640. doi: 10.1177/0146621616666008URLpmid: 29882535 |
| [13] | de la Torre, J., & Chiu,C. Y. (2016). A General method of empirical Q-Matrix validation. Psychometrika, 81(2), 253-273. doi: 10.1007/s11336-015-9467-8URLpmid: 25943366 |
| [14] | Henson, R., & Douglas, J. (2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29(4), 262-277. doi: 10.1177/0146621604272623URL |
| [15] | Henson, R., Roussos, L., Douglas, J., & He, X. (2008). Cognitive diagnostic attribute-level discrimination indices. Applied Psychological Measurement, 32(4), 275-288. doi: 10.1177/0146621607302478URL |
| [16] | Kang, H.-A., Zhang, S., & Chang, H.-H. (2017). Dual- objective item selection criteria in cognitive diagnostic computerized adaptive testing. Journal of Educational Measurement, 54(2), 165-183. doi: 10.1111/jedm.12139URL |
| [17] | Kaplan, M., de la Torre, J., & Barrada, J. R. (2015). New item selection methods for cognitive diagnosis computerized adaptive testing. Applied Psychological Measurement, 39(3), 167-188. doi: 10.1177/0146621614554650URLpmid: 29881001 |
| [18] | Kuo, B.-C., Pai, H.-S., & de la Torre, J. (2016). Modified cognitive diagnostic index and modified attribute-level discrimination index for test construction. Applied Psychological Measurement, 40(5), 315-330. doi: 10.1177/0146621616638643URLpmid: 29881056 |
| [19] | Lin., C.-J., & Chang, H.-H. (2018). Item selection criteria with practical constraints in cognitive diagnostic computerized adaptive testing. Educational and Psychological Measurement, 79(2), 335-357. doi: 10.1177/0013164418790634URLpmid: 30911196 |
| [20] | Liu, H. Y., You, X. F., Wang, W. Y., Ding, S. L., & Chang, H. H. (2013). The development of computerized adaptive testing with cognitive diagnosis for an English achievement test in China. Journal of classification, 30(2), 152-172. doi: 10.1007/s00357-013-9128-5URL |
| [21] | Lin, J. (1991). Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1), 145-151. doi: 10.1109/18.61115URL |
| [22] | McGlohen, M., & Chang, H.-H. (2008). Combining computer adaptive testing technology with cognitively diagnostic assessment. Behavior Research Methods, 40(3), 808-821. doi: 10.3758/BRM.40.3.808URL |
| [23] | Rupp, A. A., Templin, J., & Henson, R. A. . (2010). Diagnostic Measurement: Theory, Methods, and Applications New York: Guiford Press Theory, Methods, and Applications. New York: Guiford Press. |
| [24] | Swanson,L., & Stocking,M. L. (1993). A model and heuristic for solving very large item selection problems. Applied Psychological Measurement, 17(2), 151-166. doi: 10.1177/014662169301700205URL |
| [25] | Wang, C. (2013). Mutual information item selection method in cognitive diagnostic computerized adaptive testing with short test length. Educational and Psychological Measurement, 73(6), 1017-1035. doi: 10.1177/0013164413498256URL |
| [26] | Wang, C., Chang, H. -H., & Douglas, J. (2012). Combining CAT with cognitive diagnosis: A weighted item selection approach. Behavior Research Methods, 44(1), 95-109. doi: 10.3758/s13428-011-0143-3URL |
| [27] | Wang, C., Chang, H.-H., & Huebner, A. (2011). Restrictive stochastic item selection methods in cognitive diagnostic computerized adaptive testing. Journal of Educational Measurement, 48(3), 255-273. doi: 10.1111/jedm.2011.48.issue-3URL |
| [28] | Wang, C., Zheng, C., & Chang, H.-H. (2014). An enhanced approach to combine item response theory with cognitive diagnosis in adaptive testing. Journal of Educational Measurement, 51(4), 358-380. doi: 10.1111/jedm.12057URL |
| [29] | Xu, X. L., Chang, H. H., & Douglas, J. (2003). A simulation study to compare CAT strategies for cognitive diagnosis. Paper presented at the annual meeting of National Council on Measurement in Education, Montreal, Canada. |
| [30] | Zheng, C., & Chang, H.-H. (2016). High-efficiency response distribution-based item selection algorithms for short-length cognitive diagnostic computerized adaptive testing. Applied Psychological Measurement, 40(8), 608-624. doi: 10.1177/0146621616665196URLpmid: 29881073 |
| [31] | Zheng, C., He, G., & Gao, C. (2018). The information product methods: A unified approach to dual-purpose computerized adaptive testing. Applied Psychological Measurement, 42(4), 321-324. doi: 10.1177/0146621617730392URLpmid: 29882539 |
相关文章 9
| [1] | 张雪琴, 毛秀珍, 李佳. 基于CAT的在线标定:设计与方法[J]. 心理科学进展, 2020, 28(11): 1970-1978. |
| [2] | 高一珠, 陈孚, 辛涛, 詹沛达, 姜宇. 心理测量学模型在学习进阶中的应用：理论、途径和突破[J]. 心理科学进展, 2017, 25(9): 1623-1630. |
| [3] | 高旭亮;涂冬波;王芳;张龙;李雪莹. 可修改答案的计算机化自适应测验的方法[J]. 心理科学进展, 2016, 24(4): 654-664. |
| [4] | 陈孚;辛涛;刘彦楼;刘拓;田伟. 认知诊断模型资料拟合检验方法和统计量[J]. 心理科学进展, 2016, 24(12): 1946-1960. |
| [5] | 毛秀珍;辛涛. 多维计算机化自适应测验：模型、技术和方法[J]. 心理科学进展, 2015, 23(5): 907-918. |
| [6] | 陈平;张佳慧;辛涛. 在线标定技术在计算机化自适应测验中的应用[J]. 心理科学进展, 2013, 21(10): 1883-1892. |
| [7] | 唐小娟;丁树良;俞宗火. 计算机化自适应测验在认知诊断中的应用[J]. 心理科学进展, 2012, 20(4): 616-626. |
| [8] | 毛秀珍;辛涛. 计算机化自适应测验选题策略述评[J]. 心理科学进展, 2011, 19(10): 1552-1562. |
| [9] | 陈秋梅; 张敏强. 认知诊断模型发展及其应用方法述评[J]. 心理科学进展, 2010, 18(03): 522-529. |
PDF全文下载地址:
http://journal.psych.ac.cn/xlkxjz/CN/article/downloadArticleFile.do?attachType=PDF&id=5264
