基于条件随机域和语义类的中文组块分析方法

孙广路^1,2, 郎非³, 薛一波¹

1.清华大学信息技术研究院;2.哈尔滨理工大学计算机科学与技术学院;3.哈尔滨理工大学外国语学院



摘要:

为了解决中文组块分析精度不高和未利用词的语义信息的问题,提出了一种基于条件随机域模型和语义类的中文组块分析方法.该方法通过研究中文组块分析任务及其序列化特性,采用条件随机域模型融合不同类型特征,克服标记偏置问题,将语义词典中抽取的语义类特征应用到中文组块分析中,提高分析精度.实验表明,该方法取得了F值为92.77%的中文组块分析性能,实验进一步还表明了特征模板的选取和训练语料的规模对于分析性能的影响.

关键词:  条件随机域  中文组块分析  特征模板  语义词典

DOI：10.11918/j.issn.0367-6234.2011.07.028

分类号:TP391.1

基金项目:国家自然科学基金资助项目(60903083);黑龙江省自然科学基金项目(F200936);黑龙江省高等学校新世纪优秀人才基金资助项目(1155-ncet-008)



Chinese chunking method based on conditional random fields and semantic classes

SUN Guang-lu^1,2, LANG Fei³, XUE Yi-bo¹

1.Research Institute of Information Technology,Tsinghua University,100084 Beijing,China;2.School of Computer Science and Technology,Harbin University of Science and Technology,150080 Harbin,China;3.School of Foreign Languages,Harbin University of Science and Technology,150080 Harbin,China)

Abstract:

To improve the accuracy of Chinese chunking and utilize the semantic information of words,a new Chinese chunking method is proposed based on conditional random fields and semantic classes.Through the analysis of Chinese chunking task and its sequential characteristics,conditional random fields that could incorporate various types of features were applied to overcome the label bias problem.Semantic features were utilized to improve the chunking performance.Experimental results show that the algorithm achieves impressive accuracy of 92.77% in terms of the F-score.A further experiment indicates the effects of feature template selection and training data′s scales on the aspect of chunking performance.

Key words:  conditional random fields  Chinese chunking  feature template  semantic dictionary