基于栈式预训练模型的中文序列标注

删除或更新信息，请邮件至freekaoyan#163.com(#换成@)

本站小编 Free考研考试/2024-10-07

作者:刘宇鹏，李国栋
Authors:LIU Yu-peng,LI Guo-dong摘要:\n\t摘要:序列标注(sequence labelling)是自然语言处理(natural language processing)中的一类重要任务。在文中，根据任务的相关性，使用栈式预训练模型进行特征提取，分词，命名实体识别/语块标注。并且通过对BERT内部框架的深入研究，在保证原有模型的准确率下进行优化，降低了BERT模型的复杂度，减少了模型在训练和预测过程中的时间成本。上层结构上，相比于传统的长短期记忆络（LSTM），采用的是双层双向LSTM结构，底层使用双向长短期记忆网络（Bi-LSTM）用来分词，顶层用来实现序列标注任务。在新式半马尔可夫条件随机场(new semiconditional random field，NSCRF)上，将传统的半马尔可夫条件随机场（Semi-CRF）和条件随机场（CRF）相结合，同时考虑分词和单词的标签，在训练和解码上提高了准确率。将模型在CCKS2019、MSRANER和BosonNLP数据集上进行训练并取得了很大的提升，F1测度分别达到了92.37%、95.69%和93.75%。\n

Abstract:\n\tAbstract:Sequence labeling is an important task in natural language processing. In this paper, according to the relevance of tasks, we use stacking pretraining model to extract features, segment words, and name entity recognition/chunk tagging.Through in-depth research on the internal structure of BERT, while ensuring the accuracy of the original model, the Bidirectional Encoder Representation from Transformers （BERT） is optimized, which reduces the\n
\n\n\tcomplexity and the time cost of the model in the training and prediction process.In the upper layer structure, compared with the traditional long-short-term memory network (LSTM), this paper uses a two-layer bidirectional LSTM structure, the bottom layer uses a bidirectional long-short-term memory network (Bi-LSTM) for word segmentation, and the top layer is used for sequence labeling tasks.On the New Semi-Conditional Random Field (NSCRF), the traditional semi-Markov Conditional Random Field (Semi-CRF) and Conditional Random Field (CRF) are combined while considering the segmentation.The labeling of words improves accuracy in training and decoding. We trained the model on the CCKS2019, MSRANER, and BosonNLP datasets and achieved great improvements. The F1 measures reached 92.37%, 95.69%, and 93.75%, respectively.\n

PDF全文下载地址:

可免费Download/下载PDF全文

基于栈式预训练模型的中文序列标注

本站小编 Free考研考试/2024-10-07

相关话题/

领限时大额优惠券,享本站正版考研考试资料!