情报科学 ›› 2022, Vol. 40 ›› Issue (6): 108-114.

• 业务研究 • 上一篇    下一篇

基于增强对抗网络和多模态融合的谣言检测方法 

  

  • 出版日期:2022-06-01 发布日期:2022-06-12

  • Online:2022-06-01 Published:2022-06-12

摘要: 【目的/意义】随着社交网络的复杂化,当前谣言往往是由描述事件的文本、对应的图片或者视频组成,多种
模态的谣言更容易给用户传达一种错误的认知。现有谣言检测的研究往往只使用谣言文本特征,且未能充分挖掘
谣言与事件存在的联系。【方法
/过程】因此本文提出一种基于增强对抗网络和多模态融合的谣言检测方法,使用
BERT Text-CNN 提取文本特征,使用 VGG-19网络提取图像特征,再通过注意力机制捕捉多个模态的特征交
互,最后使用增强对抗网络来挖掘谣言和事件之间联系。【结果
/结论】在公开的微博多模态数据集上进行对比实
验,实验结果表明该方法检测的准确率达到了
92.5%,相较于传统单模态和现有多模态模型,提升了约 10%~20%
【创新
/局限】本文将对抗网络和多模态特征融入谣言检测中,有效提升了谣言检测的效果,但目前仅尝试了文本和
图像两种模态的结合,如何融合更多模态的特征后续有待研究。

Abstract: Purpose/significanceWith the complexity of social networks,current rumors are often composed of text describing events,corresponding pictures or videos,and multimodal rumors are more likely to convey a wrong cognition to users.The existing research on rumor detection only uses the text features of rumor,and fails to fully explore the relationship between rumor and event. Method/pro⁃cessTherefore,this paper proposes a rumor detection method based on enhanced adversary network and multimodal fusion.It uses BERT and Text CNN to extract text features,VGG-19 network to extract image features,and then uses attention mechanism to capture multi-modal feature interaction.Finally,it uses enhanced adversary network to mine the relationship between rumors and events. Re⁃
sult/conclusion
The experimental results show that the detection accuracy of this method reaches 92.5%, which is about 10~20% higher than that of traditional single-mode and existing multi-mode models. Innovation/limitationAt present,only the combination of text and image modes has been tried.There are still many short video and voice data on social media.How to integrate more modal fea⁃tures needs to be studied in the future.