情报科学 ›› 2021, Vol. 39 ›› Issue (5): 169-175.

• 博士论坛 • 上一篇    下一篇

灾害事件下基于多模态融合的社交媒体
信息丰富型推文识别研究

  

  • 出版日期:2021-05-01 发布日期:2021-05-12

  • Online:2021-05-01 Published:2021-05-12

摘要:

【目的/意义】灾害事件期间,反映事件实时状态的多模态信息被广泛发布在社交媒体平台中,有效识别这
些蕴含丰富灾害信息的推文对于灾害事件的危机预警、态势感知和应急决策都具有重要的作用。【方法/过程】本文
提出一种基于多模态融合的信息丰富型推文识别框架,将推文中图文的形式特征和语义特征作为融合输入,利用
机器学习和深度学习的方法构建多种识别信息丰富型推文的二元分类器,进而分析不同推文特征和不同融合方法
在信息丰富型推文识别中的效果。【结果/结论】实验结果表明,基于语义特征的识别方法要优于基于形式特征的识
别方法,基于图文融合的识别方法优于基于单一模态文本或图像的识别方法,同时融合文本和图像的形式特征和
语义特征来识别灾害事件信息丰富型推文的效果更好。【创新/局限】本文在综合考虑推文中多模态信息的形式特
征和语义特征的基础上,提出基于特征层和决策层的多模态融合方法来识别社交媒体中的信息丰富型推文,但是
该方法在不同灾害事件场景下的适用性还有待进一步检验。

Abstract:

【Purpose/significance】During the disasters, multimodal information reflecting the real-time situation is widely published
in the social media. How to effectively identify informative tweets is important to disasters response.【Method/process】This paper pro⁃poses a social media informative tweets identification framework based on multimodal fusion. Formal features and semantic features of text and text in tweets are used as fusion inputs. Then, a binary classifier to identify informative is constructed through machine learn⁃ing and deep learning. Finally, the role of different tweet features and fusion methods in informative tweets identification are analyzed.【Result/conclusion】The results of informative tweets identification experiment show that the effect of semantic features is better than formal features, and the effect of multimodal fusion are better than single modal such as text or images. At the same time, the best way of identifying informative tweets related disasters is fusing formal and semantic features of text and images.【Innovation/limitation】Considering the formal and semantic features of multimodal information in tweets, we propose a multimodal fusion method of feature layer and decision layer to identify informative tweets in social media. However, the applicability of this method in different disasters needs to be further tested.