Abstract: 【Purpose/significance】Based on the underlying semantic feature data composed of external features such as color and tex‐
ture and local visual features, this paper uses the random forest method to automatically label the medical image information, which
can not only facilitate the general public to better understand the medical knowledge and personal health, provide clinical decision-making reference for medical workers, and facilitate the general public to understand the medical knowledge and personal health.It
can also expand the scope of information organization and processing by researchers in the field of Library and Information in the big
data environment, promote interdisciplinary and integration, improve the development of intelligent medicine, and provide intellectual
and technical support for the healthy China strategy.【Method/process】Integrating library and information domain knowledge and
medical knowledge, image semantic annotation is regarded as a multi class classification problem.Firstly, external features such as
color and texture and low-level semantic features such as local visual features are extracted; Then, using the method of random forest,an automatic labeling scheme of medical images based on random forest is designed【. Result/conclusion】The experiment shows that the automatic annotation scheme of medical image information integrating the underlying semantic features has better effect than the random tree annotation scheme【. Innovation/limitation】Visual semantic dictionary is introduced into image annotation as the underly‐ing semantic feature of medical images; Medical image annotation scheme based on random forest; The limitation is that only BreaK‐His data set is used as experimental data。