Abstract: 【Purpose/significance】Large language model has multi-domain knowledge and can achieve deep semantic understanding and generation of cross-domain text, offering opportunities to enhance the automation level of science and technology novelty search. This article will explore the potential of deep application of large language models in novelty search work.【Method/process】This ar⁃ ticle evaluates the ability of large language models in relation to the crucial aspects of novelty search work. First, four evaluation tasks were established, including novelty point generation, keyword generation, method comparison and conclusion comparison. Then we se⁃ lected general large language models with different parameter sizes for evaluation, and attempted to build different prompt templates to fully unlock the ability of the large language models. Finally, we compared the general models with the model fine-tuned on the do⁃ main corpus to further evaluate the large language models.【Result/conclusion】Experimental results show that different models need to understand the learning task through few-shot learning, and the four-shot setting is effective than the one-shot setting. In the method comparison and conclusion comparison tasks, fine-tuning can significantly optimize the results, while there is no improvement when fine-tuning the model for the novelty point generation and keyword generation tasks. The long text understanding and processing capa⁃ bilities of ChatGLM3-6B are insufficient. The GPT-4 model has sufficient long text understanding and processing capabilities.【Inno⁃ vation/limitation】We confirms that large language models have significant application potential in science and technology novelty search. In the future, detailed research will be conducted to further enhance the depth of their integration with novelty search.