Research advances in multi-modality medical image registration and fusion methods and their clinical application
Gu Heng,Nie Shengdong
Institute of Medical Imaging Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China (Gu HL,Nie SHD);Department of Radiation Oncology,Shanghai Chest Hospital,Shanghai Jiao Tong University,Shanghai 200030,China (Gu HL)
Abstract：Multi-modality medical image processing has become a hot topic for research in the field of image processing and plays an important role in clinical diagnosis and treatment. Images with different modalities provide different information on patients. Anatomical images (such as computed tomography and magnetic resonance imaging) provide information on anatomical morphology and the structure of human body, and functional images (such as single-photon emission computed tomography and positron emission tomography) provide the functional information on the distribution of radioactive concentration within human body. Such information needs to be fused to obtain comprehensive fusion images, and the images with different modalities need to be registered to obtain useful fusion images. This article reviews several image registration and fusion techniques used in the medical field, points out their advantages and shortcomings, and introduces the application of various processing techniques in clinical practice.
Gu Heng,Nie Shengdong. Research advances in multi-modality medical image registration and fusion methods and their clinical application[J]. Chinese Journal of Radiation Oncology, 2016, 25(8): 902-906.
[1] Ireland RH,Dyker KE,Barber DC,et al. Nonrigid image registration for head and neck cancer radiotherapy treatment planning with PET/CT[J].Int J Radiat Oncol Biol Phys,2007,68(3):952-957.DOI:10.1016/j.ijrobp.2007.02.017 [2] Maintz JB,Viergever MA.A survey of medical image registration[J].Med Image Anal,1998,2(1):1-36.DOI:10.1016/S1361-8415(01)80026-8 [3] Maurer Jr CR,McCrory JJ,Fitzpatrick JM.Estimation of accuracy in localizing externally attached markers in multimodal volume head images[A]//Loew MH.Proceedings of the SPIE medical imaging 1993:image processing[C].Newport Beach,CA:SPIE,1993.DOI:10.1117/12.154535 [4] Hill DL,Hawkes DJ,Gleeson MJ,et al. Accurate frameless registration of MR and CT images of the head:applications in planning surgery and radiation therapy[J].Radiology,1994,191(2):447-454.DOI:10.1148/radiology.191.2.8153319 [5] Meyer CR,Leichtman GS,Brunberg JA,et al. Simultaneous usage of homologous points,lines,and planes for optimal,3-D,linear registration of multimodality imaging data[J].IEEE Trans Med Imaging,1995,14(1):1-11.DOI:10.1109/42.370397 [6] Maurer Jr CR,Aboutanos GB,Dawant BM,et al. Registration of 3-D images using weighted geometrical features[J].IEEE Trans Med Imaging,1996,15(6):836-849.DOI:10.1109/42.544501 [7] Small C,Lu JWT.Estimation and vicarious validation of urban vegetation abundance by spectral mixture analysis[J].Remote Sens Environ,2006,100(4):441-456.DOI:10.1016/j.rse.2005.10.023 [8] Chow CK,Tsui HT,Lee T,et al. Medical image registration and model construction using genetic algorithms[A]//Proceedings of the International Workshop on Medical Imaging and Augmented Reality[C].Shatin,Hong Kong:IEEE,2001:174-179.DOI:10.1109/MIAR.2001.930282 [9] Ito K,Aoki T,Kosuge E,et al. Medical image registration using phase-only correlation for distorted dental radiographs[A]//Proceedings of the 19th International Conference on Pattern Recognition[C].Tampa,FL:IEEE,2008:1-4.DOI:10.1109/ICPR.2008.4761734 [10] Xu R,Chen YW,Tang SY,et al. Parzen-window based normalized mutual information for medical image registration[J].IEICE Trans Inf Syst,2008,91(1):132-144.DOI:10.1093/ietisy/e91-d.1.132 [11] Chen GTY,Pelizzari CA.Image correlation techniques in radiation therapy treatment planning[J].Comput Med Imaging Graph,1989,13(3):235-240.DOI:10.1016/0895-6111(89)90130-4 [12] Maes F,Vandermeulen D,Suetens P.Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximization of mutual information[J].Med Image Anal,1999,3(4):373-386.DOI:10.1016/S1361-8415(99)80030-9 [13] Livyatan H,Yaniv Z,Joskowicz L.Gradient-based 2-D/3-D rigid registration of fluoroscopic X-ray to CT[J].IEEE Trans Med Imaging,2003,22(11):1395-1406.DOI:10.1109/TMI.2003.819288 [14] Auer M,Regitnig P,Holzapfel GA.An automatic nonrigid registration for stained histological sections[J].IEEE Trans Image Process,2005,14(4):475-486.DOI:10.1109/TIP.2005.843756 [15] Karaali B.Information theoretic deformable registration using local image information[J].Int J Comput Vis,2007,72(3):219-237.DOI:10.1007/s11263-006-8704-0 [16] King AP,Rhode KS,Ma Y,et al. Registering preprocedure volumetric images with intraprocedure 3-D ultrasound using an ultrasound imaging model[J].IEEE Trans Med Imaging,2010,29(3):924-937.DOI:10.1109/TMI.2010.2040189 [17] Fu DS,Kuduvalli G.A fast,accurate,and automatic 2D-3D image registration for image-guided cranial radiosurgery[J].Med Phys,2008,35(5):2180-2194.DOI:10.1118/1.2903431 [18] Hou JD,Guerrero M,Chen WJ,et al. Deformable planning CT to cone-beam CT image registration in head-and-neck cancer[J].Med Phys,2011,38(4):2088-2094.DOI:10.1118/1.3554647 [19] Gu XJ,Dong B,Wang J,et al. A contour-guided deformable image registration algorithm for adaptive radiotherapy[J].Phys Med Biol,2013,58(6):1889-1901.DOI:10.1088/0031-9155/58/6/1889 [20] Kadoya N,Fujita Y,Katsuta Y,et al. Evaluation of various deformable image registration algorithms for thoracic images[J].J Radiat Res,2014,55(1):175-182.DOI:10.1093/jrr/rrt093 [21] Staring M,van der Heide UA,Klein S,et al. Registration of cervical MRI using multifeature mutual information[J].IEEE Trans Med Imaging,2009,28(9):1412-1421.DOI:10.1109/TMI.2009.2016560 [22] Bavachan B,Krishnan DP.A Survey on image fusion techniques[J].Int J Res Comput Commun Technol,2014,3(3):49-52 [23] Wang WC,Chang FL.A multi-focus image fusion method based on Laplacian pyramid[J].J Comput,2011,6(12):2559-2566.DOI:10.4304/jcp.6.12.2559-2566 [24] Naidu VPS,Elias B.A Novel Image Fusion Technique using DCT based Laplacian Pyramid[J].Int J Invent Eng Sci,2013,1(2):17-24 [25] Kumar DR,Nelson S.Fusion of MRI and CT images using multi-wavelet based image fusion[A]//Proceedings of the National Conference on System Design and Information Processing[C].Bonfring,2013 [26] Singh R,Srivastava R,Prakash O,et al. DTCWT based multimodal medical image fusion[A]//Proceedings of International Conference on Signal,Image and Video Processing[C].Patna:ⅡT Patna,2012 [27] Li ST,Yang B,Hu JW.Performance comparison of different multi-resolution transforms for image fusion[J].Inf Fusion,2011,12(2):74-84.DOI:10.1016/j.inffus.2010.03.002 [28] Li MJ,Dong YB,Wang XL.Image fusion algorithm based on wavelet transform and laplacian pyramid[J].Adv Mater Res,2014,860-863:2846-2849.DOI:10.4028/www.scientific.net/AMR.860-863.2846 [29] Ling T,Qian ZY.An improved medical image fusion algorithm based on wavelet transform[A]//Proceedings of the 7th International Conference on Natural Computation[C].Shanghai:IEEE,2011:76-78.DOI:10.1109/ICNC.2011.6022024 [30] Prakash C,Rajkumar S,Chandra Mouli PVSSR.Medical image fusion based on redundancy DWT and Mamdani type min-sum mean-of-max techniques with quantitative analysis[A]//Proceedings of the International Conference on Recent Advances in Computing and Software Systems[C].Chennai:IEEE,2012:54-59.DOI:10.1109/RACSS.2012.6212697 [31] Singh R,Srivastava R,Prakash O,et al. Mixed scheme based multimodal medical image fusion using Daubechies Complex Wavelet Transform[A]//Proceedings of the International Conference on Informatics,Electronics& Vision[C].Dhaka:IEEE,2012:304-309.DOI:10.1109/ICIEV.2012.6317394 [32] Cao YQ,Lei ZM,Huang XS.Region-based algorithm for non-sampling morphological wavelet medical image fusion[J].Appl Res Comput,2012,29(6):2379-2381.DOI:10.3969/j.issn.1001-3695.2012.06.101 [33] 杨艳春,党建武,王阳萍.基于提升小波变换与自适应PCNN的医学图像融合方法[J].计算机辅助设计与图形学学报,2012,24(4):494-499.DOI:10.3969/j.issn.1003-9775.2012.04.010 Yang YCH,Dang JW,Wang YP.Medical image fusion method based on lifting wavelet transform and adaptive PCNN[J].J Comput Aid Des Comput Grap,2012,24(4):494-499.DOI:10.3969/j.issn.1003-9775.2012.04.010 [34] Daneshvar S,Ghassemian H.MRI and PET image fusion by combining IHS and retina-inspired models[J].Inf Fusion,2010,11(2):114-123.DOI:10.1016/j.inffus.2009.05.003 [35] Sharmila K,Rajkumar S,Vijayarajan V.Hybrid method for multimodality medical image fusion using Discrete Wavelet Transform and Entropy concepts with quantitative analysis[A]//Proceedings of the International Conference on Communications and Signal Processing[C].Melmaruvathur:IEEE,2013:489-493.DOI:10.1109/iccsp.2013.6577102 [36] Ding L,Han CZ.Fusion for CT image and MR image based on nonsubsampled transformation[A]//Proceedings of the 2nd International Conference on Advanced Computer Control[C].Shenyang:IEEE,2010:372-374.DOI:10.1109/ICACC.2010.5487087 [37] Kavitha CT,Chellamuthu C.Fusion of SPECT and MRI images using integer wavelet transform in combination with curvelet transform[J].Imaging Sci J,2015,63(1):17-23.DOI:10.1179/1743131X14Y.0000000092 [38] Amini N,Fatemizadeh E,Behnam H.MRI-PET image fusion based on NSCT transform using local energy and local variance fusion rules[J].J Med Eng Technol,2014,38(4):211-219.DOI:10.3109/03091902.2014.904014
2016, Vol. 25 
