Medical image registration based on PCA and M_PSNR
Jin Jin1 , Xinrui Bi2 , Miaomiao Jiang3 , Jincai Chang4 , Jianzhong Cui5
1, 2, 4College of Sciences, North China University of Science and Technology, Tangshan, 063210, China
1, 2, 43D Modeling and Application Innovation Laboratory, North China University of Technology, Tangshan, 063210, China
3, 5Department of Neurosurgery, Tangshan Gongren Hospital, Tangshan, 063000, China
4, 5Corresponding authors
Journal of Complexity in Health Sciences, Vol. 3, Issue 1, 2020, p. 62-72.
Received 23 December 2019; received in revised form 1 February 2020; accepted 10 February 2020; published 30 June 2020
Medical image registration is not only the premise of medical image fusion, but also the focus of medical image processing. It plays an important role in clinical diagnosis and treatment planning. Aiming at the problem of mutual information value in image registration, a PCA neural network and M_PSNR based method for rigid registration of reference and floating images was proposed. In this method, the centroid of the image needs to be figured out through the image matrix, and the rotation Angle and translation of the image registration can be calculated by PCA method to obtain the optimal rotation transformation and the initial registration value. Finally, M_PSNR method improved by PSNR was used for similarity test to increase the speed and efficiency of registration and simplify the calculation process.
Keywords: medical image registration, PCA neural network, PSNR, mutual information value.
In recent years, medical image registration has played an important role in clinical diagnosis and treatment planning, domestic and foreign experts and scholars in the medical image registration study phase has made many achievements, in a growing number of surgical navigation system and virtual surgery system, rapid registration is played a crucial role, at present most allocation algorithm between MRI and CT, the existing medical image registration methods mainly include two categories: the method based on gray level and based on the characteristics of the method .
3D Slicer software in the field of medicine has strong function of Registration, the user simply import the CT and MRI data, select the General Registration Registration (Elastix) module, can quickly achieve the same patients with different mode of image Registration, also can according to the actual situation to adjust the required parameters and templates, as shown in the Figs. 1-3.
Fig. 1. CT
Fig. 2. MR
Fig. 3. CT and MR registration
Medical image registration is a kind of typical problem in the field of image processing. Typically, patients receive multiple tomographic radiography for medical diagnosis. For example, images from different acquisition devices, different time, different shooting angles and so on, these are called multimode images, and multimode images are very useful in medical image analysis, because images in different modes provide supplementary information. Multimodal image registration is a method of recording the volume of different modes, for example, computed tomography (CT) and magnetic resonance (MR). Since the 20th century, medical imaging technology has experienced rapid development from static to dynamic , from shape to function, and from plane to three-dimensional. Combining various images to display their information on the same image and provide diversified images for clinical medical diagnosis will be a technology of great application value, while accurate and efficient image matching criteria is a major difficulty.
2. The problem of image registration based on mutual information values
Define a stereo coordinate system: the -axis – along the line scan direction; the -axis – along the column scan direction; the -axis – from the bottom to the top. General with MRI image as a reference image (), CT images are floating images (). The image sizes of and are both , grayscale, and the grayscale values at the point are respectively and . Under some transformation , the joint probability density of and images can be obtained through normalization processing :
where, refers to the joint gray histogram of and under the transformation of , and the probability density of is:
Similarly, the probability density of is:
The information contained in the gray distribution of the image is defined as the entropy of the image, denotation as . In the process of medical image registration, when the space of and images is consistent, their corresponding gray mutual information reaches the maximum value, and the mutual information calculation formula is as follows:
where is the entropy value of the reference image, is the entropy value of the floating image, and is the joint entropy of the two. That is:
According to the above equation, an optimal transformation can be found under the rigid transformation to obtain the maximum mutual trust value :
Since is translation and rotation in the three-dimensional space of medical images:
where is the Angle of rotation as change.
It can be seen that the process of finding the best transformation is actually a multi-parameter optimization problem. Unconstrained method is usually used to find the optimal solution in such problems. By referring to the literature  and using the mutual information of images as the similarity test, it is necessary to use the multi-parameter optimization method of Powell  and simplex method to obtain the parameters in translation and rotation changes. Due to the fact that many local minima are involved in the calculation of mutual information, these optimization methods tend to get people into the trouble of local optimization. In the calculation of the optimal solution, the calculation amount is very large, the process is very tedious, and a lot of cycles and iterations are required, which consumes a lot of search time. More importantly, this algorithm needs an accurate and proper initial value. If it is not selected properly, it will consume more search time and may even cause the failure of medical image registration. Therefore, it is necessary to find a method with low registration time.
3. PCA neural network model
PCA is a tool for finding patterns of high-dimensional data, images, etc. In machine learning practice, PCA is often used to preprocess the input neural network data. PCA is the nature of the direction of maximum variance as the main characteristics, through the gathering, rotation and scaling of data in each orthogonal direction will be “related to”, which make them in different orthogonal directions without correlation, this algorithm can get rid of some low variance of dimension and dimension reduction effect, this operation can improve the convergence rate of the neural network and the overall effect.
In order to improve the registration efficiency and increase the probability of registration success, the first task is to find the optimal initial value needed for medical image registration, which can greatly reduce the search time. First, PCA neural network  was used to calculate the first principal direction of the feature image, and then the first principal direction of the feature image was matched with the center of mass to achieve image registration. The following are the basic methods and steps of PCA algorithm in image registration.
3.1. Determine the centroid coordinates of the image
Let the order matrix of two-dimensional discrete function be:
Among them, , 0, 1, 2,….
Its zero-order moment is defined as , when , :
When the first moment and the higher moment are divided by the zero moment , the center of mass of the image is independent of the size of the image.
The first moment is defined as or , when , and , :
According to the above equation, is the center of mass of the object in the image.
Similarly, in medical image registration, the zero-order moment and first-order moment of images and can be calculated to obtain their centroid coordinates as and , respectively.
3.2. Calculate rotation angle
The calculation of rotation Angle is an important step in medical image registration. As long as the image is rotated, it is obvious that the rotation Angle is equal to the rotation Angle of the first main direction. The first main direction of the reference feature image () and floating feature image () was calculated by using PCA neural network method, and then the rotation Angle, namely the Angle between the two main directions, was obtained .
The calculation of the first principal direction requires the principal element analysis neural network method of , which can be used to calculate the principal direction of the correlation matrix with a given random data set.
Let’s say the input matrix:
It’s a zero mean random matrix.
If is defined, then PCA learning algorithm of can be described by the following random difference equation:
where, is the learning rate, and meets the following formula:
If the learning rate meets some basic conditions, will converge to the first main direction. The learning algorithm of PCA neural network model is shown in Fig. 4.
Fig. 4. PCA neural network model algorithm schematic diagram
The reference image () and the floating image () are respectively included in the corresponding matrix and , which satisfy the following equation:
UsingPCA principal component analysis neural network method under , the first principal direction of image can be obtained:
And , similarly, the first principal direction of image is obtained:
is the absolute value of the Angle.
The calculation formula of the Angle between these two vectors is as follows:
Function is defined as follows:
In general, the rotation Angle is within [–30°, 30°], so the value of the rotation Angle can be obtained according to the function:
3.3. Registration translation calculation
According to the centroid coordinates of reference image () and floating image () determined in Section 3.1 and the rotation Angle derived in Section 3.2, a method for calculating the registration translation is given.
The process of shift calculation is as follows: the center of mass of the reference image minus the center of mass of the floating image:
is the distance that we shifted in the direction, is the distance that we shifted in the direction.
3.4. Application and necessity of neural network
Neural network has the ability to use training data sets to approximate complex human input output, which is widely used in image segmentation, and is often used in the segmentation of lung CT, brain MRI and breast cancer X-ray images. Similarly, neural network is widely used in medical image registration. Generally, a spatial transformation is sought to make the spatial position of two images consistent, and the relatively mature one is the registration of brain CT and MRI. In addition, neural network also has strong nonlinear mapping function and better ability to obtain data rules, which can be used in all aspects of CAD to find the diagnosis rules and improve the diagnosis efficiency. Combining neural network with other techniques can improve the performance of the diagnosis system.
3.5. Registration translation calculation
Translation and Rotation are important steps in the image registration process, and the Translation and Rotation operations in the Transforms module in 3D Slicer software can translate and rotate slices, taking MR as an example. The section was rotated and adjusted by Rotation. The axial midline adjustment (IS) and coronal midline adjustment (PA) were performed to make the yellow line of the cross line coincide with the midline, and the sagittal position was adjusted by Angle (LR). The comparison before and after translation and rotation is shown in Fig. 5.
Fig. 5. MR contrast diagram before and after translation rotation
3.6. The advantages of PCA neural networks
The characteristics and advantages of artificial neural networks are mainly manifested in the ability to find optimal solutions at high speed. Finding the optimal solution of a complex problem often requires a large amount of computations. By using a feedback artificial neural network designed for a certain problem and giving full play to the high-speed computing capacity of computers, the optimal solution will soon be found.
4. Image similarity test based on PSNR method
Based on the problem of mutual information value in medical image registration, it requires a large amount of calculation, a complex process and a long time, so it cannot directly reflect the difference between the reference image and the floating image as a whole. In order to evaluate the image similarity after registration more objectively, the PSNR method (image peak signal-to-noise ratio)  was finally adopted to test the similarity, so as to further reduce the time of image registration.
PSNR is a commonly used objective standard for evaluating images, which can directly show the gray scale difference between and on the whole. Generally, the larger the calculated PSNR is, the smaller the gray difference between and is, and the more similar the image is. On the contrary, the smallerPSNR is, the bigger the gray scale difference between and images is, and the bigger the difference between the two images is. The mathematical formula of is expressed as follows:
where, MSE is the mean square error between and images. Combining with Eq. (8), we can get:
Since PSNR algorithm has more advantages in terms of computational load, it is selected as the similarity test of and images, and then refers to 1 to find the best transformation of the two images, so as to maximize the PSNR value of the image after registration:
Self-registration was carried out for the direction, direction and Angle rotation in Fig. 1, and the following PSNR change curve (Fig. 6) was obtained.
As can be seen from the Fig. 6, the PSNR curve has a break point and needs to be corrected.
Fig. 6. PSNR curve
5. Improved M_PSNR method
Theoretically, the value of could be 0, in which case PSND, it is like an image registration with itself; But in the actual situation and the two images on time, is impossible to be 0.
It is necessary to modify the above Eq. (22) and add a modified value to obtain the improvedPSNR, which is defined as M_PSNR algorithm. The calculation of M_PSNR method is as follows:
The optimal transformation of the corresponding two images becomes:
According to Eq. (24), the modified value is:
Because of the preconditions:
The norm of the modified value is obtained:
A large number of experiments have shown that:
As a result:
Substitute into Eq. (24), and get:
The following M_PSNR curve graph (Fig. 7) was obtained by self-registration of Fig. 1.
After adding the modified value, the M_PSNR curve is smoother, and the registered image will not show the break point. When testing the similarity degree of images, the curve of the improved M_PSNR method is relatively smooth, and there is no breakpoint in the image after registration, and the registration speed is greatly improved. Secondly, the algorithm is simple and universal with strong portability.
This paper is based on the problem of mutual information value in medical image registration. For example, when calculating the maximum gray mutual information , it involves the solution of some local minima, which requires a large amount of calculation, a tedious process and a lot of time. A method based onPCA neural network and M_PSNR model is proposed.
When determining the centroid coordinates of the image, the zero-order moment and the first-order moment are calculated. Then PCA neural network method was used to calculate the rotation Angle of the first principal direction, and the rotation invariance of PCA was used to get the initial value, and then the Angle between the two principal directions, namely the rotation Angle, was obtained. The initial registration value obtained  by this method is close to the optimal, which can reduce the search time, avoid getting into the local optimal problem when searching for the optimal rotation, and obtain better registration accuracy.
In order to further reduce the time of image registration, M_PSNR method is required to conduct similarity test for the two images and find the best transformation of the two images, so that the M_PSNR value of the image after registration can reach the maximum. At the same time, an improved M_PSNR method was proposed. The calculation amount was about the same, but the modified was added to improve the registration speed.
Fig. 7. M_PSNR curve
In conclusion, for the registration methods used for mutual information, the image registration methods used in this paper have more advantages. The larger the image size, the higher the registration effect. PCA neural network model was used to complete the registration of its first principal direction and the centroid of the image, and then the improve PSNR d method was used to test the similarity of and images. A rigid registration method based on PCA neural network and M_PSNR was proposed, which was simple and efficient. CT and MRI image registration will provide doctors with more medical information, including diagnosis and treatment information. Accurate registration is helpful for diagnosis and treatment of craniocerebral tumor and other problems .
This work was supported by the National Science Foundation of China (51674121, 61702184), Tangshan Innovation Team Project (18130209B, 18130210B), the Hebei Key Laboratory of Science and Application, Research and practice project of higher education teaching reform in Hebei Province (2017GJJG111) and Ministry of education production university cooperation education project (201802305012).
- Zou Maoyang, Yang Hao, Pan Guanghui, Zhong Yong Research progress and challenges of deep learning in medical image registration. Journal of Biomedical Engineering, Vol. 36, Issue 4, 2019, p. 677-683. [Search CrossRef]
- Ding Min Multi-Mode Medical Image Registration Algorithm Based on Mutual Information Correlation Ratio. Nanjing University of Science and Technology, 2018. [Search CrossRef]
- Ke Xinhua, Ou Dong, Geng Bo, Zeng Fan, Huang Hao, Yu Hongfei Networked management of medical photography. China Medical Education Technology, Vol. 6, 2004, p. 391-392. [Search CrossRef]
- Liang Zhigang, Gu Junhua Improved brainstorm optimization algorithm combined with Powell algorithm for medical image registration. Computer Application, Vol. 38, Issue 9, 2018, p. 2683-2688. [Search CrossRef]
- Zhang Weiguo, Wang Guihua Study and implementation of clinical image recognition of liver ultrasound based on PCA multi-feature fusion. Computer Application and Software, Vol. 31, Issue 4, 2014, p. 239-243. [Search CrossRef]
- Huang Ximou, Wan Lei Quantitative study of cervical vertebra manipulative rotation Angle based on MR model. Journal of Taishan Medical College, Vol. 40, Issue 8, 2019, p. 581-583. [Search CrossRef]
- Tang Yi Ling, Shun Liang Jiang, Shao Ping Xu, Liu Tingyun, Li Chongxi Based on the asymmetric distortion of ocular dominance stereo image quality assessment. Acta Automatica Sinica, 2019, https://doi.org/10.16383/j.aas.c190124. [Search CrossRef]
- Pan Meisen, Tang Jingtian, Yang Xiaoli Medical image registration using PCA and PSNR. Infrared and Laser Engineering, Vol. 40, Issue 2, 2011, p. 355-364. [Search CrossRef]
- Wang Dongdong Research on Medical Image Registration Based on Mutual Information Measure and Multi-Resolution Strategy. Jiangsu University, 2017. [Search CrossRef]
- Casella A., De Falco I., Della Cioppa A., Scafuri U., Tarantino E. Exploiting multi-core and GPU hardware to speed up the registration of range images by means of differential evolution. Journal of Parallel and Distributed Computing, Vol. 133, 2019, p. 307-318. [Publisher]
- Li Kuncheng, Li Yongcheng, Zhao Fuqiang, Zhao Xin Preliminary application of CT and MRI image registration and fusion in craniocerebral tumors. Chinese Medical Imaging Technology, Vol. 9, 2001, p. 823-825. [Search CrossRef]
- Wang Y. C., Wu Z. Q., Shi J. Y., Li Z. M., Shan F., Li Z. Y., Ji J. F. Evaluation of postoperative complications registration status of gastric cancer by medical information: a single center feasibility study. Chinese Journal of Gastrointestinal Surgery, Vol. 22, Issue 8, 2019, p. 729-735. [Search CrossRef]
A method of 3D‐3D multi‐stage non‐rigid registration of the spine based on binocular structured light
The International Journal of Medical Robotics and Computer Assisted SurgeryLong Chen, Xin Zhang, Yuhao He, Wencong Wang, Fengfeng Zhang, Lining Sun