In recent years, with the rapid development of computer image processing technology, medical image technology and location tracking technology, computer-aided navigation system helps doctors to determine the best surgical path through preoperative diagnosis, operation path planning, registration and intraoperative navigation, which makes the operation more accurate, rapid and safe, and opens up a new field for the development of surgery. In this paper, the key technologies, process analysis, function realization and clinical application progress of surgical navigation system are briefly summarized. Finally, this paper also makes a prospect for the development trend of surgical navigation technology.
- This article analyzes the workflow of the surgical navigation system and the key technologies involved in each module.
- The application of surgical navigation system is summarized and the future development trend is made.
- This article can provide a reference for beginners who want to understand the surgical navigation system.
The operation navigation system is based on CT, MRI and other medical image data. Before operation, doctors use medical image equipment and computer graphics methods to reconstruct and visualize the multimodal fusion image data of patients to obtain 3D model . Surgeons operate related software on the basis of this model to analyze the condition and formulate the preoperative model. Preoperative simulation is planned; during the operation, the three-dimensional model is unified in one coordinate system with the actual position of the patient and the real-time position of the surgical instrument in the space, as shown in Fig. 1.
Fig. 1Computer aided navigation system architecture
The current position of the surgical instrument relative to the patient’s anatomical structure is tracked in real time by using the three-dimensional positioning system, and the surgeon uses a high-resolution display screen from all directions (axial position, vector) . Observing the current surgical path and various parameters (angle, depth, etc.) in the shape, coronal position and the perspective level in front of the surgical field, so as to avoid the dangerous area to the greatest extent, reach the target lesion in the shortest time, reduce the surgical injury and complications, complete the minimally invasive surgery in the true sense, and greatly improve the success rate of the operation. Therefore, the operation navigation system is also known as the GPS system in the field of medical devices. At present, it is mainly used in surgery, minimally invasive interventional surgery and robot surgery.
2. Basic condition
With the development of modern medicine, more and more detection methods have been developed for human disease monitoring, such as X-ray examination, computed tomography (CT), magnetic resonance imaging (MRI), optical coherence examination (OCT), ultrasound examination and so on . These tests can help doctors obtain various data of patients, provide guidance for disease diagnosis, and help doctors provide a basic operation direction
In the aspect of application software, in the era of information sharing, there are various kinds of relevant software for medical image processing and visualization. These software equipment for the secondary processing of images directly exported from the workstation is an essential process in the surgical navigation system.
In terms of hardware equipment, the optical positioning navigator can obtain the space position of surgical tools in real time, and convert it into coordinate data in real time to store and display in the computer, so as to achieve the effect of assisting doctors in surgery.
3. Key technologies of surgical navigation system
3.1. Multimodal fusion and visual analysis technology
Medical image processing and three-dimensional reconstruction visualization are the key parts of preoperative planning, and also the important foundation of navigation during operation. The three-dimensional reconstruction and visualization analysis of patients’ various image inspection data (such as X-ray, CT, MRI, Oct, ultrasound, etc.) are carried out by using medical imaging technology According to the results of the three-dimensional model, doctors analyze the shape of the focus area to prepare for the operation path planning analysis.
Fig. 2a) Comparison of CT modeling and b) MRI modeling
The basic processing of medical image is also essential for focus analysis and operation plan. By denoising the low resolution image or super-resolution image, the doctor can analyze and study the patient’s condition more clearly, such as the relationship between the focus area and other tissues, the volume and shape of the focus area and other information, so as to make the best treatment plan. Medical image segmentation technology is also very important, for example, for brain CT, the segmentation of skull region is more convenient for doctors to measure its wall thickness.
3.2. Image registration technology
Since the late 1980s, medical image registration technology has been widely developed in the world. Image registration technology also belongs to one of image processing, among which multimodal medical image registration technology plays a very important role in image navigation system. Its purpose is to find the best mapping between series of images in different positions. Generally speaking, it is to find the similar relationship between two images, which can be expressed as a function:
Among them, is the reference image (or fixed image), is the target image (or floating image), and is the transformation function of the image.
For the convenience of description, is usually defined, and is used to represent the transformed target image, that is to say:
Due to the different information expressed by medical images of different modes, the accuracy of relative position between different tissues can only depend on the accuracy of registration of two types of image sequences. In the process of operation, a single modal image sequence is used as the baseline of the lesion area. If we want to see the information expressed by another modal image at this time, we need to register multimodality medical images before operation. For example, the clear relationship between the skull and the relative position of the nerve bundle depends on the image registration technology, and the specific effect is shown in Fig. 3.
Fig. 3Comparison of relative position of skull and nerve bundle before and after image registration
3.3. Registration of spatial coordinate system in operation navigation
In the operation navigation system, image registration can be divided into two stages, one is the registration of CT and MRI in the preoperative stage, the other is the registration of TCM images, patients and surgical instruments in the intraoperative stage . The former is the registration of single-mode image, which combines high-resolution image with the image reflecting the metabolism information of human body, and provides more comprehensive diagnosis information for doctors. The latter refers to the determination of the relative relationship between the surgical instrument and the patient during the operation through the stereo positioning system, and the integration with the medical image into a common coordinate system. The coordinate transformation relationship is shown in Fig. 4. These combined information will be used by surgeons to accurately determine the focus area or avoid the dangerous position during the operation.
Fig. 4Analysis chart of coordinate transformation in spatial registration
Generally, the registration methods are mainly divided into three categories, feature-based registration methods (external features, internal features), pixel density-based registration methods (spindle algorithm, correlation algorithm, mutual information algorithm) and path-based registration methods (Initial registration, precise registration). According to the different ways of selecting feature points, there are mainly the following registration methods:
(1) Several common feature-based registration methods: the frame registration method, that is, the use of a head frame to fix the patient's head to ensure that its position remains unchanged during the surgical formulation and implementation process to achieve system registration. After the patient puts on the headgear, he performs a CT scan (or MRI, etc.) and cannot take it off until the operation is completed, and the frame fixed by the head nail will cause some damage to the patient.
External marking method, this method does not use a frame, and directly attaches a mark with a metal ball (or button-shaped plastic) on the skin of the patient's head or face. This method has certain limitations and requires that the skin at the mark must be Approximately rigid body, generally used for the skin of the skull, and the accuracy of registration is about 4 mm.
The anatomical landmark method, which uses anatomical landmarks as features, is applicable to both rigid and non-rigid body research objects under the premise of accurately extracting features, but the precise segmentation and feature extraction of most dynamic blurred images are still a problem that has not yet been completely solved. The registration error of anatomical marks is about 5mm. The accuracy of the registration has an extremely important influence on the accuracy of the navigation system, and its error is the main source of the positioning error of the system. There are at least 4 mark points selected during registration. The more mark points selected, the higher the registration accuracy and the lower the average error.
(2) Registration method based on pixel density: images of different modals are composed of pixels. Although the texture and shape of the image are different under different modalities, the density value of the image and the corresponding spatial position still have a great correlation. The configuration method based on pixel density does not need to extract the feature information of the image. The pixel density data and corresponding spatial position data of the image in different modals are used to construct the objective function, and the transformation parameters can be obtained by solving the objective function.
(3) Path-based registration method: This method is mainly aimed at the blood vessel model. The initial registration is based on the registration of the catheter front end on the path generated in the two parts of the aorta and the aortic arch. The conversion parameters obtained by the initial registration are an important initialization parameter for fine registration; fine registration is the initial registration Based on the registration, the front end of the catheter is registered on the path generated in the left subclavian artery. Fine registration is a dynamic registration in real time, and as EMT outputs more and more path information, a fine registration algorithm the calculation will get more and more accurate conversion parameters.
3.4. Location and tracking technology
The basic principle of navigation, positioning and tracking technology is to measure and display the space position and posture of surgical instruments with marked points in real time. It is a bridge between medical images, surgical sites and surgical instruments. It is a system to determine the spatial position of the target. It can obtain the three-dimensional coordinates of the target in its measurement range in real time . It is directly related to the accuracy of the surgical navigation system and the success or failure of the operation. Its significance is particularly important. After that, the surgeon relies on the system to provide real-time positioning and the guidance of the pre-operative plan. Operations should be carried out accordingly.
Functional magnetic resonance technology can be performed before surgery, with repeatability. The functional images obtained by it and the white matter fiber bundles obtained by DTI can be combined with a neuronavigation system to provide neurosurgeons with spatial positioning information of central cortical functions and structures. However, due to the limitations of visual observation, complete removal of tumors from the normal brain parenchyma is very challenging.
Electromagnetic tracking  is a method that uses magnetic field information to obtain object position and spatial posture. It has a wide range of important applications in biomedicine, virtual reality and other applications, especially for real-time navigation of minimally invasive surgical medical equipment. With help, avoid the danger zone to the greatest extent and reduce surgical trauma and complications. Electromagnetic tracking technology has the advantages of easy to use, high precision, and is not blocked by line of sight.
At present, in various kinds of computer-assisted surgery, navigation and positioning methods widely used include optical positioning, mechanical positioning, ultrasonic positioning and electromagnetic positioning. In practice, in order to minimize the limitations of each method, the hybrid positioning method is often used, even if two or three basic methods are combined to make up for their shortcomings.
4. Workflow of surgical navigation system
4.1. Preoperative planning
The image information of human tissues and organs is obtained by CT and MRI, and the mathematical model is established according to the reference system of these devices and the characteristics of human tissues and organs. The image is segmented, reconstructed and fused, so that different modes of images of the same part can be displayed simultaneously, and various display modes such as surface display, volume display and arbitrary section display are provided. Then the surgeon uses the relevant operation software to analyze the lesion area according to these multi-mode image data, and formulates the best operation plan, as shown in Fig. 5, and previews and simulates the operation.
Fig. 5Basic workflow diagram of surgical navigation system
4.2. Intraoperative navigation
This stage is the core stage of operation navigation system, and also the longest period of the whole system. During the operation navigation, the system screen shows the current position of the probe and the images of different sections around the probe in real time. According to the need of surgery, doctors select different types of section images and image ranges to display them in three-dimensional, and assist surgeons to carry out the scheduled operation plan. At the same time, the system detects whether the registration error is within the allowable range, and if it exceeds the allowable range, the system prompts the doctor to ensure the safety of the operation.
5. Clinical application
Since the emergence of surgical navigation system, it has been continuously tested in clinical application. Many reports describe various application steps. Optimal access to the lesion, determination of the degree of excision, assurance of the completion of the established surgical plan and protection of non-pathological important structures are basic principles in any operation, but also the focus of minimally invasive surgery. Surgical navigation system is an important guarantee for these principles. The following are the main applications of surgical navigation system in various clinics.
Neurosurgery is the earliest and most common application of surgical navigation system. When a hospital carries out craniocerebral surgery, there is a problem that it cannot accurately locate the location of hematomas and tumors, and intracranial hemorrhage has the characteristics of acute onset and rapid development of the disease. By using operation navigation system to model hematomas and tumors, neurosurgeons can observe the location of hematomas and tumors in real time, facilitate intracranial puncture and other operations, and provide rapid and effective treatment for them. To a certain extent, it improves the success rate of craniocerebral surgery, maximizes the rescue of patients, and significantly increases the safety factor.
5.2. Orthopaedic surgery
Orthopaedic diseases (such as spine surgery, joint replacement surgery, etc.) are usually detected by X-ray, but two-dimensional images cannot clearly show the surface contour and spatial anatomical relationship. Fractures in human body are complex, fracture must be reduced first, fracture must be fully joined, the traditional method is to rely on X-ray and doctor’s experience; while the operation navigation system can provide three-dimensional images before and after operation, showing the joint of broken bones, to avoid blind exploration of traditional surgery. At present, interlocking intramedullary nail is a rapidly developing field in fracture internal fixation . The intramedullary nail was inserted through the fracture or one end of the bone to connect the proximal and distal ends of the fracture. The use of navigation system to guide the insertion of interlocking intramedullary nails can reduce the mistake of nails and effectively improve the quality of surgery.
5.3. Vascular malformation surgery
Traditional pathology cannot reflect vascular malformations, such as arteriovenous malformations (AVM). Vascular anatomy is clear. Considering their special vascular structure, surgical removal of malformed vessels is at risk of massive hemorrhage . Through DSA or FMRI, an integrated database of blood vessels and their surrounding tissues was established, and preoperative plans were made to determine the location of malformed blood vessels. Recognition of known markers during surgery will further increase the confidence of surgeons. For deep AVM, optimizing the surgical route, similar to neurosurgery, can guide doctors to reach malformed blood vessels with minimal damage to surrounding tissues. Under the guidance of accurate route, both the opening and the operative channel can be optimized, which greatly reduces the surgical errors.
5.4. Endoscopic surgery
In the 1990s, endoscopic surgery has made remarkable progress, and its advantages of minimally invasive are widely accepted. But endoscopy has limited field of vision. Doctors cannot observe what he is doing, and it is difficult to know the exact location of endoscopes. The combination of endoscopy and surgical navigation system can produce good results. Through stereo tracking and positioning, the endoscope images can be combined with standard medical images to achieve the orientation of local images . Although endoscopy is minimally invasive, due to its deep penetration into the human body, the potential risk of wrong orientation is very high. System navigation technology can assist doctors to confirm the local and adjacent areas of nose and skull base according to the best preoperative plan during the operation, and reach all cysts in a certain sequence, so as to prevent the damage of important structures such as internal carotid artery and reduce the complications during the operation. It greatly improves the accuracy of the operation, so the operation navigation system has a good prospect in the minimally invasive surgery such as endoscope.
5.5. Plastic surgery
Because the anatomical structure of cranium and oral and maxillofacial region is complex and delicate, and the treatment process and purpose involve plastic and cosmetic surgery, the surgical treatment must complete accurate preoperative prediction design and accurate intraoperative positioning operation . Serious complications will cause irreparable physiological and psychological defects for patients. Traditional two-dimensional simulation and prediction system and orthognathic surgery methods are reluctant to reconstruct the shape and function of some complex patients. Combining with surgical navigation system, the risk of surgery is reduced, so that plastic surgery is more safe and accurate.
In addition to some of the clinical applications listed above, surgical navigation system can also be used for teaching. Surgeons or medical students can use them to preview and simulate the operation . They can operate repeatedly according to the operation records until they are proficient, not afraid of mistakes. The system cooperates with virtual reality and augmented reality technology. The surgeon can not only see the operation area from the virtual glasses, but also feel the patients limbs and organs by means of feedback devices, and have a sense of immersion. Through simulation training, learners can improve surgical skills, accumulate surgical experience, reduce teaching costs, simplify the teaching process, which has a positive significance for the future improvement of medical quality.
6. Trend of development
With the continuous maturity of technology, surgical navigation technology will be more and more widely used in clinical application, which is the future direction of surgery development. There are mainly the following development directions:
– The scope of application continues to expand.
At present, surgical navigation technology is mainly used in the field of hard tissue surgery such as brain neurosurgery and orthopaedics. With the development of computer technology, navigation technology will also be useful in other fields such as liver surgery, heart surgery and other soft tissue surgery.
– Multi-information fusion.
Current surgical navigation systems are mostly based on a medical image, mainly the physiological structure of CT/MRI/US. With the development of image fusion technology, the navigation information will be more comprehensive if functional images (PET, SPECT, MEG, f-MRI, etc.) and visual images (such as real-time camera images or microscopic images) are superimposed on structural images . Even the force feedback information of the surgical tool will be integrated into the navigation system to make the operation safer and more accurate.
– Distance expert consultation and medical teaching.
Navigation system is a fully digital technology, information can be transmitted through the network. Surgeons can be trained in surgery, and remote experts can see the diagnosis and treatment in real time, and give recommendations for diagnosis and treatment.
– Cooperate with medical robots to complete the operation.
Operational navigation system and medical high-precision robots, manipulators and other combination of manipulation of surgical instruments, autonomous completion of the operation, so that the operation is more stable and fine, to a large extent, reducing the error with preoperative simulation and prediction results, will open a new era of clinical medicine.
This work was supported by the National Science Foundation of China (51674121, 61702184), Tangshan Innovation Team Project (18130209B, 18130210B), the Hebei Key Laboratory of Science and Application, Research and Practice Project of Higher Education Teaching Reform in Hebei Province (2017GJJG111) and Ministry of Education Production University Cooperation Education Project (201802305012).
Breeuwer M., Wadley J. P., De Bliek H. L. T. EASI project-improving the effectiveness and quality of image-guided surgery. IEEE Transaction on Information Technology in Biomedicine, Vol. 2, Issue 3, 1998, p. 158.
Brandt Guido, Zimolong Andreas, Carrat Lionel CRIGOS: A compact robot for image-guided orthopedic surgery. IEEE Transaction on Information Technology in Biomedicine, Vol. 3, Issue 4, 1999, p. 252.
Russell Taylor H., Dan Stoianovici Medical robotics in computer integrated surgery. IEEE Transaction on Robotics and Automation, Vol. 19, Issue 5, 2003, p. 765.
Zhang J., Levesque M. F., Wilson C. L., et al. Multimodality imaging of brain structures for stereo tactic surgery. Radiol, Vol. 175, Issue 2, 1990, p. 435-441.
Gu L., Liu Xingxin Simulation Method of Surgical Navigation Based on Electromagnetic Positioning Technology and Intraoperative Image Guidance. CN108420529A, Shanghai, 2018.
Kong Xiangzhan Research on Key Technologies of Craniofacial Puncture Diagnosis and Surgery Robot. Beijing University of Technology, 2015.
Xianghua Research on Three-Dimensional Registration Technology of Surgical Navigation. Tsinghua University, 2012.
Xue Fei Research on Surgical Navigation System and Its Key Technologies. Xi’an University of Electronic Science and Technology, 2007.
Tian Wei, Liu Yajun, Liu Bo, Li Qin, Hu Lin, Li Zhiyu, Yuan Qiang Experimental and clinical studies of computer navigation in spinal surgery. Chinese Journal of Orthopedics, Vol. 10, 2006, p. 671-675.
Niu Yanhua Augmented Reality Technology in Surgical Navigation. Zhejiang University, 2004.
Zhang Xuemin Research on Navigation and Visual Localization of Robotic Neurosurgery. Huazhong University of Science and Technology, 2005.