He Zhao,  Zhu Ya-Nan, Chen Yu, Chen Yi, HeYuchen, Sun Yuhao, Wang Tao, Zhang Chengcheng, Sun Bomin, Yan Fuhua, Zhang Xiaoqun, Sun QingFang, *Yang Guang-Zhong, *Feng Yuan

 

Nature Communications 2023, 14, 8257.

 

Abstract

Accurate navigation and targeting are critical for neurological interventions including biopsy and deep brain stimulation. Real-time image guidance further improves surgical planning and MRI is ideally suited for both pre- and intra-operative imaging. However, balancing spatial and temporal resolution is a major challenge for real-time interventional MRI (i-MRI). Here, we proposed a deep unrolled neural network, dubbed as LSFP-Net, for real-time i-MRI reconstruction. By integrating LSFP-Net and a custom-designed, MR-compatible interventional device into a 3 T MRI scanner, a real-time MRI-guided brain intervention system is proposed. The performance of the system was evaluated using phantom and cadaver studies. 2D/3D real-time i-MRI was achieved with temporal resolutions of 80/732.8 ms, latencies of 0.4/3.66 s including data communication, processing and reconstruction time, and in-plane spatial resolution of 1 × 1 mm2. The results demonstrated that the proposed method enables real-time monitoring of the remote-controlled brain intervention, and showed the potential to be readily integrated into diagnostic scanners for image-guided neurosurgery.

 

Introduction and Methods

Deep learning (DL) can greatly improve reconstruction quality and accelerate computation speed, making it especially useful in fast MRI. Typical DL networks include AUTOMAP, GAN, U-nets, transformers, and diffusion models39. For further utilizing temporal information, a convolutional recurrent neural network (CRNN) was proposed for dynamic MRI. Similarly, Jaubert et al. developed a deep artifact suppression method using recurrent U-Nets for real-time cardiac MRI. A DL-based image reconstruction and motion estimation from undersampled radial k-space has also been applied to real-time MRI-guided radiotherapy. However, these data-driven networks rely on large-scale training datasets and have limited interpretability and generalizability. To overcome these limitations, unrolled networks were proposed. Typical models include cascaded networks, ISTA-Net (unrolling of the iterative shrinkage-thresholding algorithm), ADMM-Net (unrolling of alternating direction method of multipliers method), and variational network (unrolling of gradient descent algorithm). An unrolled variational network with an undersampled spiral k-space trajectory was also developed for real-time cardiac MRI reconstruction. However, only exploiting sparse prior limits the performance of these networks. By utilizing both low-rank and sparse priors, SLR-Net and L + S-Net have become two state-of-the-art unrolled networks for dynamic MRI reconstruction. However, SLR-Net and L + S-Net are designed for retrospective reconstruction with Cartesian sampling, which cannot satisfy the online reconstruction requirement of real-time i-MRI.

 

In this study, we proposed LSFP-Net for i-MRI reconstruction by unrolling the iterative LSFP algorithm into a neural network. The low-rank and sparse priors and spatial sparsity of both low-rank and sparse components are utilized. The group-based reconstruction with periodic radial sampling makes LSFP-Net satisfy the online reconstruction requirement for real-time i-MRI. Simulated and clinical images were used to train and test the LSFP-Net. By deploying the trained LSFP-Net on a 3 T MRI scanner, we used a custom-designed interventional device to demonstrate the feasibility of the proposed i-MRI system for brain intervention. Imaging performance was evaluated using interventional phantoms and cadaver studies.

 

Key Results and Conclusions

In this study, we proposed LSFP-Net for real-time i-MRI-guided neurosurgery. For clinical application, a total of 2400 coronal slices (5 frames for each slice) generated from 23 patients with deep brain stimulation surgery were used for model training and validation. In addition, to demonstrate the performance and applicability in clinical scenarios, a custom-designed, MRI-compatible interventional device was used to construct an experimental system for brain intervention. A cadaver head was placed in a diagnostic scanner to further validate the performance of the model in neuro-intervention. With temporal resolutions of 80/732.8 ms for 2D/3D real-time MRI and in-plane spatial resolution of 1 × 1 mm2, the proposed method showed the potential to be integrated into diagnostic scanners for image-guided neurosurgery. However, for application in live patients, a fully fledged robotic system with required regulatory and ethical approval is required to ensure the safety of the patients. Future works include training and testing with a larger sample size, and integration of the proposed model to a robotic system for additional validations.

 

 

 

Fig. 1: LSFP-Net for real-time i-MRI reconstruction.

a A golden-angle radial sampling pattern (golden angle = 111.25°) was used for k-space data acquisition. b Multiple interventional images were reconstructed simultaneously in a group-wise way. c LSFP-Net was trained on a simulated dataset. d For inference, interventional images were reconstructed in real-time using the trained LSFP-Net.

 

 

Fig. 2: The preparation of the training dataset and a comparison of different methods.

a A simulated dataset of brain intervention was prepared. b A comparison of different methods. 10 spokes were used for the reconstruction of each frame (acceleration factor R = 20). Five frames per group were used for L + S, LSFP, CRNN, SLR-Net, L + S-Net, and LSFP-Net. c The magnified view of the interventional features. The area of the interventional feature is indicated by the red dashed box in b. d–f The quantitative metrics of the different methods. The pixel values were normalized without dimension. d, e The data are presented as mean values±standard deviation, and the sample size is n = 64. Source data of d–f are provided as a Source Data file.

 

 

Fig. 3: Results of different parameters for LSFP-Net.

a–c A comparison of different methods with R = 40, 25, and 10. d–f The results of different iterations of LSFP-Net. g–i The results of different convolution layers for the sparsity transform of LSFP-Net. j–l The results of different combinations of spokes per frame (SPF) and frames per group (FPG). a, b, g, h, j, k The data are presented as mean values±standard deviation, and the sample size is n = 64. Source data of a–l are provided as a Source Data file.

 

 

Fig. 4: A comparison of different methods on the simulated DBS electrode placement dataset.

a The dataset preparation is based on the postoperative MRI of DBS electrode placement. b The reconstruction results from different methods using 10 spokes. c–d The quantitative metrics of different methods. The pixel values were normalized without dimension. c the data are presented as mean values ±standard deviation, and the sample size is n = 188. Source data of c–e are provided as a Source Data file.

 

 

Fig. 5: Real-time MRI-guided brain intervention system.

a The MR-compatible components of the system were located in the MR scanner room. The ferromagnetic components were located in the control room. b The custom-designed interventional device. c The components in the control room. d The components in the MR scanner room. e The partially magnified view of d.

 

 

Fig. 6: Results of the interventional experiments with a fruit phantom and a porcine-brain phantom.

a In the fruit phantom, several red cherry tomatoes and green grapes were embedded into a cylinder filled with gelatin. b The interventional device was fixed on the cover of the cylinder and the combination was placed in the MR head coil. c Two porcine brains were embedded into a 3D-printed human skull model. d The porcine-brain phantom was placed in the MR head coil, and the interventional device was prepared. e Fully sampled 2D MR images were acquired before the intervention. f Real-time 2D MRI monitored the intervention using LSFP-Net for reconstruction. g Fully sampled 2D MR images after the intervention. h, i Comparison of the theoretical interventional depth and the measurements from real-time images. j A comparison of the reconstruction time of different methods on the fruit phantom experiment. k A comparison of different methods on two phantom intervention experiments. Source data of h–j are provided as a Source Data file.

 

 

Fig. 7: Results of the cadaver head interventional experiment.

a The interventional device was fixed to the cadaver head. An MR-compatible camera was used to monitor the movement of the interventional device. b The view from the MR-compatible camera. c A gelatin-filled glass fiber tube (indicated by a yellow triangle) was used for the trajectory planning. d Whole-brain T1W MR scan after intervention shows the final position of the ceramic needle (indicated by a yellow triangle). e Whole-brain T1W MR scan after needle withdrawal. f Fully sampled MR images with 3D GRE radial sequence showing the initial needle position before the intervention. g Real-time MRI monitors the interventional procedure with a temporal resolution of 732.8 ms/volume (3.66 s/group). h Fully sampled MR images with 3D GRE radial sequence indicate the final needle position after the intervention.

 

 

https://www.nature.com/articles/s41467-023-43966-w#Abs1