Abstract:

Photoacoustic microscopy (PAM) has gained increasing popularity in biomedical imaging, providing new opportunities for tissue monitoring and characterization. With the development of deep learning techniques, convolutional neural networks have been used for PAM image resolution enhancement and denoising. However, there exist several inherent challenges for this approach. This work presents a Unified PhotoAcoustic Microscopy image reconstruction Network (UPAMNet) for both PAM image super-resolution and denoising. The proposed method takes advantage of deep image priors by incorporating three effective attention-based modules and a mixed training constraint at both pixel and perception levels. The generalization ability of the model is evaluated in details and experimental results on different PAM datasets demonstrate the superior performance of the method. Experimental results show improvements of 0.59 dB and 1.37 dB, respectively, for 1/4 and 1/16 sparse image reconstruction, and 3.9 dB for image denoising in peak signal-to-noise ratio. (351 patients) deferred after FFR measurement with available RWS data and coronary computed tomography angiography. On coronary computed tomography angiography, HRP was defined as a lesion with both minimum lumen area ＜4 mm² and plaque burden ≥70%. The primary outcome was target vessel failure (TVF), a composite of target vessel revascularization, target vessel myocardial infarction, or cardiac death.

Fig. 1 Illustration of the network architecture of the proposed method. The upper part shows the key components of our UPAMNet. UPAMNet consists of three modules, i.e., the feature contraction module, the feature connection module, and the feature expansion module. Based on deep image priors, we design three attention blocks to improve the performance and exploit the semantic segmentation to propose a combined training constraint at both pixel and perception levels. To better evaluate the generalization ability of pre-trained models, we implement few-shot and zero-shot transfer learning to conduct experiments on unseen datasets. The detailed architecture of the ResConv block is shown at the left bottom.

Fig. 2  Visualization of the detailed architecture of three proposed attention blocks. a. The orange part represents the spatial attention block. b. The green part represents the positional attention block. c. The blue part represents the oriented attention block.

Fig. 3   Visualization results of in vivo PAM image super-resolution (×4) and denoising. (a,b)-1: Low-quality images with image size 256 × 128 collected by 80 nJ excitation light energy. (a,b)-2: Reconstructed image by our method with image size 1024 × 512. (a,b)-3: Ground truth images with image size 1024 × 512 collected by 320 nJ excitation light energy. (c)-1: Low-quality images with image size 256 × 136. (c)-2: Reconstructed image by our method with image size 1024 × 544. (d)-1: Low-quality images with image size 256 × 184. (d)-2: Reconstructed image by our method with image size 1024 × 736. We show the zoomed images of the colorful shaded areas on the right.

DOI：10.1016/j.pacs.2024.100608