DeepPhD: Physics-informed self-supervised denoising for ultrasensitive fluorescence microscopy
By explicitly modeling heterogeneous noise components within a learnable flow and informing the image restoration module of noise parameters, DeepPhD reinforces noise decoupling and signal estimation without requiring any clean images.
Comprehensive noise modeling and self-supervised parameter learning.
DeepPhD is grounded in a comprehensive physical dissection of image degradation.
We designed a normalizing flow backbone that learns two reciprocal transformations
to implement explicit noise modeling and parameter learning. Heterogeneous noise
components can be automatically learned from raw noisy data, providing a solid
noise model for denoising.
Physics-informed noise decoupling and image restoration.
By informing the image restoration module of the exact noise model, DeepPhD
accurately decouples and removes heterogeneous noise components, including
fixed-pattern noise (FPN), row noise (RN), and mixed Poisson–Gaussian noise (MPGN).
Fluorescence signals can be recovered from severe noise with enhanced performance
and reliability.
State-of-the-art performance and advanced biological applications.
DeepPhD achieves state-of-the-art denoising performance on different noise
configurations and noise levels with superior physical fidelity. We demonstrated
the capability of DeepPhD on various imaging modalities and experimental scenarios,
including light-sheet imaging of larval zebrafish, neural recording of freely
behaving mice, and multiphoton imaging of immune cell migration. Extensive
qualitative and quantitative evaluations demonstrate that DeepPhD can extend the
performance and interpretability of fluorescence image denoising.
Code and datasets.
We provide a companion webpage for DeepPhD and relevant resources. All source code
and data (~300 GB, including simulated datasets, experimentally obtained images of
light-sheet microscopy, head-mounted miniaturized microscopy, and multiphoton
microscopy) are being made publicly available to the community to facilitate
relevant research.
Physics-informed principle
DeepPhD is grounded in a comprehensive physical dissection of image degradation, directly linking noise-formation mechanisms to computational noise removal.
We designed a normalizing flow backbone that learns two reciprocal transformations to implement explicit noise modeling and parameter estimation.
By combining the flow-based noise model with a denoising network, DeepPhD can accurately decompose and remove complex noise components.
More information
From biological specimens to digital images, fluorescence photons undergo multiple stages of photoelectric conversion, amplification, readout, and digitization, during which distinct noise sources are accumulated.
More information
In the physical modeling module, noise parameters are estimated, including the system gain α, the Gaussian noise variance β, and the FPN pattern. For image restoration, we designed a two-stage scheme composed of noise decoupling and signal estimation.
Comprehensive evaluation
Physical model validation
Synthetic data
Beyond denoising performance, we evaluated the physical consistency and interpretability of the physics-based features learned within the noise-flow. Consistent with theoretical expectations, FPN and RN can be accurately estimated at the pixel level, and the decoupled intermediate MPGN images show high statistical agreement with GT, highlighting the ability of DeepPhD to estimate instrument-related noise characteristics.
More information
Estimated consistency of FPN and RN: GT versus DeepPhD learned noise patterns. Statistical consistency of MPGN modeling: probability density functions of the DeepPhD learned MPGN image and GT are overlaid on the same axes. Denoising performance of synthetic 3D lysosome images with complex noise components: raw, GT, and DeepPhD.
Extensive numerical experiments show that the system gain (α) and the Gaussian variance (β) can be accurately estimated over a wide range of values. Across a wide range of noise intensities, DeepPhD can learn the exact patterns of FPN and RN, thereby removing these two noise components with small residuals.
More information
Scatterplots showing MPGN parameters learned by DeepPhD versus GT. α and β represent the system gain and the Gaussian variance in the MPGN model, respectively. Estimation accuracy of FPN and RN at different noise levels: mean absolute error between learned noise patterns and GT, and estimation errors when the noise standard deviation is 10, 50, and 100.
Experimental data
The multi-frame average projection of the raw image stack is compared with the FPN pattern estimated by DeepPhD, alongside a representative raw frame and the corresponding denoised frame. MPGN and RN vary between different frames, whereas FPN remains fixed. Averaging multiple frames therefore reduces MPGN and RN while retaining FPN, which is especially apparent in signal-free regions.
More information
Magnified views of the boxed signal-free regions are shown below. Scale bars, 100 μm for the whole FOV and 10 μm for magnified views.
Denoising performance
Synthetic data
Among these methods, DeepPhD produced results with superior fidelity to the GT images and achieved the highest and most stable output SNR across all noise configurations. DeepPhD demonstrates superior robustness to structured noise across the evaluated range.
More information
Denoising results on synthetic volumetric imaging data of interphase HeLa cells. Magnified views of boxed regions show subcellular structures restored by different denoising methods. From left to right: raw images, SN2N, SUPPORT, DeepInterpolation, DeepCAD, SRDTrans, DeepPhD, and GT. Quantitative comparison of DeepPhD and other denoising methods on different noise configurations and at different noise levels.
Experimental data
Compared with existing methods, DeepPhD not only suppresses random noise but also effectively eliminates spatiotemporally structured artifacts without distorting the underlying biological morphology, thereby unveiling the spatial distribution of GABAergic neurons in larval zebrafish.
More information
Comparing different denoising methods on light-sheet imaging of larval zebrafish. Representative light-sheet images for comparing different denoising methods on GABAergic neurons in larval zebrafish. Magnified views of the boxed regions are shown below each panel.
Biological applications
Ultrasensitive light-sheet imaging of GABAergic neurons in larval zebrafish
Although the raw images are heavily contaminated by both random and structured noise, DeepPhD can effectively suppress noise components and restore underlying fluorescence signals. GABAergic neurons in the brain and spinal cord can be clearly reconstructed in 3D after denoising.
After DeepPhD denoising, fine neuronal morphology can be preserved in close correspondence with the high-SNR reference, and individual neurons are clearly resolved. Statistical analysis and hypothesis testing confirmed significant increases in image correlation for all three dimensions after denoising.
DeepPhD was used to process the calcium imaging data for ultrasensitive characterization of calcium transients. Many subtle and clustered spiking events previously invisible in the raw images become clearly discernible after denoising. The kinetics of the calcium indicator are also well preserved.
High-fidelity neural recording of freely behaving mice with head-mounted miniaturized microscopy
We built a neuroethological recording platform combining miniaturized widefield microscopy and unsupervised animal tracking for simultaneous behavioral and neural analysis.
Although only a limited number of neurons are visible in the raw data due to severe noise, calcium transients from many neurons reemerge after denoising. Fluorescence traces extracted from consecutive frames also capture spiking events with higher fidelity.
DeepPhD facilitates more accurate neuron extraction and substantially reduces false-positive pixels in the segmentation mask. The neuron diameters obtained from denoised images are concentrated in the range of 10–15 µm, which is consistent with previously reported values.
After DeepPhD denoising, the spike rate of the observed neurons significantly increased when companions were nearby. DeepPhD enables high-fidelity neural recording in neuroethological research and reveals the phenomenon that cortical neurons tend to be more active when the mouse is surrounded by companions.
DeepPhD restores dendritic continuity and enables reliable identification of individual postsynaptic spines, with spine positions and morphologies closely matching the high-SNR references. It preserves rapid and low-amplitude calcium transients.
These traces clearly indicate that calcium signals in spines on the same dendritic branch still exhibit different dynamics, which could provide evidence for the functional heterogeneity of dendritic spines.
Trained solely on the low-SNR raw data, DeepPhD can learn to restore the morphological structures and migration dynamics of neutrophils in the FOV. It can reveal the evolution of neutrophil uropods and resolve polarized rear compartments that coordinate contraction, adhesion release, and forward cell translocation during migration.
Improved delineation of cell boundaries enables more accurate neutrophil segmentation, resulting in reduced false-negative and false-positive pixels in segmented cell masks. Segmentation accuracy is quantified with F1 score and Intersection-over-Union (IoU) score.
DeepPhD further enhances neutrophil tracking to obtain more continuous migration trajectories. DeepPhD increases the number of trajectories lasting longer than 30 s from 176 to 243, and those lasting longer than 3 min from 16 to 51.
