First-hand information for everyone
This paper describes a compact, synchronization-free system that uses a single projected light pattern and an AI model to estimate depth in real time for laparoscopic (minimally invasive) surgery. The team replaced complex multi-shot projection systems with a passive LED-illuminated binary mask and a neural network that learns depth directly from the patterned images. The goal is practical, video-rate depth sensing inside small endoscopes without the hardware and timing demands of conventional methods.
The researchers mounted a small projection module on one channel of a dual-channel laparoscope while the other channel imaged the scene lit by the binary pattern. They collected 722 paired images of surgical phantoms together with reference depth maps from a Zivid 3D camera. Those Zivid depth maps were reprojected into the single-shot structured-light endoscopy (SSLE) image frame and used as supervised ground truth for training and testing the AI model.
On the algorithm side they used a vector-quantized variational autoencoder (VQ-VAE) to compress each input image into a small set of discrete codes. A U-Net style network operating in that compressed, or latent, space then predicted depth maps. In plain terms: the system first turns the raw image into a compact codebook representation and then decodes depth from that compact code, without a separate step to predict the illumination mask or to segment the scene.
On a fixed train/validation/test split the method produced a mean absolute error (MAE) of 3.70 mm and an absolute relative error (AbsRel) of 0.0326. Two common threshold accuracy measures reported as delta=1.1 and delta=1.1^2 were 0.962 and 0.970, respectively; these “delta” numbers indicate the fraction of depth estimates that fall within specified relative error bounds. The pipeline ran at about 26.0 frames per second on an NVIDIA A100 GPU across 301 consecutive frames. The authors report lower MAE than a baseline made of two U-Nets (one for mask prediction and one for depth) and better performance than off-the-shelf monocular depth models on their test set.
Important caveats are noted. The experiments used phantom objects rather than live tissue, so real surgical scenes may behave differently. The paper emphasizes the importance of dataset size and the calibration accuracy between the SSLE camera and the Zivid reference camera; errors in that calibration or too-small training sets will hurt performance. Finally, the results come from a laboratory setup and a single GPU, so clinical use would require further validation and engineering.
In short, the work shows a practical path to video-rate depth sensing in compact laparoscopes by combining a simple LED pattern with a latent-space neural decoder. The results are promising for phantom tests, but more data, careful calibration, and testing on real surgical scenes are needed before clinical deployment.