Fat-U-Net: Non-Contracting U-Net for Free-Space Optical Neural Networks

In brief

Applies the FatNet approach to U-Net for image segmentation, removing the pooling steps so that resolution stays high throughout. On a 4f optical system, Fat-U-Net is estimated to run 538 times faster than U-Net on the same optical hardware and 37 times faster than U-Net on a GPU, with IoU reductions of 4.24% on Oxford-IIIT Pet and 1.76% on HeLa cell nuclei.

Two grey electron-microscopy images of HeLa cells side by side, labelled Fat-U-Net and U-Net, with segmented cell nuclei highlighted in orange. The two segmentations look very similar. — Fat-U-Net (left) and U-Net (right) nucleus segmentation on an 8192 × 8192 HeLa cell image. Figure 4 from the paper.

Abstract

This paper describes the advantages and disadvantages of adapting the U-Net architecture from a traditional GPU to a 4f free-space optical environment. The implementation is based on an optical-based acceleration called FatNet and thus this adaption is called Fat-U-Net. Fat-U-Net neglects the pooling operations in U-Net, but maintains a similar number of weights and pixels per layer as U-Net. Our results demonstrate that the conversion to Fat-U-Net offers significant improvement in speed for segmentation tasks, with Fat-U-Net achieving a ×538 acceleration in inference compared to U-Net when both are run on optical devices and ×37 acceleration in inference compared to the results provided by U-Net on GPU. The performance loss after conversion remains minimal in two datasets, with reductions of 4.24% in IoU for the Oxford IIIT Pet dataset and 1.76% in IoU of HeLa cells nucleus segmentation.

Citation

Riad Ibadulla, Constantino C. Reyes-Aldasoro, Thomas M. Chen. “Fat-U-Net: Non-Contracting U-Net for Free-Space Optical Neural Networks.” Proc. SPIE 12903, AI and Optical Data Sciences V (SPIE Photonics West), 2024. doi:10.1117/12.3008618