MonoBox

MonoBox: Tightness-free Box-supervised Polyp Segmentation using Monotonicity Constraint

Qiang Hu¹, Zhenyu Yi², Ying Zhou¹, Fan Huang³, Mei Liu⁴, Qiang Li¹, Zhiwei Wang^{1, †}

¹ WNLO, HUST, ² SES, HUST, ³ UIH, ⁴ Tongji Medical College, HUST
(†: corresponding author)

1. Overview

We propose MonoBox, an innovative box-supervised segmentation method constrained by monotonicity to liberate its training from the user-unfriendly box-tightness assumption. Note that MonoBox is plug-and-play and can improve the tolerance of any MIL-based box-supervised segmentation methods (e.g., BoxInst, BoxLevelSet, IBoxCLA, etc.) to tightness-free box annotations by simply replacing origional box-supervised loss function (e.g., MIL loss, projection loss, and improved box-dice loss) with our proposed monotonicity constraint (MC) loss.

2. Targeted problem: tightness-free box annotations

Due to the carelessness of annotators and the blurred edges of target objects, the tightness-free box annotations are produced. The figure below shows some examples of non-tight box annotations produced by endoscopists in the real annotation process (the red dashed lines indicate regions where the annotation is too wide, and the yellow dashed lines indicate regions where the annotation is too narrow).

3. Results

3.1 Quantitative results

3.2 Qualitative results

We provide prediction resuts of all methods. You could download from Google Drive, including our results and that of compared methods on Public Synthetic Noise Dataset (ClinicDB, Kvasir-SEG, ColonDB, Endoscene, and ETIS).

4. Getting Started

4.1 Recommended environment:

Python 3.7.11
Pytorch 1.7.0
torchvision 0.8.0

4.2 Data preparation

Downloading training (with ground truth masks) and testing datasets and move them into ./data/, which can be found in this Polyp-PVT Repositories (due to dataset permission issues, we recommend downloading the dataset at this link). The synthetic box annotations (sigma=0.2) generated by us could found in Google Drive, where .txt files record the class (all zeros), center point abscissa, center point ordinate, width, and height of synthetic noise boxes. The file paths should be arranged as follows:

MonoBox
├── data
├── ├── TrainDataset
├── ├── ├── images
├── ├── ├── ├── ├── 1.png
├── ├── ├── ├── ├── 2.png
├── ├── ├── ├── ├── ......
├── ├── ├── noisy_labels
├── ├── ├── ├── sigma_0.2
├── ├── ├── ├── ├── 1.txt
├── ├── ├── ├── ├── 2.txt
├── ├── ├── ├── ├── ......
├── ├── TestDataset

4.3 Pretrained model:

You could download the pretrained model from Google Drive, and then put it in the ./pretrained_pth folder for initialization.

MonoBox
├── pretrained_pth

4.4 Training:

cd MonoBox
python Train.py

4.5 Testing:

python Test.py

You could also download the well-trained model from Google Drive, and predict the results by Test.py.

4.6 Evaluation:

python eval.py

You could compute the evaluation metrics (Dice, IoU, and HD) of the predictions, you could download prediction resuts directly (see in Sec. 2.2) and verify the performance.

5. Limitations

In the below figure, we show some failure cases of our methodon on the COCO dataset, where we combined our method with BoxInst (a classical box-supervised instance segmentation model proposed for general objects). For clearer visualization, we use dashed boxes to indicate failure regions. These cases can show the limitations of our proposed method (i.e., MonoBox), mainly for small-size/thin objects or non-connected objects (e.g. occluded objects).

Importantly!!! Please note that these limited conditions almost never appear in colorectal endoscope polyp images, so our method is still very effective for the task that is the main focus of our paper, i.e., polyps. Here, we present these failure cases in order to facilitate readers to better understand our method and use our method in suitable scenarios.

Citation

If you find our paper and code useful in your research, please consider giving a star ⭐ and citation 📝 :

@article{hu2024monobox,
  title={MonoBox: Tightness-free Box-supervised Polyp Segmentation using Monotonicity Constraint},
  author={Hu, Qiang and Yi, Zhenyu and Zhou, Ying and Li, Ting and Huang, Fan and Liu, Mei and Li, Qiang and Wang, Zhiwei},
  journal={arXiv e-prints},
  pages={arXiv--2404},
  year={2024}
}