Unsupervised-MIseg

This repository contains the code to implement unsupervised medical image segmentation using edge mapping and adversarial learning as described in our paper: Unsupervised Medical Image Segmentation with Adversarial Networks: From Edge Diagrams to Segmentation Maps.

Note that this version was written so that it is easier to follow the logic of the approach. It is therefore meant to be used for scripting. It should be fairly easy to adapt it for different purposes, but we may add other versions for different kinds of users (especially if requested).

Requirements

For now, different parts of this project were adapted from different libraries, and so the dependencies vary by which step of the process is being run. The following are needed to run all parts of the code:

• Python 3
• Tensorflow
• Keras
• Caffe
• Pytorch
• Matlab
• scipy
• Scikit-learn
• Scikit-image
• Open-CV

Additional Resources (borrowed and adapted code)

We use models and algorithms from the following repositories:

• nVidia's pix2pixHD for our GAN architecture
• Richer Convolutional Features for edge detection
• Structured Edge Detection for skeletonizing the output of RCF
• Mask R-CNN for the ISIC 2018 segmentation model

Comparison with W-net:

• This W-net repo for our W-net comparison
• We adapted the Dense CRF from this repo for our dcrf.py implementation, used for post-processing the W-net outputs.
• We also used this for segmentation-grouping to clean up the output (and consolidate segmentation classes for the W-net)

Using this method

Most of the functions can be kept as is and easily adapted to other datasets. However ,the following two things need to be done for each dataset:

• Preprocess the images according to your needs. Our models currently take images in 256x256 pixels, but that can be changed in the code and the models can be retrained.
• You will need to write a custom function for construction edge diagrams according to the shape of the objects that need to be segmented in your images.
• Once these two steps are done, follow the steps laid our in one of the workflow.py files to train the GAN and your preferred segmentation model.

Some results from our paper

Real (first four columns) vs. Synthetic (last four columns) kidney ultrasounds

Real (first four columns) vs. Synthetic (last four columns) ISIC skin lesion images

Real test kidney ultrasound segmentation results (green: clinician-provided labels, red: supervised model, blue: our unsupervised approach)

Real test ISIC skin lesion segmentation results (red: supervised model, blue: our unsupervised approach)