Does Khronos Perform Automatic Segmentation or Require User-Specified Classes?

[yongyun-iastate]

Khronos is using open-set segmentation, which combines CLIP and SAM. Does it require input from the user, such as specific object classes, or does it automatically segment any objects?

[Schmluk]

Hi @yongyun-iastate

Khronos can use any segmentation as input, in our open-set case we used this repository: https://github.com/MIT-SPARK/semantic_inference, where we clustered masks from Fast-SAM using CLIPS features. You can of course add other specifications, such as filtering for specific object classes of interest or filtering to remove background using the CLIP features (we played around with some of this). If you use the default launch files provided in khronos, no additional input will be needed.

Hope this helps!