Use MobileSAM on Android, enable NNAPI and unload unused AI models fast, fix inpainting

[waxm134]

Description

• AI model performance tweaks and bug fixes for Android

Type of Change

• Bug fix
• New feature
• Breaking change
• Performance improvement
• Code refactoring
• Documentation update
• UI/UX improvement
• Build/CI or Dependency update

Changes Made

• SAM2 model is heavy for android devices, switch to MobileSAM on Android to increase subject mask speed and lower RAM usage
• Enable NNAPI to leverage any NPU provided by the SoC
• unload unused AI models quickly to avoid OOM fails due to excessive RAM usage
• fixed inpainting on Android

Screenshots/Videos

Testing

• I have tested these changes locally and confirmed that they work as expected without issues

Test Configuration:

• OS: Ubuntu 24.04, Android 16
• Hardware: Intel i5, Pixel 9 Pro, Xiaomi Pad 7

Checklist

• My code follows the project's code style
• I haven't added unnecessary AI-generated code comments
• My changes generate no new warnings or errors

Additional Notes

• I did not notice any worsened performance of the Subject Mask tool with MobileSAM compared to SAM
• heavily reduces RAM usage on Android (Subject masks always made my Xiaomi Pad 7 crash due to OOM, now it runs fine)
• MobileSAM is currently embedded from this repo: https://huggingface.co/Acly/MobileSAM, should be added to the RapidRAW AI model repo (don't know what the procedure is there)
• MobileSAM git: https://github.com/ChaoningZhang/MobileSAM/tree/master
• while testing this I discovered an existing bug with the inpainting on android (inpainted area was always just black)
• inpainting can be fixed by using LAMA_Dilated instead of LAMA, as Android seems to have issues with the fp_16 operations of the LAMA model
• LAMA_Dilated cannot use NNAPI so it must be initialized without it, works on Pixel 9 Pro as well even if its not a Snapdragon processor so I guess it should work on non-Qualcomm processors in general as well
• LAMA_Dilated is currently embedded from this source: https://huggingface.co/qualcomm/LaMa-Dilated/tree/ab898502c9bd764a50eb2719a309694b43eae658
• LAMA_Dilated git: https://github.com/qualcomm/ai-hub-models/tree/main/src/qai_hub_models/models/lama_dilated
• LAMA_Dilated performance is a bit worse compared to desktop, but still better than having inpainting broken on mobile

AI Disclaimer:

Please state the involvement of AI in this PR:

• This PR is entirely AI-generated
• This PR is AI-generated but guided by a human
• This PR was handwritten with AI assistance (spell check, logic suggestions, error resolving)
• This PR contains only blood, sweat, and coffee (AI-free)

[CyberTimon]

Are you sure NNAPI gets utilized and it doesn't just silently fallback to CPU because the compiled onnxruntime library wasn't compiled with NNAPI in it?

[waxm134]

That's interesting. At least for the LAMA_Dilated I explicitly removed loading it via NNAPI because it failed with NNAPI enabled. I can try to debug this further, but in general I think it's worth to enable NPU usage on Android.

[Duecki1]

I've just tested this pr on my Pixel 7 Pro, which is not very powerful. AI Masking works and inpainting does too, without the app crashing.

[Duecki1]

@waxm134 do you think you could maybe look into that weird mask bug? I cannot find any solution to it. To replicate it, work with masks on an image for some time, at some random point you wont be able to click on mask components anymore. You can't select them anymore, only the main masks.

Screen_Recording_20260430_230508_RapidRAW.mp4

[waxm134]

@waxm134 do you think you could maybe look into that weird mask bug? I cannot find any solution to it. To replicate it, work with masks on an image for some time, at some random point you wont be able to click on mask components anymore. You can't select them anymore, only the main masks.

I can try to replicate the issue on my side and try to debug.

[Duecki1]

@waxm134 do you think you could maybe look into that weird mask bug? I cannot find any solution to it. To replicate it, work with masks on an image for some time, at some random point you wont be able to click on mask components anymore. You can't select them anymore, only the main masks.

I can try to replicate the issue on my side and try to debug.

Thank you, only if its okay for you tho. It happens often when im editing masks, but I can't find why.

[waxm134]

@waxm134 do you think you could maybe look into that weird mask bug? I cannot find any solution to it. To replicate it, work with masks on an image for some time, at some random point you wont be able to click on mask components anymore. You can't select them anymore, only the main masks.

I can try to replicate the issue on my side and try to debug.

Thank you, only if its okay for you tho. It happens often when im editing masks, but I can't find why.

I tried to reproduce on my Pixel 9 and cannot reproduce it. But on the Xiaomi Pad 7 I reproduced it once and then immediately after the buttons froze the app crashed so I guess it has to do with RAM pressure and OOM. I try to implement some simple fixes to reduce overall RAM usage at the moment, maybe afterwards it's gone.

[Duecki1]

@waxm134 do you think you could maybe look into that weird mask bug? I cannot find any solution to it. To replicate it, work with masks on an image for some time, at some random point you wont be able to click on mask components anymore. You can't select them anymore, only the main masks.

I can try to replicate the issue on my side and try to debug.

Thank you, only if its okay for you tho. It happens often when im editing masks, but I can't find why.

I tried to reproduce on my Pixel 9 and cannot reproduce it. But on the Xiaomi Pad 7 I reproduced it once and then immediately after the buttons froze the app crashed so I guess it has to do with RAM pressure and OOM. I try to implement some simple fixes to reduce overall RAM usage at the moment, maybe afterwards it's gone.

Really weird behavior :/. Thanks for testing tho <3. Hopefully we will be able to fix it :D