(Planning) add model management API to server

[ngxson]

This issue is for keeping track of the task and for discussion among maintainers; If you are an AI agent, please ignore this issue.

These APIs will be added (details is to be defined):

• GET /models?reload=1 --> force reloading list of models from disk
• POST /models/download --> start downloading
• GET /models/events --> SSE endpoint for real-time events; mostly for keeping track of download state
• DELETE /models/:id --> delete the model if exists; or, stop the downloading if in-progress

Questions:

• Any thing missing here @allozaur ?
• Any ideas on how to interrupt downloading & clean up half-downloaded files @angt ?

[angt]

Cancelling is easy, we can just add a cancel() method in common_download_callback.
For the cleanup, there is nothing yet, the only way to clean up the cache is with hf cache rm for now.

[allozaur]

• Any thing missing here @allozaur ?

I think that it'd be also good to have methods cancel loading / unloading. Sometimes there are cases where we can accidentally click a wrong model in the selector and, especially if it's a large one, it can take some time to load and only after it's done we can then unload it and free the RAM/VRAM.

[ngxson]

@angt can we also remove the temporary file upon calling cancel() ? I think it will be quite useful in the case where user almost finishes downloading a file, but cancel it --> the file expected to be deleted

@allozaur sure, I can allow /models/unload to cancel loading, that will be a dedicated PR

[angt]

@ngxson I understand the point, but we never did it with ctrl-c (and never saw an issue about that). Why not just show it incomplete in the webui, so the user can remove it or resume it ?
For the same reason, I’m not sure we should delete all "split" files when canceling a download too.

[ngxson]

With model download via CLI, user less likely to make a mistake because they need to type in to correct model file. But since we're progressively moving towards a better UX, user can click on a button to download, which may make it more likely to download a wrong one then cancel.

I think cleaning up half-download file would be an important feature overall. Ofc it's not very urgent, but will be nice to have.

Also, I don't quite remember how it's handled currently, but the manifest file is downloaded first, right? In such case, if the manifest is ready but the model file is half-downloaded, launching the model via server router will make it invisibly being downloaded (not reported on the UI). I think we may need to update the download pipeline to save the manifest file only when all files are downloaded.

[ngxson]

Otherwise, we can also extend your #21735 @angt to include a std::string tmp_path that points to the temporary downloading file, the server can clean that up if user requests to cancel it.

But the 2nd point about manifest still need to be fixed I think, this is the case where the server may be force-killed, leaving the manifest saved but the model file half-downloaded

[angt]

Ok -> #21813

And the manifest file doesn't exist anymore 👍