Scaffold of inference related documentation#378
Conversation
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
Updated contact information and added a brief introduction to inference services.
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
Adding description of inference services, as well as having both Stefano and Pablo as entry points for service design. |
Updated the title and introductory text for the LLM Inference API service documentation.
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
I made a few tweaks to file names and link names. |
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
Updated the LLM Inference API service documentation to clarify beta status, usage requirements, and token consumption details. Added information on service limitations, access requests, and future improvements.
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
|
||
| !!! todo | ||
| Instead of adding a screenshot, can't we | ||
| Add a screenshot to see how to obtain the API key |
There was a problem hiding this comment.
does this need a screenshot?
And, is the method for obtaining API keys documented elsewhere in our docs? If it is, link to that (and improve those docs if needed). That way we document everything once.
There was a problem hiding this comment.
@naevtamarkus I am aware of the process but I don't have access to it to take printscreens. Can you do this?
There was a problem hiding this comment.
We should say that this is temporary implementation (e.g., the API key is managed in a strange way vs. standards... i.e. the key is visible (not hidden), and not revokable, etc...)
There was a problem hiding this comment.
This is already explained in the limitations... I would avoid having "construction site" warnings all over the doc, but rather explain how it works "today" and change the doc when we change things... would you agree?
Updated the API documentation to include more details on participation in the Beta and clarified project key management.
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
Clarified data usage policies and removed redundancy regarding sensitive data.
This comment has been minimized.
This comment has been minimized.
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
|
|
||
| ### Obtain your authentication token | ||
|
|
||
| Approved projects are given an authentication token, which can be retrieved and managed through [project management portal][ref-account-waldur]. |
There was a problem hiding this comment.
The portal page currently has no info on how to retrieve and manage the token. Can that be added (here or on the portal page)?
| !!! example "Claude Code CLI" | ||
| Example environment configuration to be set before starting a `claude` session. | ||
| ```console | ||
| $ export ANTHROPIC_API_KEY=<AUTHENTICATION_TOKEN> | ||
| $ export ANTHROPIC_BASE_URL=https://ai-gateway.svc.cscs.ch/v1 | ||
| $ export ANTHROPIC_MODEL=apertus-70b-instruct | ||
| ``` |
There was a problem hiding this comment.
This seems to not work currently work with the llm-proxy url, which is openai compatible. One option seems to be to use a litellm proxy locally. This may need documenting or better: the anthropic url, if it exists, should be documented.
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
|
preview available: https://cscs-docs-preview.svc.cscs.ch/378 |
4 participants
I am creating this PR primarily to get a rendered version for discussion with Pablo and team.
Feel free to adapt to fit the overall guidelines of our docs.
https://cscs-docs-preview.svc.cscs.ch/378/services/inference