Embabel Ollama: Support OAuth2 Bearer + custom TLS

[biiyamn]

Hi, We’re using Embabel 0.3.4 with an Ollama endpoint that requires an OAuth2 Bearer token and a custom TLS certificate. At startup, Embabel can’t discover models because the /api/tags request is unauthenticated, so it returns 401 and zero models are registered.

What is the supported way to add Bearer/OAuth2 auth and apply custom trust/SSL to all Ollama requests (both /api/tags discovery and the OllamaApi chat/embedding endpoints)?

11:13:09.911 [main] INFO OllamaModelsConfig - Using default Ollama instance at [url]
11:16:03.987 [main] WARN OllamaModelsConfig - Failed to load models from [url] : 401 Unauthorized:...
11:16:03.987 [main] WARN OllamaModelsConfig - No Ollama models discovered from default instance at [url]. Check server configuration.
11:16:03.998 [main] INFO OllamaModelsConfig - Ollama: Initialized 0 LLM(s) and 0 embedding(s)

[poutsma]

To resolve this issue, the RestClient created at OllamaModelsConfig.kt#92 needs to be customizable. The obvious approach would be to accept a RestClient.Builder as a constructor argument, consistent with how OllamaApi is already configured
in the same class (lines 154–157 and 201–204).

The problem is that there is already a ClientHttpRequestFactory accepted as a constructor argument here, which is used to configure the OllamaApi (see above). The question is if and how these two arguments (RestClient.Builder and ClientHttpRequestFactory) can coexist, or if we replace the ClientHttpRequestFactory with the RestClient.Builder, since the latter has a larger configuration scope than the former.

Thoughts, @alexheifetz, @igordayen?

[igordayen]

Request factory is a property within RestClient Builder:

RestClient.Builder
    ├── .requestFactory(...)       ← TLS certificates  lives here ==> truststore pwd can be added
    └── .requestInterceptor(...)   ← OAuth lives here  ==> can be added 

if i'm not mistaken - happyly coexist:)

@biiyamn - are you planning to use spring security yaml to configure client secret, token, and truststore password?

[biiyamn]

Thank you for your responses, yes we use Spring Security for OAuth2/Bearer token handling, and Spring SSL/truststore configuration

[poutsma]

Request factory is a property within RestClient Builder:

RestClient.Builder
    ├── .requestFactory(...)       ← TLS certificates  lives here ==> truststore pwd can be added
    └── .requestInterceptor(...)   ← OAuth lives here  ==> can be added 

if i'm not mistaken - happyly coexist:)

I am not sure if you meant requestFactory coexisting with interceptors, or with a RestClient.Builder, but I think both are a bad idea 😉:

• requestFactory coexisting with RestClient.Builder: accepting both as separate constructor arguments creates an ambiguity: should the class apply the ClientHttpRequestFactory to the provided builder, or does it assume the caller has already done so? What if both are provided but configured inconsistently?
• requestFactory coexisting with requestInterceptors: if we added a dedicated interceptor parameter to support OAuth2, the next reporter might need to configure timeouts, a custom UriBuilderFactory, or something else entirely, and we'll end up re-exposing RestClient.Builder piece by piece.

Instead, I think we should replace the ClientHttpRequestFactory parameter with a RestClient.Builder parameter entirely. The caller can set the factory on the builder themselves if needed.

[igordayen]

@biiyamn - may i inquire whether your security setup is using yaml, if yes - could you please share it with values of course substitute by XXXXXXX

[igordayen]

@poutsma - Arjen, could you please elaborate:

val restClient = restClientBuilder
    .requestInterceptor(OAuth2ClientHttpRequestInterceptor(...))  // OAuth
    .requestFactory(HttpComponentsClientHttpRequestFactory(...))  // TLS
    .build()

or

    .apply(oauth("ollama"))    // adds interceptor to builder
    .apply(tls("ollama"))      // adds requestFactory to builder
    .build()                   // one RestClient with both configured

i was referring to code snippets above as "co-existence", perhaps wording is confusing.
Thank you

[poutsma]

@igordayen I misread your original comment, and think we are actually in agreement.

I was looking at this from the perspective of the OllamaModelsConfig constructor, where ClientHttpRequestFactory is currently a separate parameter. My concern was about having both that and a RestClient.Builder as distinct constructor arguments, which would be ambiguous.

But yes, your snippets show exactly the right approach: inject a single RestClient.Builder into the constructor, and the caller configures whatever they need on it: interceptor or default headers for OAuth2, requestFactory for TLS, or both together.

[igordayen]

and think we are actually in agreement.

yes!

[biiyamn]

Here is the configuration style that works for us in Spring AI :

spring:
  security:
    oauth2:
      client:
        registration:
          authentik:
            authorization-grant-type: refresh_token
            client-id: X-Ollama
            scope: openid,entitlements,offline_access
        provider:
          authentik:
            token-uri: https://<token-uri>.com/application/o/token/
  ssl:
    bundle:
      pem:
        authentik:
          truststore:
            certificate: classpath:authentik.crt
        ollama:
          truststore:
            certificate: classpath:ollama.crt

[igordayen]

@biiyamn - thank you.

i would like to incept broader discussion here whether authentication problem specific to Ollama or having broader implications.

Imagine corporate cloud deployment, multitenant cluster. agent-microservice is not alowed to use external Open AI compatible URLs or external URLs got intercepted by outgress controller / proxy server. Proxy server got maintained by corporate cloud security team. Discussion points:
a. LLM Authentication Token is managed centrally by corporate security team, token injection gets applied on proxy server, is it valid scenario?
b. AFAIK - even within the same cluster agent-micro / proxy communication channel is still subject to TLS SSL encryption.

Should we follow up with community on "general" forum on security implication in general on using Open AI URL behind the firewall?