Is btw compatibile with chat_openai_compatible() ?

[Malarkey73]

I can't get any btw functions to work with local models using chat_openai_compatible()

I tested the model below and it's fine as a basic interactive chat when used staright but the following produces no error or any response

chat <-  btw_client(
  client = chat_openai_compatible(
    base_url = "http://localhost:1234/v1",
    name = "OpenAI-compatible",
    system_prompt = "You are a helpful assistant.",
    api_key = NULL,
    credentials = NULL,
    model = "olmo-3-7b-instruct-sft-bfloat16",
    params = NULL,
    api_args = list(),
    api_headers = character(),
    echo = c("none", "output", "all")
  )
)

chat$chat(
  "How can I replace `stop()` calls with functions from the cli package?"
)

Am I misunderstanding something? What is missing?

[gadenbuie]

Does this work for you? (i.e. does everything work okay if you take btw out of the equation?)

client <- chat_openai_compatible(
  base_url = "http://localhost:1234/v1",
  name = "OpenAI-compatible",
  system_prompt = "You are a helpful assistant.",
  api_key = NULL,
  credentials = NULL,
  model = "olmo-3-7b-instruct-sft-bfloat16",
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = c("none", "output", "all")
)

client$chat("Just say hello if you can hear me")

(By "it's fine as a basic interactive chat when used staright" you probably meant that this works fine, but I'm just double checking.)

[gadenbuie]

Maybe echo = c("none", "output", "all") is the culprit? That would likely result in no output being shown in the console (effectively, echo = "none"), which might appear like "produces no error or any response".

[Malarkey73]

Maybe echo = c("none", "output", "all") is the culprit? That would likely result in no output being shown in the console (effectively, echo = "none"), which might appear like "produces no error or any response".

No I deleted that. Nothing.

Then tried it with echo = "output" and echo = "all". Nothing.

It's working fine without btw

i.e. as you suggest:

client <- chat_openai_compatible(
  base_url = "http://localhost:1234/v1",
  name = "OpenAI-compatible",
  system_prompt = "You are a helpful assistant.",
  api_key = NULL,
  credentials = NULL,
  model = "olmo-3-7b-instruct-sft-bfloat16",
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = c("none", "output", "all")
)

client$chat("Just say hello if you can hear me")
Hello! I can hear you. How can I assist you today?

[gadenbuie]

Are you using LMStudio? I didn't see that specific model ID but found a similar one and can replicate the issue without btw:

library(ellmer)

client <- chat_openai_compatible(
  base_url = "http://localhost:1234/v1",
  name = "OpenAI-compatible",
  system_prompt = "You are a helpful assistant.",
  api_key = NULL,
  credentials = NULL,
  model = "olmo-3-7b-instruct-mlx",
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = c("none", "output", "all")
)
client$register_tool(tool(
  function() sample(1:100, 1),
  name = "pick_random_number",
  description = "Picks a random number between 1 and 100"
))

client$chat("Pick a random number for me")
#> I'm sorry, but I can't assist with that. Based on the available API, I can only
#> pick a random number between 1 and 100 if you request it explicitly using the 
#> `pick_random_number` function. Let me know how you'd like to proceed!

[Malarkey73]

Thanks
Yes I'm using lm-studio too. And I get much the same response with the nearly the same mlx model olmo-3-7b-instruct-sft-bfloat16.

client$chat("Pick_random_number for me")
"I can't pick a random number directly, but I can assist by instructing the API to 
do so. Would you like me to generate a random number between 1 and 100 for you?"
client$chat("Yes")
"I'm sorry, but I can't assist with that. The API only allows me to pick a random 
number between 1 and 100, not specifically for you. Let me know if there's 
anything else you need help with!"
client$chat("Please pick_random_number")
"I'm sorry, but I can't assist with that. The API only allows me to pick a random 
number between 1 and 100, not specifically for you. Let me know if there's 
anything else I can help with!"

I think that's working that way but it's just being very obtuse??? Not sure how to actually make it answer.

Completely inexplicably I just deleted the client, ran your same code again.... and now it's working

client$chat("Pick_random_number for me")
"◯ [tool call] pick_random_number()
● #> 51
You picked the number: 51."

client$chat("Pick a random number for me")
"◯ [tool call] pick_random_number()
● #> 74
You picked the number: 74."

That is indeed so random. So weird. Trying to use btw_client() still breaks it.

Same behaviour with mistralai/devstral-small-2-2512 model. Will do random number code above but not btw_client().