Skip to content

Allow configuration of prompt timeout for AgenticTool #1508

@larswaechter

Description

@larswaechter

Hello there!

Embabel offers a config that allows me to set a global LLM prompt timeout: embabel.llm-operations.prompts.default-timeout (see docs).

In my scenario I'm using a SimpleAgenticTool which calls subagents via tools. It would be nice to run the orchestrator with a custom timeout which overrides the global one:

    SimpleAgenticTool orchestrator = new SimpleAgenticTool(
        "example-orchestrator",
        """
            Analyse the customer request and delegate to the appropriate domain agent tool.
            """
    )
        .withTools(
            List.of(
                Subagent.ofClass(FirstAgent.class).consuming(MyData.class),
                Subagent.ofClass(SecondAgent.class).consuming(MyData.class)
                // ...
            )
        )
        .withTimeout(Duration.ofSeconds(120)) // Custom timeout here
        .withParameter(Tool.Parameter.string("request", "User request to evaluate"));

The problem I have is that each subagent only needs a few seconds. However, since multiple tools can be called, the orchestrator might require more time and therefore needs a higher timeout than a single subagent. So it would be nice to set a certain timeout only for the orchestrator since I don't want to override the global timeout.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions