Skip to content

[Bug] openai_agents/sandbox activities throw only retryable exceptions #1548

@mfateev

Description

@mfateev

Describe the bug

Sandbox integration activities pass all exceptions from underlying library as they are. So they are all treated as retryable. This can lead to situations when workflows will get stuck on exceptions that can never succeed. Here is an exception I've got after the sandbox was stopped by an external command:

"stackTrace": "  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/.venv/lib/python3.12/site-packages/temporalio/worker/_activity.py\", line 359, in _handle_start_activity_task
    result = await self._execute_activity(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/.venv/lib/python3.12/site-packages/temporalio/worker/_activity.py\", line 714, in _execute_activity
    return await impl.execute_activity(input)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/.venv/lib/python3.12/site-packages/temporalio/contrib/openai_agents/_trace_interceptor.py\", line 323, in execute_activity
    return await self.next.execute_activity(input)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/.venv/lib/python3.12/site-packages/temporalio/worker/_activity.py\", line 909, in execute_activity
    return await input.fn(*input.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/.venv/lib/python3.12/site-packages/temporalio/contrib/openai_agents/sandbox/_sandbox_client_provider.py\", line 218, in session_stop
    await session.stop()

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/sandbox/session/sandbox_session.py\", line 60, in _wrapped
    return await self._annotate(
           ^^^^^^^^^^^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/sandbox/session/sandbox_session.py\", line 400, in _annotate
    value = await run()
            ^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/sandbox/session/sandbox_session.py\", line 495, in stop
    await self._inner.stop()

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/sandbox/session/base_sandbox_session.py\", line 275, in stop
    await self._persist_snapshot()

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/sandbox/session/base_sandbox_session.py\", line 292, in _persist_snapshot
    await snapshot_lifecycle.persist_snapshot(self)

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/sandbox/session/snapshot_lifecycle.py\", line 31, in persist_snapshot
    workspace_archive = await session.persist_workspace()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/extensions/sandbox/daytona/sandbox.py\", line 1036, in persist_workspace
    raise snapshot_error

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/extensions/sandbox/daytona/sandbox.py\", line 993, in persist_workspace
    raw = await self._run_persist_workspace_command(tar_cmd, tar_path)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/sandbox/util/retry.py\", line 104, in wrapped
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^

  File \"/Users/maxim/workarea/workspaces/demos/tasks/openai-agents-demo/openai-agents-python/src/agents/extensions/sandbox/daytona/sandbox.py\", line 960, in _run_persist_workspace_command
    raise WorkspaceArchiveReadError(path=self._workspace_root_path(), cause=e) from e
",
    "cause": {
      "message": "Failed to execute command: bad request: failed to resolve container IP after 3 attempts: no IP address found. Is the Sandbox started?",
      "stackTrace": "  File \"/Users/maxim/workarea/workspaces/demos/t",
      "applicationFailureInfo": {
        "type": "DaytonaError"
      }
    },
    "applicationFailureInfo": {
      "type": "WorkspaceArchiveReadError"
    }
  },

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions