Swarming: Implement JSON env var encapsulation and v2 API protos#5170
Merged
Swarming: Implement JSON env var encapsulation and v2 API protos#5170
Conversation
jardondiego
reviewed
Feb 25, 2026
Collaborator
|
I think it's important to be more specific with both the title and description. |
jardondiego
reviewed
Feb 25, 2026
| @@ -0,0 +1,18 @@ | |||
| # Copyright 2023 Google LLC | |||
Collaborator
There was a problem hiding this comment.
Sorry I am not sure if these gRPC should be on this PR. Are they needed for changes in swarming/__init__.py?
Collaborator
Author
There was a problem hiding this comment.
All of the used protos in this repo live under src/_internal/protos, i just added the ones that were missing but maybe @letitz can give more context here
IvanBM18
changed the title
Collaborator
Author
Yeah i guess it was a little bit to generic, thanks for the feedback |
jardondiego
reviewed
Feb 25, 2026
| @@ -35,8 +36,10 @@ def _requires_gpu() -> bool: | |||
|
|
|||
| def is_swarming_task(command: str, job_name: str): | |||
Collaborator
There was a problem hiding this comment.
Are moving this file logic to be under a swarming service that implements the RemoteTaskInterface?
Collaborator
There was a problem hiding this comment.
Yes. The plan is to eventually migrate it to use the Remote Task interface.
There was a problem hiding this comment.
Yes. The plan is to eventually migrate it to use the Remote Task interface.
How it's not me...?
letitz
reviewed
Feb 26, 2026
Collaborator
letitz
left a comment
letitz
left a comment
There was a problem hiding this comment.
Good stuff! A couple comments below.
letitz
requested changes
Feb 27, 2026
|
Ok 💭 but liitle bit time need to understand about the theme |
letitz
approved these changes
Mar 2, 2026
Collaborator
letitz
left a comment
letitz
left a comment
There was a problem hiding this comment.
One last question around the environment variables, and a couple debug statements to be removed. LGTM otherwise.
Collaborator
|
It would be good testing it in the dev environment. |
IvanBM18
added a commit
that referenced
this pull request
Mar 2, 2026
## Finishes the last details in the base of clusterfuzz-swarming integration. This PR finalizes the base integration for Clusterfuzz-Swarming. The primary focus is ensuring environment variable persistence across Docker/ swarming bots and laying the groundwork for the Swarming v2 API. - Encapsulates all of the env vars in a single JSON env var, so that this JSON is be processed by a startup script. This is done so that the env vars are not lost in swarming bots that launch clusterfuzz in a docker container. - Simplified the way we check if a task is a swarming task to avoid creating unused newTaskRequest objects and too many quries into the db, resulting in a refactor of various related methods - Adds new protos for the swarming v2 api - Also adds some unit tests. This integration will later be refactored to use remote_tasks once the remote task gate allows to send tasks to specific backends always that some conditions are given. Bug: b/479476219
jardondiego
added a commit
that referenced
this pull request
Mar 3, 2026
IvanBM18
pushed a commit
that referenced
this pull request
Mar 4, 2026
IvanBM18
added a commit
that referenced
this pull request
Mar 6, 2026
integration. This PR finalizes the base integration for Clusterfuzz-Swarming. The primary focus is ensuring environment variable persistence across Docker/ swarming bots and laying the groundwork for the Swarming v2 API. - Encapsulates all of the env vars in a single JSON env var, so that this JSON is be processed by a startup script. This is done so that the env vars are not lost in swarming bots that launch clusterfuzz in a docker container. - Simplified the way we check if a task is a swarming task to avoid creating unused newTaskRequest objects and too many quries into the db, resulting in a refactor of various related methods - Adds new protos for the swarming v2 api - Also adds some unit tests. This integration will later be refactored to use remote_tasks once the remote task gate allows to send tasks to specific backends always that some conditions are given. Bug: b/479476219
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finishes the last details in the base of clusterfuzz-swarming integration.
This PR finalizes the base integration for Clusterfuzz-Swarming. The primary focus is ensuring environment variable persistence across Docker/ swarming bots and laying the groundwork for the Swarming v2 API.
This integration will later be refactored to use remote_tasks once the remote task gate allows to send tasks to specific backends always that some conditions are given.
Bug: b/479476219