feature: Move quota workflow to a script#2360
Open
tmeschter wants to merge 1 commit into
Open
Conversation
One of the workflows supported by the azure-quotas skill is to verify quota limits and current usage. This workflow has definite, limited inputs and is very regular, with the LM instructed to execute a series of four `az` commands one after the other. In the end, the relevant output is a series of resource names with the current usage and limit for each. The LM can generally follow this workflow but there are some downsides with this approach. First, each `az` command will produce an "intermediate" output on the way to the final output. We don't really care about these intermediate outputs but they take up space in the token context window. Second, if one of those intermediate outputs is large (greater than 20KB or so) Copilot won't provide it directly to the LM but will instead put it in a temporary file, and tell the LM to read the file. This requires additional tool invocations and opportunities for things to go wrong. Third, despite the clarity of the instructions the LM will not always be consistent in which steps it follows, or exactly how it follows them. Related to this, sometimes it will make a mistake and may or may not recover. To address these issues, this change is an attempt to change this one specific workflow to be implemented as a script. The LM no longer executes individual `az` commands; instead, it runs the script with the necessary inputs. This should be more consistent, faster, and use fewer tokens. If this change is successful we will move more of these well-defined workflows over to scripts.
JasonYeMSFT
approved these changes
May 21, 2026
Member
Author
|
@rakal-dyh Would you be able to take a look at this change? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One of the workflows supported by the azure-quotas skill is to verify quota limits and current usage. This workflow has definite, limited inputs and is very regular, with the LM instructed to execute a series of four
azcommands one after the other. In the end, the relevant output is a series of resource names with the current usage and limit for each.The LM can generally follow this workflow but there are some downsides with this approach. First, each
azcommand will produce an "intermediate" output on the way to the final output. We don't really care about these intermediate outputs but they take up space in the token context window.Second, if one of those intermediate outputs is large (greater than 20KB or so) Copilot won't provide it directly to the LM but will instead put it in a temporary file, and tell the LM to read the file. This requires additional tool invocations and opportunities for things to go wrong.
Third, despite the clarity of the instructions the LM will not always be consistent in which steps it follows, or exactly how it follows them. Related to this, sometimes it will make a mistake and may or may not recover.
To address these issues, this change is an attempt to change this one specific workflow to be implemented as a script. The LM no longer executes individual
azcommands; instead, it runs the script with the necessary inputs. This should be more consistent, faster, and use fewer tokens.If this change is successful we will move more of these well-defined workflows over to scripts.
Description
Checklist
cd tests && npm test)npm run test:skills:integration -- <skill>)USE FOR/DO NOT USE FOR/PREFER OVERclauses: confirmed no routing regressions for competing skills