feature: simplify CLI deployment

[psschwei]

Currently, deploying a workflow via the CLI looks something like this:

maestro deploy demos/workflows/weather-checker.ai/agents.yaml demos/workflows/weather-checker.ai/workflow.yaml --k8s BEE_API=http://172.21.0.1:4000 BEE_API_KEY=sk-proj-testkey

I'd like to suggest two ways to clean this up:

1. Remove the need to specify an agents file and a workflow file (i.e. maestro should be able to parse which resource is which)
2. Envvars should be able to be set via a configmap (rather than needing to specify on the command line)

This would allow the deployment to look something like this (more in line with how it's done when using kubectl):

maestro deploy -f resources/

where resources/ contains the following files:

• agents.yaml
• workflow.yaml

[maximilien]

I like this @psschwei feel free to take it if you can.

CC: @akihikokuroda who created the simple deployment we have now.

[maximilien]

@psschwei can you do this as part of cbom.ai work or thereafter. Trying to look at all issues and make sure we are making progress.

[psschwei]

I don't really have time to work on this right now, but I did investigate how it might be done in case someone else wants to pick this up.

Our CLI functions look in general like this: maestro deploy AGENTS_FILE WORKFLOW_FILE [options] [ENV...]

and we store the file locations in the class args (e.g. if AGENTS_FILE=/home/paul/agents.yaml, then in the class we'll have self.args["AGENTS_FILE"] = "/home/paul/agents.yaml"

Instead, I think we should consider storing the resource yamls instead of file locations. Right now, we assume separate files for agents and workflows, but in theory they could both be stored in the same file. Or a directory could have multiple files (agent1.yaml, agent2.yaml, etc.) so it would probably be better to process that up front, for each file grabbing the yaml and appending to a list of either workflows or agents after reading it in. The fact that we've hardcoded the deploy / create / run functions to only accept a single file is going to be limiting going forward I think, so might as well clean that up now.

We can then update the function call to this: maestro deploy -f FILE | --file FILE [options] [ENV...] and then add something like this in the optons: -f --file File to create | deploy | run

Storing the yamls rather than the file locations might also let us drop the writes to /tmp and instead we could do something along the lines of : subprocess.run(["kubectl", "apply", "-f", "-"], input=agent_defs, text=True)

I think it's a pretty decent sized change that's going to be needed to implement this. It could also be possible to implement alongside the existing one. so basically having two deploy functions -- one if -f is used, and another for AGENTS_FILE / WORKFLOW_FILE. That would allow for smaller changes (which would make for smaller PRs to review AND not break the existing cli) and then the old functionality could be deprecated once it's all in.

Anyways, just some (very rambling) thoughts on how someone might approach this.

[maximilien]

Wait for new CLI.

[ajbozarth]

We should apply this to all the commands that accept both agent and workflow files, like run and serve

Also this issue will need to be moved to the new maestro-cli repo once it's opened and addressed there