Use an api server for generation in textgen
Load a simple model like a 1b or 7b set the context to 1 just so the backend doesn't eat resources and error out.
put your api key in .env
This is just a poc version, will add in gradio ui and use of state vars next.