Skip to content

README.md

Latest commit

 

History

History
195 lines (138 loc) · 9.18 KB

README.md

File metadata and controls

195 lines (138 loc) · 9.18 KB

AI My API

AIMyAPI is a typescript code sandbox for your LLM using QuickJS. It allows you to expose a global API object and allows your agent to write functions against it without giving it access to the internet or your filesystem except through the methods you define.

Requires an OpenAI compatible API

AIMyAPI will handle async operations and timers in your API object.

About version 1.0 rewrite and motivation

AIMyAPI was originally written at a time (circa ChatGPT 3.5 / GPT-4) before tool calling existed, before Claude code and codex and just before OpenAI released their code interpreter. LLM models have also gotten drastically better since. Some of the problems it originally aimed to solve are better solved today with simple tool calling. However, there is still a need to safely run small programs in a sandboxed way, reliable way. Tools are good if you have a set interface but there may be times that an agent needs to do some sorting or filtering, or an algorithm or analysis, or some repeatable workflow which isn't implemented by the available tool functions. Writing code allows the agent to do things it might not otherwise be able to do with tools alone. Having the ability to write code allows the tools API surface to be smaller; it could simply be a set of CRUD operations.

The other advantage of writing code is privacy and security. When an agent executes code in a sandbox, it doesn't see any of the results*, API keys or other things you may want to keep hidden. When an agent uses tools, it sees all the parameters and results of the tool calls. Code can be audited for security and correctness before running it. Finally, code can encompass a series of operations, whereas tool calls need to be approved one at a time or just trusted, often without knowing what will come next.

Version 1.0 updates AIMyAPI code generation to use the tool calling process for writing code. This allows the agent to check the code and run some tests or API calls before committing to a final version. This should result in better code since the previous model relied on the agent always being able to one-shot the code, which depended on it having a complete view of the problem before writing the code which wasn't always realistic. We could also update this process a bit to add "Plan" / "Build" / "Test" phases instead of just one code generation prompt.

*You'll have the option to hide logs and errors from the agent - logging can allow the agent to fix it's own coding mistakes and test the code, but it also allows information to leave the sandbox which you may want to limit depending on the sensitivity the data your api is handling. Another approach would be to implement a mocked version of your API that the agent can use to write code and debug against while keeping your real data hidden and safe from potentially destructive operations.

Quickstart

Install the package

npm i aimyapi --save

Setup: create a .env file in the root of your project and add your OpenAI api key:

OPENAI_API_KEY=XXXX
OPENAI_API_BASE_URL=XXXX       // If using your own server, otherwise omit to use the default
OPENAI_API_MODEL=XXXX              // defaults to "gpt-5-mini"

Simply define your strongly-typed typescript API:

sales_api.ts

// Sales data for each month, per business unit
export interface SalesData {
    sales: number;
    month: number;  // 1...12
    year: number;
    businessUnit: "Advertising" | "Hardware" | "Software";
}

export interface APIInterface {
    getSalesData(): SalesData[];
    // Returns an array of sales data.

    print(text: string): void;
    // Prints the specified text to the user.
};

// Global object you will use for accessing the APIInterface. Any other globals can go here too if you declare them in the `apiGlobals` parameter.
declare global {
    const api: APIInterface;
}

Implement it (sales_api_impl.ts):

export class API implements APIInterface {

    getSalesData() {
    ...

Document it (sales_api.md):

Examples of how you should respond:

user: How are you today?

assistant:

api.print("I'm great, thanks for asking!");

Instantiate your api and AIMyAPI. AIMyAPI will wrap your API so that it can be called from within the QuickJS sandbox.

import { AIMyAPI } from "aimyapi"

const api = new API();

const aimyapi = await AIMyAPI.createWithAPI({
    apiObject: api,
    apiExports: APIExports,
    apiDefFilePath: path.join(__dirname, "./sales_api.ts"),
    apiDocsPath: path.join(__dirname, "./sales_api.md"),
})

And now you can make requests against the API in natural language. Any code generated by the LLM will be run within the sandbox. The LLM will ONLY have access to functions provided within your API and cannot use javascript to make http requests etc.

await aimyapi.processRequest("Which business unit had the highest sales in 2020?");

This would produce the following code generated by the LLM prompt, which is run in the sandbox and makes calls on your API object:

let salesData = api.getSalesData();
let maxSales = 0;
let businessUnitWithMaxSales = "";
for (let i = 0; i < salesData.length; i++) {
    if (salesData[i].year === 2020 && salesData[i].sales > maxSales) {
        maxSales = salesData[i].sales;
        businessUnitWithMaxSales = salesData[i].businessUnit;
    }
}
api.print(\`The business unit with the highest sales in 2020 was ${businessUnitWithMaxSales}.\`);

This example does not take advantage of the chat history and treats every request as a fresh chat. See example2 for a more complex example that uses the chat history.

APIs should be kept simple and understandable. If you have a complicated API you may want to write a simpler facade for it.

API Reference for AIMyAPI

createWithAPI(options: AIMyAPIOptions): Promise

Primary function you will use to create a module for working your API API. For a simple example with no chat history, see example1. For an example with chat history see example2.

Options:

export interface AIMyAPIOptions {
    apiObject: object;
    apiExports: object;
    apiWhitelist?: string[];
    apiGlobals?: object;
    apiDefFilePath: string;
    apiGlobalName?: string;    
    apiDocsPath?: string;
    debug?: boolean;
    model?: string;
    additionalModelOptions?: object;
    hideLogsFromAgent?: boolean; 
};
  • apiObject: An object representing the API to be used.
  • apiExports: An object representing the exported functions and variables from the API.
  • apiGlobals (optional): An object representing global variables for the API.
  • apiDefFilePath: A string representing the full path to the API definition file.
  • apiGlobalName (optional): A string representing the name of the global variable for the API.
  • apiDocsPath (optional): A string representing the path to the documentation for the API.
  • debug (optional): A boolean indicating whether to output debugging information.

The AIMyAPI module (type: CreateWithAPIExports)

The module created by the createWithAPI function:

Methods

export interface AIMyAPIInstance {
    options:AIMyAPIOptions;
    generateCode: (queryText:string, userChatHistory:ChatCompletionMessageParam[]) => Promise<GenerateCodeResult | null>;
    runCode: (code:string) => Promise<CodeRunResult>;
    checkCode: (code:string) => Promise<CodeRunResult>;
    // run a single request with no history
    processRequest: (userQuery:string, context?: object) => Promise<GenerateCodeResult | null>;
}
  • generateCode(): A function that generates code to be run in the sandbox for the given query text and user chat history.
  • runCode(): A function that runs the generated code in the sandbox.
  • checkCode(): Typecheck / compile the code
  • processRequest(): A single function that generates and runs code assuming no chat history.

Safety notes:

  • If the user is giving sensitive information in a query, that information isn't private. Refer to the OpenAI policies and plan accordingly.
  • The LLM will do stupid things with your API. You should idiot proof your functions, make sure you sanity check any inputs, etc.
  • Even though we are running the code in sandbox, you should still treat anything coming back from the LLM as untrusted code.
  • You should never give the LLM access to anything mission critical through your API such as live production data or services.
  • Give as few permissions as is necessary - read only access would be preferred.
  • Users can trick the LLM to do things that you may not want or intend. Again, access controls are critical. You shouldn't let users access anything they wouldn't normally be able to access via your API.
  • Any 'dangerous' operations should involve some kind of human oversight and approval.
  • A way to audit and rollback anything the LLM does would be wise. The LLM can misunderstand requests due to false assumptions or incomplete information (or trickery). If you allow it to modify your data it could mess things up.
  • By using this library you acknowledge that you take full responsibility for anything that happens as a result of it's use. We cannot gaurantee what will happen based on your instructions to the LLM or how the LLM interprets them etc.