Skip to content

Commit cf5cd2e

Browse files
aarneCopilot
aarne
and
Copilot
authored
Sparse fields as part of executeBridge api (#82)
* feat: add `requestedFields` sparse fieldset filtering to both execution engines (#81) * Initial plan * feat: add requestedFields (sparse fieldsets) to both runtime and compiler engines - Add requestedFields option to ExecuteBridgeOptions in bridge-core and bridge-compiler - Implement matchesRequestedFields utility with dot-separated path and wildcard support - Runtime: filter output fields in ExecutionTree.run() and resolveNestedField() - Compiler: filter output wires and use backward reachability for dead code elimination - Update compiler cache key to include sorted requestedFields - Add 20 shared parity tests covering field filtering, tool skipping, A||B→C chains, wildcards Co-authored-by: aarne <82001+aarne@users.noreply.github.com> * docs: add sparse fieldsets documentation and changeset Co-authored-by: aarne <82001+aarne@users.noreply.github.com> * refactor: address code review feedback (validation, perf, docs) Co-authored-by: aarne <82001+aarne@users.noreply.github.com> * Lint & build * Move graphql tests * refactor: remove check exports script and related commands * Reorder and update CI steps in test workflow --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: aarne <82001+aarne@users.noreply.github.com> Co-authored-by: Aarne Laur <aarne.laur@gmail.com> * CLI mode in the Playground * docs: Review --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: aarne <82001+aarne@users.noreply.github.com>
1 parent ec5d6f8 commit cf5cd2e

36 files changed

+2799
-684
lines changed

‎.changeset/sparse-fieldsets.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
"@stackables/bridge-core": minor
3+
"@stackables/bridge-compiler": minor
4+
---
5+
6+
Add `requestedFields` option to `executeBridge()` for sparse fieldset filtering.
7+
8+
When provided, only the listed output fields (and their transitive tool dependencies) are resolved.
9+
Tools that feed exclusively into unrequested fields are never called, reducing latency and upstream
10+
bandwidth.
11+
12+
Supports dot-separated paths and a trailing wildcard (`["id", "price", "legs.*"]`).

‎.github/workflows/test.yml‎

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,9 @@ jobs:
2020
- uses: pnpm/action-setup@v4
2121
- name: Install dependencies
2222
run: pnpm install
23-
- name: Build
24-
run: pnpm build
25-
- name: Check Exports
26-
run: pnpm check:exports
27-
- name: Lint Types
28-
run: pnpm lint:types
2923
- name: Test
3024
run: pnpm test
25+
- name: Build
26+
run: pnpm build
3127
- name: Lint with ESLint
32-
run: pnpm lint:eslint
28+
run: pnpm lint

‎AGENTS.md‎

Lines changed: 47 additions & 289 deletions
Large diffs are not rendered by default.

‎package.json‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,8 @@
55
"packageManager": "pnpm@10.30.3+sha256.ff0a72140f6a6d66c0b284f6c9560aff605518e28c29aeac25fb262b74331588",
66
"scripts": {
77
"test": "pnpm -r test",
8-
"build": "pnpm -r build",
9-
"lint:types": "pnpm -r --filter './packages/*' lint:types",
10-
"lint:eslint": "eslint .",
11-
"check:exports": "node scripts/check-exports.mjs",
8+
"build": "pnpm -r --filter './packages/*' lint:types",
9+
"lint": "eslint .",
1210
"smoke": "node scripts/smoke-test-packages.mjs",
1311
"e2e": "pnpm -r e2e",
1412
"depcheck": "pnpm -r exec pnpm dlx depcheck",

‎packages/bridge-compiler/README.md‎

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -73,16 +73,17 @@ console.log(code); // Prints the raw `export default async function...` string
7373

7474
## API: `ExecuteBridgeOptions`
7575

76-
| Option | Type | What it does |
77-
| ---------------- | --------------------- | -------------------------------------------------------------------------------- |
78-
| `document` | `BridgeDocument` | The parsed AST from `@stackables/bridge-parser`. |
79-
| `operation` | `string` | Which bridge to run, e.g. `"Query.myField"`. |
80-
| `input?` | `Record<string, any>` | Input arguments — equivalent to GraphQL field args. |
81-
| `tools?` | `ToolMap` | Your custom tool functions (merged with built-in `std`). |
82-
| `context?` | `Record<string, any>` | Shared data available via `with context as ctx` in `.bridge` files. |
83-
| `signal?` | `AbortSignal` | Pass an `AbortSignal` to cancel execution and upstream HTTP requests mid-flight. |
84-
| `toolTimeoutMs?` | `number` | Fails the execution if a single tool takes longer than this threshold. |
85-
| `logger?` | `Logger` | Structured logger for tool calls. |
76+
| Option | Type | What it does |
77+
| ------------------ | --------------------- | -------------------------------------------------------------------------------- |
78+
| `document` | `BridgeDocument` | The parsed AST from `@stackables/bridge-parser`. |
79+
| `operation` | `string` | Which bridge to run, e.g. `"Query.myField"`. |
80+
| `input?` | `Record<string, any>` | Input arguments — equivalent to GraphQL field args. |
81+
| `tools?` | `ToolMap` | Your custom tool functions (merged with built-in `std`). |
82+
| `context?` | `Record<string, any>` | Shared data available via `with context as ctx` in `.bridge` files. |
83+
| `signal?` | `AbortSignal` | Pass an `AbortSignal` to cancel execution and upstream HTTP requests mid-flight. |
84+
| `toolTimeoutMs?` | `number` | Fails the execution if a single tool takes longer than this threshold. |
85+
| `logger?` | `Logger` | Structured logger for tool calls. |
86+
| `requestedFields?` | `string[]` | Sparse fieldset filter — only resolve the listed output fields. Supports dot-separated paths and a trailing `*` wildcard (e.g. `["id", "legs.*"]`). Omit to resolve all fields. |
8687

8788
_Returns:_ `Promise<{ data: T }>`
8889

‎packages/bridge-compiler/src/codegen.ts‎

Lines changed: 88 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ import type {
3030
NodeRef,
3131
ToolDef,
3232
} from "@stackables/bridge-core";
33+
import { matchesRequestedFields } from "@stackables/bridge-core";
3334

3435
const SELF_MODULE = "_";
3536

@@ -38,6 +39,12 @@ const SELF_MODULE = "_";
3839
export interface CompileOptions {
3940
/** The operation to compile, e.g. "Query.livingStandard" */
4041
operation: string;
42+
/**
43+
* Sparse fieldset filter — only emit code for the listed output fields.
44+
* Supports dot-separated paths and a trailing `*` wildcard.
45+
* Omit or pass an empty array to compile all output fields.
46+
*/
47+
requestedFields?: string[];
4148
}
4249

4350
export interface CompileResult {
@@ -88,7 +95,7 @@ export function compileBridge(
8895
(i): i is ToolDef => i.kind === "tool",
8996
);
9097

91-
const ctx = new CodegenContext(bridge, constDefs, toolDefs);
98+
const ctx = new CodegenContext(bridge, constDefs, toolDefs, options.requestedFields);
9299
return ctx.compile();
93100
}
94101

@@ -231,16 +238,20 @@ class CodegenContext {
231238
/** Map from ToolDef dependency tool name to its emitted variable name.
232239
* Populated lazily by emitToolDeps to avoid duplicating calls. */
233240
private toolDepVars = new Map<string, string>();
241+
/** Sparse fieldset filter for output wire pruning. */
242+
private requestedFields: string[] | undefined;
234243

235244
constructor(
236245
bridge: Bridge,
237246
constDefs: Map<string, string>,
238247
toolDefs: ToolDef[],
248+
requestedFields?: string[],
239249
) {
240250
this.bridge = bridge;
241251
this.constDefs = constDefs;
242252
this.toolDefs = toolDefs;
243253
this.selfTrunkKey = `${SELF_MODULE}:${bridge.type}:${bridge.field}`;
254+
this.requestedFields = requestedFields?.length ? requestedFields : undefined;
244255

245256
for (const h of bridge.handles) {
246257
switch (h.kind) {
@@ -452,7 +463,7 @@ class CodegenContext {
452463
}
453464

454465
// Separate wires into tool inputs, define containers, and output
455-
const outputWires: Wire[] = [];
466+
const allOutputWires: Wire[] = [];
456467
const toolWires = new Map<string, Wire[]>();
457468
const defineWires = new Map<string, Wire[]>();
458469

@@ -465,7 +476,7 @@ class CodegenContext {
465476
? `${w.to.module}:${w.to.type}:${w.to.field}`
466477
: toKey;
467478
if (toTrunkNoElement === this.selfTrunkKey) {
468-
outputWires.push(w);
479+
allOutputWires.push(w);
469480
} else if (this.defineContainers.has(toKey)) {
470481
// Wire targets a define-in/out container
471482
const arr = defineWires.get(toKey) ?? [];
@@ -478,6 +489,19 @@ class CodegenContext {
478489
}
479490
}
480491

492+
// ── Sparse fieldset filtering ──────────────────────────────────────
493+
// When requestedFields is provided, drop output wires for fields that
494+
// weren't requested. Kahn's algorithm will then naturally eliminate
495+
// tools that only feed into those dropped wires.
496+
const outputWires = this.requestedFields
497+
? allOutputWires.filter((w) => {
498+
// Root wires (path length 0) and element wires are always included
499+
if (w.to.path.length === 0) return true;
500+
const fieldPath = w.to.path.join(".");
501+
return matchesRequestedFields(fieldPath, this.requestedFields);
502+
})
503+
: allOutputWires;
504+
481505
// Ensure force-only tools (no wires targeting them from output) are
482506
// still included in the tool map for scheduling
483507
for (const [tk] of forceMap) {
@@ -618,38 +642,68 @@ class CodegenContext {
618642
lines.push(` }`);
619643

620644
// ── Dead tool detection ────────────────────────────────────────────
621-
// Detect tools whose output is never referenced by any output wire,
622-
// other tool wire, or define container wire. These are dead code
623-
// (e.g. a pipe-only handle whose forks are all element-scoped).
624-
const referencedToolKeys = new Set<string>();
625-
const allWireSources = [...outputWires, ...bridge.wires];
626-
for (const w of allWireSources) {
627-
if ("from" in w) referencedToolKeys.add(refTrunkKey(w.from));
628-
if ("cond" in w) {
629-
referencedToolKeys.add(refTrunkKey(w.cond));
630-
if (w.thenRef) referencedToolKeys.add(refTrunkKey(w.thenRef));
631-
if (w.elseRef) referencedToolKeys.add(refTrunkKey(w.elseRef));
632-
}
633-
if ("condAnd" in w) {
634-
referencedToolKeys.add(refTrunkKey(w.condAnd.leftRef));
635-
if (w.condAnd.rightRef)
636-
referencedToolKeys.add(refTrunkKey(w.condAnd.rightRef));
637-
}
638-
if ("condOr" in w) {
639-
referencedToolKeys.add(refTrunkKey(w.condOr.leftRef));
640-
if (w.condOr.rightRef)
641-
referencedToolKeys.add(refTrunkKey(w.condOr.rightRef));
642-
}
643-
// Also count falsy/nullish/catch fallback refs
644-
if ("falsyFallbackRefs" in w && w.falsyFallbackRefs) {
645-
for (const ref of w.falsyFallbackRefs)
646-
referencedToolKeys.add(refTrunkKey(ref));
647-
}
648-
if ("nullishFallbackRef" in w && w.nullishFallbackRef) {
649-
referencedToolKeys.add(refTrunkKey(w.nullishFallbackRef));
645+
// Detect which tools are reachable from the (possibly filtered) output
646+
// wires. Uses a backward reachability analysis: start from tools
647+
// referenced in output wires, then transitively follow tool-input
648+
// wires to discover all upstream dependencies. Tools not in the
649+
// reachable set are dead code and can be skipped.
650+
651+
/**
652+
* Extract all tool trunk keys referenced as **sources** in a set of
653+
* wires. A "source key" is the trunk key of a node that feeds data
654+
* into a wire (the right-hand side of `target <- source`). This
655+
* includes pull refs, ternary branches, condAnd/condOr operands,
656+
* and all fallback refs. Used by the backward reachability analysis
657+
* to discover which tools are transitively needed by the output.
658+
*/
659+
const collectSourceKeys = (wires: Wire[]): Set<string> => {
660+
const keys = new Set<string>();
661+
for (const w of wires) {
662+
if ("from" in w) keys.add(refTrunkKey(w.from));
663+
if ("cond" in w) {
664+
keys.add(refTrunkKey(w.cond));
665+
if (w.thenRef) keys.add(refTrunkKey(w.thenRef));
666+
if (w.elseRef) keys.add(refTrunkKey(w.elseRef));
667+
}
668+
if ("condAnd" in w) {
669+
keys.add(refTrunkKey(w.condAnd.leftRef));
670+
if (w.condAnd.rightRef) keys.add(refTrunkKey(w.condAnd.rightRef));
671+
}
672+
if ("condOr" in w) {
673+
keys.add(refTrunkKey(w.condOr.leftRef));
674+
if (w.condOr.rightRef) keys.add(refTrunkKey(w.condOr.rightRef));
675+
}
676+
if ("falsyFallbackRefs" in w && w.falsyFallbackRefs) {
677+
for (const ref of w.falsyFallbackRefs) keys.add(refTrunkKey(ref));
678+
}
679+
if ("nullishFallbackRef" in w && w.nullishFallbackRef) {
680+
keys.add(refTrunkKey(w.nullishFallbackRef));
681+
}
682+
if ("catchFallbackRef" in w && w.catchFallbackRef) {
683+
keys.add(refTrunkKey(w.catchFallbackRef));
684+
}
650685
}
651-
if ("catchFallbackRef" in w && w.catchFallbackRef) {
652-
referencedToolKeys.add(refTrunkKey(w.catchFallbackRef));
686+
return keys;
687+
};
688+
689+
// Seed: tools directly referenced by output wires + forced tools
690+
const referencedToolKeys = collectSourceKeys(outputWires);
691+
for (const tk of forceMap.keys()) referencedToolKeys.add(tk);
692+
693+
// Transitive closure: walk backward through tool input wires
694+
const visited = new Set<string>();
695+
const queue = [...referencedToolKeys];
696+
while (queue.length > 0) {
697+
const tk = queue.pop()!;
698+
if (visited.has(tk)) continue;
699+
visited.add(tk);
700+
const deps = toolWires.get(tk);
701+
if (!deps) continue;
702+
for (const key of collectSourceKeys(deps)) {
703+
if (!visited.has(key)) {
704+
referencedToolKeys.add(key);
705+
queue.push(key);
706+
}
653707
}
654708
}
655709

‎packages/bridge-compiler/src/execute-bridge.ts‎

Lines changed: 37 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,20 @@ export type ExecuteBridgeOptions = {
5757
* - `"full"` — everything including input and output
5858
*/
5959
trace?: TraceLevel;
60+
/**
61+
* Sparse fieldset filter.
62+
*
63+
* When provided, only the listed output fields (and their transitive
64+
* dependencies) are compiled and executed. Tools that feed exclusively
65+
* into unrequested fields are eliminated by the compiler's dead-code
66+
* analysis (Kahn's algorithm).
67+
*
68+
* Supports dot-separated paths and a trailing wildcard:
69+
* `["id", "price", "legs.*"]`
70+
*
71+
* Omit or pass an empty array to resolve all fields (the default).
72+
*/
73+
requestedFields?: string[];
6074
};
6175

6276
export type ExecuteBridgeResult<T = unknown> = {
@@ -91,20 +105,37 @@ const AsyncFunction = Object.getPrototypeOf(async function () {})
91105
.constructor as typeof Function;
92106

93107
/**
94-
* Cache: one compiled function per (document identity × operation).
108+
* Cache: one compiled function per (document identity × operation × requestedFields).
95109
* Uses a WeakMap keyed on the document object so entries are GC'd when
96110
* the document is no longer referenced.
97111
*/
98112
const fnCache = new WeakMap<BridgeDocument, Map<string, BridgeFn>>();
99113

100-
function getOrCompile(document: BridgeDocument, operation: string): BridgeFn {
114+
/** Build a cache key that includes the sorted requestedFields. */
115+
function cacheKey(
116+
operation: string,
117+
requestedFields?: string[],
118+
): string {
119+
if (!requestedFields || requestedFields.length === 0) return operation;
120+
return `${operation}:${[...requestedFields].sort().join(",")}`;
121+
}
122+
123+
function getOrCompile(
124+
document: BridgeDocument,
125+
operation: string,
126+
requestedFields?: string[],
127+
): BridgeFn {
128+
const key = cacheKey(operation, requestedFields);
101129
let opMap = fnCache.get(document);
102130
if (opMap) {
103-
const cached = opMap.get(operation);
131+
const cached = opMap.get(key);
104132
if (cached) return cached;
105133
}
106134

107-
const { functionBody } = compileBridge(document, { operation });
135+
const { functionBody } = compileBridge(document, {
136+
operation,
137+
requestedFields,
138+
});
108139

109140
let fn: BridgeFn;
110141
try {
@@ -133,7 +164,7 @@ function getOrCompile(document: BridgeDocument, operation: string): BridgeFn {
133164
opMap = new Map();
134165
fnCache.set(document, opMap);
135166
}
136-
opMap.set(operation, fn);
167+
opMap.set(key, fn);
137168
return fn;
138169
}
139170

@@ -202,7 +233,7 @@ export async function executeBridge<T = unknown>(
202233
logger,
203234
} = options;
204235

205-
const fn = getOrCompile(document, operation);
236+
const fn = getOrCompile(document, operation, options.requestedFields);
206237

207238
// Merge built-in std namespace with user-provided tools, then flatten
208239
// so the generated code can access them via dotted keys like tools["std.str.toUpperCase"].

0 commit comments

Comments
 (0)