C# SDK: re-land x-opaque-json → JsonElement mapping with object boundary at RPC params by SteveSandersonMS · Pull Request #1359 · github/copilot-sdk

SteveSandersonMS · 2026-05-21T11:17:54Z

What this PR does

1. Codegen: `x-opaque-json` → `JsonElement` (`scripts/codegen/csharp.ts`)

The runtime schema generator now stamps x-opaque-json: true on every field/type that legitimately holds free-form JSON (tool args, hook payloads, telemetry, form schemas, etc.) and rejects everything else at build time. The codegen consumes that marker:

DTO fields with x-opaque-json: true → JsonElement / JsonElement? (was object?).
All bare-object fallbacks removed from the codegen; reaching an unmappable shape is now a hard error.
Hand-written types that round-trip opaque JSON (ToolInvocation.Arguments, hook in/out, related serialization plumbing in Session.cs, SessionFsProvider.cs, Client.cs) switched to JsonElement on the wire. New TestJsonContext for tests.

2. RPC-boundary special-case (the bit that made #1343 painful)

The reverted PR forced consumers to pass JsonElement for outbound RPC method parameters (e.g. Rpc.Tools.HandlePendingToolCallAsync(... JsonElement result, ...)). That's worse UX than the old object? surface: callers naturally hold a typed CLR value (string, dictionary, generated DTO) and don't want to serialize before each call.

Fixed by introducing an asymmetric boundary:

Inbound opaque-JSON (events, hook inputs, tool args, generated event DTOs, DTO fields) stays JsonElement(?) — consumers can introspect the wire payload directly.
Outbound opaque-JSON at top-level RPC method parameters now accepts object/object?. The codegen generates a call to a new helper:
```
public static JsonElement? ToJsonElementForWire(object? value)
```
which short-circuits if value is already a JsonElement, otherwise serializes the runtime type using SerializerOptionsForMessageFormatter (the chained source-gen JsonSerializerContexts the SDK uses for all JSON-RPC traffic). This matches pre-revert serialization exactly.

DTO field types stay JsonElement(?) — the object only appears at the public RPC surface. A generated call site looks like:
```
public async Task<HandlePendingToolCallResult> HandlePendingToolCallAsync(
    string requestId, object? result = null, string? error = null, ...)
{
    var request = new HandlePendingToolCallRequest {
        ..., Result = CopilotClient.ToJsonElementForWire(result), ...
    };
    ...
}
```
ToolResultObject.ToolTelemetry reverted to IDictionary<string, object>? for the same reason (it's a consumer-constructed outbound type). Tests updated.

Validation

dotnet build clean across net472 / net8.0 / net10.0.
98 (net8.0) + 91 (net472) unit tests pass; failures match baseline (4 ConnectionTokenTests rely on the test-harness proxy subprocess and aren't relevant to this change).
TS / Python / Go / Rust codegens regenerate cleanly against the updated runtime schema.

Dependencies

This PR requires runtime changes to be merged first (or for the runtime's regenerated generated/*.schema.json to ship a copy). The branch already contains regenerated artifacts from the runtime PR's schema; once the runtime merges and re-emits, no codegen changes will be needed here.

stephentoub · 2026-05-21T14:28:49Z

+    {
+        null => null,
+        JsonElement je => je,
+        _ => JsonSerializer.SerializeToElement(value, value.GetType(), SerializerOptionsForMessageFormatter)


You can avoid the need for the suppress messages by passing in a JsonTypeInfo instead of Type, e.g. value.GetType() => SerializerOptionsForMessageFormatter.GetTypeInfo(value.GetType())

Copilot

Pull request overview

This PR updates the SDK code generators and generated artifacts to (1) treat schema-marked opaque JSON (x-opaque-json) as structured “wire JSON” (not object fallbacks), and (2) propagate/express visibility: "internal" consistently across languages, while keeping the .NET public RPC method surface ergonomic by converting outbound object inputs at the RPC boundary.

Changes:

Codegen: add internal-visibility propagation and marker emission; add x-opaque-json helpers and strip/annotation steps where needed (TS/Python).
.NET: map opaque-JSON DTO fields to JsonElement, add CopilotClient.ToJsonElementForWire(object?), and apply an RPC-boundary conversion for outbound parameters.
Regenerate/update generated types and tests across Node/TS, Python, Go, Rust, and .NET to reflect the new schema markers and visibility rules.

Show a summary per file

File	Description
scripts/codegen/utils.ts	Adds helpers for internal visibility, opaque JSON markers, schema mutation passes, and Python post-processing helpers.
scripts/codegen/typescript.ts	Applies internal visibility propagation, strips opaque-json markers pre-compile, and emits `@internal/@experimental` via descriptions/JSDoc.
scripts/codegen/rust.ts	Emits `pub(crate)` + hidden docs for internal items and propagates internal visibility before generation.
scripts/codegen/python.ts	Renames internal Python symbols/types, prefixes internal fields, and annotates internal fields on public types.
scripts/codegen/package-lock.json	Lockfile adjustment (dependency metadata).
scripts/codegen/go.ts	Adds internal/experimental field markers, propagates internal visibility, and excludes internal types from public re-exports.
scripts/codegen/csharp.ts	Maps `x-opaque-json` to `JsonElement`, removes `object` fallbacks, adds internal-property handling, and adds outbound RPC boundary conversion.
rust/src/generated/session_events.rs	Regenerated with internal visibility + experimental doc markers applied.
rust/src/generated/rpc.rs	Regenerated RPC surface with internal visibility applied.
rust/src/generated/api_types.rs	Regenerated API types with internal visibility + experimental doc markers applied.
python/copilot/generated/session_events.py	Regenerated session events with internal symbol renames/field prefixing and experimental/internal markers.
python/copilot/generated/rpc.py	Regenerated RPC types with internal renames/annotations; internal RPC methods/types made private-ish via naming.
python/copilot/client.py	Updates client handshake path to use internal `_connect` and `_ConnectRequest` types.
nodejs/src/generated/session-events.ts	Regenerated session event types with `@internal/@experimental` tags and simplified opaque JSON shapes to `unknown`.
nodejs/src/generated/rpc.ts	Regenerated RPC types with marker tags and simplified opaque JSON shapes to `unknown`.
go/zsession_events.go	Removes re-export aliases for internal session-event types.
go/rpc/zsession_events.go	Adds internal/experimental doc markers on fields/types in generated session events.
go/rpc/zrpc.go	Adds internal/experimental doc markers on relevant RPC fields.
dotnet/test/Unit/SessionEventSerializationTests.cs	Updates tests for `JsonElement`-typed opaque JSON and internalized fields.
dotnet/test/E2E/ToolResultsE2ETests.cs	Updates tool telemetry construction to align with opaque JSON handling.
dotnet/test/E2E/SessionFsE2ETests.cs	Updates SQLite bind parameter typing to `object?`.
dotnet/test/E2E/RpcTasksAndHandlersE2ETests.cs	Updates pending-tool-call result handling to match new RPC boundary behavior.
dotnet/test/E2E/PendingWorkResumeE2ETests.cs	Updates pending-tool-call result handling for resumed sessions.
dotnet/test/E2E/InMemorySessionFsSqliteHandler.cs	Updates SQLite bind parameter typing to `object?`.
dotnet/src/Types.cs	Switches handwritten opaque JSON fields (tool args/results, hook payloads) to `JsonElement`.
dotnet/src/SessionFsProvider.cs	Converts SQLite request params from `JsonElement` to CLR values and converts results back to `JsonElement`.
dotnet/src/Session.cs	Updates elicitation/tool plumbing to use `JsonElement` for inbound opaque JSON and wire conversions for outbound.
dotnet/src/Generated/SessionEvents.cs	Regenerated session-events DTOs with `JsonElement` opaque JSON fields and internal visibility.
dotnet/src/Generated/Rpc.cs	Regenerated RPC DTOs/methods with `JsonElement` mapping and outbound boundary conversion usage.
dotnet/src/Client.cs	Adds `CopilotClient.ToJsonElementForWire(object?)` using the shared JSON-RPC serializer contexts.

Copilot's findings

Files not reviewed (4)

go/rpc/zrpc.go: Language not supported
go/rpc/zsession_events.go: Language not supported
go/zsession_events.go: Language not supported
scripts/codegen/package-lock.json: Language not supported

Files reviewed: 17/30 changed files
Comments generated: 11

+                Rows = result?.Rows?.Select(row => (IDictionary<string, JsonElement>)row.ToDictionary(
+                    kvp => kvp.Key,
+                    kvp => CopilotClient.ToJsonElementForWire(kvp.Value)!.Value)).ToList() ?? [],


+                Properties = new Dictionary<string, JsonElement>
                {
-                    ["confirmed"] = new Dictionary<string, object> { ["type"] = "boolean", ["default"] = true }
+                    ["confirmed"] = JsonDocument.Parse("""{"type":"boolean","default":true}""").RootElement.Clone()
                },


                Type = "object",
-                Properties = new Dictionary<string, object>
+                Properties = new Dictionary<string, JsonElement>
                {
-                    ["selection"] = new Dictionary<string, object> { ["type"] = "string", ["enum"] = options }
+                    ["selection"] = JsonDocument.Parse($$"""{"type":"string","enum":{{enumJson}}}""").RootElement.Clone()
                },


            {
                Type = "object",
-                Properties = new Dictionary<string, object> { ["value"] = field },
+                Properties = new Dictionary<string, JsonElement>
+                {
+                    ["value"] = JsonDocument.Parse(fieldNode.ToJsonString()).RootElement.Clone()
+                },


+                Properties = elicitationParams.RequestedSchema.Properties.ToDictionary(
+                    kvp => kvp.Key,
+                    kvp => CopilotClient.ToJsonElementForWire(kvp.Value)!.Value),


                ToolTelemetry = new Dictionary<string, object>
                {
-                    ["metrics"] = new Dictionary<string, object> { ["analysisTimeMs"] = 150 },
-                    ["properties"] = new Dictionary<string, object> { ["analyzer"] = "eslint" },
+                    ["metrics"] = JsonDocument.Parse("""{"analysisTimeMs":150}""").RootElement.Clone(),
+                    ["properties"] = JsonDocument.Parse("""{"analyzer":"eslint"}""").RootElement.Clone(),


        var tool = await session.Rpc.Tools.HandlePendingToolCallAsync(
            requestId: "missing-tool-request",
-            result: "tool result");
+            result: JsonDocument.Parse("\"tool result\"").RootElement.Clone());


            var toolResult = await session2.Rpc.Tools.HandlePendingToolCallAsync(
                toolEvent.Data.RequestId,
-                result: "EXTERNAL_RESUMED_BETA");
+                result: JsonDocument.Parse("\"EXTERNAL_RESUMED_BETA\"").RootElement.Clone());


            var resumedResult = await session2.Rpc.Tools.HandlePendingToolCallAsync(
                toolEvent.Data.RequestId,
-                result: "EXTERNAL_RESUMED_BETA");
+                result: JsonDocument.Parse("\"EXTERNAL_RESUMED_BETA\"").RootElement.Clone());
            Assert.True(resumedResult.Success);


            var resultB = await session2.Rpc.Tools.HandlePendingToolCallAsync(
                toolB.Data.RequestId,
-                result: "PARALLEL_B_BETA");
+                result: JsonDocument.Parse("\"PARALLEL_B_BETA\"").RootElement.Clone());
            Assert.True(resultB.Success);
            var resultA = await session2.Rpc.Tools.HandlePendingToolCallAsync(
                toolA.Data.RequestId,
-                result: "PARALLEL_A_ALPHA");
+                result: JsonDocument.Parse("\"PARALLEL_A_ALPHA\"").RootElement.Clone());


…RPC params Also makes visibility: internal handling consistent across all codegens: - C# session-events now honors type-level visibility=internal (was missing; caused CS9032 build failures with required internal members on public types). - New transitive-visibility pass propagates internal from a referenced type to its referencing fields (fixes CS0053-style inconsistencies). - Rust honors type-level visibility=internal via pub(crate) on struct + fields (replaces the previous doc-hidden-only on fields). Internal RPC methods are also pub(crate). - Python prefixes internal types and fields with underscore (the universal Python no-stability-guarantee convention); replaces the no-op # Internal: comment that consumers never saw. - Go skips internal types from the copilot package re-exports so consumers using the canonical copilot.* namespace never see them. Lowercase types in the rpc package was evaluated but rejected (requires non-trivial runtime refactoring in go/client.go). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-05-22T19:24:06Z

Cross-SDK Consistency Review ✅

This PR is primarily a .NET SDK change (asymmetric JsonElement/object boundary for opaque JSON), and the modifications to Go, Node.js, Python, and Rust files are auto-generated artifacts from the updated runtime schema — not manual feature changes.

The JsonElement ↔ object design decision is inherently C#-specific. Other SDKs handle opaque JSON with their own idiomatic types (e.g., any in TypeScript, interface{} / json.RawMessage in Go, dict in Python), so no equivalent change is needed in other language implementations.

No cross-SDK consistency issues found. The PR maintains appropriate parity — each SDK uses its language's natural type for opaque JSON at the RPC boundary.

Generated by SDK Consistency Review Agent for issue #1359 · ● 1.2M · ◷

github-code-quality Bot found potential problems May 21, 2026

View reviewed changes

Comment thread scripts/codegen/csharp.ts Fixed

Comment thread scripts/codegen/csharp.ts Fixed

Comment thread scripts/codegen/csharp.ts Fixed

Comment thread scripts/codegen/csharp.ts Fixed

stephentoub reviewed May 21, 2026

View reviewed changes

SteveSandersonMS force-pushed the stevesandersonms/stevesa-fallback-on-jsonelement branch from 78d5ff2 to 12d3837 Compare May 21, 2026 14:45