feat(csharp): add gRPC mock server test generation#14100
feat(csharp): add gRPC mock server test generation#14100Swimburger wants to merge 38 commits intomainfrom
Conversation
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review.
Tip: disable this comment in your organization's Code Review settings.
…ate reflection in GrpcMockServerBuilder - Fix stub request types resolving to Empty instead of actual proto types by deriving proto type from inlined request body name + proto namespace - Remove double-await pattern in generated gRPC tests (MethodInvocation already emits await) - Replace reflection-based MapGrpcService call with direct generic call - Use fully-qualified type names to avoid namespace conflicts - Add Microsoft.AspNetCore.Routing using directive Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…ace conflicts Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…rd, fix orphan stubs Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
generators/csharp/base/src/asIs/test/GrpcMockServer.Template.cs
Outdated
Show resolved
Hide resolved
| if (isSupportedResponse) { | ||
| writer.write("var response = "); | ||
| writer.writeNodeStatement(endpointSnippet); | ||
| writer.writeNodeStatement( | ||
| this.csharp.invokeMethod({ | ||
| on: this.Types.JsonAssert, | ||
| method: "AreEqual", | ||
| arguments_: [this.csharp.codeblock("response"), this.csharp.codeblock("mockResponse")] | ||
| }) | ||
| ); | ||
| } else { | ||
| writer.write("Assert.DoesNotThrowAsync(async () => "); | ||
| writer.writeNode(endpointSnippet); | ||
| writer.write(");"); | ||
| } |
There was a problem hiding this comment.
🟡 Text response type incorrectly asserted with JsonAssert instead of string equality
In GrpcMockServerTestGenerator.ts:184-193, when isSupportedResponse is true, the generated test always uses JsonAssert.AreEqual(response, mockResponse) regardless of whether the response is JSON or text. The existing HTTP MockServerTestGenerator.ts:149-162 correctly differentiates: it uses JsonAssert.AreEqual for JSON responses but Assert.That(response, Is.EqualTo(mockResponse)) for text responses. If a gRPC endpoint returns a text response, the generated test would call JsonAssert.AreEqual on a plain string, which would likely fail or produce a misleading comparison. While gRPC endpoints typically return protobuf messages (JSON response type), the code explicitly handles the text case in isSupportedResponse (line 82) and in the mock response writing (line 127-129), making this an inconsistency.
Prompt for agents
In generators/csharp/sdk/src/test-generation/mock-server/GrpcMockServerTestGenerator.ts, lines 184-198, the isSupportedResponse branch should differentiate between JSON and text responses, matching the pattern used in MockServerTestGenerator.ts lines 149-162. Specifically, when responseBodyType is 'json', use JsonAssert.AreEqual(response, mockResponse). When responseBodyType is 'text', use Assert.That(response, Is.EqualTo(mockResponse)) instead. Add a local variable for responseBodyType (from this.endpoint.response?.body?.type) and use a conditional to select the correct assertion method.
Was this helpful? React with 👍 or 👎 to provide feedback.
…arning Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…points (Fetch, List) Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…st files Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
| // Skip if multi-URL environments without default (snippets don't support setting environment) | ||
| if ( | ||
| context.ir.environments?.environments.type === "multipleBaseUrls" && | ||
| context.ir.environments.defaultEnvironment == null | ||
| ) { | ||
| return []; | ||
| } |
There was a problem hiding this comment.
🔴 Missing union base properties check in gRPC mock server test generation
The new generateGrpcMockServerTests function is missing the "unions with base properties" guard that exists in the HTTP mock server test path. The HTTP version at generators/csharp/sdk/src/generateSdkTests.ts:152-159 (shouldSkipMockServerTestsForService) checks Object.values(context.ir.types).find(t => t.shape.type === "union" && t.shape.baseProperties.length > 0) and skips all test generation when this is true, because the snippet generator doesn't support base properties on unions. The gRPC test generation function uses the same underlying AbstractEndpointGenerator.generateEndpointSnippet infrastructure (via GrpcEndpointGenerator.generateGrpcEndpointSnippet), so the same limitation applies. When an API has both gRPC endpoints and unions with base properties, the gRPC tests will be generated with broken snippets while the HTTP tests are correctly skipped.
| // Skip if multi-URL environments without default (snippets don't support setting environment) | |
| if ( | |
| context.ir.environments?.environments.type === "multipleBaseUrls" && | |
| context.ir.environments.defaultEnvironment == null | |
| ) { | |
| return []; | |
| } | |
| // Skip if multi-URL environments without default (snippets don't support setting environment) | |
| if ( | |
| context.ir.environments?.environments.type === "multipleBaseUrls" && | |
| context.ir.environments.defaultEnvironment == null | |
| ) { | |
| return []; | |
| } | |
| // Skip if unions with base properties exist (snippets don't support them) | |
| if ( | |
| Object.values(context.ir.types).find( | |
| (typeReference) => typeReference.shape.type === "union" && typeReference.shape.baseProperties.length > 0 | |
| ) | |
| ) { | |
| return []; | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
…r test generation Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…service base classes Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
… with unsupported snippet types Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…id type conflicts Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…atch when returnTypeName is null Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…lusion target Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
| if (this.context.hasGrpcEndpoints() && this.protobufSourceFilePaths.length > 0) { | ||
| const pathToProtobufDirectory = path.win32.normalize(`../../${this.generation.constants.folders.protobuf}`); | ||
| for (const protoFilePath of this.protobufSourceFilePaths) { | ||
| if (EXTERNAL_PROTO_FILE_PREFIXES.some((prefix) => protoFilePath.startsWith(prefix))) { | ||
| continue; | ||
| } | ||
| const windowsPath = path.win32.normalize(protoFilePath); | ||
| protoServerItems.push( | ||
| `<Protobuf Include="${pathToProtobufDirectory}\\${windowsPath}" GrpcServices="Server" ProtoRoot="${pathToProtobufDirectory}" />` | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| const testCsProjContents = eta.renderString(testCsProjTemplateContents, { | ||
| projectName: this.name, | ||
| testProjectName, | ||
| projectReferencePath: projectReferenceRelativePath | ||
| projectReferencePath: projectReferenceRelativePath, | ||
| grpc: this.context.hasGrpcEndpoints(), | ||
| protoServerItems |
There was a problem hiding this comment.
🟡 Model generator test projects get unnecessary gRPC server compilation and packages
The CsharpProject.createTestProject() method unconditionally adds gRPC packages (Grpc.AspNetCore, Microsoft.AspNetCore.Mvc.Testing) and proto GrpcServices="Server" compilation to all test projects where hasGrpcEndpoints() is true — including model generator test projects (visible in seed/csharp-model/csharp-grpc-proto/ and seed/csharp-model/csharp-grpc-proto-exhaustive/). The model generator never generates GrpcMockServer, GrpcMockServerBuilder, stub classes, or mock server test files (its getCoreTestAsIsFiles() at generators/csharp/model/src/ModelGeneratorContext.ts:113 doesn't include them). This means model test projects now compile proto server stubs that are never referenced, adding unnecessary build-time dependencies and proto code generation overhead.
Prompt for agents
In generators/csharp/base/src/project/CsharpProject.ts, the createTestProject method at lines 429-447 unconditionally adds gRPC test infrastructure packages and proto server compilation when hasGrpcEndpoints() is true. This should be gated so that only SDK test projects (not model test projects) get these additions. Consider adding a method like shouldGenerateGrpcTestInfrastructure() that returns true only for the SDK generator context, or check whether the context has mock server test generation enabled before adding the gRPC-specific items. The grpc template variable (line 446) and protoServerItems (line 447) should both be conditional on being in the SDK context.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Valid observation — model test projects get gRPC packages/proto compilation they don't use. This is a cleanup/optimization rather than a bug (the model tests compile fine). Deferring to the maintainer on whether to gate this behind an SDK-only check.
…sserting stub handlers Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…on to handle auth params Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…generation - Create BaseGrpcMockServerTestGenerator that generates a base class handling all ceremony (stub creation, mock server building, client instantiation, server disposal) - Individual tests now inherit from BaseGrpcMockServerTest and only contain unique code (mock response JSON, stub configuration, method call, assertion) - Add ParseProtoJson<T> helper with tolerant JsonParser settings (WithIgnoreUnknownFields) - Register BaseGrpcMockServerTest in generation-info.ts type system - Update generateSdkTests.ts to generate the base class file Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
When all examples are filtered out (null snippets or unsupported patterns), the test generator now correctly skips file generation instead of producing empty test classes and orphan stub files. Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
ParseProtoJson expects proto3 JSON format (camelCase field names), but mock JSON examples use SDK format (snake_case). Switch to JsonUtils.Deserialize<SdkType>(mockResponse).ToProto() which properly handles the snake_case format and converts to proto types. Also removes the now-unused ParseProtoJson helper from base class. Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…t closing syntax Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…gRPC mock server tests Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…rly async Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
| _serviceRegistrations.Add(services => services.AddSingleton(implementation)); | ||
| _endpointRegistrations.Add(endpoints => endpoints.MapGrpcService<TService>()); |
There was a problem hiding this comment.
Service registration mismatch will cause runtime failure. The code registers the service instance with its concrete type (AddSingleton(implementation) registers as the runtime type), but then MapGrpcService<TService>() attempts to resolve using the base class type parameter. This causes the gRPC middleware to fail finding the service in DI.
Fix:
_serviceRegistrations.Add(services => services.AddSingleton<TService>(implementation));This ensures the service is registered under the TService type (the base class) so that MapGrpcService<TService>() can correctly resolve it from the DI container.
| _serviceRegistrations.Add(services => services.AddSingleton(implementation)); | |
| _endpointRegistrations.Add(endpoints => endpoints.MapGrpcService<TService>()); | |
| _serviceRegistrations.Add(services => services.AddSingleton<TService>(implementation)); | |
| _endpointRegistrations.Add(endpoints => endpoints.MapGrpcService<TService>()); |
Spotted by Graphite
Is this helpful? React 👍 or 👎 to let us know.
…azy-evaluated Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…S4008) Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
| const responseBodyType = this.endpoint.response?.body?.type; | ||
| if (responseBodyType === "json") { | ||
| writer.writeLine('const string mockResponse = """'); | ||
| writer.writeLine(JSON.stringify(jsonExampleResponse, null, 2).replace(/"\\{1,2}\$ref"/g, '"$ref\"')); |
There was a problem hiding this comment.
Invalid JSON replacement that produces malformed JSON. The regex replacement .replace(/"\\{1,2}\$ref"/g, '"$ref\"') replaces "\$ref" or "\\$ref" with "$ref\", which creates invalid JSON due to the trailing backslash-quote sequence. This will cause JSON parsing to fail when the mock response is parsed.
Fix:
writer.writeLine(JSON.stringify(jsonExampleResponse, null, 2).replace(/"\\{1,2}\$ref"/g, '"$ref"'));Remove the trailing backslash from the replacement string to produce valid JSON.
| writer.writeLine(JSON.stringify(jsonExampleResponse, null, 2).replace(/"\\{1,2}\$ref"/g, '"$ref\"')); | |
| writer.writeLine(JSON.stringify(jsonExampleResponse, null, 2).replace(/"\\{1,2}\$ref"/g, '"$ref"')); |
Spotted by Graphite
Is this helpful? React 👍 or 👎 to let us know.
…fields Response types containing proto oneof fields or google.protobuf.Struct don't round-trip cleanly through JsonParser.Default.Parse<>(): - oneof: mock JSON may set multiple members simultaneously (violates constraint) - Struct: SDK JSON format doesn't match proto JSON format exactly Added responseHasProtoIncompatibleFields() which recursively checks the IR type declarations for undiscriminatedUnion (oneof) and well-known protobuf types (Struct, Value, Any). Affected endpoints (Create, Query, Fetch) now use the non-assertion path. Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
| } catch { | ||
| // Type does not have a protobuf source; fall back to undefined | ||
| return undefined; | ||
| } |
There was a problem hiding this comment.
🟡 Empty catch block swallows error without logging, violating CLAUDE.md TypeScript rules
The resolveProtoTypeById method catches all exceptions and returns undefined without logging the error. Per the repository's CLAUDE.md rule: "Never swallow errors with empty catch blocks — at minimum log the error." While there is a comment explaining intent, the error is not captured or logged, which could hide unexpected failures (e.g., misconfigured protobuf resolver) during generation.
| } catch { | |
| // Type does not have a protobuf source; fall back to undefined | |
| return undefined; | |
| } | |
| } catch (error) { | |
| // Type does not have a protobuf source; fall back to undefined | |
| this.context.logger.debug(`Could not resolve protobuf type for ${typeId}: ${error}`); | |
| return undefined; | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
This is expected behavior — getProtobufClassReference throws for non-protobuf types (e.g. SDK-only wrapper types), which is a normal control flow path. Adding debug logging here would be noisy since many types legitimately don't have protobuf sources. The comment explains the intent.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… in gRPC mock server tests - Added ParseProtoJson<T> to BaseGrpcMockServerTest that uses proto reflection to detect and strip extra oneof fields before parsing - Removed responseHasProtoIncompatibleFields() and related helpers from GrpcMockServerTestGenerator (only keeping google.protobuf.Any detection) - All tests with oneof responses (Create, Query, Fetch) now assert responses instead of falling back to smoke tests - Updated all seed output files across 4 configs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
| writer.writeLine(" var node = JsonNode.Parse(json);"); | ||
| writer.writeLine(" if (node is JsonObject obj)"); | ||
| writer.writeLine(" {"); | ||
| writer.writeLine(" foreach (var oneof in descriptor.Oneofs)"); | ||
| writer.writeLine(" {"); | ||
| writer.writeLine(" var seen = false;"); | ||
| writer.writeLine(" foreach (var field in oneof.Fields)"); | ||
| writer.writeLine(" {"); | ||
| writer.writeLine( | ||
| " var key = obj.ContainsKey(field.JsonName) ? field.JsonName : field.Name;" | ||
| ); | ||
| writer.writeLine(" if (!obj.ContainsKey(key)) continue;"); | ||
| writer.writeLine(" if (!seen) { seen = true; continue; }"); | ||
| writer.writeLine(" obj.Remove(key);"); | ||
| writer.writeLine(" }"); | ||
| writer.writeLine(" }"); | ||
| writer.writeLine(" }"); | ||
| writer.writeLine(" return JsonParser.Default.Parse<T>(node!.ToJsonString());"); |
There was a problem hiding this comment.
If JsonNode.Parse(json) returns null or a non-object JSON value (array/primitive), the code skips the oneof sanitization (line 156-171) but then unconditionally calls node!.ToJsonString() at line 172. This will throw a NullReferenceException at runtime if the JSON is null, and may not handle non-object JSON correctly.
Fix:
var node = JsonNode.Parse(json);
if (node == null)
{
return JsonParser.Default.Parse<T>(json);
}
if (node is JsonObject obj)
{
// ... existing sanitization logic ...
}
return JsonParser.Default.Parse<T>(node.ToJsonString());Spotted by Graphite
Is this helpful? React 👍 or 👎 to let us know.
Description
Adds gRPC mock server test generation for C# SDKs, mirroring the existing HTTP/WireMock-based mock server tests. Tests use a base class pattern (
BaseGrpcMockServerTest) so each test method only contains the unique stub setup, client call, and assertion — all ceremony (server building, client instantiation, stub creation) is inherited.Changes Made
Infrastructure (as-is templates)
GrpcMockServer.Template.cs— wraps an in-process ASP.NET CoreTestServer, exposes aGrpcChannelandHttpClient, implementsIAsyncDisposableGrpcMockServerBuilder.Template.cs— fluent builder:.Configure().WithService<TService>(stub).BuildAsync()using direct genericMapGrpcService<TService>()call; includesResponseVersionHandlerto patch HTTP/1.1→HTTP/2 version headers forTestServercompatibilityAsIs.ts; conditionally emitted inSdkGeneratorContext.getCoreTestAsIsFileswhen gRPC endpoints existCode generators
BaseGrpcMockServerTestGenerator.ts— generates aBaseGrpcMockServerTestclass with[SetUp]/[TearDown]lifecycle, exposingClientand per-service stub properties. Includes aParseProtoJson<T>helper that uses proto reflection to sanitize oneof conflicts at runtime (see below).GrpcStubGenerator.ts— generates a per-service stub class extending the protobufServiceBase(withglobal::prefix to avoid namespace conflicts). Each RPC gets a fluent.OnMethodName(handler), request capture inList<TRequest>, andRpcException(Unimplemented)fallback. Resolves correct proto request types via three paths: inlined request body name,extendstype IDs, and sdkRequest wrapper name fallback (for query-only endpoints likeFetch/List)GrpcMockServerTestGenerator.ts— generates per-endpoint[TestFixture]classes inheritingBaseGrpcMockServerTest. All tests useParseProtoJson<T>()for response parsing, with automatic fallback to non-assertion path only forgoogle.protobuf.Anyresponses (which require valid@typeURLs)Proto oneof sanitization via
ParseProtoJson<T>Auto-generated mock JSON from IR examples can violate proto3
oneofconstraints by setting multiple fields in the same oneof group simultaneously (e.g., bothsuccessanderror_messageinCreateResponse). Proto'sJsonParserstrictly rejects this.Rather than fixing mock JSON generation upstream (which would require proto metadata in the TypeScript IR layer), the base test class includes a
ParseProtoJson<T>helper that uses proto reflection at C# runtime to:JsonParser.Default.Parse<T>()This allows 12 of 13 test methods to assert responses. The only remaining non-assertion test is
UpdateTestwhich containsgoogle.protobuf.Anywith an invalid@typeURL.Wiring
generateSdkTests.ts— addedgenerateGrpcMockServerTests()alongside existing HTTP path; skips multi-URL environments without defaults; only generates stubs for services with actual testsTemplate.Test.csproj— conditionally includesGrpc.AspNetCoreandMicrosoft.AspNetCore.Mvc.TestingNuGet packages via Eta<% if (it.grpc) %>block; MSBuild target excludes proto message types from test compilation (resolves from SDK project reference instead)CsharpProject.ts— passesgrpc: this.context.hasGrpcEndpoints()andprotoServerItemsto template rendererSeed output
csharp-grpc-proto-exhaustivefixture configs (no-custom-config, package-id, read-only-memory, include-exception-handler)Human review checklist
ParseProtoJson<T>oneof sanitization logic — it keeps the first oneof field encountered and strips the rest. This is arbitrary but sufficient for mock testing; confirm this is acceptablenode!.ToJsonString()null-forgiving operator inParseProtoJson<T>assumesJsonNode.Parse(json)never returns null for valid mock JSON — verify this holdsgoogle.protobuf.Anydetection (hasProtoAnyInResponse) checks for@typekey in the mock JSON string — confirm this doesn't false-positive on JSON payloads that mention@typeas a regular fieldgoogle.protobuf.Any) is non-assertion; all other 12 tests assertendpoint.requestBody.name.originalNameandendpoint.sdkRequest.shape.wrapperName.originalNamematch the proto message name holds for all gRPC APIs[SetUp]/[TearDown]lifecycle — confirm stub, server, and client are properly scoped per testResponseVersionHandlerinjection — patches HTTP/1.1→HTTP/2 version headers forTestServercompatibility; verify this doesn't affect request/response semanticsGrpcMockServer.DisposeAsync(Channel → HttpClient → TestServer)Testing
google.protobuf.Anyuses non-assertion path)dotnet build)dotnet test)Link to Devin session: https://app.devin.ai/sessions/a07bc8a996e7471b89fe59b7fe8a6175
Requested by: @Swimburger