Phase 36: Auth, Observability & System Management Test Hardening (+33 unit tests)

EnterpriseIntegrationPlatform/rules/completion-log.md

@@ -4,6 +4,89 @@ Detailed record of completed chunks, files created/modified, and notes.
 
 See `milestones.md` for current phase status and next chunk.
 
+## Chunk 362 — ControlBusPublisher & DlqManagementService Tests
+
+- **Date**: 2026-04-09
+- **Phase**: 36 — Auth, Observability & System Management Test Hardening
+- **Status**: done
+- **Goal**: Dedicated unit tests for ControlBusPublisher (10 tests) covering publish success/failure, command intent, topic routing, argument validation, subscribe registration. DlqManagementService (2 tests) covering replay delegation and filter passthrough.
+- **Files created**:
+  - `tests/UnitTests/ControlBusPublisherTests.cs` — 12 tests across 2 fixtures: ControlBusPublisherTests (PublishCommandAsync success/topic/intent/failure/null command/empty type, SubscribeAsync registration/null handler/empty type, constructor null producer/consumer), DlqManagementServiceTests (ResubmitAsync delegates to replayer, passes filter)
+- **Notes**:
+  - ControlBusPublisher.PublishCommandAsync sets MessageIntent.Command on the envelope
+  - NSubstitute When..Do pattern used to capture generic envelope argument
+  - All 12 tests pass. UnitTests total: 1919 (cumulative with chunks 360–361)
+
+## Chunk 361 — LokiObservabilityEventLog Tests
+
+- **Date**: 2026-04-09
+- **Phase**: 36 — Auth, Observability & System Management Test Hardening
+- **Status**: done
+- **Goal**: Dedicated unit tests for LokiObservabilityEventLog covering RecordAsync (Loki push + in-memory fallback), GetByCorrelationId (Loki query + fallback), GetByBusinessKey (fallback + case-insensitive matching).
+- **Files created**:
+  - `tests/UnitTests/LokiObservabilityEventLogTests.cs` — 11 tests: RecordAsync (posts to push endpoint, Loki unavailable doesn't throw, always stores in fallback, correct content type, payload contains correlation_id, multiple events stored), GetByCorrelationId (Loki returns results, no matching returns empty), GetByBusinessKey (Loki unavailable uses fallback, case-insensitive fallback)
+- **Notes**:
+  - Uses MockLokiHandler (DelegatingHandler) for HTTP interception with separate push/query status codes
+  - Uses reflection to clear the static FallbackStore between tests for isolation
+  - Loki query_range response format: values are [timestamp_string, json_string] — log line is a JSON-escaped string
+  - All 11 tests pass
+
+## Chunk 360 — ApiKeyAuthenticationHandler Tests
+
+- **Date**: 2026-04-09
+- **Phase**: 36 — Auth, Observability & System Management Test Hardening
+- **Status**: done
+- **Goal**: Dedicated unit tests for the security-critical ApiKeyAuthenticationHandler covering all authentication paths.
+- **Files created/modified**:
+  - `tests/UnitTests/ApiKeyAuthenticationHandlerTests.cs` — 10 tests: missing header fails, invalid key fails, valid key succeeds, valid key sets Admin role claim, sets Name claim, sets apikey_prefix claim with masking, multiple configured keys accepts any, case-sensitive comparison rejects wrong case, empty configured keys rejects all, short key masks entirely
+  - `src/Admin.Api/Admin.Api.csproj` — added `<InternalsVisibleTo Include="UnitTests" />`
+  - `tests/UnitTests/UnitTests.csproj` — added `<FrameworkReference Include="Microsoft.AspNetCore.App" />`
+- **Notes**:
+  - ApiKeyAuthenticationHandler is `internal sealed` — InternalsVisibleTo enables direct test instantiation
+  - Tests use `AuthenticationHandler.InitializeAsync` + `AuthenticateAsync` pattern with `DefaultHttpContext`
+  - MaskKey masks keys longer than 4 chars to `first4****`, short keys to `****`
+  - All 10 tests pass
+
+## Chunk 352 — HttpEnrichmentSource & DatabaseEnrichmentSource Tests
+
+- **Date**: 2026-04-09
+- **Phase**: 35 — Observability, Validation & Enrichment Test Hardening
+- **Status**: done
+- **Goal**: Dedicated unit tests for HttpEnrichmentSource (6 tests) and DatabaseEnrichmentSource (6 tests) covering constructor validation, successful enrichment, URL key substitution, HTTP error handling, empty DB results, and column-to-JSON mapping.
+- **Files created**:
+  - `tests/UnitTests/HttpDatabaseEnrichmentSourceTests.cs` — 12 tests: HttpEnrichmentSource (constructor null factory/options/logger, FetchAsync success returns JsonNode, FetchAsync replaces key placeholder, FetchAsync non-success throws), DatabaseEnrichmentSource (constructor null factory, empty SQL, empty param, null logger, FetchAsync no rows returns null, FetchAsync with row returns JsonObject)
+- **Notes**:
+  - HttpEnrichmentSource tests use a lightweight MockHandler (DelegatingHandler subclass) for HTTP interception
+  - DatabaseEnrichmentSource tests use concrete ADO.NET test doubles (InMemoryDbConnection/Command/Reader) since DbCommand.Parameters is non-virtual
+  - All 12 tests pass. UnitTests total: 1886 (cumulative with chunks 350–351)
+
+## Chunk 351 — Activity Service & Message Validation Tests
+
+- **Date**: 2026-04-09
+- **Phase**: 35 — Observability, Validation & Enrichment Test Hardening
+- **Status**: done
+- **Goal**: Dedicated unit tests for DefaultMessageValidationService (8 tests), DefaultCompensationActivityService (2 tests), MessageValidationResult factory methods (2 tests), and MessageHistoryEntry record (2 tests).
+- **Files created**:
+  - `tests/UnitTests/ActivityServiceTests.cs` — 14 tests across 4 fixtures: DefaultMessageValidationServiceTests (empty/whitespace type, empty/whitespace payload, non-JSON, valid JSON object, valid JSON array, whitespace-prefixed JSON), DefaultCompensationActivityServiceTests (valid input returns true, any step returns true), MessageValidationResultTests (Success/Failure factories), MessageHistoryEntryTests (constructor properties, enum values)
+- **Notes**:
+  - DefaultMessageValidationService validates: non-empty message type, non-empty payload, payload starts with `{` or `[`
+  - DefaultCompensationActivityService always returns true (logs compensation; real implementations override)
+  - All 14 tests pass
+
+## Chunk 350 — CorrelationPropagator, PlatformMeters & MessageTracer Tests
+
+- **Date**: 2026-04-09
+- **Phase**: 35 — Observability, Validation & Enrichment Test Hardening
+- **Status**: done
+- **Goal**: Dedicated unit tests for CorrelationPropagator (6 tests), PlatformMeters (5 tests), and MessageTracer (6 tests) covering trace context injection/extraction, metric recording via MeterListener, and message lifecycle tracing.
+- **Files created**:
+  - `tests/UnitTests/CorrelationPropagatorTests.cs` — 17 tests across 3 fixtures: CorrelationPropagatorTests (inject without Activity, inject with Activity sets TraceId/SpanId, extract with valid headers, extract without headers, extract default kind, extract explicit kind), PlatformMetersTests (RecordReceived, RecordProcessed, RecordFailed, RecordDeadLettered, RecordRetry via MeterListener), MessageTracerTests (TraceIngestion/Routing/Transformation/Delivery, CompleteSuccess sets Ok status, CompleteFailed sets Error status)
+- **Notes**:
+  - CorrelationPropagator tests use ActivityListener to capture Activities from DiagnosticsConfig.ActivitySource
+  - PlatformMeters tests use System.Diagnostics.Metrics.MeterListener to verify counter/histogram/gauge recordings
+  - MessageTracer tests verify that trace stages create properly tagged Activity spans
+  - All 17 tests pass
+
 ## Chunk 344 — AI & Remaining DI Tests
 
 - **Date**: 2026-04-09

EnterpriseIntegrationPlatform/rules/milestones.md

@@ -248,3 +248,62 @@ Tests verify every `Add*` method registers the correct service types in the cont
 ### Next Chunk
 
 Phase 34 is complete. No remaining chunks.
+
+---
+
+## Phase 35 — Observability, Validation & Enrichment Test Hardening
+
+> **Origin:** Audit revealed that CorrelationPropagator, PlatformMeters, and MessageTracer
+> (core observability infrastructure used by every service) had **zero dedicated unit tests**.
+> DefaultMessageValidationService and DefaultCompensationActivityService (core activity services)
+> were also untested. HttpEnrichmentSource and DatabaseEnrichmentSource (external enrichment
+> patterns) lacked tests. This phase closes these test gaps.
+
+| Chunk | Description | Status |
+|-------|-------------|--------|
+| 350 | **CorrelationPropagator, PlatformMeters & MessageTracer Tests** — see `rules/completion-log.md` | `done` |
+| 351 | **Activity Service & Message Validation Tests** — see `rules/completion-log.md` | `done` |
+| 352 | **HttpEnrichmentSource & DatabaseEnrichmentSource Tests** — see `rules/completion-log.md` | `done` |
+
+### Summary
+
+Phase 35 complete — 3 chunks (350–352). 43 new unit tests. UnitTests total: 1886 (was 1843).
+CorrelationPropagator now has 6 tests covering trace context injection/extraction.
+PlatformMeters has 5 tests verifying counter/histogram/gauge recordings via MeterListener.
+MessageTracer has 6 tests covering all 4 trace stage methods plus success/failure completion.
+DefaultMessageValidationService has 8 tests covering all validation paths.
+DefaultCompensationActivityService has 2 tests. MessageValidationResult factory methods tested.
+MessageHistoryEntry record and enum values tested. HttpEnrichmentSource has 6 tests with mock
+HTTP handler. DatabaseEnrichmentSource has 6 tests with mock ADO.NET infrastructure.
+
+---
+
+## Phase 36 — Auth, Observability & System Management Test Hardening
+
+> **Origin:** Audit revealed that `ApiKeyAuthenticationHandler` (security-critical auth handler)
+> had **zero unit tests**. `LokiObservabilityEventLog` (core observability event storage with
+> Loki HTTP API + in-memory fallback) was untested. `ControlBusPublisher` (EIP Control Bus pattern)
+> and `DlqManagementService` (DLQ resubmission) lacked dedicated test fixtures. This phase
+> closes these critical test gaps.
+
+| Chunk | Description | Status |
+|-------|-------------|--------|
+| 360 | **ApiKeyAuthenticationHandler Tests** — see `rules/completion-log.md` | `done` |
+| 361 | **LokiObservabilityEventLog Tests** — see `rules/completion-log.md` | `done` |
+| 362 | **ControlBusPublisher & DlqManagementService Tests** — see `rules/completion-log.md` | `done` |
+
+### Summary
+
+Phase 36 complete — 3 chunks (360–362). 33 new unit tests. UnitTests total: 1919 (was 1886).
+ApiKeyAuthenticationHandler now has 10 tests covering missing header, invalid key, valid key
+with claim verification, case-sensitive comparison, multiple keys, empty config, short key masking.
+LokiObservabilityEventLog has 11 tests covering RecordAsync (push, fallback, error handling),
+GetByCorrelationId (Loki results, fallback, empty), GetByBusinessKey (fallback, case-insensitive).
+ControlBusPublisher has 10 tests covering publish success/failure/intent, subscribe, arg validation.
+DlqManagementService has 2 tests covering replay delegation and filter passthrough.
+
+---
+
+### Next Chunk
+
+Phase 36 is complete. No remaining chunks.

EnterpriseIntegrationPlatform/src/Admin.Api/Admin.Api.csproj

@@ -1,4 +1,7 @@
 <Project Sdk="Microsoft.NET.Sdk.Web">
+  <ItemGroup>
+    <InternalsVisibleTo Include="UnitTests" />
+  </ItemGroup>
   <ItemGroup>
     <ProjectReference Include="..\ServiceDefaults\ServiceDefaults.csproj" />
     <ProjectReference Include="..\Observability\Observability.csproj" />

EnterpriseIntegrationPlatform/tests/UnitTests/ActivityServiceTests.cs

@@ -0,0 +1,168 @@
+using EnterpriseIntegrationPlatform.Activities;
+using EnterpriseIntegrationPlatform.Contracts;
+using Microsoft.Extensions.Logging;
+using Microsoft.Extensions.Logging.Abstractions;
+using NSubstitute;
+using NUnit.Framework;
+
+namespace EnterpriseIntegrationPlatform.Tests.Unit;
+
+[TestFixture]
+public sealed class DefaultMessageValidationServiceTests
+{
+    private DefaultMessageValidationService _sut = null!;
+
+    [SetUp]
+    public void SetUp()
+    {
+        _sut = new DefaultMessageValidationService();
+    }
+
+    [Test]
+    public async Task ValidateAsync_EmptyMessageType_ReturnsFailure()
+    {
+        var result = await _sut.ValidateAsync("", """{"id":1}""");
+
+        Assert.That(result.IsValid, Is.False);
+        Assert.That(result.Reason, Is.EqualTo("Message type must not be empty."));
+    }
+
+    [Test]
+    public async Task ValidateAsync_WhitespaceMessageType_ReturnsFailure()
+    {
+        var result = await _sut.ValidateAsync("   ", """{"id":1}""");
+
+        Assert.That(result.IsValid, Is.False);
+        Assert.That(result.Reason, Is.EqualTo("Message type must not be empty."));
+    }
+
+    [Test]
+    public async Task ValidateAsync_EmptyPayload_ReturnsFailure()
+    {
+        var result = await _sut.ValidateAsync("OrderCreated", "");
+
+        Assert.That(result.IsValid, Is.False);
+        Assert.That(result.Reason, Is.EqualTo("Payload must not be empty."));
+    }
+
+    [Test]
+    public async Task ValidateAsync_WhitespacePayload_ReturnsFailure()
+    {
+        var result = await _sut.ValidateAsync("OrderCreated", "   ");
+
+        Assert.That(result.IsValid, Is.False);
+        Assert.That(result.Reason, Is.EqualTo("Payload must not be empty."));
+    }
+
+    [Test]
+    public async Task ValidateAsync_NonJsonPayload_ReturnsFailure()
+    {
+        var result = await _sut.ValidateAsync("OrderCreated", "plain text");
+
+        Assert.That(result.IsValid, Is.False);
+        Assert.That(result.Reason, Is.EqualTo("Payload is not valid JSON."));
+    }
+
+    [Test]
+    public async Task ValidateAsync_ValidJsonObject_ReturnsSuccess()
+    {
+        var result = await _sut.ValidateAsync("OrderCreated", """{"id":1}""");
+
+        Assert.That(result.IsValid, Is.True);
+        Assert.That(result.Reason, Is.Null);
+    }
+
+    [Test]
+    public async Task ValidateAsync_ValidJsonArray_ReturnsSuccess()
+    {
+        var result = await _sut.ValidateAsync("OrderCreated", "[1,2,3]");
+
+        Assert.That(result.IsValid, Is.True);
+        Assert.That(result.Reason, Is.Null);
+    }
+
+    [Test]
+    public async Task ValidateAsync_WhitespaceBeforeJson_ReturnsSuccess()
+    {
+        var result = await _sut.ValidateAsync("OrderCreated", """  {"id":1}""");
+
+        Assert.That(result.IsValid, Is.True);
+        Assert.That(result.Reason, Is.Null);
+    }
+}
+
+[TestFixture]
+public sealed class DefaultCompensationActivityServiceTests
+{
+    private DefaultCompensationActivityService _sut = null!;
+
+    [SetUp]
+    public void SetUp()
+    {
+        var logger = NullLogger<DefaultCompensationActivityService>.Instance;
+        _sut = new DefaultCompensationActivityService(logger);
+    }
+
+    [Test]
+    public async Task CompensateAsync_ValidInput_ReturnsTrue()
+    {
+        var result = await _sut.CompensateAsync(Guid.NewGuid(), "DebitAccount");
+
+        Assert.That(result, Is.True);
+    }
+
+    [Test]
+    public async Task CompensateAsync_AnyStepName_ReturnsTrue()
+    {
+        var result = await _sut.CompensateAsync(Guid.NewGuid(), "AnyArbitraryStep");
+
+        Assert.That(result, Is.True);
+    }
+}
+
+[TestFixture]
+public sealed class MessageValidationResultTests
+{
+    [Test]
+    public void Success_IsValid_ReturnsTrue()
+    {
+        var result = MessageValidationResult.Success;
+
+        Assert.That(result.IsValid, Is.True);
+        Assert.That(result.Reason, Is.Null);
+    }
+
+    [Test]
+    public void Failure_IsNotValid_ReturnsFalseWithReason()
+    {
+        var result = MessageValidationResult.Failure("Something went wrong.");
+
+        Assert.That(result.IsValid, Is.False);
+        Assert.That(result.Reason, Is.EqualTo("Something went wrong."));
+    }
+}
+
+[TestFixture]
+public sealed class MessageHistoryEntryTests
+{
+    [Test]
+    public void Constructor_SetsProperties()
+    {
+        var timestamp = DateTimeOffset.UtcNow;
+        var entry = new MessageHistoryEntry("Validate", timestamp, MessageHistoryStatus.Completed, "All good");
+
+        Assert.That(entry.ActivityName, Is.EqualTo("Validate"));
+        Assert.That(entry.Timestamp, Is.EqualTo(timestamp));
+        Assert.That(entry.Status, Is.EqualTo(MessageHistoryStatus.Completed));
+        Assert.That(entry.Detail, Is.EqualTo("All good"));
+    }
+
+    [Test]
+    public void Status_Enum_HasExpectedValues()
+    {
+        Assert.That((int)MessageHistoryStatus.Completed, Is.EqualTo(0));
+        Assert.That((int)MessageHistoryStatus.Skipped, Is.EqualTo(1));
+        Assert.That((int)MessageHistoryStatus.Failed, Is.EqualTo(2));
+        Assert.That((int)MessageHistoryStatus.InProgress, Is.EqualTo(3));
+    }
+}