You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[PUBLIC API BREAKING CHANGE] Feature/cache enhancements (#9)
* feat: CacheInteraction enum has been introduced to classify data request fulfillment;
refactor: RangeResult structure has been updated to include CacheInteraction property;
docs: documentation has been updated to reflect changes in CacheInteraction and RangeResult
* feat: strong and hybrid consistency modes for WindowCacheExtensions have been validated
* refactor: remove outdated TODO comment from WindowCacheExtensions
* feat: runtime options update mechanism has been implemented;
refactor: cache execution controllers have been updated to utilize runtime options holder;
docs: runtime options update functionality has been documented in relevant files
* refactor: simplify syntax in test classes and improve readability;
feat: add CurrentRuntimeOptions property to expose runtime option snapshots;
chore: introduce RuntimeOptionsValidator for shared validation logic;
test: add unit tests for RuntimeOptionsSnapshot and RuntimeOptionsValidator
* feat: built-in wrappers for data source configuration have been implemented;
refactor: renamed WindowCacheExtensions to WindowCacheConsistencyExtensions for clarity;
docs: updated documentation to reflect new extension methods and their behaviors
* feat(cache): implement layered cache builder and data source adapter for multi-layer caching
* feat: implement FuncDataSource for inline data source creation;
docs: update README with usage examples for FuncDataSource
* refactor: simplify variable declarations and update cache builder references
* fix: validate debounce delay to ensure non-negative values in cache options
* docs: update invariants documentation to reflect changes in system invariants count;
refactor: improve validation remarks in RuntimeOptionsValidator and update example in IWindowCache
---------
Co-authored-by: Mykyta Zotov <mykyta.zotov@ihsmarkit.com>
-**Hybrid consistency**: `GetDataAndWaitOnMissAsync` — waits for idle only on `PartialHit` or `FullMiss`; returns immediately on `FullHit`. Use for warm-cache guarantees without always paying the idle-wait cost.
224
+
-**Strong consistency**: `GetDataAndWaitForIdleAsync` — always waits for idle regardless of `CacheInteraction`
225
+
226
+
**Serialized Access Requirement for Hybrid/Strong Modes:**
227
+
`GetDataAndWaitOnMissAsync` and `GetDataAndWaitForIdleAsync` provide their warm-cache guarantee only under **serialized (one-at-a-time) access**. Under parallel access, `WaitForIdleAsync`'s "was idle at some point" semantics (Invariant H.49) may return the old completed TCS, missing the rebalance triggered by the concurrent request. These methods remain safe (no crashes/hangs) but the guarantee degrades under parallelism.
228
+
221
229
**Lock-Free Operations:**
222
230
```csharp
223
231
// Intent management using Volatile and Interlocked
A read-only, range-based, sequential-optimized cache with decision-driven background rebalancing, smart eventual consistency, and intelligent work avoidance.
3
+
A read-only, range-based, sequential-optimized cache with decision-driven background rebalancing, three consistency modes (eventual/hybrid/strong), and intelligent work avoidance.
- Smart eventual consistency: cache converges to optimal configuration while avoiding unnecessary work
20
+
- Opt-in hybrid or strong consistency via extension methods (`GetDataAndWaitOnMissAsync`, `GetDataAndWaitForIdleAsync`)
20
21
21
22
For the canonical architecture docs, see `docs/architecture.md`.
22
23
@@ -126,7 +127,7 @@ Key points:
126
127
2.**Decision happens in background** — CPU-only validation (microseconds) in the intent processing loop
127
128
3.**Work avoidance prevents thrashing** — validation may skip rebalance entirely if unnecessary
128
129
4.**Only I/O happens asynchronously** — debounce + data fetching + cache updates run in background
129
-
5.**Smart eventual consistency** — cache converges to optimal state while avoiding unnecessary operations
130
+
5.**Smart eventual consistency** — cache converges to optimal state while avoiding unnecessary operations; opt-in hybrid or strong consistency via extension methods
130
131
131
132
## Materialization for Fast Access
132
133
@@ -143,33 +144,65 @@ For detailed comparison and guidance, see `docs/storage-strategies.md`.
143
144
144
145
```csharp
145
146
usingSlidingWindowCache;
146
-
usingSlidingWindowCache.Configuration;
147
+
usingSlidingWindowCache.Public.Cache;
148
+
usingSlidingWindowCache.Public.Configuration;
147
149
usingIntervals.NET;
148
150
usingIntervals.NET.Domain.Default.Numeric;
149
151
150
-
varoptions=newWindowCacheOptions(
151
-
leftCacheSize: 1.0, // Cache 100% of requested range size to the left
152
-
rightCacheSize: 2.0, // Cache 200% of requested range size to the right
153
-
leftThreshold: 0.2, // Rebalance if <20% left buffer remains
154
-
rightThreshold: 0.2// Rebalance if <20% right buffer remains
Implement `IDataSource<TRange, TData>` to connect the cache to your backing store. The `FetchAsync` single-range overload is the only method you must provide; the batch overload has a default implementation that parallelizes single-range calls.
168
+
169
+
### FuncDataSource — inline without a class
170
+
171
+
`FuncDataSource<TRange, TData>` wraps an async delegate so you can create a data source in one expression:
172
+
173
+
```csharp
174
+
usingSlidingWindowCache.Public;
175
+
usingSlidingWindowCache.Public.Dto;
176
+
177
+
// Unbounded source — always returns data for any range
For **bounded** sources (database with min/max IDs, time-series with temporal limits), return a `RangeChunk` with `Range = null` when no data is available — never throw:
For sources where a dedicated class is warranted (custom batch optimization, retry logic, dependency injection), implement `IDataSource<TRange, TData>` directly. See `docs/boundary-handling.md` for the full boundary contract.
202
+
170
203
## Boundary Handling
171
204
172
-
`GetDataAsync` returns `RangeResult<TRange, TData>` where `Range` may be `null` when the data source has no data for the requested range. Always check before accessing data:
205
+
`GetDataAsync` returns `RangeResult<TRange, TData>` where `Range` may be `null` when the data source has no data for the requested range, and `CacheInteraction` indicates whether the request was a `FullHit`, `PartialHit`, or `FullMiss`. Always check`Range` before accessing data:
@@ -267,6 +300,51 @@ var options = new WindowCacheOptions(
267
300
);
268
301
```
269
302
303
+
## Runtime Options Update
304
+
305
+
Cache sizing, threshold, and debounce options can be changed on a live cache instance without recreation. Updates take effect on the **next rebalance decision/execution cycle**.
306
+
307
+
```csharp
308
+
// Change left and right cache sizes at runtime
309
+
cache.UpdateRuntimeOptions(update=>
310
+
update.WithLeftCacheSize(3.0)
311
+
.WithRightCacheSize(3.0));
312
+
313
+
// Change debounce delay
314
+
cache.UpdateRuntimeOptions(update=>
315
+
update.WithDebounceDelay(TimeSpan.Zero));
316
+
317
+
// Change thresholds — or clear a threshold to null
318
+
cache.UpdateRuntimeOptions(update=>
319
+
update.WithLeftThreshold(0.15)
320
+
.ClearRightThreshold());
321
+
```
322
+
323
+
`UpdateRuntimeOptions` uses a **fluent builder** (`RuntimeOptionsUpdateBuilder`). Only fields explicitly set via builder calls are changed — all other options remain at their current values.
324
+
325
+
**Constraints:**
326
+
-`ReadMode` and `RebalanceQueueCapacity` are creation-time only and cannot be changed at runtime.
327
+
- All validation rules from construction still apply (`ArgumentOutOfRangeException` for negative sizes, `ArgumentException` for threshold sum > 1.0, etc.). A failed update leaves the current options unchanged — no partial application.
328
+
- Calling `UpdateRuntimeOptions` on a disposed cache throws `ObjectDisposedException`.
329
+
330
+
**`LayeredWindowCache`** delegates `UpdateRuntimeOptions` to the outermost (user-facing) layer. To update a specific inner layer, use the `Layers` property (see Multi-Layer Cache below).
331
+
332
+
## Reading Current Runtime Options
333
+
334
+
Use `CurrentRuntimeOptions` to inspect the live option values on any cache instance. It returns a `RuntimeOptionsSnapshot` — a read-only point-in-time copy of the five runtime-updatable values.
The snapshot is immutable. Subsequent calls to `UpdateRuntimeOptions` do not affect previously obtained snapshots — obtain a new snapshot to see updated values.
346
+
347
+
- Calling `CurrentRuntimeOptions` on a disposed cache throws `ObjectDisposedException`.
270
348
## Diagnostics
271
349
272
350
⚠️ **CRITICAL: You MUST handle `RebalanceExecutionFailed` in production.** Rebalance operations run in background tasks. Without handling this event, failures are silently swallowed and the cache stops rebalancing with no indication.
6.`docs/state-machine.md` — formal state transitions and mutation ownership
321
399
7.`docs/actors.md` — actor responsibilities and execution contexts
322
400
323
-
## Strong Consistency Mode
401
+
## Consistency Modes
324
402
325
-
By default, `GetDataAsync` is **eventually consistent**: data is returned immediately while the cache window converges asynchronously in the background. For scenarios where you need the cache to be fully converged before proceeding, use the `GetDataAndWaitForIdleAsync` extension method:
403
+
By default, `GetDataAsync` is **eventually consistent**: data is returned immediately while the cache window converges asynchronously in the background. Two opt-in extension methods provide stronger consistency guarantees. Both require a `using SlidingWindowCache.Public;` import.
404
+
405
+
> **Serialized access requirement:** The hybrid and strong consistency modes provide their warm-cache guarantee only when requests are made one at a time (serialized). Under concurrent/parallel callers they remain safe (no crashes or hangs) but the guarantee degrades — due to `AsyncActivityCounter`'s "was idle at some point" semantics (Invariant H.49) and a brief gap between the counter increment and TCS publication in `IncrementActivity`, a concurrent waiter may observe a previously completed idle TCS and return without waiting for the new rebalance.
406
+
407
+
### Eventual Consistency (Default)
408
+
409
+
```csharp
410
+
// Returns immediately; rebalance converges asynchronously in background
// Waits for idle only if the request was a PartialHit or FullMiss; returns immediately on FullHit
422
+
varresult=awaitcache.GetDataAndWaitOnMissAsync(
423
+
Range.Closed(100, 200),
424
+
cancellationToken);
425
+
426
+
// result.CacheInteraction tells you which path was taken:
427
+
// CacheInteraction.FullHit → returned immediately (no wait)
428
+
// CacheInteraction.PartialHit → waited for cache to converge
429
+
// CacheInteraction.FullMiss → waited for cache to converge
430
+
if (result.Range.HasValue)
431
+
ProcessData(result.Data);
432
+
```
433
+
434
+
**When to use:**
435
+
- Warm-cache fast path: pays no penalty on cache hits, still waits on misses
436
+
- Access patterns where most requests are hits but you want convergence on misses
437
+
438
+
**When NOT to use:**
439
+
- First request (always a miss — pays full debounce + I/O wait)
440
+
- Hot paths with many misses
441
+
442
+
> **Cancellation:** If the cancellation token fires during the idle wait (after `GetDataAsync` has already returned data), the method catches `OperationCanceledException` and returns the already-obtained result gracefully — degrading to eventual consistency for that call. The background rebalance is not affected.
@@ -347,17 +466,30 @@ This is a thin composition of `GetDataAsync` followed by `WaitForIdleAsync`. The
347
466
**When NOT to use:**
348
467
- Hot paths or rapid sequential requests — each call waits for full rebalance, which includes the debounce delay plus data fetching. For normal usage, the default eventual consistency model is faster.
349
468
469
+
> **Cancellation:** If the cancellation token fires during the idle wait (after `GetDataAsync` has already returned data), the method catches `OperationCanceledException` and returns the already-obtained result gracefully — degrading to eventual consistency for that call. The background rebalance is not affected.
470
+
350
471
### Deterministic Testing
351
472
352
473
`WaitForIdleAsync()` provides race-free synchronization with background operations for tests. Uses "was idle at some point" semantics — does not guarantee still idle after completion. See `docs/invariants.md` (Activity tracking invariants).
353
474
475
+
### CacheInteraction on RangeResult
476
+
477
+
Every `RangeResult` carries a `CacheInteraction` property classifying the request:
|`FullHit`| Entire requested range was served from cache |
482
+
|`PartialHit`| Request partially overlapped the cache; missing part fetched from `IDataSource`|
483
+
|`FullMiss`| No overlap (cold start or jump); full range fetched from `IDataSource`|
484
+
485
+
This is the per-request programmatic alternative to the `UserRequestFullCacheHit` / `UserRequestPartialCacheHit` / `UserRequestFullCacheMiss` diagnostics callbacks.
486
+
354
487
## Multi-Layer Cache
355
488
356
489
For workloads with high-latency data sources, you can compose multiple `WindowCache` instances into a layered stack. Each layer uses the layer below it as its data source, allowing you to trade memory for reduced data-source I/O.
.AddLayer(newWindowCacheOptions( // L2: deep background cache
362
494
leftCacheSize: 10.0,
363
495
rightCacheSize: 10.0,
@@ -375,6 +507,21 @@ var result = await cache.GetDataAsync(range, ct);
375
507
376
508
`LayeredWindowCache` implements `IWindowCache` and is `IAsyncDisposable` — it owns and disposes all layers when you dispose it.
377
509
510
+
**Accessing and updating individual layers:**
511
+
512
+
Use the `Layers` property to access any specific layer by index (0 = innermost, last = outermost). Each layer exposes the full `IWindowCache` interface:
513
+
514
+
```csharp
515
+
// Update options on the innermost (deep background) layer
-`ProportionalRangePlanner<TRange, TDomain>` — computes `DesiredCacheRange`; reads configuration from `RuntimeCacheOptionsHolder` at invocation time
81
+
-`NoRebalanceSatisfactionPolicy<TRange>` / `NoRebalanceRangePlanner<TRange, TDomain>` — computes `NoRebalanceRange`; reads configuration from `RuntimeCacheOptionsHolder` at invocation time
0 commit comments