Skip to content

Commit aff1721

Browse files
blaze6950Mykyta Zotov
andauthored
[PUBLIC API BREAKING CHANGE] Feature/cache enhancements (#9)
* feat: CacheInteraction enum has been introduced to classify data request fulfillment; refactor: RangeResult structure has been updated to include CacheInteraction property; docs: documentation has been updated to reflect changes in CacheInteraction and RangeResult * feat: strong and hybrid consistency modes for WindowCacheExtensions have been validated * refactor: remove outdated TODO comment from WindowCacheExtensions * feat: runtime options update mechanism has been implemented; refactor: cache execution controllers have been updated to utilize runtime options holder; docs: runtime options update functionality has been documented in relevant files * refactor: simplify syntax in test classes and improve readability; feat: add CurrentRuntimeOptions property to expose runtime option snapshots; chore: introduce RuntimeOptionsValidator for shared validation logic; test: add unit tests for RuntimeOptionsSnapshot and RuntimeOptionsValidator * feat: built-in wrappers for data source configuration have been implemented; refactor: renamed WindowCacheExtensions to WindowCacheConsistencyExtensions for clarity; docs: updated documentation to reflect new extension methods and their behaviors * feat(cache): implement layered cache builder and data source adapter for multi-layer caching * feat: implement FuncDataSource for inline data source creation; docs: update README with usage examples for FuncDataSource * refactor: simplify variable declarations and update cache builder references * fix: validate debounce delay to ensure non-negative values in cache options * docs: update invariants documentation to reflect changes in system invariants count; refactor: improve validation remarks in RuntimeOptionsValidator and update example in IWindowCache --------- Co-authored-by: Mykyta Zotov <mykyta.zotov@ihsmarkit.com>
1 parent 0d62e14 commit aff1721

76 files changed

Lines changed: 7042 additions & 1314 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

AGENTS.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,14 @@ catch (Exception ex)
218218
- WindowCache is **NOT designed for multiple users sharing one cache** (violates coherent access pattern)
219219
- Multiple threads from the SAME logical consumer CAN call WindowCache safely (read-only User Path)
220220

221+
**Consistency Modes (three options):**
222+
- **Eventual consistency** (default): `GetDataAsync` — returns immediately, cache converges in background
223+
- **Hybrid consistency**: `GetDataAndWaitOnMissAsync` — waits for idle only on `PartialHit` or `FullMiss`; returns immediately on `FullHit`. Use for warm-cache guarantees without always paying the idle-wait cost.
224+
- **Strong consistency**: `GetDataAndWaitForIdleAsync` — always waits for idle regardless of `CacheInteraction`
225+
226+
**Serialized Access Requirement for Hybrid/Strong Modes:**
227+
`GetDataAndWaitOnMissAsync` and `GetDataAndWaitForIdleAsync` provide their warm-cache guarantee only under **serialized (one-at-a-time) access**. Under parallel access, `WaitForIdleAsync`'s "was idle at some point" semantics (Invariant H.49) may return the old completed TCS, missing the rebalance triggered by the concurrent request. These methods remain safe (no crashes/hangs) but the guarantee degrades under parallelism.
228+
221229
**Lock-Free Operations:**
222230
```csharp
223231
// Intent management using Volatile and Interlocked

README.md

Lines changed: 169 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Sliding Window Cache
22

3-
A read-only, range-based, sequential-optimized cache with decision-driven background rebalancing, smart eventual consistency, and intelligent work avoidance.
3+
A read-only, range-based, sequential-optimized cache with decision-driven background rebalancing, three consistency modes (eventual/hybrid/strong), and intelligent work avoidance.
44

55
[![CI/CD](https://github.com/blaze6950/SlidingWindowCache/actions/workflows/slidingwindowcache.yml/badge.svg)](https://github.com/blaze6950/SlidingWindowCache/actions/workflows/slidingwindowcache.yml)
66
[![NuGet](https://img.shields.io/nuget/v/SlidingWindowCache.svg)](https://www.nuget.org/packages/SlidingWindowCache/)
@@ -17,6 +17,7 @@ Optimized for access patterns that move predictably across a domain (scrolling,
1717
- Single-writer architecture: only rebalance execution mutates shared cache state
1818
- Decision-driven execution: multi-stage analytical validation prevents thrashing and unnecessary I/O
1919
- Smart eventual consistency: cache converges to optimal configuration while avoiding unnecessary work
20+
- Opt-in hybrid or strong consistency via extension methods (`GetDataAndWaitOnMissAsync`, `GetDataAndWaitForIdleAsync`)
2021

2122
For the canonical architecture docs, see `docs/architecture.md`.
2223

@@ -126,7 +127,7 @@ Key points:
126127
2. **Decision happens in background** — CPU-only validation (microseconds) in the intent processing loop
127128
3. **Work avoidance prevents thrashing** — validation may skip rebalance entirely if unnecessary
128129
4. **Only I/O happens asynchronously** — debounce + data fetching + cache updates run in background
129-
5. **Smart eventual consistency** — cache converges to optimal state while avoiding unnecessary operations
130+
5. **Smart eventual consistency** — cache converges to optimal state while avoiding unnecessary operations; opt-in hybrid or strong consistency via extension methods
130131

131132
## Materialization for Fast Access
132133

@@ -143,33 +144,65 @@ For detailed comparison and guidance, see `docs/storage-strategies.md`.
143144

144145
```csharp
145146
using SlidingWindowCache;
146-
using SlidingWindowCache.Configuration;
147+
using SlidingWindowCache.Public.Cache;
148+
using SlidingWindowCache.Public.Configuration;
147149
using Intervals.NET;
148150
using Intervals.NET.Domain.Default.Numeric;
149151

150-
var options = new WindowCacheOptions(
151-
leftCacheSize: 1.0, // Cache 100% of requested range size to the left
152-
rightCacheSize: 2.0, // Cache 200% of requested range size to the right
153-
leftThreshold: 0.2, // Rebalance if <20% left buffer remains
154-
rightThreshold: 0.2 // Rebalance if <20% right buffer remains
155-
);
156-
157-
var cache = WindowCache<int, string, IntegerFixedStepDomain>.Create(
158-
dataSource: myDataSource,
159-
domain: new IntegerFixedStepDomain(),
160-
options: options,
161-
readMode: UserCacheReadMode.Snapshot
162-
);
152+
await using var cache = WindowCacheBuilder.For(myDataSource, new IntegerFixedStepDomain())
153+
.WithOptions(o => o
154+
.WithCacheSize(left: 1.0, right: 2.0) // 100% left / 200% right of requested range
155+
.WithReadMode(UserCacheReadMode.Snapshot)
156+
.WithThresholds(0.2)) // rebalance if <20% buffer remains
157+
.Build();
163158

164159
var result = await cache.GetDataAsync(Range.Closed(100, 200), cancellationToken);
165160

166161
foreach (var item in result.Data.Span)
167162
Console.WriteLine(item);
168163
```
169164

165+
## Implementing a Data Source
166+
167+
Implement `IDataSource<TRange, TData>` to connect the cache to your backing store. The `FetchAsync` single-range overload is the only method you must provide; the batch overload has a default implementation that parallelizes single-range calls.
168+
169+
### FuncDataSource — inline without a class
170+
171+
`FuncDataSource<TRange, TData>` wraps an async delegate so you can create a data source in one expression:
172+
173+
```csharp
174+
using SlidingWindowCache.Public;
175+
using SlidingWindowCache.Public.Dto;
176+
177+
// Unbounded source — always returns data for any range
178+
IDataSource<int, string> source = new FuncDataSource<int, string>(
179+
async (range, ct) =>
180+
{
181+
var data = await myService.QueryAsync(range, ct);
182+
return new RangeChunk<int, string>(range, data);
183+
});
184+
```
185+
186+
For **bounded** sources (database with min/max IDs, time-series with temporal limits), return a `RangeChunk` with `Range = null` when no data is available — never throw:
187+
188+
```csharp
189+
IDataSource<int, Record> bounded = new FuncDataSource<int, Record>(
190+
async (range, ct) =>
191+
{
192+
var available = range.Intersect(Range.Closed(minId, maxId));
193+
if (available is null)
194+
return new RangeChunk<int, Record>(null, []);
195+
196+
var records = await db.FetchAsync(available, ct);
197+
return new RangeChunk<int, Record>(available, records);
198+
});
199+
```
200+
201+
For sources where a dedicated class is warranted (custom batch optimization, retry logic, dependency injection), implement `IDataSource<TRange, TData>` directly. See `docs/boundary-handling.md` for the full boundary contract.
202+
170203
## Boundary Handling
171204

172-
`GetDataAsync` returns `RangeResult<TRange, TData>` where `Range` may be `null` when the data source has no data for the requested range. Always check before accessing data:
205+
`GetDataAsync` returns `RangeResult<TRange, TData>` where `Range` may be `null` when the data source has no data for the requested range, and `CacheInteraction` indicates whether the request was a `FullHit`, `PartialHit`, or `FullMiss`. Always check `Range` before accessing data:
173206

174207
```csharp
175208
var result = await cache.GetDataAsync(Range.Closed(100, 200), ct);
@@ -267,6 +300,51 @@ var options = new WindowCacheOptions(
267300
);
268301
```
269302

303+
## Runtime Options Update
304+
305+
Cache sizing, threshold, and debounce options can be changed on a live cache instance without recreation. Updates take effect on the **next rebalance decision/execution cycle**.
306+
307+
```csharp
308+
// Change left and right cache sizes at runtime
309+
cache.UpdateRuntimeOptions(update =>
310+
update.WithLeftCacheSize(3.0)
311+
.WithRightCacheSize(3.0));
312+
313+
// Change debounce delay
314+
cache.UpdateRuntimeOptions(update =>
315+
update.WithDebounceDelay(TimeSpan.Zero));
316+
317+
// Change thresholds — or clear a threshold to null
318+
cache.UpdateRuntimeOptions(update =>
319+
update.WithLeftThreshold(0.15)
320+
.ClearRightThreshold());
321+
```
322+
323+
`UpdateRuntimeOptions` uses a **fluent builder** (`RuntimeOptionsUpdateBuilder`). Only fields explicitly set via builder calls are changed — all other options remain at their current values.
324+
325+
**Constraints:**
326+
- `ReadMode` and `RebalanceQueueCapacity` are creation-time only and cannot be changed at runtime.
327+
- All validation rules from construction still apply (`ArgumentOutOfRangeException` for negative sizes, `ArgumentException` for threshold sum > 1.0, etc.). A failed update leaves the current options unchanged — no partial application.
328+
- Calling `UpdateRuntimeOptions` on a disposed cache throws `ObjectDisposedException`.
329+
330+
**`LayeredWindowCache`** delegates `UpdateRuntimeOptions` to the outermost (user-facing) layer. To update a specific inner layer, use the `Layers` property (see Multi-Layer Cache below).
331+
332+
## Reading Current Runtime Options
333+
334+
Use `CurrentRuntimeOptions` to inspect the live option values on any cache instance. It returns a `RuntimeOptionsSnapshot` — a read-only point-in-time copy of the five runtime-updatable values.
335+
336+
```csharp
337+
var snapshot = cache.CurrentRuntimeOptions;
338+
Console.WriteLine($"Left: {snapshot.LeftCacheSize}, Right: {snapshot.RightCacheSize}");
339+
340+
// Useful for relative updates — double the current left size:
341+
var current = cache.CurrentRuntimeOptions;
342+
cache.UpdateRuntimeOptions(u => u.WithLeftCacheSize(current.LeftCacheSize * 2));
343+
```
344+
345+
The snapshot is immutable. Subsequent calls to `UpdateRuntimeOptions` do not affect previously obtained snapshots — obtain a new snapshot to see updated values.
346+
347+
- Calling `CurrentRuntimeOptions` on a disposed cache throws `ObjectDisposedException`.
270348
## Diagnostics
271349

272350
⚠️ **CRITICAL: You MUST handle `RebalanceExecutionFailed` in production.** Rebalance operations run in background tasks. Without handling this event, failures are silently swallowed and the cache stops rebalancing with no indication.
@@ -320,9 +398,50 @@ Canonical guide: `docs/diagnostics.md`.
320398
6. `docs/state-machine.md` — formal state transitions and mutation ownership
321399
7. `docs/actors.md` — actor responsibilities and execution contexts
322400

323-
## Strong Consistency Mode
401+
## Consistency Modes
324402

325-
By default, `GetDataAsync` is **eventually consistent**: data is returned immediately while the cache window converges asynchronously in the background. For scenarios where you need the cache to be fully converged before proceeding, use the `GetDataAndWaitForIdleAsync` extension method:
403+
By default, `GetDataAsync` is **eventually consistent**: data is returned immediately while the cache window converges asynchronously in the background. Two opt-in extension methods provide stronger consistency guarantees. Both require a `using SlidingWindowCache.Public;` import.
404+
405+
> **Serialized access requirement:** The hybrid and strong consistency modes provide their warm-cache guarantee only when requests are made one at a time (serialized). Under concurrent/parallel callers they remain safe (no crashes or hangs) but the guarantee degrades — due to `AsyncActivityCounter`'s "was idle at some point" semantics (Invariant H.49) and a brief gap between the counter increment and TCS publication in `IncrementActivity`, a concurrent waiter may observe a previously completed idle TCS and return without waiting for the new rebalance.
406+
407+
### Eventual Consistency (Default)
408+
409+
```csharp
410+
// Returns immediately; rebalance converges asynchronously in background
411+
var result = await cache.GetDataAsync(Range.Closed(100, 200), cancellationToken);
412+
```
413+
414+
Use for all hot paths and rapid sequential access. No latency beyond data assembly.
415+
416+
### Hybrid Consistency — `GetDataAndWaitOnMissAsync`
417+
418+
```csharp
419+
using SlidingWindowCache.Public;
420+
421+
// Waits for idle only if the request was a PartialHit or FullMiss; returns immediately on FullHit
422+
var result = await cache.GetDataAndWaitOnMissAsync(
423+
Range.Closed(100, 200),
424+
cancellationToken);
425+
426+
// result.CacheInteraction tells you which path was taken:
427+
// CacheInteraction.FullHit → returned immediately (no wait)
428+
// CacheInteraction.PartialHit → waited for cache to converge
429+
// CacheInteraction.FullMiss → waited for cache to converge
430+
if (result.Range.HasValue)
431+
ProcessData(result.Data);
432+
```
433+
434+
**When to use:**
435+
- Warm-cache fast path: pays no penalty on cache hits, still waits on misses
436+
- Access patterns where most requests are hits but you want convergence on misses
437+
438+
**When NOT to use:**
439+
- First request (always a miss — pays full debounce + I/O wait)
440+
- Hot paths with many misses
441+
442+
> **Cancellation:** If the cancellation token fires during the idle wait (after `GetDataAsync` has already returned data), the method catches `OperationCanceledException` and returns the already-obtained result gracefully — degrading to eventual consistency for that call. The background rebalance is not affected.
443+
444+
### Strong Consistency — `GetDataAndWaitForIdleAsync`
326445

327446
```csharp
328447
using SlidingWindowCache.Public;
@@ -347,17 +466,30 @@ This is a thin composition of `GetDataAsync` followed by `WaitForIdleAsync`. The
347466
**When NOT to use:**
348467
- Hot paths or rapid sequential requests — each call waits for full rebalance, which includes the debounce delay plus data fetching. For normal usage, the default eventual consistency model is faster.
349468

469+
> **Cancellation:** If the cancellation token fires during the idle wait (after `GetDataAsync` has already returned data), the method catches `OperationCanceledException` and returns the already-obtained result gracefully — degrading to eventual consistency for that call. The background rebalance is not affected.
470+
350471
### Deterministic Testing
351472

352473
`WaitForIdleAsync()` provides race-free synchronization with background operations for tests. Uses "was idle at some point" semantics — does not guarantee still idle after completion. See `docs/invariants.md` (Activity tracking invariants).
353474

475+
### CacheInteraction on RangeResult
476+
477+
Every `RangeResult` carries a `CacheInteraction` property classifying the request:
478+
479+
| Value | Meaning |
480+
|--------------|---------------------------------------------------------------------------------|
481+
| `FullHit` | Entire requested range was served from cache |
482+
| `PartialHit` | Request partially overlapped the cache; missing part fetched from `IDataSource` |
483+
| `FullMiss` | No overlap (cold start or jump); full range fetched from `IDataSource` |
484+
485+
This is the per-request programmatic alternative to the `UserRequestFullCacheHit` / `UserRequestPartialCacheHit` / `UserRequestFullCacheMiss` diagnostics callbacks.
486+
354487
## Multi-Layer Cache
355488

356489
For workloads with high-latency data sources, you can compose multiple `WindowCache` instances into a layered stack. Each layer uses the layer below it as its data source, allowing you to trade memory for reduced data-source I/O.
357490

358491
```csharp
359-
await using var cache = LayeredWindowCacheBuilder<int, byte[], IntegerFixedStepDomain>
360-
.Create(realDataSource, domain)
492+
await using var cache = WindowCacheBuilder.Layered(realDataSource, domain)
361493
.AddLayer(new WindowCacheOptions( // L2: deep background cache
362494
leftCacheSize: 10.0,
363495
rightCacheSize: 10.0,
@@ -375,6 +507,21 @@ var result = await cache.GetDataAsync(range, ct);
375507

376508
`LayeredWindowCache` implements `IWindowCache` and is `IAsyncDisposable` — it owns and disposes all layers when you dispose it.
377509

510+
**Accessing and updating individual layers:**
511+
512+
Use the `Layers` property to access any specific layer by index (0 = innermost, last = outermost). Each layer exposes the full `IWindowCache` interface:
513+
514+
```csharp
515+
// Update options on the innermost (deep background) layer
516+
layeredCache.Layers[0].UpdateRuntimeOptions(u => u.WithLeftCacheSize(8.0));
517+
518+
// Inspect the outermost (user-facing) layer's current options
519+
var outerOptions = layeredCache.Layers[^1].CurrentRuntimeOptions;
520+
521+
// cache.UpdateRuntimeOptions() is shorthand for Layers[^1].UpdateRuntimeOptions()
522+
layeredCache.UpdateRuntimeOptions(u => u.WithRightCacheSize(1.0));
523+
```
524+
378525
**Recommended layer configuration pattern:**
379526
- **Inner layers** (closest to the data source): `CopyOnRead`, large buffer sizes (5–10×), handles the heavy prefetching
380527
- **Outer (user-facing) layer**: `Snapshot`, small buffer sizes (0.3–1.0×), zero-allocation reads
@@ -393,8 +540,7 @@ var result = await cache.GetDataAsync(range, ct);
393540
394541
**Three-layer example:**
395542
```csharp
396-
await using var cache = LayeredWindowCacheBuilder<int, byte[], IntegerFixedStepDomain>
397-
.Create(realDataSource, domain)
543+
await using var cache = WindowCacheBuilder.Layered(realDataSource, domain)
398544
.AddLayer(l3Options) // L3: 10× CopyOnRead — network/disk absorber
399545
.AddLayer(l2Options) // L2: 2× CopyOnRead — mid-level buffer
400546
.AddLayer(l1Options) // L1: 0.5× Snapshot — user-facing

benchmarks/SlidingWindowCache.Benchmarks/Benchmarks/ExecutionStrategyBenchmarks.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
using Intervals.NET.Domain.Extensions.Fixed;
55
using SlidingWindowCache.Benchmarks.Infrastructure;
66
using SlidingWindowCache.Public;
7+
using SlidingWindowCache.Public.Cache;
78
using SlidingWindowCache.Public.Configuration;
89

910
namespace SlidingWindowCache.Benchmarks.Benchmarks;

benchmarks/SlidingWindowCache.Benchmarks/Benchmarks/RebalanceFlowBenchmarks.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
using Intervals.NET.Domain.Extensions.Fixed;
55
using SlidingWindowCache.Benchmarks.Infrastructure;
66
using SlidingWindowCache.Public;
7+
using SlidingWindowCache.Public.Cache;
78
using SlidingWindowCache.Public.Configuration;
89

910
namespace SlidingWindowCache.Benchmarks.Benchmarks;

benchmarks/SlidingWindowCache.Benchmarks/Benchmarks/ScenarioBenchmarks.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
using Intervals.NET.Domain.Default.Numeric;
44
using SlidingWindowCache.Benchmarks.Infrastructure;
55
using SlidingWindowCache.Public;
6+
using SlidingWindowCache.Public.Cache;
67
using SlidingWindowCache.Public.Configuration;
78

89
namespace SlidingWindowCache.Benchmarks.Benchmarks;

benchmarks/SlidingWindowCache.Benchmarks/Benchmarks/UserFlowBenchmarks.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
using Intervals.NET.Domain.Extensions.Fixed;
55
using SlidingWindowCache.Benchmarks.Infrastructure;
66
using SlidingWindowCache.Public;
7+
using SlidingWindowCache.Public.Cache;
78
using SlidingWindowCache.Public.Configuration;
89

910
namespace SlidingWindowCache.Benchmarks.Benchmarks;

docs/actors.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ Invariant ownership
5050
- 24f. Delivered data represents what user actually received
5151

5252
Components
53-
- `WindowCache<TRange, TData, TDomain>` (facade / composition root)
53+
- `WindowCache<TRange, TData, TDomain>` (facade / composition root; also owns `RuntimeCacheOptionsHolder` and exposes `UpdateRuntimeOptions`)
5454
- `UserRequestHandler<TRange, TData, TDomain>`
5555
- `CacheDataExtensionService<TRange, TData, TDomain>`
5656

@@ -77,8 +77,8 @@ Invariant ownership
7777
- 35. Threshold sum constraint (leftThreshold + rightThreshold ≤ 1.0)
7878

7979
Components
80-
- `ProportionalRangePlanner<TRange, TDomain>` — computes `DesiredCacheRange`
81-
- `NoRebalanceSatisfactionPolicy<TRange>` / `NoRebalanceRangePlanner<TRange, TDomain>` — computes `NoRebalanceRange`
80+
- `ProportionalRangePlanner<TRange, TDomain>` — computes `DesiredCacheRange`; reads configuration from `RuntimeCacheOptionsHolder` at invocation time
81+
- `NoRebalanceSatisfactionPolicy<TRange>` / `NoRebalanceRangePlanner<TRange, TDomain>` — computes `NoRebalanceRange`; reads configuration from `RuntimeCacheOptionsHolder` at invocation time
8282

8383
---
8484

0 commit comments

Comments
 (0)