A read-only, range-based, sequential-optimized cache with decision-driven background rebalancing, smart eventual consistency, and intelligent work avoidance.
Optimized for access patterns that move predictably across a domain (scrolling, playback, time-series inspection):
- Serves
GetDataAsyncimmediately; background work converges the cache window asynchronously - Single-writer architecture: only rebalance execution mutates shared cache state
- Decision-driven execution: multi-stage analytical validation prevents thrashing and unnecessary I/O
- Smart eventual consistency: cache converges to optimal configuration while avoiding unnecessary work
For the canonical architecture docs, see docs/architecture.md.
dotnet add package SlidingWindowCacheTraditional caches work with individual keys. A sliding window cache operates on continuous ranges of data:
- User requests a range (e.g., records 100–200)
- Cache fetches more than requested (e.g., records 50–300) based on left/right cache coefficients
- Subsequent requests within the window are served instantly from materialized data
- Window automatically rebalances when the user moves outside threshold boundaries
Requested Range (what user asks for):
[======== USER REQUEST ========]
Actual Cache Window (what cache stores):
[=== LEFT BUFFER ===][======== USER REQUEST ========][=== RIGHT BUFFER ===]
← leftCacheSize requestedRange size rightCacheSize →
Current Cache Window:
[========*===================== CACHE ======================*=======]
↑ ↑
Left Threshold (20%) Right Threshold (20%)
Scenario 1: Request within thresholds → No rebalance
[========*===================== CACHE ======================*=======]
[---- new request ----] ✓ Served from cache
Scenario 2: Request outside threshold → Rebalance triggered
[========*===================== CACHE ======================*=======]
[---- new request ----]
↓
🔄 Rebalance: Shift window right
Example: User requests range of size 100
leftCacheSize = 1.0, rightCacheSize = 2.0
[==== 100 ====][======= 100 =======][============ 200 ============]
Left Buffer Requested Range Right Buffer
Total Cache Window = 100 + 100 + 200 = 400 items
leftThreshold = 0.2 (20% of 400 = 80 items)
rightThreshold = 0.2 (20% of 400 = 80 items)
Key insight: Threshold percentages are calculated based on the total cache window size, not individual buffer sizes.
The cache uses a decision-driven model where rebalance necessity is determined by analytical validation, not by blindly executing every user request. This prevents thrashing, reduces unnecessary I/O, and maintains stability under rapid access pattern changes.
User Request
│
▼
┌─────────────────────────────────────────────────┐
│ User Path (User Thread — Synchronous) │
│ • Read from cache or fetch missing data │
│ • Return data immediately to user │
│ • Publish intent with delivered data │
└────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────┐
│ Decision Engine (Background Loop — CPU-only) │
│ Stage 1: Current NoRebalanceRange check │
│ Stage 2: Pending coverage check │
│ Stage 3: DesiredCacheRange computation │
│ Stage 4: Desired == Current check │
│ → Decision: SKIP or SCHEDULE │
└────────────┬────────────────────────────────────┘
│
├─── If SKIP: return (work avoidance) ✓
│
└─── If SCHEDULE:
│
▼
┌─────────────────────────────────────┐
│ Background Rebalance (ThreadPool) │
│ • Debounce delay │
│ • Fetch missing data (I/O) │
│ • Normalize cache to desired range │
│ • Update cache state atomically │
└─────────────────────────────────────┘
Key points:
- User requests never block — data returned immediately, rebalance happens later
- Decision happens in background — CPU-only validation (microseconds) in the intent processing loop
- Work avoidance prevents thrashing — validation may skip rebalance entirely if unnecessary
- Only I/O happens asynchronously — debounce + data fetching + cache updates run in background
- Smart eventual consistency — cache converges to optimal state while avoiding unnecessary operations
The cache always materializes data in memory. Two storage strategies are available:
| Strategy | Read | Write | Best For |
|---|---|---|---|
Snapshot (UserCacheReadMode.Snapshot) |
Zero-allocation (ReadOnlyMemory<TData> directly) |
Expensive (new array allocation) | Read-heavy workloads |
CopyOnRead (UserCacheReadMode.CopyOnRead) |
Allocates per read (copy) | Cheap (List<T> operations) |
Frequent rebalancing, memory-constrained |
For detailed comparison and guidance, see docs/storage-strategies.md.
using SlidingWindowCache;
using SlidingWindowCache.Configuration;
using Intervals.NET;
using Intervals.NET.Domain.Default.Numeric;
var options = new WindowCacheOptions(
leftCacheSize: 1.0, // Cache 100% of requested range size to the left
rightCacheSize: 2.0, // Cache 200% of requested range size to the right
leftThreshold: 0.2, // Rebalance if <20% left buffer remains
rightThreshold: 0.2 // Rebalance if <20% right buffer remains
);
var cache = WindowCache<int, string, IntegerFixedStepDomain>.Create(
dataSource: myDataSource,
domain: new IntegerFixedStepDomain(),
options: options,
readMode: UserCacheReadMode.Snapshot
);
var result = await cache.GetDataAsync(Range.Closed(100, 200), cancellationToken);
foreach (var item in result.Data.Span)
Console.WriteLine(item);GetDataAsync returns RangeResult<TRange, TData> where Range may be null when the data source has no data for the requested range. Always check before accessing data:
var result = await cache.GetDataAsync(Range.Closed(100, 200), ct);
if (result.Range != null)
{
// Data available
foreach (var item in result.Data.Span)
ProcessItem(item);
}
else
{
// No data available for this range
}Canonical guide: docs/boundary-handling.md.
WindowCache implements IAsyncDisposable. Always dispose when done:
// Recommended: await using
await using var cache = new WindowCache<int, string, IntegerFixedStepDomain>(
dataSource, domain, options, cacheDiagnostics
);
var data = await cache.GetDataAsync(Range.Closed(0, 100), ct);
// DisposeAsync called automatically at end of scopeAfter disposal, all operations throw ObjectDisposedException. Disposal is idempotent and concurrent-safe. Background operations are cancelled gracefully, not forcibly terminated.
leftCacheSize — multiplier of requested range size for left buffer. 1.0 = cache as much to the left as the user requested.
rightCacheSize — multiplier of requested range size for right buffer. 2.0 = cache twice as much to the right.
leftThreshold / rightThreshold — percentage of the total cache window size that triggers rebalancing when crossed. E.g., with a total window of 400 items and rightThreshold: 0.2, rebalance triggers when the request moves within 80 items of the right edge.
leftThreshold + rightThreshold must not exceed 1.0 when both are specified. Exceeding this creates overlapping stability zones (impossible geometry). Examples:
- ✅
leftThreshold: 0.3, rightThreshold: 0.3(sum = 0.6) - ✅
leftThreshold: 0.5, rightThreshold: 0.5(sum = 1.0) - ❌
leftThreshold: 0.6, rightThreshold: 0.6(sum = 1.2 — throwsArgumentException)
debounceDelay (default: 100ms) — delay before background rebalance executes. Prevents thrashing when the user rapidly changes access patterns.
rebalanceQueueCapacity (default: null) — controls rebalance serialization strategy:
| Value | Strategy | Backpressure | Use Case |
|---|---|---|---|
null (default) |
Task-based (lock-free task chaining) | None | Recommended for 99% of scenarios |
>= 1 |
Channel-based (bounded queue) | Blocks when full | Extreme high-frequency with I/O latency |
Forward-heavy scrolling:
var options = new WindowCacheOptions(
leftCacheSize: 0.5,
rightCacheSize: 3.0,
leftThreshold: 0.25,
rightThreshold: 0.15
);Bidirectional navigation:
var options = new WindowCacheOptions(
leftCacheSize: 1.5,
rightCacheSize: 1.5,
leftThreshold: 0.2,
rightThreshold: 0.2
);High-latency data source with stability:
var options = new WindowCacheOptions(
leftCacheSize: 2.0,
rightCacheSize: 3.0,
leftThreshold: 0.1,
rightThreshold: 0.1,
debounceDelay: TimeSpan.FromMilliseconds(150)
);RebalanceExecutionFailed in production. Rebalance operations run in background tasks. Without handling this event, failures are silently swallowed and the cache stops rebalancing with no indication.
public class LoggingCacheDiagnostics : ICacheDiagnostics
{
private readonly ILogger _logger;
public LoggingCacheDiagnostics(ILogger logger) => _logger = logger;
public void RebalanceExecutionFailed(Exception ex)
{
// CRITICAL: always log rebalance failures
_logger.LogError(ex, "Cache rebalance failed. Cache may not be optimally sized.");
}
// Other methods can be no-op if you only care about failures
}If no diagnostics instance is provided, the cache uses NoOpDiagnostics — zero overhead, JIT-optimized away completely.
Canonical guide: docs/diagnostics.md.
- Snapshot mode: O(1) reads, O(n) rebalance (array allocation)
- CopyOnRead mode: O(n) reads (copy cost), cheaper rebalance operations
- Rebalancing is asynchronous — does not block user reads
- Debouncing: multiple rapid requests trigger only one rebalance operation
- Diagnostics overhead: zero when not used (NoOpDiagnostics); ~1–5 ns per event when enabled
README.md— you are heredocs/boundary-handling.md— RangeResult usage, bounded data sourcesdocs/storage-strategies.md— choose Snapshot vs CopyOnRead for your use casedocs/glossary.md— canonical term definitions and common misconceptionsdocs/diagnostics.md— optional instrumentation
docs/glossary.md— start here for canonical terminologydocs/architecture.md— single-writer, decision-driven execution, disposaldocs/invariants.md— formal system invariantsdocs/components/overview.md— component catalog with invariant implementation mappingdocs/scenarios.md— temporal behavior walkthroughsdocs/state-machine.md— formal state transitions and mutation ownershipdocs/actors.md— actor responsibilities and execution contexts
By default, GetDataAsync is eventually consistent: data is returned immediately while the cache window converges asynchronously in the background. For scenarios where you need the cache to be fully converged before proceeding, use the GetDataAndWaitForIdleAsync extension method:
using SlidingWindowCache.Public;
// Returns only after cache has converged to its desired window geometry
var result = await cache.GetDataAndWaitForIdleAsync(
Range.Closed(100, 200),
cancellationToken);
// Cache geometry is now stable — safe to inspect, assert, or rely on
if (result.Range.HasValue)
ProcessData(result.Data);This is a thin composition of GetDataAsync followed by WaitForIdleAsync. The returned RangeResult is identical to what GetDataAsync would return.
When to use:
- Cold start synchronization: waiting for the initial cache window to be built before proceeding
- Integration testing: asserting on cache geometry after a request
- Any scenario where you want to know the cache has finished rebalancing before moving on
When NOT to use:
- Hot paths or rapid sequential requests — each call waits for full rebalance, which includes the debounce delay plus data fetching. For normal usage, the default eventual consistency model is faster.
WaitForIdleAsync() provides race-free synchronization with background operations for tests. Uses "was idle at some point" semantics — does not guarantee still idle after completion. See docs/invariants.md (Activity tracking invariants).
For workloads with high-latency data sources, you can compose multiple WindowCache instances into a layered stack. Each layer uses the layer below it as its data source, allowing you to trade memory for reduced data-source I/O.
await using var cache = LayeredWindowCacheBuilder<int, byte[], IntegerFixedStepDomain>
.Create(realDataSource, domain)
.AddLayer(new WindowCacheOptions( // L2: deep background cache
leftCacheSize: 10.0,
rightCacheSize: 10.0,
readMode: UserCacheReadMode.CopyOnRead,
leftThreshold: 0.3,
rightThreshold: 0.3))
.AddLayer(new WindowCacheOptions( // L1: user-facing cache
leftCacheSize: 0.5,
rightCacheSize: 0.5,
readMode: UserCacheReadMode.Snapshot))
.Build();
var result = await cache.GetDataAsync(range, ct);LayeredWindowCache implements IWindowCache and is IAsyncDisposable — it owns and disposes all layers when you dispose it.
Recommended layer configuration pattern:
- Inner layers (closest to the data source):
CopyOnRead, large buffer sizes (5–10×), handles the heavy prefetching - Outer (user-facing) layer:
Snapshot, small buffer sizes (0.3–1.0×), zero-allocation reads
Important — buffer ratio requirement: Inner layer buffers must be substantially larger than outer layer buffers, not merely slightly larger. When the outer layer rebalances, it fetches missing ranges from the inner layer via
GetDataAsync. Each fetch publishes a rebalance intent on the inner layer. If the inner layer'sNoRebalanceRangeis not wide enough to contain the outer layer's fullDesiredCacheRange, the inner layer will also rebalance — and re-center toward only one side of the outer layer's gap, leaving it poorly positioned for the next rebalance. With undersized inner buffers this becomes a continuous cycle (cascading rebalance thrashing). Use a 5–10× ratio andleftThreshold/rightThresholdof 0.2–0.3 on inner layers to ensure the inner layer's stability zone absorbs the outer layer's rebalance fetches. Seedocs/architecture.md(Cascading Rebalance Behavior) anddocs/scenarios.md(Scenarios L6 and L7) for the full explanation.
Three-layer example:
await using var cache = LayeredWindowCacheBuilder<int, byte[], IntegerFixedStepDomain>
.Create(realDataSource, domain)
.AddLayer(l3Options) // L3: 10× CopyOnRead — network/disk absorber
.AddLayer(l2Options) // L2: 2× CopyOnRead — mid-level buffer
.AddLayer(l1Options) // L1: 0.5× Snapshot — user-facing
.Build();For detailed guidance see docs/storage-strategies.md.
MIT