Skip to content

availability.md

Latest commit

 

History

History
162 lines (123 loc) · 5.56 KB

availability.md

File metadata and controls

162 lines (123 loc) · 5.56 KB

Availability Methods

Availability methods determine how the LoadBalancer selects endpoints and handles failover.

fail-forward (Default)

Always tries endpoints in order. Fails over to the next endpoint only when the response status matches configured failure codes.

const lb = new LoadBalancer({
  endpoints: [
    new Endpoint("https://primary.example.com"),
    new Endpoint("https://secondary.example.com"),
    new Endpoint("https://tertiary.example.com"),
  ],
  availability: {
    type: "fail-forward",
    options: {
      failoverOnStatuses: [502, 503, 504], // these are the default values
    },
  },
});

Behavior:

  • Tries the first endpoint
  • If the response status is in failoverOnStatuses, tries the next endpoint
  • Non-matching status codes (e.g., 403, 404, 500) are returned as-is
  • Network errors trigger failover to the next endpoint

Options:

Option Type Default Description
failoverOnStatuses number[] [502, 503, 504] HTTP status codes that trigger failover

async-block

Sequentially checks each endpoint's health before selecting one. Returns the first healthy endpoint.

const lb = new LoadBalancer({
  endpoints: [
    new Endpoint("https://api1.example.com", {
      healthCheckPathname: "/health",
    }),
    new Endpoint("https://api2.example.com", {
      healthCheckPathname: "/health",
    }),
  ],
  availability: { type: "async-block" },
});

Behavior:

  • Checks endpoints one by one in order
  • Returns the first endpoint that responds with 2xx on its health check
  • Skips endpoints that fail health checks or have network errors
  • Throws "No available endpoints" if all health checks fail

Requirements:

  • Each endpoint must have healthCheckPathname configured

promise.any

Checks all endpoints' health in parallel. Returns the first endpoint that responds successfully.

const lb = new LoadBalancer({
  endpoints: [
    new Endpoint("https://api1.example.com", {
      healthCheckPathname: "/health",
    }),
    new Endpoint("https://api2.example.com", {
      healthCheckPathname: "/health",
    }),
  ],
  availability: { type: "promise.any" },
});

Behavior:

  • Fires health checks to all endpoints simultaneously
  • Returns the first endpoint to respond with 2xx
  • Faster than async-block when you have multiple endpoints
  • Throws "No available endpoints" if all health checks fail

Requirements:

  • Each endpoint must have healthCheckPathname configured

Comparison

Method Health Check Selection Strategy Best For
fail-forward Skips Entirely Sequential, lazy Primary/backup setups
async-block Before request Sequential, eager Ordered preference with health validation
promise.any Before request Parallel, fastest Lowest latency endpoint selection

Response Headers

The LoadBalancer adds headers to successful responses to help with debugging and monitoring:

Header Description Always Present
X-Load-Balancer-Endpoint URL of the endpoint that handled the request Yes
X-Load-Balancer-Latency Total time in ms from request start to response Yes
X-Load-Balancer-Endpoint-Gather-Latency Time in ms to select the endpoint Yes
X-Load-Balancer-Tried-Count Number of endpoints tried before success Only on failover
X-Load-Balancer-Tried-Endpoints Comma-separated list of endpoint URLs tried Only on failover

Example response headers (after failover):

X-Load-Balancer-Endpoint: https://api3.example.com
X-Load-Balancer-Latency: 150
X-Load-Balancer-Endpoint-Gather-Latency: 5
X-Load-Balancer-Tried-Count: 3
X-Load-Balancer-Tried-Endpoints: https://api1.example.com, https://api2.example.com, https://api3.example.com

The Tried-Count and Tried-Endpoints headers are only present when failover occurred (i.e., more than one endpoint was tried).

Recovery Function

All availability methods support a recovery function that runs when all endpoints fail:

const lb = new LoadBalancer({
  endpoints: [...],
  availability: { type: "fail-forward" },
  recoveryFn: async (request, context) => {
    // Log the failure with tried endpoints
    console.error("All endpoints failed for:", request.url);
    console.error("Tried endpoints:", context.triedEndpoints.map(e => e.url));

    // Optionally dump to R2 for replay later
    await env.BUCKET.put(`failed/${Date.now()}`, request.body);

    // Return undefined to throw the default error,
    // or return a Response to handle gracefully
    return undefined;
  },
});

The recovery function receives:

  • request - The original request
  • context - An object containing:
    • triedEndpoints - Array of Endpoint objects that were tried before failing

The recovery function can:

  • Log failures to external services
  • Store failed requests for later replay
  • Return undefined to throw the default "No available endpoints" error
  • Return a Response to handle the failure gracefully