Skip to content

developer-guide.md

Latest commit

 

History

History
573 lines (426 loc) · 19.1 KB

developer-guide.md

File metadata and controls

573 lines (426 loc) · 19.1 KB

Developer Guide

This guide provides comprehensive documentation for developers working with or contributing to the @ivotoby/openapi-mcp-server codebase. It covers key concepts, internal architecture, and development workflows.

Table of Contents

Architecture Overview

The MCP OpenAPI Server consists of several key components:

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   OpenAPIServer │    │   ToolsManager  │    │ OpenAPISpecLoader│
│                 │    │                 │    │                 │
│ - Server setup  │───▶│ - Tool filtering│───▶│ - Spec parsing  │
│ - Transport mgmt│    │ - Tool lookup   │    │ - Tool creation │
│ - Request routing│   │ - Tool metadata │    │ - Schema processing│
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│    ApiClient    │    │   AuthProvider  │    │   Tool ID Utils │
│                 │    │                 │    │                 │
│ - HTTP requests │    │ - Dynamic auth  │    │ - ID generation │
│ - Parameter     │    │ - Token refresh │    │ - ID parsing    │
│   handling      │    │ - Error recovery│    │ - Hyphen escaping│
└─────────────────┘    └─────────────────┘    └─────────────────┘

Core Concepts

ExtendedTool Interface

The server extends the standard MCP Tool interface with metadata for efficient filtering:

interface ExtendedTool extends Tool {
  /** OpenAPI tags associated with this tool's operation */
  tags?: string[]
  /** HTTP method for this tool (GET, POST, etc.) */
  httpMethod?: string
  /** Primary resource name extracted from the path */
  resourceName?: string
  /** Original OpenAPI path before toolId conversion */
  originalPath?: string
}

This metadata is computed during tool creation and enables efficient filtering without re-parsing tool IDs or accessing the raw OpenAPI specification.

Tools Loading Modes

The server supports three distinct tools loading modes:

  1. "all" (default): Load all tools from the OpenAPI spec, applying any specified filters
  2. "dynamic": Load only meta-tools for API exploration (list-api-endpoints, get-api-endpoint-schema, invoke-api-endpoint)
  3. "explicit": Load only tools explicitly listed in includeTools, ignoring all other filters

Tool ID System

Overview

Tool IDs uniquely identify API endpoints and have the format: METHOD::pathPart

Examples:

  • GET::users → GET /users
  • POST::api__v1__users → POST /api/v1/users
  • GET::api__resource-name__items → GET /api/resource-name/items

Path Separation Scheme

Critical for developers: The tool ID system uses double underscores (__) as a separator for path segments. This approach is robust and avoids the complexities of hyphen-escaping schemes.

The Problem (Simplified)

OpenAPI paths (e.g., /api/v1/users, /api/resource-name/items) need to be converted into a flat string format for tool IDs. A clear separator is needed to distinguish between different segments of the original path. Legitimate hyphens within path segments (e.g., resource-name) must be preserved.

The Solution

  • Double underscores (__) are used to replace slashes (/) from the original path.
  • Legitimate hyphens within path segments are preserved as-is.

Examples

Original Path Tool ID Parsed Back
/users GET::users /users
/api/v1/users GET::api__v1__users /api/v1/users
/api/resource-name/items GET::api__resource-name__items /api/resource-name/items
/user-profile/data GET::user-profile__data /user-profile/data
/a_b/c-d/e_f-g GET::a_b__c-d__e_f-g /a_b/c-d/e_f-g

Implementation Details

Generation (generateToolId):

const cleanPath = path
  .replace(/^\//, "") // Remove leading slash
  .replace(/\/+/g, "/") // Collapse multiple consecutive slashes to single slash
  .replace(/\{([^}]+)\}/g, "$1") // Remove curly braces from path params
  .replace(/\//g, "__") // Convert slashes to double underscores

const sanitizedPath = sanitizeForToolId(cleanPath) // Apply further sanitization

return `${method.toUpperCase()}::${sanitizedPath}`

Parsing (parseToolId):

const [method, pathPart] = toolId.split("::", 2)
// Simply replace double underscores with slashes
const path = pathPart.replace(/__/g, "/")
return { method, path: "/" + path }

Character Sanitization

Tool IDs are sanitized by the sanitizeForToolId helper function to ensure they contain only safe characters [A-Za-z0-9_-]. The process involves:

  • Removing disallowed characters: Any character not in A-Za-z0-9_- is removed.
  • Collapsing underscores: Sequences of three or more underscores (___, ____, etc.) are collapsed to a double underscore (__). This preserves the __ path separator if an original path segment happened to contain multiple underscores that were then joined by __.
  • Trimming: Leading or trailing underscores (_) and hyphens (-) are removed from the final sanitized path part.
  • Original path structure: Note that operations like collapsing multiple slashes (// to /) in the original path happen before sanitization during the generateToolId's path cleaning phase.

Known Limitations

(This section can be removed as the previous limitations were specific to the hyphen-escaping scheme. The double underscore system is much simpler and avoids those issues. If new limitations are identified, they can be added here.)

Tool Name Abbreviation System

Overview

Tool names are generated from OpenAPI operationId, summary, or fallback patterns and must be ≤64 characters with format [a-z0-9-]+.

Abbreviation Process

The abbreviation system follows a multi-step process:

  1. Initial Sanitization: Replace non-alphanumeric characters with hyphens
  2. Word Splitting: Split by underscores, camelCase, and numbers
  3. Common Word Removal: Remove words like "controller", "api", "service"
  4. Standard Abbreviations: Apply predefined abbreviations
  5. Vowel Removal: For long words (>5 chars) that aren't abbreviations
  6. Truncation & Hashing: Add hash suffix if original was long or result exceeds limit

Common Words Removed

const REVISED_COMMON_WORDS_TO_REMOVE = [
  "controller",
  "api",
  "operation",
  "handler",
  "endpoint",
  "action",
  "perform",
  "execute",
  "retrieve",
  "specify",
  "for",
  "and",
  "the",
  "with",
  "from",
  "into",
  "onto",
  "out",
]

Standard Abbreviations

const WORD_ABBREVIATIONS = {
  service: "Svc",
  user: "Usr",
  management: "Mgmt",
  authority: "Auth",
  group: "Grp",
  update: "Upd",
  delete: "Del",
  create: "Crt",
  configuration: "Config",
  resource: "Res",
  authentication: "Authn", // ... and more
}

Examples

Original Process Result
getUserDetails get-user-details get-user-details
ServiceUsersManagementController_updateServiceUsersAuthorityGroup Split → Remove common → Abbreviate → Hash svc-usrs-mgmt-upd-svc-usrs-auth-grp-a1b2
UpdateUserConfigurationManagement Split → Abbreviate upd-usr-config-mgmt

Disabling Abbreviation

Set disableAbbreviation: true to disable the abbreviation system:

  • No common word removal
  • No standard abbreviations
  • No vowel removal
  • No length limits (may cause errors if names exceed 64 characters)

Resource Name Extraction

Algorithm

Resource names are extracted from OpenAPI paths for filtering purposes:

private extractResourceName(path: string): string | undefined {
  const segments = path.replace(/^\//, "").split("/")

  // Find the last non-parameter segment
  for (let i = segments.length - 1; i >= 0; i--) {
    const segment = segments[i]
    if (!segment.includes("{") && !segment.includes("}") && segment.length > 0) {
      return segment
    }
  }

  return segments[0] || undefined
}

Examples

Path Resource Name
/users users
/users/{id} users
/api/v1/users/{id}/posts posts
/api/v1/user-profile-settings settings
/health health

Filtering System

Filter Application Order

Filters are applied in a specific order with different precedence:

  1. includeTools (highest priority): If specified, overrides all other filters
  2. includeOperations: Filter by HTTP methods (AND operation with remaining filters)
  3. includeResources: Filter by resource names (AND operation)
  4. includeTags: Filter by OpenAPI tags (AND operation)

Filter Modes

All Mode (default)

toolsMode: "all"
// Apply filters as AND operations
// Empty filter arrays = no filtering for that dimension

Explicit Mode

toolsMode: "explicit"
includeTools: ["GET::users", "POST::users"]
// ONLY load explicitly listed tools
// Ignore all other filters

Dynamic Mode

toolsMode: "dynamic"
// Load only meta-tools:
// - list-api-endpoints
// - get-api-endpoint-schema
// - invoke-api-endpoint

Case Sensitivity

All filtering is case-insensitive:

  • Tool IDs: GET::Users matches filter get::users
  • Tool names: getUsers matches filter getusers
  • Resource names: Users matches filter users
  • Tags: ADMIN matches filter admin
  • HTTP methods: GET matches filter get

Authentication System

AuthProvider Interface

interface AuthProvider {
  /**
   * Get authentication headers for the current request
   * Called before each API request to get fresh headers
   */
  getAuthHeaders(): Promise<Record<string, string>>

  /**
   * Handle authentication errors from API responses
   * Called when the API returns 401 or 403 errors
   * Return true to retry the request, false otherwise
   */
  handleAuthError(error: AxiosError): Promise<boolean>
}

Authentication Flow

  1. Before each request: getAuthHeaders() is called
  2. On auth errors (401/403): handleAuthError() is called
  3. If handleAuthError() returns true: Request is retried once with fresh headers
  4. If handleAuthError() returns false: Error is propagated to user

Static vs Dynamic Authentication

Static Authentication (backward compatible):

const config = {
  headers: { Authorization: "Bearer token" },
}
// Internally creates StaticAuthProvider

Dynamic Authentication:

const config = {
  authProvider: new MyAuthProvider(),
  // No headers property when using AuthProvider
}

OpenAPI Processing

Reference Resolution

The server resolves OpenAPI $ref references:

  • Parameter references: $ref: "#/components/parameters/MyParam"
  • Schema references: $ref: "#/components/schemas/MySchema"
  • Recursive references: Circular reference detection prevents infinite loops
  • External references: Gracefully handled (returns empty schema)

Schema Composition

Supports OpenAPI schema composition keywords:

  • allOf: Schemas are merged into a single object
  • oneOf/anyOf: Composition is preserved in the input schema
  • not: Preserved as-is in the input schema

Parameter Inheritance

Path-level parameters are inherited by operations:

  • Path parameters are added to all operations in the path
  • Operation parameters can override path parameters (same name + location)
  • Merging logic combines both sets without duplication

Input Schema Generation

The server creates unified input schemas by merging:

  1. Path parameters (from URL path)
  2. Query parameters (from URL query string)
  3. Header parameters (with x-parameter-location: "header")
  4. Cookie parameters (with x-parameter-location: "cookie")
  5. Request body (flattened into schema or wrapped in body property)

Content Type Handling

For request bodies with multiple content types:

  • Priority: application/json > application/x-www-form-urlencoded > multipart/form-data > others
  • File uploads: multipart/form-data with type: string, format: binary

Development Workflow

Setup

git clone <repository>
cd mcp-openapi-server
npm install

Development Commands

npm run dev              # Watch mode with auto-rebuild
npm run inspect-watch    # Debug mode with auto-reload
npm run build           # Build TypeScript
npm run typecheck       # Type checking only
npm run lint            # ESLint
npm run test            # Run tests
npm run test:watch      # Watch mode tests
npm run clean           # Remove build artifacts

Project Structure

src/
├── config.ts           # Configuration loading and validation
├── server.ts           # Main OpenAPIServer class
├── tools-manager.ts    # Tool filtering and management
├── openapi-loader.ts   # OpenAPI spec parsing and tool creation
├── api-client.ts       # HTTP client for API requests
├── auth-provider.ts    # Authentication interfaces and implementations
├── transport-http.ts   # HTTP transport implementation
└── utils/
    ├── tool-id.ts      # Tool ID generation and parsing
    └── abbreviations.ts # Name abbreviation rules

test/
├── *.test.ts          # Unit tests for each module
└── fixtures/          # Test data and mock OpenAPI specs

docs/
├── developer-guide.md  # This document
├── auth-provider-guide.md # AuthProvider documentation
└── plans/             # Development plans and improvements

examples/
├── basic-library-usage/    # Simple library usage example
├── auth-provider-example/  # AuthProvider implementations
└── beatport-example/       # Real-world production example

Testing Guidelines

Test Organization

Tests are organized by module with comprehensive coverage:

  • Unit tests: Test individual functions and classes
  • Integration tests: Test component interactions
  • Edge case tests: Test error conditions and boundary cases
  • Regression tests: Prevent known issues from reoccurring

Key Testing Areas

  1. Tool ID System: Round-trip consistency, hyphen escaping, edge cases
  2. Abbreviation System: All processing steps, edge cases, hash generation
  3. Filtering Logic: All filter combinations, precedence, case sensitivity
  4. OpenAPI Processing: Reference resolution, schema composition, parameter inheritance
  5. Authentication: Static and dynamic auth, error handling, retry logic

Running Tests

npm test                    # All tests
npm test -- --watch        # Watch mode
npm test tool-id-utils     # Specific test file
npm test -- --coverage     # Coverage report

Test Patterns

Parameterized tests for comprehensive coverage:

const testCases = [
  { input: "GET::users", expected: { method: "GET", path: "/users" } },
  { input: "POST::api-v1-users", expected: { method: "POST", path: "/api/v1/users" } },
]

for (const { input, expected } of testCases) {
  const result = parseToolId(input)
  expect(result).toEqual(expected)
}

Mock management for isolated testing:

const mockSpecLoader = {
  loadOpenAPISpec: vi.fn(),
  parseOpenAPISpec: vi.fn(),
}

Contributing

Code Style

  • TypeScript: Strict mode enabled
  • Formatting: Prettier with project configuration
  • Linting: ESLint with TypeScript rules
  • Naming: camelCase for variables/functions, PascalCase for classes, kebab-case for files

Documentation Standards

Follow Google's Technical Writing Style Guide:

  • Use active voice and present tense
  • Write clear, concise sentences
  • Define terminology when needed
  • Use lists and tables for complex information
  • Include examples for all concepts

JSDoc Requirements

All code must have comprehensive JSDoc documentation:

/**
 * Parse a tool ID into HTTP method and path
 *
 * Tool IDs have the format: METHOD::pathPart where pathPart has slashes
 * converted to hyphens and legitimate hyphens escaped as double hyphens.
 *
 * @param toolId - Tool ID in format METHOD::pathPart
 * @returns Object containing method and path
 *
 * @example
 * parseToolId("GET::users") → { method: "GET", path: "/users" }
 * parseToolId("GET::api__resource-name__items") → { method: "GET", path: "/api/resource-name/items" }
 */

Pull Request Process

  1. Fork the repository
  2. Create a feature branch from main
  3. Implement changes with tests
  4. Run npm run typecheck && npm run lint && npm test
  5. Update documentation if needed
  6. Submit pull request with clear description

Commit Message Format

Follow conventional commit format:

feat: add support for OpenAPI 3.1 specifications

- Implement OpenAPI 3.1 parser compatibility
- Add tests for new specification features
- Update documentation with 3.1 examples

Closes #123

Adding New Features

When adding new features:

  1. Design: Consider backward compatibility
  2. Test: Add comprehensive test coverage
  3. Document: Update relevant documentation
  4. Examples: Add usage examples if applicable
  5. Performance: Consider impact on existing functionality

This developer guide should be updated as the codebase evolves to ensure it remains accurate and comprehensive.