Multi Repository Wiki Support

Multi-Repository Wiki Support

Wikigen enables grouping multiple repositories into a single unified wiki. This feature allows teams to generate cross-repository documentation that spans multiple services, libraries, or components, showing how they interact and fit into the larger system architecture.

Overview

Multi-repository wiki support addresses the need for integrated documentation across loosely-coupled services. Rather than generating separate wikis for each repository, wikigen can combine documentation from multiple repositories into a single, cohesive wiki where shared concepts, architecture, and inter-service interactions are documented holistically.

Multi-repo wikis are specified in repos.txt using a project grouping syntax:

myproject:owner/frontend-repo
myproject:owner/backend-repo
myproject:owner/shared-library

All repositories in the same group are documented within a single wiki output directory, with cross-repository architecture pages and unified navigation. Standalone repositories continue to work as before when no project prefix is used.

Related pages: System Overview | Architecture & Design | Input Formats & Repository Configuration | CLI Usage & Commands

Multi-Repository Specification Format

Syntax

Wikigen supports three input formats for repositories:

# Standalone wiki (one wiki per repository)
owner/repo1
owner/repo2

# Multi-repository wiki (multiple repositories grouped into one)
projectname:owner/frontend
projectname:owner/backend
projectname:owner/shared-lib

# Local directories (standalone or grouped)
/path/to/local/repo
projectname:/path/to/local/component

The project prefix before the colon groups repositories together. All repositories with the same project prefix are processed as a single wiki project. Sources: main.go:204-225

Parsing Logic

The parseRepoList function distinguishes between standalone and grouped repositories:

• Standalone format: owner/repo or local path
• Grouped format: prefix:owner/repo or prefix:/local/path
• The colon separates the project group name from the repository identifier
• The project name cannot contain / (to avoid ambiguity with file paths)

func parseRepoList(lines []string) (standalone []RepoEntry, groups map[string][]string) {
	groups = make(map[string][]string)
	for _, line := range lines {
		line = strings.TrimSpace(line)
		if line == "" || strings.HasPrefix(line, "#") {
			continue
		}
		// Check for project group prefix (project:something)
		if before, after, ok := strings.Cut(line, ":"); ok && !strings.Contains(before, "/") && !isLocalPath(before) {
			groups[before] = append(groups[before], after)
		} else if isLocalPath(line) {
			// standalone local directory
			name := filepath.Base(line)
			standalone = append(standalone, RepoEntry{Repo: name, LocalDir: line})
		} else {
			// standalone owner/repo format
			standalone = append(standalone, RepoEntry{Repo: line})
		}
	}
	return
}

Sources: main.go:703-725

Processing Pipeline for Multi-Repository Projects

When wikigen processes a multi-repository project, it follows a unified pipeline that treats all repositories as a cohesive unit:

flowchart TD
    A["Input: repos.txt<br/>multiproject:repo1<br/>multiproject:repo2"] --> B["Parse grouped repositories"]
    B --> C["Create single task<br/>name=multiproject<br/>repos=[repo1,repo2]"]
    C --> D["Clone/fetch all repos"]
    D --> E["Create output directory<br/>wiki-output/multiproject/"]
    E --> F["Phase 1: Structure<br/>Claude Code analyzes<br/>ALL repositories"]
    F --> G["Generate unified<br/>wiki structure"]
    G --> H["Write Home.md<br/>_Sidebar.md"]
    H --> I["Phase 2: Content<br/>Generate pages in parallel<br/>with all repo directories"]
    I --> J["Complete multi-repo wiki"]

Loading

Key Differences from Standalone Processing

1. Repository Directory Passing: All repositories in a group are passed to Claude Code via multiple --add-dir flags, allowing Claude to analyze cross-repository relationships.

2. Unified Output Directory: A single output directory is created for the project group (e.g., wiki-output/myproject/), not separate directories for each repository.

3. Structure Determination: Claude Code analyzes all repositories together when determining wiki structure, enabling it to create cross-repository architecture pages.

4. Page Generation: When generating each page, Claude Code has access to all repositories in the group, enabling references to multiple code sources.

Sources: main.go:495-577

Data Structures for Multi-Repository Support

RepoEntry Structure

The RepoEntry type represents each repository in the input list:

type RepoEntry struct {
	Project  string // group name (empty = standalone)
	Repo     string // owner/repo or display name for local
	LocalDir string // local directory path (empty = remote repo)
}

The Project field is populated only for grouped repositories. For standalone repositories, Project remains empty. Sources: main.go:72-76

Task Organization

Internally, wikigen creates a single task for each project group:

type task struct {
	name  string   // project group name
	repos []string // list of repository references (owner/repo or paths)
}

All repositories in the group are added to the repos slice of a single task. The task is then processed by the generateWiki function with all repository directories. Sources: main.go:1000-1030

Claude Code Integration for Multi-Repositories

When wikigen invokes Claude Code for a multi-repository project, it passes all repository directories simultaneously:

func claudeCall(claudePath, model string, repoDirs []string, systemPrompt, prompt, workDir string) (string, error) {
	args := []string{"-p", "--output-format", "text", "--dangerously-skip-permissions"}
	if model != "" {
		args = append(args, "--model", model)
	}
	for _, dir := range repoDirs {
		args = append(args, "--add-dir", dir)  // Multiple directories
	}
	if systemPrompt != "" {
		args = append(args, "--system-prompt", systemPrompt)
	}
	// ... command execution
}

Example invocation for a three-repository project:

claude -p --add-dir /path/to/frontend --add-dir /path/to/backend --add-dir /path/to/shared

This allows Claude Code to access the source code of all repositories simultaneously, enabling it to understand cross-service communication patterns, shared data models, and architectural relationships. Sources: main.go:137-164

Structure Determination for Multi-Repository Projects

Multi-Repository Structure Prompt

The structure determination prompt includes special instructions for multi-repository projects:

## Rules for Multi-Repository Projects
- When multiple repositories form one project, create CROSS-REPOSITORY documentation
- Show how repositories interact with each other (e.g., frontend calls backend API)
- Create architecture pages that span all repositories
- Individual repository details should still get their own focused pages
- Clearly indicate which repository each page primarily covers

The prompt explicitly instructs Claude Code to:

1. Analyze all repositories together
2. Create pages about inter-repository interactions
3. Document shared patterns and dependencies
4. Design a unified architecture view

Sources: main.go:246-251

Typical Multi-Repository Wiki Structure

A multi-repository wiki typically includes:

1. System Architecture — Overall architecture spanning all services
2. API Contracts — How services communicate
3. Data Models — Shared schemas across repositories
4. Deployment & Infrastructure — How services are deployed together
5. Inter-Service Communication — Service-to-service interaction patterns
6. Repository Overview — Individual repository documentation
7. Integration Guide — How to develop across multiple repositories

Example structure for a frontend-backend-shared library project:

Home.md                           # Project overview, lists repositories
System-Architecture.md            # Cross-service architecture
API-Specification.md              # Backend API spec (primary: backend repo)
Frontend-Architecture.md          # Frontend design (primary: frontend repo)
Data-Models.md                    # Shared data structures
Frontend-Backend-Integration.md   # How frontend calls backend
Testing-Strategy.md               # Testing across all three repos
Build-Deployment.md               # Deployment pipeline
_Sidebar.md                       # Navigation sidebar
_errors.log                       # Error details (if any)

Output Structure for Multi-Repository Wikis

Directory Organization

For a multi-repository project named myproject, the output structure is:

wiki-output/
  myproject/
    Home.md                  # Lists all repositories
    _Sidebar.md              # Navigation
    System-Architecture.md
    API-Specification.md
    ... (all pages)
    _errors.log              # Only if errors occurred

Compare with standalone (single-repository) output:

wiki-output/
  repo-name/
    Home.md
    _Sidebar.md
    ... (pages)

Home Page for Multi-Repository Projects

The Home.md file for multi-repository wikis includes a dedicated section listing all repositories:

func writeHomeAndSidebar(wikiDir, projectName, structureContent string, allPages []WikiPage, repos []string) {
	var home strings.Builder
	home.WriteString(fmt.Sprintf("# %s\n\n", projectName))
	desc := extractTag(structureContent, "description")
	if desc != "" {
		home.WriteString(fmt.Sprintf("%s\n\n", desc))
	}
	if len(repos) > 1 {
		home.WriteString("## Repositories\n\n")
		for _, r := range repos {
			home.WriteString(fmt.Sprintf("- %s\n", r))
		}
		home.WriteString("\n")
	}
	// ... write pages list
}

When multiple repositories are present, the Home.md explicitly lists them under a "Repositories" section. This helps readers understand the scope of the documentation. Sources: main.go:669-696

Repository Cloning and Setup for Multi-Repository Projects

Parallel Repository Cloning

When a multi-repository project contains remote repositories, wikigen clones them all into the clone directory with ownership-prefixed names:

.repos/
  tomohiro_wikigen/        # owner_repo naming
  anthropic_sdk-js/        # Another repository
  another-owner_shared/    # Shared library

Each repository is cloned with --depth=1 --single-branch to minimize download size. Sources: main.go:168-192

Local Directory Support

Multi-repository projects can also combine local directories:

myproject:/home/user/projects/frontend
myproject:/home/user/projects/backend
myproject:/home/user/projects/shared

Local directories are validated but not cloned. They are used directly. Sources: main.go:82-113

Mixed Mode Support

A multi-repository project can combine remote and local repositories:

backend:owner/backend-repo
backend:/home/user/shared-library
backend:owner/infra-config

This is useful when some repositories are external while others are local development directories. Sources: main.go:513-546

Parallelism in Multi-Repository Processing

Multi-repository wikis benefit from two levels of parallelism:

1. Repository-Level: Multiple multi-repo projects can be processed concurrently
2. Page-Level: Pages within a multi-repo wiki can be generated in parallel

graph TD
    A["Multiple Projects<br/>(parallelism -p)"] --> B["project-1"]
    A --> C["project-2"]
    A --> D["project-3"]

    B --> E["Page 1<br/>parallelism -pp"]
    B --> F["Page 2"]
    B --> G["Page 3"]

    C --> H["Page 1"]
    C --> I["Page 2"]

    D --> J["Page 1"]
    D --> K["Page 2"]
    D --> L["Page 3"]
    D --> M["Page 4"]

Loading

Configuration

Use the CLI flags to control parallelism:

# 2 projects in parallel, 5 pages per project in parallel
./wikigen -f repos.txt -p 2 -pp 5

# Or via environment variables
WIKI_PARALLEL=2 WIKI_PAGE_PARALLEL=5 ./wikigen -f repos.txt

All repositories in a single multi-repository project are analyzed together during structure determination, so they share a single Structure Claude Code invocation. Page generation uses the -pp flag to control how many pages are generated simultaneously. Sources: main.go:594-646

Error Handling in Multi-Repository Projects

Per-Project Error Logging

Errors in multi-repository projects are logged to _errors.log in the project directory:

wiki-output/myproject/_errors.log

This file contains timestamped error details for each failed page within the project. Sources: main.go:639

Partial Success

If some pages fail in a multi-repository project, the successful pages are preserved. The _errors.log documents which pages failed. This is particularly valuable for multi-repo projects since failure in one repository's documentation should not prevent documentation of other repositories from being published.

Retry Mechanism

The -retry flag can be used to regenerate failed pages across all multi-repository projects:

./wikigen -retry

This scans all project directories in wiki-output/ and identifies pages with content less than 100 characters, treating them as failed. It then regenerates only those pages. Sources: main.go:778-912

Use Cases for Multi-Repository Wikis

Microservices Documentation

Generate a unified wiki that documents an entire microservices architecture:

api:owner/api-gateway
api:owner/user-service
api:owner/product-service
api:owner/order-service
api:owner/payment-service

The resulting wiki includes how each service exposes APIs, how they communicate with each other, shared authentication patterns, and deployment pipelines.

Full-Stack Projects

Document frontend and backend together:

myapp:owner/frontend-react
myapp:owner/backend-go
myapp:owner/shared-types

The wiki can show architecture spanning both sides, API contracts, and end-to-end flows.

Monorepo Alternatives

When code is split across multiple repositories for organizational reasons, multi-repo support allows treating them as a logical unit:

platform:owner/core
platform:owner/plugins
platform:owner/integrations
platform:owner/cli

Library + Consumer Projects

Document a library and its consumers together to show how the library is used:

sdk:owner/typescript-sdk
sdk:owner/example-app
sdk:owner/integration-tests

Comparison: Standalone vs. Multi-Repository

graph TD
    A["Input Format"] --> B["Standalone"]
    A --> C["Multi-Repository"]

    B --> B1["owner/repo"]
    B --> B2["Separate wikis"]
    B --> B3["No cross-repo docs"]

    C --> C1["project:owner/repo"]
    C --> C2["Single unified wiki"]
    C --> C3["Cross-repo architecture"]

Loading

Aspect	Standalone	Multi-Repository
Input Format	owner/repo	project:owner/repo
Output Directory	wiki-output/repo/	wiki-output/project/
Claude Code Access	Single repository	All repositories simultaneously
Structure Pages	Repository-specific	Cross-repository + repository-specific
Use Case	Individual project documentation	Integrated multi-service systems
Number of Output Wikis	One per repository	One per project group

Sources: main.go:1014-1030

Command-Line Usage Examples

Generate Multi-Repository Wiki from repos.txt

./wikigen -f repos.txt

With repos.txt containing:

# Standalone
owner/single-repo

# Multi-repo group
myproject:owner/frontend
myproject:owner/backend
myproject:owner/shared

This generates three wikis: one for single-repo and one for the myproject group containing all three repositories.

Dry-Run for Multi-Repository Project

./wikigen -dry-run myproject:owner/frontend myproject:owner/backend

Determines the wiki structure without generating content.

Multi-Repository with Custom Parallelism

./wikigen -f repos.txt -p 2 -pp 5

Processes 2 projects in parallel, generating 5 pages per project in parallel.

Multi-Repository with Local Directories

./wikigen \
  backend:/home/user/api \
  backend:/home/user/worker \
  backend:owner/shared-lib

Combines local development directories with remote repositories.

JSON Output for Multi-Repository Projects

./wikigen -json -f repos.txt 2>/dev/null | jq '.[] | select(.project == "myproject")'

Outputs JSON results; filtering for a specific project.

Repository Configuration File Example

A complete repos.txt demonstrating mixed standalone and multi-repo:

# Single-repository wikis
owner/utility-lib
owner/cli-tool

# Multi-repository wiki: Platform
platform:owner/core-service
platform:owner/auth-service
platform:owner/api-gateway

# Multi-repository wiki: Web Stack
web:owner/frontend-app
web:owner/backend-api
web:/home/dev/shared-components

# Multi-repository wiki: Infrastructure
infra:owner/terraform-modules
infra:owner/docker-configs

This configuration generates:

• 2 standalone wikis (utility-lib, cli-tool)
• 3 multi-repo wikis (platform, web, infra)
• Total: 5 output directories in wiki-output/

Internal Processing: Task Creation

When wikigen processes a mixed configuration, it creates separate tasks:

var tasks []task

// Standalone entries
for _, entry := range standalone {
	tasks = append(tasks, task{
		name:  filepath.Base(entry.LocalDir or entry.Repo),
		repos: []string{entry.LocalDir or entry.Repo},
	})
}

// Grouped entries
for project, repoList := range groups {
	tasks = append(tasks, task{
		name:  project,
		repos: repoList,  // All repos in the group
	})
}

Each task is then processed independently but concurrently via goroutines, with -p controlling how many execute in parallel. Sources: main.go:1028-1093

Best Practices for Multi-Repository Wikis

1. Consistent Naming

Use descriptive project names that reflect the system:

Good:  platform, web-api, infrastructure, data-pipeline
Poor:  proj1, multi, group-a

2. Logical Grouping

Group repositories by functionality or system boundary:

# Good: grouped by tier
frontend:owner/react-app
backend:owner/api-server
database:owner/migrations

# Good: grouped by service
auth:owner/auth-service
auth:owner/auth-ui
auth:owner/auth-mobile

3. Document Inter-Service Contracts

When generating multi-repo documentation, Claude Code should have clear guidance about documenting:

• API contracts between services
• Shared data models or types
• Authentication and authorization flows
• Deployment dependencies

4. Use Parallelism Appropriately

For large multi-repo projects:

# CPU-bound structure determination (phase 1)
./wikigen -f repos.txt -p 2 -pp 8

This processes 2 projects in parallel while generating 8 pages per project in parallel, maximizing resource utilization.

5. Monitor Errors

After generation, check for partial failures:

ls wiki-output/*/_ errors.log

Use -retry to regenerate failed pages:

./wikigen -retry

Related Pages

• System Overview — Project purpose and core concepts
• Architecture & Design — System architecture and data structures
• Input Formats & Repository Configuration — Complete input format specification
• CLI Usage & Commands — All CLI flags and usage patterns
• Configuration & Environment — Environment variables and configuration
• Output Format & Wiki Structure — Output directory structure and file organization
• Parallel Processing & Performance — Concurrency control and parallelism
• Claude Code Integration — How Claude Code is invoked
• Wiki Generation Processing Flow — Two-phase generation pipeline