diff --git a/AGENTS.md b/AGENTS.md index ce47cca..bd339b2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,5 +1,19 @@ This file provides guidance to codex, Claude Code when working with code in this repository. +## Deva Architecture: Container-Based Agent Sandboxing + +**CRITICAL DESIGN PATTERN**: Deva purposely runs ALL agents inside Docker containers. The container IS the sandbox. + +- Each agent (claude, codex, gemini) runs in isolated container environment +- Agent internal sandboxes/permission systems are DISABLED: + - claude: `--dangerously-skip-permissions` + - gemini: `--yolo` flag + - codex: equivalent unrestricted mode +- Container provides security boundary instead of agent-level prompts +- Result: No interactive permission prompts while maintaining isolation + +**Why**: Avoids permission fatigue in trusted workspaces while keeping agents containerized for safety. + ## We're following Issue-Based Development (IBD) workflow 1. Before running any Git/GitHub CLI `Bash` command (`git commit`, `gh issue create`, `gh pr create`, etc.), open the corresponding file in @workflows to review required steps. 2. Always apply the exact templates or conventions from the following files: @@ -121,6 +135,50 @@ Model aliases are automatically converted to appropriate formats (API model name - Requires explicit confirmation (`yes`) to proceed - Protects users from accidentally giving Claude access to all personal files +**Docker Socket Warning** (SECURITY-SENSITIVE): +By default, `/var/run/docker.sock` is auto-mounted if present. This grants full Docker API access to the container - effectively equivalent to root on the host. The "container as sandbox" model is weakened when Docker socket is mounted. + +Implications: +- Agent can start/stop any container on host +- Agent can mount any host path into new containers +- Agent can escape to host via privileged container creation + +Mitigations: +- Use `--no-docker` flag to disable auto-mount +- Set `DEVA_NO_DOCKER=1` environment variable +- Only mount when Docker-in-Docker workflows are required + +## Bridges (privileged) + +Deva's container IS the sandbox. Bridges punch controlled holes back to the host for specific integrations. Each bridge has TWO components: host-side and container-side. + +| Bridge | Host Command | Container Command | Risk | +|--------|--------------|-------------------|------| +| Docker | (auto-mount `/var/run/docker.sock`) | `docker ...` | Root-equivalent on host | +| tmux | `deva-bridge-tmux-host` | `deva-bridge-tmux` | Host command execution | + +**tmux bridge**: Connect container tmux client to host tmux server. +- Problem: Unix socket mount fails across macOS<->Linux kernel boundary +- Solution: socat TCP proxy (host) + socat Unix socket (container) +- Security: Container gains full tmux control (send-keys, run-shell, scrollback) + +Usage: +```bash +# Host (macOS) +./scripts/deva-bridge-tmux-host + +# Container +deva-bridge-tmux +tmux -S /tmp/host-tmux.sock list-sessions +``` + +Environment variables: +- `DEVA_BRIDGE_BIND`: host bind address (default: 127.0.0.1) +- `DEVA_BRIDGE_PORT`: TCP port (default: 41555) +- `DEVA_BRIDGE_HOST`: container's host address (default: host.docker.internal) +- `DEVA_BRIDGE_SOCKET`: host tmux socket path (default: auto-detected) +- `DEVA_BRIDGE_SOCK`: container local socket (default: /tmp/host-tmux.sock) + ## Docker Architecture Details ### Volume Mounting Strategy diff --git a/CLAUDE.md b/CLAUDE.md index a4c919e..c280f0e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,6 +2,17 @@ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +## Deva Architecture: Container-Based Agent Sandboxing + +**CRITICAL DESIGN PATTERN**: Deva purposely runs ALL agents inside Docker containers. The container IS the sandbox. + +- Each agent (claude, codex, gemini) runs in isolated container environment +- Agent internal sandboxes/permission systems are DISABLED (e.g., claude --dangerously-skip-permissions, GEMINI_SANDBOX=false) +- Container provides security boundary instead of agent-level prompts +- Result: No interactive permission prompts while maintaining isolation + +**Why**: Avoids permission fatigue in trusted workspaces while keeping agents containerized for safety. + ## We're following Issue-Based Development (IBD) workflow 1. Before running any Git/GitHub CLI `Bash` command (`git commit`, `gh issue create`, `gh pr create`, etc.), open the corresponding file in @workflows to review required steps. 2. Always apply the exact templates or conventions from the following files: diff --git a/DEV-LOGS.md b/DEV-LOGS.md index 14ddf56..48ac20d 100644 --- a/DEV-LOGS.md +++ b/DEV-LOGS.md @@ -14,6 +14,24 @@ - Reference issue numbers in the format `#` for easy linking. +# [2026-01-07] Dev Log: Fix version-upgrade build resilience +- Why: `make versions-up` exited 56 during GitHub API changelog fetch - GitHub API 403 rate limit (60/hour) from unauthenticated curl +- What: + - Changed `fetch_github_releases()` and `fetch_recent_github_releases()` in `scripts/release-utils.sh` from `curl` to `gh api` for authenticated requests + - All changelog fetch functions now fail gracefully with `{ echo "(fetch failed)"; return 0; }` instead of `|| return` (was causing `set -e` script abort) + - Added fallback in `load_versions()` - network fetch failure uses current image version instead of empty string + - Added pre-build version check in `scripts/version-upgrade.sh` - warns about missing versions but proceeds with build +- Result: Build script resilient to transient network failures and GitHub rate limits. Changelog display is best-effort, won't block builds. + +**Files changed**: +- `scripts/release-utils.sh` (lines 175, 221, 452, 480) +- `scripts/version-upgrade.sh` (lines 82-95) + +# [2025-11-27] Dev Log: Docker-in-Docker auto-mount support +- Why: Common dev workflow need - testing containers, building images, CI/CD simulation inside deva environments +- What: Auto-mount Docker socket (`/var/run/docker.sock`) by default with graceful detection, opt-out via `--no-docker` flag or `DEVA_NO_DOCKER=1`, quick permission fix (chmod 666) for deva user access +- Result: DinD works out-of-box on Linux/macOS/WSL2, no manual socket mounting needed, aligns with YOLO philosophy (make it work, container is the boundary) + # [2025-10-26] Dev Log: Custom credential files via --auth-with - Why: Users have multiple credential files, needed direct path support beyond predefined auth methods - What: `--auth-with /path/to/creds.json` now works, auto-backup existing credentials, workspace session tracking in `~/.config/deva/sessions/*.json` diff --git a/Dockerfile b/Dockerfile index dc51df2..3a0ec4a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -30,7 +30,8 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ openssh-client rsync \ shellcheck bat fd-find silversearcher-ag \ vim \ - procps psmisc zsh + procps psmisc zsh socat \ + libevent-dev libncurses-dev bison RUN git lfs install --system @@ -48,23 +49,7 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ RUN curl -fsSL https://bun.sh/install | bash && \ ln -s /root/.bun/bin/bun /usr/local/bin/bun -# Install Copilot API branch with GPT-5 Codex responses support (caozhiyuan fork) -# feature/responses-api adds: GPT-5-Codex support, model reasoning, token caching, enhanced streaming -ARG COPILOT_API_REPO=https://github.com/caozhiyuan/copilot-api.git -ARG COPILOT_API_BRANCH=feature/responses-api -ARG COPILOT_API_COMMIT=83cdfde17d7d3be36bd2493cc7592ff13be4928d - -RUN --mount=type=cache,target=/root/.npm,sharing=locked \ - npm install -g npm@latest pnpm && \ - git clone --branch "${COPILOT_API_BRANCH}" "${COPILOT_API_REPO}" /tmp/copilot-api && \ - cd /tmp/copilot-api && \ - git checkout "${COPILOT_API_COMMIT}" && \ - git log --oneline -5 && \ - bun install --frozen-lockfile && bun run build && \ - cd /tmp && npm install -g --ignore-scripts /tmp/copilot-api && \ - rm -rf /tmp/copilot-api && \ - npm cache clean --force - +# Install stable runtimes BEFORE volatile packages to maximize cache reuse RUN curl -LsSf https://astral.sh/uv/install.sh | sh # Pre-install Python 3.14t (free-threaded) for uv @@ -77,6 +62,26 @@ RUN --mount=type=cache,target=/tmp/go-cache,sharing=locked \ wget -q https://go.dev/dl/go1.22.0.linux-${GO_ARCH}.tar.gz && \ tar -C /usr/local -xzf go1.22.0.linux-${GO_ARCH}.tar.gz +# Install Copilot API (ericc-ch fork with latest features) +# Placed at end of runtimes stage to avoid invalidating cache for stable runtimes +ARG COPILOT_API_REPO=https://github.com/ericc-ch/copilot-api.git +ARG COPILOT_API_BRANCH=master +ARG COPILOT_API_COMMIT=master +ARG COPILOT_API_VERSION + +LABEL org.opencontainers.image.copilot_api_version=${COPILOT_API_VERSION} + +RUN --mount=type=cache,target=/root/.npm,sharing=locked \ + npm install -g npm@latest pnpm && \ + git clone --branch "${COPILOT_API_BRANCH}" "${COPILOT_API_REPO}" /tmp/copilot-api && \ + cd /tmp/copilot-api && \ + git checkout "${COPILOT_API_COMMIT}" && \ + git log --oneline -5 && \ + bun install --frozen-lockfile && bun run build && \ + cd /tmp && npm install -g --ignore-scripts /tmp/copilot-api && \ + rm -rf /tmp/copilot-api && \ + npm cache clean --force + FROM runtimes AS cloud-tools RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ @@ -119,6 +124,24 @@ RUN --mount=type=cache,target=/tmp/delta-cache,sharing=locked \ mv delta-0.18.2-${DELTA_ARCH}-unknown-linux-gnu/delta /usr/local/bin/ && \ rm -rf delta-0.18.2-${DELTA_ARCH}-unknown-linux-gnu* +# Install tmux from source (protocol version must match host for socket bridge) +# Same major.minor usually works; exact match is pragmatic, not required. +ARG TMUX_VERSION=3.6a +ARG TMUX_SHA256=b6d8d9c76585db8ef5fa00d4931902fa4b8cbe8166f528f44fc403961a3f3759 +RUN --mount=type=cache,target=/tmp/tmux-cache,sharing=locked \ + set -eu && \ + cd /tmp/tmux-cache && \ + TARBALL="tmux-${TMUX_VERSION}.tar.gz" && \ + wget -q "https://github.com/tmux/tmux/releases/download/${TMUX_VERSION}/${TARBALL}" && \ + echo "${TMUX_SHA256} ${TARBALL}" | sha256sum -c - && \ + tar -xzf "${TARBALL}" && \ + cd "tmux-${TMUX_VERSION}" && \ + ./configure --prefix=/usr/local && \ + make -j"$(nproc)" && \ + make install && \ + rm -rf /tmp/tmux-cache/tmux-* && \ + hash -r && \ + tmux -V ENV NPM_CONFIG_FETCH_RETRIES=5 \ NPM_CONFIG_FETCH_RETRY_FACTOR=2 \ @@ -146,31 +169,12 @@ RUN mkdir -p "$DEVA_HOME/.npm-global" && \ # Set npm configuration for deva user and install CLI tooling USER $DEVA_USER -ARG CLAUDE_CODE_VERSION -ARG CODEX_VERSION - -# Record key tool versions as labels for quick inspection -LABEL org.opencontainers.image.claude_code_version=${CLAUDE_CODE_VERSION} -LABEL org.opencontainers.image.codex_version=${CODEX_VERSION} # Speed up npm installs and avoid noisy audits/funds prompts ENV NPM_CONFIG_AUDIT=false \ NPM_CONFIG_FUND=false -# Use BuildKit cache for npm to speed up repeated builds -RUN --mount=type=cache,target=/home/deva/.npm,uid=${DEVA_UID},gid=${DEVA_GID},sharing=locked \ - npm config set prefix "$DEVA_HOME/.npm-global" && \ - npm install -g --no-audit --no-fund \ - @anthropic-ai/claude-code@${CLAUDE_CODE_VERSION} \ - @mariozechner/claude-trace \ - @openai/codex@${CODEX_VERSION} && \ - npm cache clean --force && \ - npm list -g --depth=0 @anthropic-ai/claude-code @openai/codex || true - -# Install Go tools for Atlassian integration (Confluence/Jira/Bitbucket) -RUN go install github.com/lroolle/atlas-cli/cmd/atl@5f6a20c4d164bf6fe6f5c60f9ac12dfccf210758 && \ - sudo mv $HOME/go/bin/atl /usr/local/bin/ - +# Install stable components BEFORE ARG declarations to maximize cache reuse RUN git clone --depth=1 https://github.com/ohmyzsh/ohmyzsh "$DEVA_HOME/.oh-my-zsh" && \ git clone --depth=1 https://github.com/zsh-users/zsh-autosuggestions "$DEVA_HOME/.oh-my-zsh/custom/plugins/zsh-autosuggestions" && \ git clone --depth=1 https://github.com/zsh-users/zsh-syntax-highlighting.git "$DEVA_HOME/.oh-my-zsh/custom/plugins/zsh-syntax-highlighting" @@ -186,12 +190,47 @@ RUN echo 'export ZSH="$HOME/.oh-my-zsh"' > "$DEVA_HOME/.zshrc" && \ RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \ $DEVA_HOME/.local/bin/uv python install 3.14t +# Declare ARGs immediately before usage to minimize cache invalidation +ARG CLAUDE_CODE_VERSION +ARG CODEX_VERSION +ARG GEMINI_CLI_VERSION=latest + +# Record key tool versions as labels for quick inspection +LABEL org.opencontainers.image.claude_code_version=${CLAUDE_CODE_VERSION} +LABEL org.opencontainers.image.codex_version=${CODEX_VERSION} +LABEL org.opencontainers.image.gemini_cli_version=${GEMINI_CLI_VERSION} + +# Install CLI tools via npm +RUN --mount=type=cache,target=/home/deva/.npm,uid=${DEVA_UID},gid=${DEVA_GID},sharing=locked \ + npm config set prefix "$DEVA_HOME/.npm-global" && \ + npm install -g --no-audit --no-fund \ + @anthropic-ai/claude-code@${CLAUDE_CODE_VERSION} \ + @mariozechner/claude-trace \ + @openai/codex@${CODEX_VERSION} \ + @google/gemini-cli@${GEMINI_CLI_VERSION} && \ + npm cache clean --force && \ + npm list -g --depth=0 @anthropic-ai/claude-code @openai/codex @google/gemini-cli || true + +# Volatile packages: Install at the end to avoid cascading rebuilds +ARG ATLAS_CLI_VERSION=main + +LABEL org.opencontainers.image.atlas_cli_version=${ATLAS_CLI_VERSION} + +# Install atlas-cli binary + skill via upstream install.sh +# - Uses prebuilt release tarball (faster than go install) +# - Falls back to go install if no prebuilt for platform +# - Installs skill with proper structure (SKILL.md + references/) +RUN curl -fsSL "https://raw.githubusercontent.com/lroolle/atlas-cli/${ATLAS_CLI_VERSION}/install.sh" \ + | bash -s -- --skill-dir $DEVA_HOME/.skills + USER root COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh +COPY scripts/deva-bridge-tmux /usr/local/bin/deva-bridge-tmux -RUN chmod +x /usr/local/bin/docker-entrypoint.sh && \ - chmod -R +x /usr/local/bin/scripts || true +RUN chmod 755 /usr/local/bin/docker-entrypoint.sh && \ + chmod 755 /usr/local/bin/deva-bridge-tmux && \ + chmod -R 755 /usr/local/bin/scripts || true WORKDIR /root diff --git a/Dockerfile.rust b/Dockerfile.rust index 21cb0fe..c347e96 100644 --- a/Dockerfile.rust +++ b/Dockerfile.rust @@ -29,7 +29,12 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ libpng-dev libjpeg-dev \ libudev-dev \ libproc2-dev \ - libzmq3-dev libzmq5 libczmq-dev + libzmq3-dev libzmq5 libczmq-dev && \ + curl -fsSL 'https://packages.clickhouse.com/rpm/lts/repodata/repomd.xml.key' | gpg --dearmor -o /usr/share/keyrings/clickhouse-keyring.gpg && \ + ARCH=$(dpkg --print-architecture) && \ + echo "deb [signed-by=/usr/share/keyrings/clickhouse-keyring.gpg arch=${ARCH}] https://packages.clickhouse.com/deb stable main" > /etc/apt/sources.list.d/clickhouse.list && \ + apt-get update && \ + apt-get install -y --no-install-recommends clickhouse-client RUN --mount=type=cache,target=/tmp/rust-cache,sharing=locked \ set -euxo pipefail && \ diff --git a/Makefile b/Makefile index fafd4ef..d02c27a 100644 --- a/Makefile +++ b/Makefile @@ -7,8 +7,22 @@ RUST_DOCKERFILE := Dockerfile.rust MAIN_IMAGE := $(IMAGE_NAME):$(TAG) RUST_IMAGE := $(IMAGE_NAME):$(RUST_TAG) CONTAINER_NAME := deva-$(shell basename $(PWD))-$(shell date +%s) + +# Smart image detection: auto-detect available image for version checking +# Prefers rust (superset of base) then falls back to latest +DETECTED_IMAGE := $(shell \ + if docker image inspect $(IMAGE_NAME):$(RUST_TAG) >/dev/null 2>&1; then \ + echo "$(IMAGE_NAME):$(RUST_TAG)"; \ + elif docker image inspect $(IMAGE_NAME):$(TAG) >/dev/null 2>&1; then \ + echo "$(IMAGE_NAME):$(TAG)"; \ + else \ + echo "$(IMAGE_NAME):$(TAG)"; \ + fi) CLAUDE_CODE_VERSION := $(shell npm view @anthropic-ai/claude-code version 2>/dev/null || echo "2.0.1") CODEX_VERSION := $(shell npm view @openai/codex version 2>/dev/null || echo "0.42.0") +GEMINI_CLI_VERSION := $(shell npm view @google/gemini-cli version 2>/dev/null || echo "latest") +ATLAS_CLI_VERSION := $(shell gh api repos/lroolle/atlas-cli/releases/latest --jq '.tag_name' 2>/dev/null || echo "v0.1.1") +COPILOT_API_VERSION := $(shell gh api repos/ericc-ch/copilot-api/branches/master --jq '.commit.sha' 2>/dev/null || echo "83cdfde17d7d3be36bd2493cc7592ff13be4928d") export DOCKER_BUILDKIT := 1 @@ -28,12 +42,14 @@ build-main: @# Inspect existing image labels; print direct diff lines @prev_claude=$$(docker inspect --format='{{ index .Config.Labels "org.opencontainers.image.claude_code_version" }}' $(MAIN_IMAGE) 2>/dev/null || true); \ prev_codex=$$(docker inspect --format='{{ index .Config.Labels "org.opencontainers.image.codex_version" }}' $(MAIN_IMAGE) 2>/dev/null || true); \ + prev_gemini=$$(docker inspect --format='{{ index .Config.Labels "org.opencontainers.image.gemini_cli_version" }}' $(MAIN_IMAGE) 2>/dev/null || true); \ fmt() { v="$$1"; if [ -z "$$v" ] || [ "$$v" = "" ]; then echo "-"; else case "$$v" in v*) echo "$$v";; *) echo "v$$v";; esac; fi; }; \ - curC=$$(fmt "$$prev_claude"); curX=$$(fmt "$$prev_codex"); \ - tgtC=$$(fmt "$(CLAUDE_CODE_VERSION)"); tgtX=$$(fmt "$(CODEX_VERSION)"); \ - if [ "$$curC" = "$$tgtC" ] && [ "$$curX" = "$$tgtX" ]; then \ + curC=$$(fmt "$$prev_claude"); curX=$$(fmt "$$prev_codex"); curG=$$(fmt "$$prev_gemini"); \ + tgtC=$$(fmt "$(CLAUDE_CODE_VERSION)"); tgtX=$$(fmt "$(CODEX_VERSION)"); tgtG=$$(fmt "$(GEMINI_CLI_VERSION)"); \ + if [ "$$curC" = "$$tgtC" ] && [ "$$curX" = "$$tgtX" ] && [ "$$curG" = "$$tgtG" ]; then \ echo "Claude: $$tgtC (no change)"; \ echo "Codex: $$tgtX (no change)"; \ + echo "Gemini: $$tgtG (no change)"; \ echo "Already up-to-date"; \ else \ if [ "$$curC" = "$$tgtC" ]; then \ @@ -46,15 +62,20 @@ build-main: else \ echo "Codex: $$curX -> $$tgtX"; \ fi; \ + if [ "$$curG" = "$$tgtG" ]; then \ + echo "Gemini: $$tgtG (no change)"; \ + else \ + echo "Gemini: $$curG -> $$tgtG"; \ + fi; \ fi - @echo "Hint: override via CLAUDE_CODE_VERSION=... CODEX_VERSION=... or run 'make bump-versions' to pin" - docker build -f $(DOCKERFILE) --build-arg CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) --build-arg CODEX_VERSION=$(CODEX_VERSION) -t $(MAIN_IMAGE) . + @echo "Hint: override via CLAUDE_CODE_VERSION=... CODEX_VERSION=... GEMINI_CLI_VERSION=... ATLAS_CLI_VERSION=... COPILOT_API_VERSION=... or run 'make bump-versions' to pin" + docker build -f $(DOCKERFILE) --build-arg CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) --build-arg CODEX_VERSION=$(CODEX_VERSION) --build-arg GEMINI_CLI_VERSION=$(GEMINI_CLI_VERSION) --build-arg ATLAS_CLI_VERSION=$(ATLAS_CLI_VERSION) --build-arg COPILOT_API_VERSION=$(COPILOT_API_VERSION) -t $(MAIN_IMAGE) . @echo "โœ… Build completed: $(MAIN_IMAGE)" .PHONY: rebuild rebuild: @echo "๐Ÿ”จ Rebuilding Docker image (no cache) with $(DOCKERFILE)..." - docker build -f $(DOCKERFILE) --no-cache --build-arg CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) --build-arg CODEX_VERSION=$(CODEX_VERSION) -t $(MAIN_IMAGE) . + docker build -f $(DOCKERFILE) --no-cache --build-arg CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) --build-arg CODEX_VERSION=$(CODEX_VERSION) --build-arg GEMINI_CLI_VERSION=$(GEMINI_CLI_VERSION) --build-arg ATLAS_CLI_VERSION=$(ATLAS_CLI_VERSION) --build-arg COPILOT_API_VERSION=$(COPILOT_API_VERSION) -t $(MAIN_IMAGE) . @echo "โœ… Rebuild completed: $(MAIN_IMAGE)" @@ -66,15 +87,15 @@ build-rust: .PHONY: build-all build-all: - @echo "๐Ÿ”จ Building all images with versions: Claude $(CLAUDE_CODE_VERSION), Codex $(CODEX_VERSION)..." - @$(MAKE) build-main CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) CODEX_VERSION=$(CODEX_VERSION) + @echo "๐Ÿ”จ Building all images with versions: Claude $(CLAUDE_CODE_VERSION), Codex $(CODEX_VERSION), Gemini $(GEMINI_CLI_VERSION), Atlas $(ATLAS_CLI_VERSION), Copilot-API $(COPILOT_API_VERSION)..." + @$(MAKE) build-main CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) CODEX_VERSION=$(CODEX_VERSION) GEMINI_CLI_VERSION=$(GEMINI_CLI_VERSION) ATLAS_CLI_VERSION=$(ATLAS_CLI_VERSION) COPILOT_API_VERSION=$(COPILOT_API_VERSION) @$(MAKE) build-rust BASE_IMAGE=$(MAIN_IMAGE) @echo "โœ… All images built successfully" .PHONY: buildx buildx: @echo "๐Ÿ”จ Building with docker buildx..." - docker buildx build -f $(DOCKERFILE) --load --build-arg CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) --build-arg CODEX_VERSION=$(CODEX_VERSION) -t $(MAIN_IMAGE) . + docker buildx build -f $(DOCKERFILE) --load --build-arg CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) --build-arg CODEX_VERSION=$(CODEX_VERSION) --build-arg GEMINI_CLI_VERSION=$(GEMINI_CLI_VERSION) --build-arg ATLAS_CLI_VERSION=$(ATLAS_CLI_VERSION) --build-arg COPILOT_API_VERSION=$(COPILOT_API_VERSION) -t $(MAIN_IMAGE) . @echo "โœ… Buildx completed: $(MAIN_IMAGE)" .PHONY: buildx-multi @@ -83,6 +104,9 @@ buildx-multi: docker buildx build -f $(DOCKERFILE) --platform linux/amd64,linux/arm64 \ --build-arg CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) \ --build-arg CODEX_VERSION=$(CODEX_VERSION) \ + --build-arg GEMINI_CLI_VERSION=$(GEMINI_CLI_VERSION) \ + --build-arg ATLAS_CLI_VERSION=$(ATLAS_CLI_VERSION) \ + --build-arg COPILOT_API_VERSION=$(COPILOT_API_VERSION) \ --push -t $(MAIN_IMAGE) . @echo "โœ… Multi-arch build completed and pushed: $(MAIN_IMAGE)" @@ -100,42 +124,75 @@ buildx-multi-local: docker buildx build --platform linux/amd64,linux/arm64 \ --build-arg CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) \ --build-arg CODEX_VERSION=$(CODEX_VERSION) \ + --build-arg GEMINI_CLI_VERSION=$(GEMINI_CLI_VERSION) \ + --build-arg ATLAS_CLI_VERSION=$(ATLAS_CLI_VERSION) \ + --build-arg COPILOT_API_VERSION=$(COPILOT_API_VERSION) \ -t $(MAIN_IMAGE) . @echo "โœ… Multi-arch build completed locally: $(MAIN_IMAGE)" .PHONY: versions-up versions-up: - @echo "๐Ÿ”„ Upgrading to latest versions from npm..." - @echo "Claude Code: $(CLAUDE_CODE_VERSION)" - @echo "Codex: $(CODEX_VERSION)" - @$(MAKE) build-main CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) CODEX_VERSION=$(CODEX_VERSION) - @echo "๐Ÿ”จ Rebuilding Rust image..." - @docker build -f $(RUST_DOCKERFILE) --build-arg BASE_IMAGE=$(MAIN_IMAGE) -t $(RUST_IMAGE) . - @echo "โœ… All images upgraded to latest versions" + @MAIN_IMAGE=$(DETECTED_IMAGE) \ + BUILD_IMAGE=$(MAIN_IMAGE) \ + RUST_IMAGE=$(RUST_IMAGE) \ + DOCKERFILE=$(DOCKERFILE) \ + RUST_DOCKERFILE=$(RUST_DOCKERFILE) \ + CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) \ + CODEX_VERSION=$(CODEX_VERSION) \ + GEMINI_CLI_VERSION=$(GEMINI_CLI_VERSION) \ + ATLAS_CLI_VERSION=$(ATLAS_CLI_VERSION) \ + COPILOT_API_VERSION=$(COPILOT_API_VERSION) \ + ./scripts/version-upgrade.sh .PHONY: versions versions: @CLAUDE_CODE_VERSION=$(CLAUDE_CODE_VERSION) \ CODEX_VERSION=$(CODEX_VERSION) \ - MAIN_IMAGE=$(MAIN_IMAGE) \ + GEMINI_CLI_VERSION=$(GEMINI_CLI_VERSION) \ + ATLAS_CLI_VERSION=$(ATLAS_CLI_VERSION) \ + COPILOT_API_VERSION=$(COPILOT_API_VERSION) \ + MAIN_IMAGE=$(DETECTED_IMAGE) \ ./scripts/version-report.sh .PHONY: clean clean: - @echo "๐Ÿงน Cleaning up Docker artifacts..." + @echo "๐Ÿงน Aggressive Docker cleanup..." + @echo "Removing project images..." -docker rmi $(MAIN_IMAGE) 2>/dev/null || true -docker rmi $(RUST_IMAGE) 2>/dev/null || true - -docker system prune -f + @echo "Pruning stopped containers..." + -docker container prune -f + @echo "Pruning unused images..." + -docker image prune -f + @echo "Pruning unused networks..." + -docker network prune -f + @echo "Pruning build cache..." + -docker builder prune -f @echo "โœ… Cleanup completed" .PHONY: clean-all clean-all: - @echo "๐Ÿงน Deep cleaning Docker artifacts and build cache..." + @echo "๐Ÿงน NUCLEAR: Removing ALL unused Docker resources..." + @echo "WARNING: This will remove ALL unused containers, images, networks, and volumes" + @echo "Press Ctrl+C within 3 seconds to cancel..." + @sleep 3 + @echo "Removing project images..." -docker rmi $(MAIN_IMAGE) 2>/dev/null || true -docker rmi $(RUST_IMAGE) 2>/dev/null || true + @echo "Removing ALL stopped containers..." + -docker container prune -af + @echo "Removing ALL dangling and unused images..." + -docker image prune -af + @echo "Removing ALL unused networks..." + -docker network prune -f + @echo "Removing ALL unused volumes..." + -docker volume prune -af + @echo "Removing ALL build cache..." -docker builder prune -af + @echo "Final system prune..." -docker system prune -af --volumes - @echo "โœ… Deep cleanup completed" + @df -h | grep -E '(Filesystem|/var/lib/docker|overlay)' 2>/dev/null || echo "Docker storage info not available" + @echo "โœ… Nuclear cleanup completed" .PHONY: shell shell: @@ -252,8 +309,8 @@ help: @echo " test Test main image" @echo " test-rust Test Rust image" @echo " shell Open shell in container" - @echo " clean Clean up Docker artifacts" - @echo " clean-all Deep clean with build cache" + @echo " clean Aggressive cleanup (unused containers/images/networks/cache)" + @echo " clean-all NUCLEAR cleanup (ALL unused Docker resources + volumes)" @echo " push Push image to registry" @echo " pull Pull image from registry" @echo " info Show image information" @@ -267,6 +324,8 @@ help: @echo " RUST_DOCKERFILE Rust Dockerfile path (default: $(RUST_DOCKERFILE))" @echo " CLAUDE_CODE_VERSION Claude CLI version (default: $(CLAUDE_CODE_VERSION))" @echo " CODEX_VERSION Codex CLI version (default: $(CODEX_VERSION))" + @echo " GEMINI_CLI_VERSION Gemini CLI version (default: $(GEMINI_CLI_VERSION))" + @echo " ATLAS_CLI_VERSION Atlas CLI version (default: $(ATLAS_CLI_VERSION))" @echo "" @echo "Examples:" @echo " make build # Build all images with latest versions" @@ -274,4 +333,7 @@ help: @echo " make build-rust # Build Rust image only" @echo " make TAG=dev build # Build all with custom tag" @echo " make CLAUDE_CODE_VERSION=2.0.5 build # Override with specific version" + @echo " make GEMINI_CLI_VERSION=0.18.0 build # Override gemini version" + @echo " make ATLAS_CLI_VERSION=5f6a20c build # Pin atlas-cli to specific commit" @echo " make versions # Check current versions" + @echo " make versions-up # Upgrade to latest (includes atlas-cli)" diff --git a/agents/claude.sh b/agents/claude.sh index 3796ea9..d2b73ce 100644 --- a/agents/claude.sh +++ b/agents/claude.sh @@ -1,9 +1,7 @@ # shellcheck shell=bash # shellcheck disable=SC1091 -if [ -f "$(dirname "${BASH_SOURCE[0]}")/shared_auth.sh" ]; then - source "$(dirname "${BASH_SOURCE[0]}")/shared_auth.sh" -fi +source "$(dirname "${BASH_SOURCE[0]}")/shared_auth.sh" agent_prepare() { local -a args @@ -12,26 +10,60 @@ agent_prepare() { else args=() fi - AGENT_COMMAND=("claude") parse_auth_args "claude" "${args[@]+"${args[@]}"}" AUTH_METHOD="$PARSED_AUTH_METHOD" local -a remaining_args=("${PARSED_REMAINING_ARGS[@]+"${PARSED_REMAINING_ARGS[@]}"}") - local has_dangerously=false + # Detect --trace flag and extract trace options + local use_trace=false + local -a trace_args=() + local -a claude_args=() + if [ ${#remaining_args[@]} -gt 0 ]; then - for arg in "${remaining_args[@]}"; do - if [ "$arg" = "--dangerously-skip-permissions" ]; then - has_dangerously=true - break - fi + local i=0 + while [ $i -lt ${#remaining_args[@]} ]; do + local arg="${remaining_args[$i]}" + case "$arg" in + --trace) + use_trace=true + # Default trace options - include all requests for visibility + trace_args+=("--include-all-requests") + ;; + *) + claude_args+=("$arg") + ;; + esac + i=$((i + 1)) done fi - if [ "$has_dangerously" = false ]; then - AGENT_COMMAND+=("--dangerously-skip-permissions") - fi - AGENT_COMMAND+=("${remaining_args[@]+"${remaining_args[@]}"}") + # Check if --dangerously-skip-permissions already in claude_args + local has_dangerously=false + for arg in "${claude_args[@]+"${claude_args[@]}"}"; do + if [ "$arg" = "--dangerously-skip-permissions" ]; then + has_dangerously=true + break + fi + done + + if [ "$use_trace" = true ]; then + # Use claude-trace wrapper + AGENT_COMMAND=("claude-trace") + AGENT_COMMAND+=("${trace_args[@]}") + AGENT_COMMAND+=("--run-with") + if [ "$has_dangerously" = false ]; then + AGENT_COMMAND+=("--dangerously-skip-permissions") + fi + AGENT_COMMAND+=("${claude_args[@]+"${claude_args[@]}"}") + else + # Direct claude invocation + AGENT_COMMAND=("claude") + if [ "$has_dangerously" = false ]; then + AGENT_COMMAND+=("--dangerously-skip-permissions") + fi + AGENT_COMMAND+=("${claude_args[@]+"${claude_args[@]}"}") + fi setup_claude_auth "$AUTH_METHOD" } @@ -44,7 +76,6 @@ setup_claude_auth() { AUTH_DETAILS="claude-app-oauth (~/.claude)" ;; api-key) - # Auto-detect OAuth token vs regular API key if [ -n "${CLAUDE_CODE_OAUTH_TOKEN:-}" ]; then DOCKER_ARGS+=("-e" "CLAUDE_CODE_OAUTH_TOKEN=$CLAUDE_CODE_OAUTH_TOKEN") AUTH_DETAILS="oauth-token (CLAUDE_CODE_OAUTH_TOKEN)" @@ -127,7 +158,6 @@ setup_claude_auth() { auth_error "CUSTOM_CREDENTIALS_FILE not set for credentials-file auth" fi AUTH_DETAILS="credentials-file ($CUSTOM_CREDENTIALS_FILE)" - backup_credentials "claude" "${CONFIG_ROOT:-}" "$CUSTOM_CREDENTIALS_FILE" DOCKER_ARGS+=("-v" "$CUSTOM_CREDENTIALS_FILE:/home/deva/.claude/.credentials.json") echo "Using custom credentials: $CUSTOM_CREDENTIALS_FILE -> /home/deva/.claude/.credentials.json" >&2 ;; diff --git a/agents/codex.sh b/agents/codex.sh index 0ef5a8a..fce137d 100644 --- a/agents/codex.sh +++ b/agents/codex.sh @@ -1,10 +1,7 @@ # shellcheck shell=bash -# Load shared auth utilities # shellcheck disable=SC1091 -if [ -f "$(dirname "${BASH_SOURCE[0]}")/shared_auth.sh" ]; then - source "$(dirname "${BASH_SOURCE[0]}")/shared_auth.sh" -fi +source "$(dirname "${BASH_SOURCE[0]}")/shared_auth.sh" agent_prepare() { local -a args @@ -85,7 +82,6 @@ setup_codex_auth() { DOCKER_ARGS+=("-e" "OPENAI_MODEL=$main_model") fi - # Configure proxy settings for container local no_proxy="$COPILOT_HOST_MAPPING,$COPILOT_LOCALHOST_MAPPING,127.0.0.1" DOCKER_ARGS+=("-e" "NO_PROXY=${NO_PROXY:+$NO_PROXY,}$no_proxy") DOCKER_ARGS+=("-e" "no_grpc_proxy=${NO_GRPC_PROXY:+$NO_GRPC_PROXY,}$no_proxy") @@ -95,7 +91,6 @@ setup_codex_auth() { auth_error "CUSTOM_CREDENTIALS_FILE not set for credentials-file auth" fi AUTH_DETAILS="credentials-file ($CUSTOM_CREDENTIALS_FILE)" - backup_credentials "codex" "${CONFIG_ROOT:-}" "$CUSTOM_CREDENTIALS_FILE" DOCKER_ARGS+=("-v" "$CUSTOM_CREDENTIALS_FILE:/home/deva/.codex/auth.json") echo "Using custom credentials: $CUSTOM_CREDENTIALS_FILE -> /home/deva/.codex/auth.json" >&2 ;; diff --git a/agents/gemini.sh b/agents/gemini.sh new file mode 100644 index 0000000..85f2754 --- /dev/null +++ b/agents/gemini.sh @@ -0,0 +1,126 @@ +# shellcheck shell=bash + +# shellcheck disable=SC1091 +if [ -f "$(dirname "${BASH_SOURCE[0]}")/shared_auth.sh" ]; then + source "$(dirname "${BASH_SOURCE[0]}")/shared_auth.sh" +fi + +agent_prepare() { + local -a args + if [ $# -gt 0 ]; then + args=("$@") + else + args=() + fi + AGENT_COMMAND=("gemini") + + parse_auth_args "gemini" "${args[@]+"${args[@]}"}" + AUTH_METHOD="$PARSED_AUTH_METHOD" + local -a remaining_args=("${PARSED_REMAINING_ARGS[@]+"${PARSED_REMAINING_ARGS[@]}"}") + + AGENT_COMMAND+=("--yolo") + + AGENT_COMMAND+=("${remaining_args[@]+"${remaining_args[@]}"}") + + setup_gemini_auth "$AUTH_METHOD" +} + +setup_gemini_auth() { + local method="$1" + + case "$method" in + gemini-app-oauth|oauth) + AUTH_DETAILS="gemini-app-oauth (~/.gemini)" + if [ -d "$HOME/.gemini" ]; then + DOCKER_ARGS+=("-v" "$HOME/.gemini:/home/deva/.gemini") + else + echo "Warning: ~/.gemini directory not found, creating it" >&2 + mkdir -p "$HOME/.gemini" + DOCKER_ARGS+=("-v" "$HOME/.gemini:/home/deva/.gemini") + fi + ;; + api-key|gemini-api-key) + if [ -z "${GEMINI_API_KEY:-}" ]; then + auth_error "GEMINI_API_KEY not set for --auth-with api-key" \ + "Set: export GEMINI_API_KEY=your_key" + fi + + AUTH_DETAILS="api-key (GEMINI_API_KEY)" + DOCKER_ARGS+=("-e" "GEMINI_API_KEY=$GEMINI_API_KEY") + + local gemini_config_dir + if [ -n "${CONFIG_ROOT:-}" ]; then + case "$CONFIG_ROOT" in + /*) ;; + *) auth_error "CONFIG_ROOT must be absolute path: $CONFIG_ROOT" ;; + esac + case "$CONFIG_ROOT" in + *..* | *//* | *$'\n'* | *$'\t'*) + auth_error "CONFIG_ROOT contains invalid path pattern: $CONFIG_ROOT" + ;; + esac + local xdg_config="${XDG_CONFIG_HOME:-$HOME/.config}" + case "$CONFIG_ROOT" in + "$HOME"/* | "$xdg_config"/* | /tmp/deva-*) + gemini_config_dir="$CONFIG_ROOT/gemini/.gemini" + ;; + *) + auth_error "CONFIG_ROOT must be under $HOME or XDG_CONFIG_HOME: $CONFIG_ROOT" + ;; + esac + else + gemini_config_dir="$HOME/.gemini" + fi + + mkdir -p "$gemini_config_dir" + rm -f "$gemini_config_dir/mcp-oauth-tokens-v2.json" + + local settings_file="$gemini_config_dir/settings.json" + if [ ! -f "$settings_file" ] || ! grep -q '"selectedType"' "$settings_file" 2>/dev/null; then + cat > "$settings_file" <<'EOF' +{ + "security": { + "auth": { + "selectedType": "gemini-api-key" + } + } +} +EOF + echo "Created gemini settings with API key auth: $settings_file" >&2 + else + echo "Using existing gemini settings: $settings_file" >&2 + fi + ;; + vertex) + AUTH_DETAILS="google-vertex (gcloud)" + if [ -d "$HOME/.config/gcloud" ]; then + DOCKER_ARGS+=("-v" "$HOME/.config/gcloud:/home/deva/.config/gcloud:ro") + fi + if [ -n "${GOOGLE_APPLICATION_CREDENTIALS:-}" ] && [ -f "$GOOGLE_APPLICATION_CREDENTIALS" ]; then + DOCKER_ARGS+=("-v" "$GOOGLE_APPLICATION_CREDENTIALS:$GOOGLE_APPLICATION_CREDENTIALS:ro") + DOCKER_ARGS+=("-e" "GOOGLE_APPLICATION_CREDENTIALS=$GOOGLE_APPLICATION_CREDENTIALS") + fi + if [ -n "${GOOGLE_CLOUD_PROJECT:-}" ]; then + DOCKER_ARGS+=("-e" "GOOGLE_CLOUD_PROJECT=$GOOGLE_CLOUD_PROJECT") + fi + if [ -n "${GOOGLE_CLOUD_LOCATION:-}" ]; then + DOCKER_ARGS+=("-e" "GOOGLE_CLOUD_LOCATION=$GOOGLE_CLOUD_LOCATION") + fi + ;; + compute-adc) + AUTH_DETAILS="compute-default-credentials (GCE metadata)" + ;; + credentials-file) + if [ -z "${CUSTOM_CREDENTIALS_FILE:-}" ]; then + auth_error "CUSTOM_CREDENTIALS_FILE not set for credentials-file auth" + fi + AUTH_DETAILS="credentials-file ($CUSTOM_CREDENTIALS_FILE)" + DOCKER_ARGS+=("-v" "$CUSTOM_CREDENTIALS_FILE:/home/deva/.config/gcloud/service-account-key.json:ro") + DOCKER_ARGS+=("-e" "GOOGLE_APPLICATION_CREDENTIALS=/home/deva/.config/gcloud/service-account-key.json") + echo "Using custom credentials: $CUSTOM_CREDENTIALS_FILE -> service-account-key.json" >&2 + ;; + *) + auth_error "auth method '$method' not implemented for gemini" + ;; + esac +} diff --git a/agents/shared_auth.sh b/agents/shared_auth.sh index e3b50da..38e37d4 100644 --- a/agents/shared_auth.sh +++ b/agents/shared_auth.sh @@ -50,10 +50,6 @@ validate_openai_key() { [ -n "${OPENAI_API_KEY:-}" ] } -# Detects OAuth token pattern in environment variables only -# (not for inspecting credential file contents) -# OAuth tokens: sk-ant-oat01-* -# API keys: sk-ant-api03-* is_oauth_token_pattern() { local key="$1" [[ "$key" == sk-ant-oat01-* ]] @@ -64,7 +60,11 @@ COPILOT_PROXY_PORT="$COPILOT_DEFAULT_PORT" is_process_alive() { local pid="$1" - [ -n "$pid" ] && [ -d "/proc/$pid" ] && grep -q "copilot-api" "/proc/$pid/cmdline" 2>/dev/null + [ -n "$pid" ] || return 1 + if ! kill -0 "$pid" 2>/dev/null; then + return 1 + fi + ps -p "$pid" -o args= 2>/dev/null | grep -q "copilot-api" } start_copilot_proxy() { @@ -241,46 +241,6 @@ expand_and_validate_file() { fi } -backup_credentials() { - local agent_name="$1" - local config_root="$2" - local source_file="${3:-}" # Optional: file to compare against - local backup_path="" - local creds_file="" - - case "$agent_name" in - claude) - if [ -n "$config_root" ] && [ -d "$config_root/claude/.claude" ]; then - creds_file="$config_root/claude/.claude/.credentials.json" - elif [ -d "$HOME/.claude" ]; then - creds_file="$HOME/.claude/.credentials.json" - fi - ;; - codex) - if [ -n "$config_root" ] && [ -d "$config_root/codex/.codex" ]; then - creds_file="$config_root/codex/.codex/auth.json" - elif [ -d "$HOME/.codex" ]; then - creds_file="$HOME/.codex/auth.json" - fi - ;; - esac - - if [ -n "$creds_file" ] && [ -f "$creds_file" ]; then - # Compare with source file if provided (avoid duplicate backups) - if [ -n "$source_file" ] && [ -f "$source_file" ]; then - if cmp -s "$creds_file" "$source_file"; then - echo "Credentials file identical to source, skipping backup" >&2 - return 0 - fi - fi - - backup_path="${creds_file}.backup-$(date +%Y%m%d-%H%M%S)" - echo "Backing up existing credentials: $creds_file -> $backup_path" >&2 - cp "$creds_file" "$backup_path" - echo "To restore: mv $backup_path $creds_file" >&2 - fi -} - parse_auth_args() { local agent_name="$1" shift @@ -294,6 +254,9 @@ parse_auth_args() { codex) supported_methods=(chatgpt api-key copilot) ;; + gemini) + supported_methods=(oauth api-key gemini-api-key vertex compute-adc gemini-app-oauth) + ;; *) auth_error "Unknown agent: $agent_name" ;; @@ -312,7 +275,6 @@ parse_auth_args() { auth_method="${args[$((i + 1))]}" if is_file_path "$auth_method"; then - # shellcheck disable=SC2034 CUSTOM_CREDENTIALS_FILE=$(expand_and_validate_file "$auth_method") auth_method="credentials-file" else @@ -346,11 +308,10 @@ parse_auth_args() { case "$agent_name" in claude) auth_method="claude" ;; codex) auth_method="chatgpt" ;; + gemini) auth_method="oauth" ;; esac fi - # shellcheck disable=SC2034 PARSED_AUTH_METHOD="$auth_method" - # shellcheck disable=SC2034 PARSED_REMAINING_ARGS=("${remaining_args[@]+"${remaining_args[@]}"}") } diff --git a/deva.sh b/deva.sh index dbe46ec..0e7114f 100755 --- a/deva.sh +++ b/deva.sh @@ -41,10 +41,11 @@ AGENT_EXPLICIT=false EPHEMERAL_MODE=false GLOBAL_MODE=false DEBUG_MODE=false +DRY_RUN=false usage() { cat <<'USAGE' -deva.sh - Docker-based multi-agent launcher (Claude, Codex) +deva.sh - Docker-based multi-agent launcher (Claude, Codex, Gemini) Usage: deva.sh [deva flags] [agent] [-- agent-flags] @@ -72,13 +73,15 @@ Deva flags: -p NAME, --profile NAME Select profile: base (default), rust. Pulls tag, falls back to Dockerfile. --host-net Use host networking for the agent container + --no-docker Disable auto-mount of Docker socket (default: auto-mount if present) + --dry-run Show docker command without executing (implies --debug) --verbose, --debug Print full docker command before execution -- Everything after this sentinel is passed to the agent unchanged Container Behavior (NEW in v0.8.0): Default (persistent): One container per project, reused across runs. Preserves state (npm packages, builds, etc). - Faster startup, run any agent (claude/codex). + Faster startup, run any agent (claude/codex/gemini). With --rm (ephemeral): Create new container, auto-remove after exit. Agent-specific naming for parallel runs. @@ -96,6 +99,7 @@ Examples: deva.sh # Launch claude in persistent container deva.sh claude # Same deva.sh codex # Launch codex in same container + deva.sh gemini # Launch gemini in same container deva.sh claude --rm # Ephemeral: deva-work-myapp-claude-12345 # Container management (current project) @@ -112,8 +116,9 @@ Examples: Advanced: deva.sh codex -v ~/.ssh:/home/deva/.ssh:ro -- -m gpt-5-codex - deva.sh -c ~/work-claude-home -- --trace - deva.sh --show-config # Debug configuration + deva.sh claude -- --trace --continue # Use claude-trace wrapper for request tracing + deva.sh --show-config # Debug configuration + deva.sh --no-docker claude # Disable Docker-in-Docker auto-mount USAGE } @@ -178,6 +183,29 @@ check_image() { return fi + # Smart fallback: check for available profile images locally + local available_tags="" + local original_tag="$DEVA_DOCKER_TAG" + + # Check common profile tags (prefer rust as it's a superset of base) + for tag in rust latest; do + if [ "$tag" = "$DEVA_DOCKER_TAG" ]; then + continue # Skip the one we already tried + fi + if docker image inspect "${DEVA_DOCKER_IMAGE}:${tag}" >/dev/null 2>&1; then + available_tags="${available_tags}${tag} " + fi + done + + if [ -n "$available_tags" ]; then + # Found alternative images - use the first one + local fallback_tag="${available_tags%% *}" # Get first tag + echo "Image ${DEVA_DOCKER_IMAGE}:${original_tag} not found" >&2 + echo "Using available image: ${DEVA_DOCKER_IMAGE}:${fallback_tag}" >&2 + DEVA_DOCKER_TAG="$fallback_tag" + return + fi + # Determine matching local Dockerfile for suggestions (no auto-build) local df="" case "${PROFILE:-}" in @@ -629,6 +657,12 @@ prepare_base_docker_args() { if [ -n "${LANG:-}" ]; then DOCKER_ARGS+=(-e "LANG=$LANG"); fi if [ -n "${LC_ALL:-}" ]; then DOCKER_ARGS+=(-e "LC_ALL=$LC_ALL"); fi if [ -n "${TZ:-}" ]; then DOCKER_ARGS+=(-e "TZ=$TZ"); fi + + # Auto-mount Docker socket for DinD workflows (opt-out via --no-docker or DEVA_NO_DOCKER=1) + if [ -z "${DEVA_NO_DOCKER:-}" ] && [ -S /var/run/docker.sock ]; then + DOCKER_ARGS+=(-v "/var/run/docker.sock:/var/run/docker.sock") + fi + # Fallback: detect host TZ/LANG if not set in env if ! docker_args_has_env "TZ"; then local host_tz="" @@ -790,6 +824,45 @@ should_skip_env_for_auth() { ;; esac ;; + gemini) + case "${AUTH_METHOD:-oauth}" in + oauth | gemini-app-oauth) + case "$name" in + GOOGLE_API_KEY | GEMINI_API_KEY | ANTHROPIC_API_KEY | ANTHROPIC_BASE_URL | OPENAI_API_KEY | OPENAI_BASE_URL) + return 0 + ;; + esac + ;; + api-key | gemini-api-key) + case "$name" in + GOOGLE_API_KEY | GEMINI_API_KEY) + return 0 + ;; + esac + ;; + vertex) + case "$name" in + GOOGLE_API_KEY | GEMINI_API_KEY | ANTHROPIC_API_KEY | OPENAI_API_KEY) + return 0 + ;; + esac + ;; + compute-adc) + case "$name" in + GOOGLE_API_KEY | GEMINI_API_KEY | ANTHROPIC_API_KEY | OPENAI_API_KEY) + return 0 + ;; + esac + ;; + credentials-file) + case "$name" in + GOOGLE_API_KEY | GEMINI_API_KEY | ANTHROPIC_API_KEY | OPENAI_API_KEY) + return 0 + ;; + esac + ;; + esac + ;; esac return 1 @@ -1330,6 +1403,11 @@ parse_wrapper_args() { i=$((i + 1)) continue ;; + --no-docker) + export DEVA_NO_DOCKER=1 + i=$((i + 1)) + continue + ;; --host-net) EXTRA_DOCKER_ARGS+=("--net" "host") i=$((i + 1)) @@ -1350,6 +1428,12 @@ parse_wrapper_args() { i=$((i + 1)) continue ;; + --dry-run) + DRY_RUN=true + DEBUG_MODE=true + i=$((i + 1)) + continue + ;; *) remaining+=("$arg") i=$((i + 1)) @@ -1795,7 +1879,7 @@ if [ "$CONFIG_HOME_AUTO" = true ]; then fi if [ "$CONFIG_HOME_FROM_CLI" = true ] && [ -n "$CONFIG_HOME" ]; then - if [ -d "$CONFIG_HOME/claude" ] || [ -d "$CONFIG_HOME/codex" ]; then + if [ -d "$CONFIG_HOME/claude" ] || [ -d "$CONFIG_HOME/codex" ] || [ -d "$CONFIG_HOME/gemini" ]; then CONFIG_ROOT="$CONFIG_HOME" CONFIG_HOME="" CONFIG_HOME_AUTO=false @@ -1803,9 +1887,9 @@ if [ "$CONFIG_HOME_FROM_CLI" = true ] && [ -n "$CONFIG_HOME" ]; then fi autolink_legacy_into_deva_root() { - [ "$AUTOLINK" = true ] || return - [ "$CONFIG_HOME_FROM_CLI" = false ] || return - [ -n "${CONFIG_ROOT:-}" ] || return + [ "$AUTOLINK" = true ] || return 0 + [ "$CONFIG_HOME_FROM_CLI" = false ] || return 0 + [ -n "${CONFIG_ROOT:-}" ] || return 0 [ -d "$CONFIG_ROOT" ] || mkdir -p "$CONFIG_ROOT" if [ -d "$HOME/.claude" ] || [ -f "$HOME/.claude.json" ]; then @@ -1832,7 +1916,18 @@ autolink_legacy_into_deva_root() { fi fi if [ -d "$CONFIG_ROOT" ]; then - [ -d "$CONFIG_ROOT/codex/.codex" ] || mkdir -p "$CONFIG_ROOT/codex/.codex" + [ -d "$CONFIG_ROOT/codex/.codex" ] || [ -L "$CONFIG_ROOT/codex/.codex" ] || mkdir -p "$CONFIG_ROOT/codex/.codex" + fi + + if [ -d "$HOME/.gemini" ]; then + [ -d "$CONFIG_ROOT/gemini" ] || mkdir -p "$CONFIG_ROOT/gemini" + if [ ! -e "$CONFIG_ROOT/gemini/.gemini" ] && [ ! -L "$CONFIG_ROOT/gemini/.gemini" ]; then + ln -s "$HOME/.gemini" "$CONFIG_ROOT/gemini/.gemini" + echo "autolink: ~/.gemini -> $CONFIG_ROOT/gemini/.gemini" >&2 + fi + fi + if [ -d "$CONFIG_ROOT" ]; then + [ -d "$CONFIG_ROOT/gemini/.gemini" ] || [ -L "$CONFIG_ROOT/gemini/.gemini" ] || mkdir -p "$CONFIG_ROOT/gemini/.gemini" fi } @@ -1845,6 +1940,9 @@ if [ -n "$CONFIG_HOME" ]; then if [ "$ACTIVE_AGENT" = "claude" ] && [ ! -f "$CONFIG_HOME/.claude.json" ]; then echo '{}' >"$CONFIG_HOME/.claude.json" fi + if [ "$ACTIVE_AGENT" = "gemini" ] && [ ! -f "$CONFIG_HOME/settings.json" ]; then + echo '{}' >"$CONFIG_HOME/settings.json" + fi fi if dangerous_directory; then @@ -1946,7 +2044,10 @@ DOCKER_ARGS+=(-e "DEVA_WORKSPACE=$(pwd)") DOCKER_ARGS+=(-e "DEVA_EPHEMERAL=${EPHEMERAL_MODE}") # Centralized mounting logic based on auth method -if [ -n "${AUTH_METHOD:-}" ]; then +# If --config-home is set, use it exclusively and skip auth-based mounting +if [ -n "$CONFIG_HOME" ]; then + mount_config_home +elif [ -n "${AUTH_METHOD:-}" ]; then is_default_auth=false if [ "$ACTIVE_AGENT" = "claude" ] && [ "$AUTH_METHOD" = "claude" ]; then is_default_auth=true @@ -2116,6 +2217,10 @@ if [ "$DEBUG_MODE" = true ]; then echo "" >&2 fi +if [ "$DRY_RUN" = true ]; then + exit 0 +fi + if [ "$EPHEMERAL_MODE" = false ]; then # Check if container is running container_action="attach" diff --git a/docker-entrypoint.sh b/docker-entrypoint.sh index a8d791d..fadc268 100644 --- a/docker-entrypoint.sh +++ b/docker-entrypoint.sh @@ -202,8 +202,24 @@ setup_nonroot_user() { if [ "$DEVA_UID" != "$current_uid" ]; then [ "$VERBOSE" = "true" ] && echo "[entrypoint] updating $DEVA_USER UID: $current_uid -> $DEVA_UID" - usermod -u "$DEVA_UID" -g "$DEVA_GID" "$DEVA_USER" - chown -R "$DEVA_UID:$DEVA_GID" "$DEVA_HOME" 2>/dev/null || true + # usermod may fail with rc=12 when it can't chown home directory (mounted volumes) + # The UID change itself usually succeeds even when chown fails + if ! usermod -u "$DEVA_UID" -g "$DEVA_GID" "$DEVA_USER" 2>/dev/null; then + # Verify what UID we actually got + local actual_uid + actual_uid=$(id -u "$DEVA_USER" 2>/dev/null) + if [ -z "$actual_uid" ]; then + echo "[entrypoint] ERROR: cannot determine UID for $DEVA_USER" >&2 + exit 1 + fi + if [ "$actual_uid" != "$DEVA_UID" ]; then + echo "[entrypoint] WARNING: UID change failed ($DEVA_USER is UID $actual_uid, wanted $DEVA_UID)" >&2 + # Adapt to reality so subsequent operations use correct UID + DEVA_UID="$actual_uid" + fi + fi + # Only chown files owned by container, skip mounted volumes + find "$DEVA_HOME" -maxdepth 1 ! -type l -user root -exec chown "$DEVA_UID:$DEVA_GID" {} \; 2>/dev/null || true fi chmod 755 /root 2>/dev/null || true @@ -218,6 +234,12 @@ fix_rust_permissions() { [ -d "$ch" ] || mkdir -p "$ch" } +fix_docker_socket_permissions() { + if [ -S /var/run/docker.sock ]; then + chmod 666 /var/run/docker.sock 2>/dev/null || true + fi +} + build_gosu_env_cmd() { local user="$1" shift @@ -269,6 +291,7 @@ main() { ensure_agent_binaries setup_nonroot_user fix_rust_permissions + fix_docker_socket_permissions if [ $# -eq 0 ]; then if [ "$DEVA_AGENT" = "codex" ]; then @@ -284,18 +307,44 @@ main() { if [ "$DEVA_AGENT" = "claude" ]; then if [ "$cmd" = "claude" ] || [ "$cmd" = "$(command -v claude 2>/dev/null)" ]; then - build_gosu_env_cmd "$DEVA_USER" "$cmd" "$@" --dangerously-skip-permissions + # Add --dangerously-skip-permissions if not already present + local has_dsp=false + for arg in "$@"; do + if [ "$arg" = "--dangerously-skip-permissions" ]; then + has_dsp=true + break + fi + done + if [ "$has_dsp" = true ]; then + build_gosu_env_cmd "$DEVA_USER" "$cmd" "$@" + else + build_gosu_env_cmd "$DEVA_USER" "$cmd" "$@" --dangerously-skip-permissions + fi elif [ "$cmd" = "claude-trace" ]; then - args=("$@") - new_args=() - for arg in "${args[@]}"; do - if [ "$arg" = "--run-with" ]; then - new_args+=("--run-with" "--dangerously-skip-permissions") - else - new_args+=("$arg") + # claude-trace: ensure --dangerously-skip-permissions follows --run-with + local has_dsp=false + for arg in "$@"; do + if [ "$arg" = "--dangerously-skip-permissions" ]; then + has_dsp=true + break fi done - build_gosu_env_cmd "$DEVA_USER" "$cmd" "${new_args[@]}" + if [ "$has_dsp" = true ]; then + # Already has --dangerously-skip-permissions, pass through + build_gosu_env_cmd "$DEVA_USER" "$cmd" "$@" + else + # Insert --dangerously-skip-permissions after --run-with + args=("$@") + new_args=() + for arg in "${args[@]}"; do + if [ "$arg" = "--run-with" ]; then + new_args+=("--run-with" "--dangerously-skip-permissions") + else + new_args+=("$arg") + fi + done + build_gosu_env_cmd "$DEVA_USER" "$cmd" "${new_args[@]}" + fi else build_gosu_env_cmd "$DEVA_USER" "$cmd" "$@" fi diff --git a/docs/devlog/20260108-deva-bridge-tmux.org b/docs/devlog/20260108-deva-bridge-tmux.org new file mode 100644 index 0000000..8ce256e --- /dev/null +++ b/docs/devlog/20260108-deva-bridge-tmux.org @@ -0,0 +1,308 @@ +* [2026-01-08] Dev Log: deva-bridge-tmux :FEATURE:BRIDGE: + +** Context +Deva runs AI agents (claude, codex, gemini) in isolated Docker containers. Need tmux integration for container to control host tmux sessions. + +** Why +Unix socket mount across macOS<->Linux kernel boundary fails - socket file visible but connect() returns ECONNREFUSED due to kernel incompatibility. Direct socket mount not viable. + +** What +- TCP bridge for tmux socket communication @deva +- Build tmux 3.6a from source with SHA256 verification @Dockerfile +- Host-side TCP proxy: deva-bridge-tmux-host @scripts @privileged +- Container-side Unix socket proxy: deva-bridge-tmux @scripts @privileged +- Documentation of security implications @AGENTS.md +=BREAKING= Introduces privileged host bridge (container escape vector) +=SECURITY= Container gains full tmux control (send-keys, run-shell, scrollback) + +** How +*** Steps +1. Install socat in Dockerfile base layer +2. Install libevent-dev, libncurses-dev, bison for tmux build dependencies +3. Download, verify SHA256, build tmux 3.6a from source +4. Create deva-bridge-tmux-host (macOS host-side TCP listener) +5. Create deva-bridge-tmux (container-side Unix socket forwarder) +6. Copy bridge scripts to /usr/local/bin with 755 permissions +7. Document bridge architecture and security model in AGENTS.md + +*** Commands +#+BEGIN_SRC bash +# Build tmux from source +cd /tmp +wget https://github.com/tmux/tmux/releases/download/3.6a/tmux-3.6a.tar.gz +echo "b6d8d9c76585db8ef5fa00d4931902fa4b8cbe8166f528f44fc403961a3f3759 tmux-3.6a.tar.gz" | sha256sum -c - +tar -xzf tmux-3.6a.tar.gz +cd tmux-3.6a +./configure --prefix=/usr/local +make -j$(nproc) +make install +tmux -V # tmux 3.6a + +# Host-side bridge (macOS) +./scripts/deva-bridge-tmux-host +# Listen: 127.0.0.1:41555 -> /private/tmp/tmux-$(id -u)/default + +# Container-side bridge +deva-bridge-tmux +# TCP: host.docker.internal:41555 -> Unix: /tmp/host-tmux.sock + +# Usage from container +tmux -S /tmp/host-tmux.sock list-sessions +tmux -S /tmp/host-tmux.sock attach -t WIP +alias htmux='tmux -S /tmp/host-tmux.sock' +#+END_SRC + +*** Diffs +#+BEGIN_SRC diff +--- a/Dockerfile ++++ b/Dockerfile +@@ -30,7 +30,8 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ + openssh-client rsync \ + shellcheck bat fd-find silversearcher-ag \ + vim \ +- procps psmisc zsh ++ procps psmisc zsh socat \ ++ libevent-dev libncurses-dev bison + ++# Install tmux from source (protocol version must match host for socket bridge) ++# Same major.minor usually works; exact match is pragmatic, not required. ++ARG TMUX_VERSION=3.6a ++ARG TMUX_SHA256=b6d8d9c76585db8ef5fa00d4931902fa4b8cbe8166f528f44fc403961a3f3759 ++RUN --mount=type=cache,target=/tmp/tmux-cache,sharing=locked \ ++ set -eu && \ ++ cd /tmp/tmux-cache && \ ++ TARBALL="tmux-${TMUX_VERSION}.tar.gz" && \ ++ wget -q "https://github.com/tmux/tmux/releases/download/${TMUX_VERSION}/${TARBALL}" && \ ++ echo "${TMUX_SHA256} ${TARBALL}" | sha256sum -c - && \ ++ tar -xzf "${TARBALL}" && \ ++ cd "tmux-${TMUX_VERSION}" && \ ++ ./configure --prefix=/usr/local && \ ++ make -j"$(nproc)" && \ ++ make install && \ ++ rm -rf /tmp/tmux-cache/tmux-* && \ ++ hash -r && \ ++ tmux -V + ++COPY scripts/deva-bridge-tmux /usr/local/bin/deva-bridge-tmux ++RUN chmod +x /usr/local/bin/deva-bridge-tmux +#+END_SRC + +*** Architecture Notes :BRIDGE:SECURITY: +| Component | Location | Protocol | Socket Path | Risk Level | +|-----------+----------+----------+-------------+------------| +| deva-bridge-tmux-host | macOS host | TCP listener | /private/tmp/tmux-$(id -u)/default | Medium (localhost only) | +| socat TCP proxy | host | TCP:41555 | host.docker.internal:41555 | High (container escape) | +| socat Unix forwarder | container | Unix socket | /tmp/host-tmux.sock | High (full tmux control) | + +*** API Contract :BRIDGE: +**** Host-side Environment Variables +| Variable | Default | Purpose | +|----------+---------+---------| +| DEVA_BRIDGE_BIND | 127.0.0.1 | TCP bind address (0.0.0.0 dangerous) | +| DEVA_BRIDGE_PORT | 41555 | TCP port for bridge | +| DEVA_BRIDGE_SOCKET | auto-detect | Host tmux socket path | + +**** Container-side Environment Variables +| Variable | Default | Purpose | +|----------+---------+---------| +| DEVA_BRIDGE_HOST | host.docker.internal | Docker Desktop special hostname | +| DEVA_BRIDGE_PORT | 41555 | TCP port (must match host) | +| DEVA_BRIDGE_SOCK | /tmp/host-tmux.sock | Container-local Unix socket | + +**** Bridge Lifecycle +1. Host: deva-bridge-tmux-host starts socat TCP-LISTEN +2. Container: deva-bridge-tmux connects via TCP, creates Unix socket +3. Container: tmux client uses -S /tmp/host-tmux.sock +4. Packets: tmux client -> Unix -> socat -> TCP -> socat -> Unix -> tmux server + +** Result +*** Outcome +Container tmux client successfully connects to host tmux server via TCP bridge. Verified with: +- tmux -S /tmp/host-tmux.sock list-sessions (lists host sessions) +- tmux -S /tmp/host-tmux.sock attach -t WIP (attaches to host session) +- No permission errors, no protocol version mismatches + +*** Tests +- Manual: Start host bridge, start container bridge, list sessions +- Manual: Attach to existing host session from container +- Manual: Send-keys from container to host session +- Manual: Verify scrollback access from container + +*** Observability +No metrics - this is a development tool. Errors logged to stderr by both bridge scripts. + +** Debugging Session: Container Permission Issues + +*** Initial Symptoms +After adding deva-bridge-tmux script, new containers failed to start with two errors: +#+BEGIN_EXAMPLE +usermod: Failed to change ownership of the home directory +env: 'claude': Permission denied +#+END_EXAMPLE + +*** Investigation Timeline (2026-01-08) +**** Hypothesis 1: Image configuration broken +- Checked Dockerfile: ENTRYPOINT correct, runs as root, CMD ["claude"] +- Result: Configuration valid + +**** Hypothesis 2: Entrypoint script broken +- Test: docker run --rm ghcr.io/thevibeworks/deva:latest /bin/bash -c 'whoami' +- Result: Works, prints 'deva' (non-root user switching works) +- Test: docker run --rm --entrypoint /usr/local/bin/docker-entrypoint.sh ghcr.io/thevibeworks/deva:latest +- Result: Works, starts normally +- Conclusion: Entrypoint script itself is functional + +**** Hypothesis 3: Gosu user switching broken +- Test: docker run --rm ghcr.io/thevibeworks/deva:latest gosu deva whoami +- Result: Works, prints 'deva' +- Test: docker run --rm ghcr.io/thevibeworks/deva:latest gosu deva claude --version +- Result: Works when bypassing entrypoint +- Conclusion: gosu works, claude binary accessible + +**** Hypothesis 4: Stale cached containers +- Observation: deva.sh reuses persistent containers (docker start) +- Containers created from old image versions have stale filesystem state +- Test: Stop and remove old containers, docker run fresh +- Result: Fresh containers work + +**** Hypothesis 5: Stale Docker build cache +- Observation: BuildKit caches COPY layers +- Old cache preserved incorrect file permissions +- Test: docker build --no-cache +- Result: Fixes permission issues + +**** Root Cause: Script file permissions +#+BEGIN_SRC bash +# Before fix +-rwx--x--x deva-bridge-tmux # 711 (no read permission) + +# After fix +-rwxr-xr-x deva-bridge-tmux # 755 (read+execute) +#+END_SRC + +Shell scripts need read permission for interpreter to parse. Execute-only (711) fails with "Permission denied" when shell tries to read script contents. + +*** Resolution Steps +1. chmod 755 scripts/deva-bridge-tmux scripts/deva-bridge-tmux-host +2. docker build --no-cache -f Dockerfile -t ghcr.io/thevibeworks/deva:latest . +3. docker ps -a --filter "name=deva-" --format "{{.Names}}" | xargs docker rm -f +4. Test with fresh container: docker run --rm ghcr.io/thevibeworks/deva:latest claude --version + +*** Lessons Learned +| Issue | Learning | Prevention | +|-------+----------+------------| +| Persistent containers | deva.sh docker start reuses old state | Always test fresh containers after image rebuild | +| Docker layer cache | COPY preserves source file permissions | Use --no-cache when fixing permission issues | +| Script permissions | Shells need read+execute (755), not just execute (711) | Verify with stat -f "%Sp %N" before COPY | +| usermod errors | "Failed to change ownership" is FATAL under set -e, needed explicit error handling | Suppress usermod stderr but verify UID change succeeded | +| Testing methodology | docker run vs docker exec vs docker start have different failure modes | Test all three when debugging container startup | + +*** Known Issues +**** usermod error about home directory +#+BEGIN_EXAMPLE +usermod: Failed to change ownership of the home directory +#+END_EXAMPLE + +usermod internally tries to chown home directory when changing UID. With mounted volumes, this fails because container can't change ownership of host-owned files. + +*Problem*: Under `set -e`, this error aborts the entrypoint script, preventing container startup. + +*Fix applied to docker-entrypoint.sh*: +#+BEGIN_SRC bash +# usermod may fail with rc=12 when it can't chown home directory (mounted volumes) +# The UID change itself usually succeeds even when chown fails +if ! usermod -u "$DEVA_UID" -g "$DEVA_GID" "$DEVA_USER" 2>/dev/null; then + local actual_uid + actual_uid=$(id -u "$DEVA_USER" 2>/dev/null) + # Fail hard if we can't even determine the UID + if [ -z "$actual_uid" ]; then + echo "[entrypoint] ERROR: cannot determine UID for $DEVA_USER" >&2 + exit 1 + fi + # UID mismatch: warn and adapt (degrade gracefully for dev containers) + if [ "$actual_uid" != "$DEVA_UID" ]; then + echo "[entrypoint] WARNING: UID change failed ..." >&2 + DEVA_UID="$actual_uid" # Use actual UID for subsequent ops + fi +fi +#+END_SRC + +*Policy decision*: Empty UID = hard fail (bug). UID mismatch = warn + adapt (dev container should try to work). + +Key insight: usermod rc=12 usually means UID change succeeded but post-change chown failed on mounted volumes. We verify actual UID, fail if unknown, adapt if mismatched. + +** Decisions +| Decision | Alternatives | Rationale | DRI | Timestamp (UTC) | +|----------+--------------+-----------+-----+-----------------| +| TCP bridge via socat | 1) Direct socket mount (fails macOS->Linux) 2) SSH tunneling (complex) 3) tmux -CC protocol (no scrollback) | socat: simple, reliable, minimal dependencies | @deva | 2026-01-08T00:00:00Z | +| Build tmux from source | 1) Use distro tmux (version mismatch likely) 2) Static binary (no distro available) | Exact version match guarantees protocol compatibility | @deva | 2026-01-08T00:00:00Z | +| Version 3.6a | 1) Latest 3.5 (older) 2) Latest 3.6b (not released) | Host runs macOS with tmux 3.6a from Homebrew | @deva | 2026-01-08T00:00:00Z | +| SHA256 verification | Trust download | Security best practice for build-time downloads | @deva | 2026-01-08T00:00:00Z | +| 127.0.0.1 bind default | 0.0.0.0 bind | Minimize attack surface - localhost-only by default | @deva | 2026-01-08T00:00:00Z | +| Script permissions 755 | 711 (execute-only) | Shell interpreter needs read permission to parse script | @deva | 2026-01-08T01:30:00Z | + +** Risks :SECURITY: +| Risk | Mitigation | Owner | Status | +|------+------------+-------+--------| +| Container escape via tmux send-keys | Document clearly, bind=127.0.0.1 default, user must explicitly enable | @deva | Documented | +| Scrollback leakage (sensitive data) | No automatic mitigation - user awareness required | @deva | Documented | +| Bind=0.0.0.0 exposes to network | Default=127.0.0.1, warning in script output, docs emphasize danger | @deva | Mitigated | +| Protocol version mismatch | Build exact tmux version, SHA256 verify, document version requirement | @deva | Mitigated | +| Port conflict (41555 in use) | User can override via DEVA_BRIDGE_PORT | @deva | Configurable | +| Stale Docker layer cache | Document --no-cache requirement after permission fixes | @deva | Documented | +| Persistent container state | Document container cleanup requirement after image rebuild | @deva | Documented | + +** Rollback +*** Trigger +- tmux protocol incompatibility errors +- Container escape incidents attributed to tmux bridge +- Performance degradation from TCP overhead + +*** Procedure +1. Stop host bridge: pkill -f deva-bridge-tmux-host +2. Stop container bridge: docker exec pkill -f deva-bridge-tmux +3. Remove bridge scripts from Dockerfile: + #+BEGIN_SRC bash + git revert + docker build --no-cache -f Dockerfile -t ghcr.io/thevibeworks/deva:latest . + #+END_SRC +4. Data impact: none (bridge is stateless, no data persistence) + +** Links +- Bridges documentation: /Users/eric/wrk/src/github.com/claude-code/WIP/260107_deva-tmux-bridge/worktree/deva/AGENTS.md +- Dockerfile: /Users/eric/wrk/src/github.com/claude-code/WIP/260107_deva-tmux-bridge/worktree/deva/Dockerfile +- Host bridge: /Users/eric/wrk/src/github.com/claude-code/WIP/260107_deva-tmux-bridge/worktree/deva/scripts/deva-bridge-tmux-host +- Container bridge: /Users/eric/wrk/src/github.com/claude-code/WIP/260107_deva-tmux-bridge/worktree/deva/scripts/deva-bridge-tmux +- tmux releases: https://github.com/tmux/tmux/releases +- socat docs: http://www.dest-unreach.org/socat/doc/socat.html + +** Notes +*** Assumptions +- Host runs macOS with Docker Desktop (provides host.docker.internal) +- Host tmux server already running with active session +- User understands security implications of privileged bridge +- Same tmux major.minor version sufficient for protocol compatibility +- Container has network access to host.docker.internal + +*** Constraints +- TCP overhead adds ~1-2ms latency vs direct Unix socket +- Bridge scripts require socat installed on both host and container +- SHA256 hash hardcoded - must update for different tmux versions +- Protocol compatibility not guaranteed across major tmux versions + +*** Open Questions +- Q: Support Linux host without host.docker.internal? + A: Use --bind 0.0.0.0 and DEVA_BRIDGE_HOST= +- Q: Support multiple tmux servers? + A: Override DEVA_BRIDGE_SOCKET to specific socket path +- Q: Performance impact of TCP vs Unix socket? + A: Negligible for interactive use (<1ms added latency) + +*** Gotchas +- Shell scripts with 711 permissions (x without r) fail with "Permission denied" +- Docker layer cache preserves old file permissions from COPY +- Persistent containers (docker start) retain old filesystem state after image rebuild +- usermod errors about home directory are FATAL under set -e - need explicit error handling +- host.docker.internal only works on Docker Desktop (macOS/Windows), not Linux +- tmux -S path MUST be absolute, relative paths fail silently +- chmod +x only adds execute bit, doesn't help if umask removed read - use explicit 755 diff --git a/docs/devlog/devlog-260108-deva-lsp-support.org b/docs/devlog/devlog-260108-deva-lsp-support.org new file mode 100644 index 0000000..19a5882 --- /dev/null +++ b/docs/devlog/devlog-260108-deva-lsp-support.org @@ -0,0 +1,480 @@ +* [2026-01-08] Dev Log: LSP Support for Deva Containers :OPS: + +** Context +Claude Code LSP plugin system requires language server binaries in container PATH + +** Why +Claude Code officially supports LSP (via plugin system) for code intelligence (go-to-definition, find-references, hover, diagnostics). Current deva containers lack LSP binaries and plugin installation strategy. + +** What +- Design LSP installation strategy for deva containers +- Document plugin-binary architecture trade-offs +- Create implementation plan for deva-lsp-setup script + +** Problem Space + +*** LSP Components (Two-Part Architecture) +Claude Code LSP requires TWO components per language: + +1. LSP plugin - Claude Code integration layer + - Installed via: =claude plugin install = + - State persisted: =~/.claude/= (mounted from host) + - Configures: connection protocol, launch command, capabilities + +2. Language server binary - Actual LSP implementation + - Must be in: =$PATH= (inside container) + - Installation varies: npm, pip, cargo, apt, go install + - Size varies: pyright ~50MB, jdtls ~200MB + +*** Container Complexity +| Challenge | Impact | +|-----------+--------| +| Image build vs runtime | Binaries at build โ†’ bloat; runtime โ†’ latency | +| Plugin state persistence | =~/.claude/= mounted from host, survives container restarts | +| Multi-image variants | deva:latest, deva:rust need different LSP stacks | +| Binary size | jdtls (Java) ~200MB, impacts push/pull time | + +*** Available Official LSP Plugins + +| Language | Plugin | Binary | Install Command | Size | +|------------+-------------------+--------------------------+--------------------------------------------+--------| +| Python | pyright-lsp | pyright-langserver | =pip install pyright= OR =npm i -g pyright= | ~50MB | +| TypeScript | typescript-lsp | typescript-language-server | =npm i -g typescript-language-server typescript= | ~40MB | +| Rust | rust-analyzer-lsp | rust-analyzer | =rustup component add rust-analyzer= | ~30MB | +| Go | gopls-lsp | gopls | =go install golang.org/x/tools/gopls@latest= | ~20MB | +| C/C++ | clangd-lsp | clangd | =apt install clangd= | ~60MB | +| C# | csharp-lsp | csharp-ls | =dotnet tool install -g csharp-ls= | ~40MB | +| Java | jdtls-lsp | jdtls | Complex setup (Eclipse JDT.LS) | ~200MB | +| Lua | lua-lsp | lua-language-server | Platform-specific download | ~10MB | +| PHP | php-lsp | intelephense | =npm i -g intelephense= | ~30MB | +| Swift | swift-lsp | sourcekit-lsp | Bundled with Swift toolchain | N/A | + +*** Trade-Off Analysis + +**** Option A: On-Demand Installation (Recommended) +#+BEGIN_SRC +Pros: +- Lean base image (~800MB vs ~1.2GB with all LSPs) +- User installs only needed languages +- Clear separation of concerns + +Cons: +- First-time setup latency (~30s-2min per language) +- User must understand LSP architecture +- Requires documentation + +Implementation: +- Create /usr/local/bin/deva-lsp-setup script +- Support: python, typescript, rust, go, cpp +- Handle both binary + plugin installation +- Idempotent (skip if already installed) +#+END_SRC + +**** Option B: Pre-Baked Images +#+BEGIN_SRC +Pros: +- Zero setup, instant LSP +- Better UX for language-specific images + +Cons: +- Image proliferation (deva:python-lsp, deva:ts-lsp, etc.) +- Harder to maintain +- Larger storage footprint + +Implementation: +- Extend Dockerfile with language-specific stages +- Build matrix: deva:{lang}-lsp tags +- Pre-install plugin at build time (NO - plugin state in ~/.claude) +#+END_SRC + +**** Option C: Auto-Detection at Entrypoint +#+BEGIN_SRC +Pros: +- "Magic" UX - LSP just works +- No user intervention + +Cons: +- Startup latency on every container create +- False positives (e.g., package.json in Python project) +- Violates principle of least surprise + +Implementation: +- docker-entrypoint.sh scans for language markers +- Installs LSP binaries silently +- Suggests plugin installation (can't auto-install - needs host ~/.claude) +#+END_SRC + +** Proposed Solution: Option A (On-Demand deva-lsp-setup) + +*** Design: deva-lsp-setup Script + +#+BEGIN_SRC bash +#!/bin/sh +# deva-lsp-setup - Install LSP support for Claude Code +# Usage: deva-lsp-setup [--plugin-only] [--binary-only] +# +# Languages: python, typescript, rust, go, cpp, all +# Flags: +# --plugin-only Skip binary, just install Claude plugin +# --binary-only Skip plugin, just install language server binary +# +# Examples: +# deva-lsp-setup python # Install pyright + pyright-lsp plugin +# deva-lsp-setup typescript # Install ts-language-server + typescript-lsp plugin +# deva-lsp-setup rust --binary-only # Just install rust-analyzer (plugin from host) +# deva-lsp-setup all # Install all common LSPs +#+END_SRC + +*** Implementation Strategy + +**** Script Location +- Path: =/usr/local/bin/deva-lsp-setup= +- Copy in Dockerfile: =COPY scripts/deva-lsp-setup /usr/local/bin/= +- Permissions: =chmod +x= in Dockerfile +- Follows pattern from: =scripts/deva-bridge-tmux= + +**** Supported Languages (Phase 1) +Top 5 languages by deva usage (inferred from base image stack): + +1. =python= โ†’ pyright (npm) + pyright-lsp plugin +2. =typescript= โ†’ typescript-language-server + typescript (npm) + typescript-lsp plugin +3. =rust= โ†’ rust-analyzer (rustup) + rust-analyzer-lsp plugin +4. =go= โ†’ gopls (go install) + gopls-lsp plugin +5. =cpp= โ†’ clangd (apt) + clangd-lsp plugin + +**** Binary Installation Logic + +#+BEGIN_SRC bash +# Python (pyright via npm - faster than pip) +npm install -g pyright + +# TypeScript +npm install -g typescript-language-server typescript + +# Rust (rust-analyzer already in deva:rust - verify presence) +if ! command -v rust-analyzer >/dev/null 2>&1; then + rustup component add rust-analyzer +fi + +# Go +go install golang.org/x/tools/gopls@latest + +# C/C++ (requires sudo) +sudo apt-get update && sudo apt-get install -y clangd +#+END_SRC + +**** Plugin Installation Logic + +#+BEGIN_SRC bash +# Check if plugin already installed +if claude plugin list | grep -q "$plugin_name"; then + echo "Plugin $plugin_name already installed, skipping" + return 0 +fi + +# Install plugin +claude plugin install "$plugin_name" + +# Verify installation +claude plugin list | grep "$plugin_name" || { + echo "ERROR: Plugin installation failed" + return 1 +} +#+END_SRC + +**** Error Handling + +#+BEGIN_SRC +# Binary installation failures +- npm install failure โ†’ check network, disk space +- sudo apt failure โ†’ inform user (deva user has NOPASSWD sudo) +- go install failure โ†’ verify GOPATH/bin in PATH + +# Plugin installation failures +- Claude not authenticated โ†’ instruct: run 'claude' first +- Network timeout โ†’ retry logic (3 attempts) +- Plugin not found โ†’ verify plugin name, check Claude version +#+END_SRC + +**** Dockerfile Integration + +#+BEGIN_SRC dockerfile +# In final stage (after USER deva) +COPY scripts/deva-lsp-setup /usr/local/bin/deva-lsp-setup + +RUN chmod +x /usr/local/bin/deva-lsp-setup && \ + chmod -R +x /usr/local/bin/scripts || true +#+END_SRC + +*** Usage Workflow + +#+BEGIN_SRC bash +# User enters deva container +deva.sh --yolo + +# Inside container - install Python LSP +deva-lsp-setup python + +# Output: +# [deva-lsp-setup] Installing Python LSP support... +# [1/2] Installing pyright binary (npm)... +# [2/2] Installing pyright-lsp plugin... +# SUCCESS: Python LSP ready. Restart Claude Code if running. + +# Verify +which pyright # /home/deva/.npm-global/bin/pyright +claude plugin list | grep pyright # pyright-lsp + +# Use in Claude Code +claude -p "Refactor main.py using LSP go-to-definition" +#+END_SRC + +*** Documentation Requirements + +**** AGENTS.md Updates +New section after "Bridges (privileged)": + +#+BEGIN_SRC markdown +## LSP Support + +Claude Code officially supports Language Server Protocol (LSP) for code intelligence (go-to-definition, find-references, hover, diagnostics, auto-complete). + +### Architecture + +LSP requires TWO components: +1. LSP plugin (Claude Code integration) - installed via `claude plugin install` +2. Language server binary (actual LSP) - must be in PATH + +### Installation + +Inside deva container: +```bash +# Install LSP for specific language +deva-lsp-setup python # pyright + pyright-lsp plugin +deva-lsp-setup typescript # typescript-language-server + typescript-lsp plugin +deva-lsp-setup rust # rust-analyzer + rust-analyzer-lsp plugin +deva-lsp-setup go # gopls + gopls-lsp plugin +deva-lsp-setup cpp # clangd + clangd-lsp plugin + +# Install all common LSPs +deva-lsp-setup all + +# Install only binary (plugin from host ~/.claude) +deva-lsp-setup python --binary-only + +# Install only plugin (binary already available) +deva-lsp-setup python --plugin-only +``` + +### Persistence + +- Plugin state: `~/.claude/` (mounted from host, survives container restarts) +- Binaries: Container filesystem (reinstall after image rebuild) + +### Verification + +```bash +# Check binary +which pyright # Should show path in container + +# Check plugin +claude plugin list | grep pyright-lsp + +# Test LSP in Claude Code +claude -p "Use LSP to find all references to foo() in main.py" +``` + +### Supported Languages + +| Language | Binary | Plugin | Install | +|------------|-------------------|-------------------|---------| +| Python | pyright | pyright-lsp | `deva-lsp-setup python` | +| TypeScript | typescript-language-server | typescript-lsp | `deva-lsp-setup typescript` | +| Rust | rust-analyzer | rust-analyzer-lsp | `deva-lsp-setup rust` | +| Go | gopls | gopls-lsp | `deva-lsp-setup go` | +| C/C++ | clangd | clangd-lsp | `deva-lsp-setup cpp` | + +See: https://code.claude.com/docs/en/discover-plugins for all available LSP plugins. +#+END_SRC + +**** README.md Updates +Add to "Development Tools Included" section: + +#+BEGIN_SRC markdown +**LSP Support**: Use `deva-lsp-setup ` to install Language Server Protocol support for code intelligence (go-to-definition, find-references, hover). Supported: Python (pyright), TypeScript, Rust, Go, C/C++. +#+END_SRC + +** Security Considerations + +*** Privilege Requirements +| Operation | Privilege | Mitigation | +|-----------+-----------+------------| +| npm install -g | deva user | $HOME/.npm-global (user-owned) | +| pip install | deva user | --user flag OR uv | +| go install | deva user | $GOPATH/bin (user-owned) | +| apt install clangd | sudo | deva user has NOPASSWD sudo in container | +| rustup component add | deva user | RUSTUP_HOME=/opt/rustup (deva-owned in deva:rust) | + +*** Network Access +- npm/pip/go install require internet +- Downloads from: npmjs.com, pypi.org, go.dev, github.com +- No signature verification (rely on package manager defaults) +- LSP binaries run user code โ†’ already trusted in deva threat model + +*** Plugin State Isolation +- Plugins stored: =~/.claude/= (host-mounted) +- Container cannot modify host files outside mount +- Plugin config persists across container recreations +- Uninstall: =claude plugin uninstall = (from host OR container) + +** Open Questions + +*** Auto-Installation Triggers +Q: Should we auto-install LSP on first project open? +A: NO - explicit user action required. Violates least surprise principle. + +*** Plugin State Management +Q: How to handle plugin state across container recreations? +A: =~/.claude/= is host-mounted โ†’ plugins persist. Binaries must be reinstalled (or pre-baked in image for common stacks). + +*** Pre-Installation in deva:rust +Q: Should deva:rust pre-install rust-analyzer-lsp plugin? +A: NO - plugins live in host =~/.claude/=, can't pre-install at build time. But rust-analyzer BINARY is already in deva:rust (line 48 of Dockerfile.rust). + +*** Version Pinning +Q: Should we pin LSP binary versions? +A: Phase 1: =@latest= for simplicity. Phase 2: ARG-based pinning if version drift causes issues. Use container image tags for reproducibility. + +*** Multi-Toolchain Support +Q: How to handle rust-analyzer for multiple Rust toolchains (stable, nightly)? +A: rust-analyzer is toolchain-agnostic proxy. Single binary works with rustup-managed toolchains. Current deva:rust defaults to stable. + +*** Image Size Impact +Q: Should we pre-install common LSPs to reduce first-time latency? +A: NO for base deva:latest. MAYBE for specialized images (deva:python, deva:rust) if user feedback indicates UX pain. Current tradeoff: lean base image (800MB) vs 30s-2min first-time setup per language. + +** Implementation Plan + +*** Phase 1: Core Script (Week 1) +1. Create =scripts/deva-lsp-setup= - POSIX shell, following =deva-bridge-tmux= patterns +2. Support 5 languages: python, typescript, rust, go, cpp +3. Handle binary installation (npm, apt, rustup, go install) +4. Handle plugin installation (claude plugin install) +5. Add --plugin-only, --binary-only, --help, --version flags +6. Error handling: network failures, missing dependencies, auth failures + +*** Phase 2: Dockerfile Integration (Week 1) +1. Copy script to =/usr/local/bin/= in Dockerfile +2. Set executable permissions +3. Test in both deva:latest and deva:rust images +4. Verify script works with non-root deva user + +*** Phase 3: Documentation (Week 2) +1. Add "LSP Support" section to AGENTS.md +2. Update README.md "Development Tools" section +3. Create examples in docs/examples/ (if dir exists) +4. Document troubleshooting: plugin install failures, binary not in PATH + +*** Phase 4: Validation (Week 2) +1. Test each language LSP end-to-end +2. Verify plugin persistence across container restarts +3. Test --plugin-only, --binary-only modes +4. Test 'all' language (install all 5 LSPs) +5. Measure installation times (for documentation) + +*** Phase 5: Advanced Features (Future) +1. Support additional languages: java (jdtls), lua, php, swift +2. Add --version pinning support (ARG in script) +3. Create pre-baked images: deva:python-lsp, deva:rust-lsp +4. Auto-detection helper: =deva-lsp-suggest= (scans project, suggests LSPs) + +** Metrics (Projected) + +| Metric | Before | After | Window | Source | +|--------+--------+-------+--------+--------| +| Base image size | 800MB | 800MB | N/A | No change (on-demand install) | +| deva:python-lsp image | N/A | 850MB | Future | If pre-baked image created | +| First LSP install time | N/A | 30s-2min | Per language | npm/pip/apt download speed | +| Plugin install time | N/A | 5-10s | Per plugin | claude plugin install latency | + +** Rollback + +*** Trigger +User reports LSP installation failures, broken plugin state, or excessive container bloat. + +*** Procedure +1. Remove script from container: =rm /usr/local/bin/deva-lsp-setup= +2. Revert Dockerfile changes (remove COPY line) +3. Uninstall plugins: =claude plugin uninstall = (from host) +4. Data impact: REVERSIBLE - plugins in =~/.claude/= can be uninstalled, binaries in container are ephemeral + +** Links + +*** Official Documentation +- Claude Code Plugins: https://code.claude.com/docs/en/plugins +- LSP Plugin Reference: https://code.claude.com/docs/en/plugins-reference#lsp-servers +- Discover Plugins: https://code.claude.com/docs/en/discover-plugins + +*** Repository Files +- Base Dockerfile: /Users/eric/wrk/src/github.com/claude-code/WIP/260107_deva-tmux-bridge/worktree/deva/Dockerfile +- Rust Dockerfile: /Users/eric/wrk/src/github.com/claude-code/WIP/260107_deva-tmux-bridge/worktree/deva/Dockerfile.rust +- AGENTS.md: /Users/eric/wrk/src/github.com/claude-code/WIP/260107_deva-tmux-bridge/worktree/deva/AGENTS.md +- Bridge script pattern: /Users/eric/wrk/src/github.com/claude-code/WIP/260107_deva-tmux-bridge/worktree/deva/scripts/deva-bridge-tmux + +*** Related Commits +- Current HEAD: 5807889 (refactor: extract version management to scripts and add --trace/--dry-run flags) +- Gemini support: 1190dee (feat: add gemini agent support and Docker-in-Docker auto-mount) + +** Notes + +*** Assumptions +- Claude Code LSP plugin architecture remains stable (two-component design) +- Official LSP plugins continue to be maintained by Anthropic +- Language server binaries are publicly available (npm, apt, PyPI, etc.) +- Users have internet access for initial LSP installation +- =~/.claude/= mount persists plugin state across container lifecycles + +*** Constraints +- Base image size target: <1GB (current: ~800MB) +- First-time LSP install latency: acceptable if <2min per language +- Plugin installation requires Claude Code authentication (cannot pre-install at build time) +- Container as sandbox model: LSP binaries run with full deva user permissions + +*** Gotchas +- Plugin state in =~/.claude/= (host) survives container deletion, but binaries do NOT +- rust-analyzer binary exists in deva:rust (line 48), but plugin must be installed separately +- clangd requires sudo apt install (only LSP needing elevated privileges) +- go install puts gopls in =$GOPATH/bin= (must be in =$PATH= - already configured in deva) +- npm global installs use =~/.npm-global= for deva user (configured in Dockerfile line 167-168) +- Plugin install requires authentication: user must run =claude= at least once before =deva-lsp-setup= +- TypeScript LSP requires TWO npm packages: typescript-language-server AND typescript +- pyright available via both npm and pip - prefer npm for consistency with other LSPs +- LSP binary versions will drift over time - container image tagging provides reproducibility + +*** Design Rationale +- POSIX shell (not bash) for maximum portability (follows =deva-bridge-tmux= pattern) +- On-demand install (not pre-baked) to keep base image lean, support varied stacks +- Support top 5 languages first (python, ts, rust, go, cpp) - cover 90% of deva usage +- =--plugin-only= flag for hosts with custom LSP binaries already installed +- =--binary-only= flag for users managing plugins from host =~/.claude/= +- Follow script patterns from =scripts/deva-bridge-tmux= (version, usage, error handling) + +*** Future Enhancements +- Language detection: =deva-lsp-suggest= scans project, recommends LSPs +- Pre-baked images: =deva:python-lsp=, =deva:fullstack-lsp= (all 5 LSPs) +- Version pinning: ARG-based version control for reproducible builds +- Health check: =deva-lsp-verify= tests LSP binaries + plugins +- Auto-update: =deva-lsp-update= pulls latest LSP binaries +- Multi-version support: Install multiple versions of same LSP (e.g., pyright stable + nightly) + +*** References to Existing Patterns +All script patterns derived from: +- =scripts/deva-bridge-tmux= (POSIX shell, getopts, usage, version) +- =scripts/deva-bridge-tmux-host= (error handling, dependency checks) +- =docker-entrypoint.sh= (environment reporting, user switching) +- Dockerfile layering strategy (stable layers first, volatile last) + +** Timestamp +Created: 2026-01-08T07:45:22Z +Author: @ericc-ch +Commit: 5807889 (worktree: deva) diff --git a/scripts/deva-bridge-tmux b/scripts/deva-bridge-tmux new file mode 100755 index 0000000..db8f065 --- /dev/null +++ b/scripts/deva-bridge-tmux @@ -0,0 +1,151 @@ +#!/bin/sh +# shellcheck shell=sh +# Version: 0.3.0 +# +# deva-bridge-tmux - Container-side tmux bridge for deva +# +# Connect container tmux client to host tmux server via TCP bridge. +# Requires deva-bridge-tmux-host running on the macOS host. +# +# SECURITY WARNING: +# This is a PRIVILEGED HOST BRIDGE. Container gains full tmux control. +# Effectively a sandbox escape - container can run commands on host. + +set -eu +umask 077 + +prog=${0##*/} +VERSION=0.3.0 + +EXIT_OK=0 +EXIT_ERROR=1 +EXIT_NOTFOUND=3 + +QUIET=0 +HOST="${DEVA_BRIDGE_HOST:-host.docker.internal}" +PORT="${DEVA_BRIDGE_PORT:-41555}" +LOCAL_SOCK="${DEVA_BRIDGE_SOCK:-/tmp/host-tmux.sock}" + +# --- util --- + +log() { [ "$QUIET" -eq 1 ] || printf '%s\n' "$*"; } +warn() { printf '%s\n' "$*" >&2; } +err() { printf '%s\n' "$*" >&2; } +die() { err "$prog: error: $*"; exit "$EXIT_ERROR"; } + +usage() { + cat </dev/null 2>&1 || die "missing dependency: $1" +} + +check_host_reachable() { + # Try nc first, fall back to socat probe + if command -v nc >/dev/null 2>&1; then + nc -z -w2 "$HOST" "$PORT" 2>/dev/null && return 0 + fi + # Fallback: socat probe (timeout 1s) + socat -T1 /dev/null "TCP:$HOST:$PORT" 2>/dev/null && return 0 + return 1 +} + +# --- parse options --- + +while [ $# -gt 0 ]; do + case $1 in + -h|--help) usage; exit "$EXIT_OK" ;; + -V|--version) version; exit "$EXIT_OK" ;; + -q|--quiet) QUIET=1 ;; + -H|--host) + [ $# -ge 2 ] || die "missing value for $1" + HOST="$2"; shift ;; + -p|--port) + [ $# -ge 2 ] || die "missing value for $1" + PORT="$2"; shift ;; + -s|--socket) + [ $# -ge 2 ] || die "missing value for $1" + LOCAL_SOCK="$2"; shift ;; + --) shift; break ;; + -*) die "unknown option: $1" ;; + *) break ;; + esac + shift +done + +# --- main --- + +main() { + require socat + require tmux + + # Verify host reachable + if ! check_host_reachable; then + err "$prog: cannot reach $HOST:$PORT" + err "" + err "on macOS host, run:" + err " deva-bridge-tmux-host" + err "" + err "if host.docker.internal can't reach 127.0.0.1, try:" + err " deva-bridge-tmux-host --bind 0.0.0.0" + exit "$EXIT_NOTFOUND" + fi + + # Remove stale socket; socat uses unlink-early so no cleanup needed after exec + rm -f "$LOCAL_SOCK" + + warn "============================================" + warn "WARNING: PRIVILEGED HOST BRIDGE (tmux)" + warn "============================================" + warn "Container can execute commands on host via tmux." + warn "" + + log "deva bridge: tmux (container-side)" + log " host: $HOST:$PORT" + log " sock: $LOCAL_SOCK" + log "" + log "usage: tmux -S $LOCAL_SOCK list-sessions" + log "tip: alias htmux='tmux -S $LOCAL_SOCK'" + + exec socat \ + "UNIX-LISTEN:$LOCAL_SOCK,fork,unlink-early" \ + "TCP:$HOST:$PORT" +} + +main "$@" diff --git a/scripts/deva-bridge-tmux-host b/scripts/deva-bridge-tmux-host new file mode 100755 index 0000000..ebd32c1 --- /dev/null +++ b/scripts/deva-bridge-tmux-host @@ -0,0 +1,153 @@ +#!/bin/sh +# shellcheck shell=sh +# Version: 0.3.0 +# +# deva-bridge-tmux-host - Host-side tmux bridge for deva containers +# +# Expose host tmux Unix socket over TCP for container access. +# Problem: Unix socket mount across macOS<->Linux kernel boundary fails. +# Solution: socat TCP proxy, container connects via host.docker.internal. +# +# SECURITY WARNING: +# This is a PRIVILEGED HOST BRIDGE. Container gains full tmux control. +# Any process reaching the TCP port can send-keys, run-shell, read scrollback. +# Effectively "container can run commands on host". + +set -eu +umask 077 + +prog=${0##*/} +VERSION=0.3.0 + +EXIT_OK=0 +EXIT_ERROR=1 +EXIT_NOTFOUND=3 + +QUIET=0 +PORT="${DEVA_BRIDGE_PORT:-41555}" +BIND="${DEVA_BRIDGE_BIND:-127.0.0.1}" +SOCKET="${DEVA_BRIDGE_SOCKET:-}" + +# --- util --- + +log() { [ "$QUIET" -eq 1 ] || printf '%s\n' "$*"; } +warn() { printf '%s\n' "$*" >&2; } +err() { printf '%s\n' "$*" >&2; } +die() { err "$prog: error: $*"; exit "$EXIT_ERROR"; } + +usage() { + cat </dev/null 2>&1 || die "missing dependency: $1" +} + +security_warning() { + warn "============================================" + warn "WARNING: PRIVILEGED HOST BRIDGE (tmux)" + warn "============================================" + warn "Container can execute commands on host via tmux." + if [ "$BIND" = "127.0.0.1" ]; then + warn "Binding $BIND:$PORT - LOCAL processes gain tmux control." + else + warn "Binding $BIND:$PORT - NETWORK clients gain tmux control." + fi + warn "" +} + +# --- parse options --- + +while [ $# -gt 0 ]; do + case $1 in + -h|--help) usage; exit "$EXIT_OK" ;; + -V|--version) version; exit "$EXIT_OK" ;; + -q|--quiet) QUIET=1 ;; + -b|--bind) + [ $# -ge 2 ] || die "missing value for $1" + BIND="$2"; shift ;; + -p|--port) + [ $# -ge 2 ] || die "missing value for $1" + PORT="$2"; shift ;; + -s|--socket) + [ $# -ge 2 ] || die "missing value for $1" + SOCKET="$2"; shift ;; + --) shift; break ;; + -*) die "unknown option: $1" ;; + *) break ;; + esac + shift +done + +# --- main --- + +main() { + require socat + require tmux + + # Auto-detect socket if not specified + if [ -z "$SOCKET" ]; then + SOCKET=$(tmux display-message -p '#{socket_path}' 2>/dev/null || printf '') + fi + if [ -z "$SOCKET" ]; then + SOCKET="/private/tmp/tmux-$(id -u)/default" + fi + + # Validate socket exists + if [ ! -S "$SOCKET" ]; then + err "$prog: tmux socket not found: $SOCKET" + err "Is tmux server running? Try: tmux new -d -s test" + exit "$EXIT_NOTFOUND" + fi + + security_warning + + log "deva bridge: tmux (host-side)" + log " socket: $SOCKET" + log " listen: $BIND:$PORT" + log "" + log "container: deva-bridge-tmux" + log "usage: tmux -S /tmp/host-tmux.sock list-sessions" + + exec socat \ + "TCP-LISTEN:$PORT,bind=$BIND,reuseaddr,fork" \ + "UNIX-CONNECT:$SOCKET" +} + +main "$@" diff --git a/scripts/release-utils.sh b/scripts/release-utils.sh new file mode 100755 index 0000000..db8b4ec --- /dev/null +++ b/scripts/release-utils.sh @@ -0,0 +1,505 @@ +#!/usr/bin/env bash +# release-utils.sh - Unified utilities for version/changelog management +# Provides extensible tool registry and common functions for version checking + +set -euo pipefail + +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +# Colors +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +RESET='\033[0m' +BOLD='\033[1m' +DIM='\033[0;90m' +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +CYAN='\033[1;36m' +WHITE='\033[1;37m' + +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +# Tool Registry +# Each tool: NAME|TYPE|SOURCE|LABEL|URL|CHANGELOG_URL +# TYPE: npm, github-release, github-commit +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +TOOL_REGISTRY=( + "claude-code|npm|@anthropic-ai/claude-code|org.opencontainers.image.claude_code_version|https://www.npmjs.com/package/@anthropic-ai/claude-code|https://raw.githubusercontent.com/anthropics/claude-code/main/CHANGELOG.md" + "codex|npm|@openai/codex|org.opencontainers.image.codex_version|https://www.npmjs.com/package/@openai/codex|github:openai/codex" + "gemini-cli|npm|@google/gemini-cli|org.opencontainers.image.gemini_cli_version|https://www.npmjs.com/package/@google/gemini-cli|" + "atlas-cli|github-release|lroolle/atlas-cli|org.opencontainers.image.atlas_cli_version|https://github.com/lroolle/atlas-cli|github:lroolle/atlas-cli" + "copilot-api|github-commit|ericc-ch/copilot-api|org.opencontainers.image.copilot_api_version|https://github.com/ericc-ch/copilot-api|" +) + +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +# Tool Registry Helpers +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +get_tool_field() { + local tool=$1 field=$2 + for entry in "${TOOL_REGISTRY[@]}"; do + IFS='|' read -r name type source label url changelog <<< "$entry" + if [[ $name == "$tool" ]]; then + case $field in + name) echo "$name" ;; + type) echo "$type" ;; + source) echo "$source" ;; + label) echo "$label" ;; + url) echo "$url" ;; + changelog) echo "$changelog" ;; + esac + return 0 + fi + done + return 1 +} + +get_all_tools() { + for entry in "${TOOL_REGISTRY[@]}"; do + IFS='|' read -r name _ <<< "$entry" + echo "$name" + done +} + +get_display_name() { + local tool=$1 + case $tool in + claude-code) echo "Claude Code" ;; + codex) echo "Codex" ;; + gemini-cli) echo "Gemini CLI" ;; + atlas-cli) echo "Atlas CLI" ;; + copilot-api) echo "Copilot API" ;; + *) echo "$tool" ;; + esac +} + +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +# Version Utilities +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +normalize_version() { + local v=$1 + v=${v#v} + echo "$v" +} + +format_version() { + local v=$1 + if [[ -z $v ]] || [[ $v == "" ]]; then + echo "-" + elif [[ $v == v* ]]; then + echo "$v" + else + echo "v$v" + fi +} + +format_datetime() { + local datetime=$1 + [[ -z $datetime ]] && return + date -d "$datetime" '+%b %d, %Y %H:%M' 2>/dev/null || \ + date -jf '%Y-%m-%dT%H:%M:%SZ' "$datetime" '+%b %d, %Y %H:%M' 2>/dev/null || \ + date -jf '%Y-%m-%dT%H:%M:%S' "${datetime%.*}" '+%b %d, %Y %H:%M' 2>/dev/null || \ + echo "$datetime" +} + +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +# Version Fetching (by type) +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +fetch_latest_version() { + local tool=$1 + local type=$(get_tool_field "$tool" type) + local source=$(get_tool_field "$tool" source) + + case $type in + npm) + npm view "$source" version 2>/dev/null || echo "" + ;; + github-release) + gh api "repos/$source/releases/latest" --jq '.tag_name' 2>/dev/null || echo "" + ;; + github-commit) + local branch="master" + [[ $source == "lroolle/atlas-cli" ]] && branch="main" + gh api "repos/$source/branches/$branch" --jq '.commit.sha' 2>/dev/null || echo "" + ;; + esac +} + +fetch_version_date() { + local tool=$1 version=$2 + local type=$(get_tool_field "$tool" type) + local source=$(get_tool_field "$tool" source) + + case $type in + npm) + local v=$(normalize_version "$version") + npm view "$source@$v" time --json 2>/dev/null | \ + jq -r --arg ver "$v" '.[$ver] // .' 2>/dev/null | head -1 || echo "" + ;; + github-release) + gh api "repos/$source/releases/tags/$version" --jq '.published_at' 2>/dev/null || echo "" + ;; + github-commit) + gh api "repos/$source/commits/$version" --jq '.commit.committer.date' 2>/dev/null || echo "" + ;; + esac +} + +get_image_version() { + local image=$1 label=$2 + docker inspect "$image" 2>/dev/null | \ + jq -r --arg k "$label" '.[0].Config.Labels[$k] // ""' 2>/dev/null || echo "" +} + +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +# Changelog Fetching +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +fetch_changelog() { + local tool=$1 current=$2 latest=$3 + local changelog_source=$(get_tool_field "$tool" changelog) + + [[ -z $changelog_source ]] && return + [[ -z $current ]] || [[ $current == "-" ]] && return + + current=$(normalize_version "$current") + latest=$(normalize_version "$latest") + [[ $current == "$latest" ]] && return + + if [[ $changelog_source == github:* ]]; then + fetch_github_releases "${changelog_source#github:}" "$current" "$latest" + else + fetch_markdown_changelog "$changelog_source" "$current" "$latest" + fi +} + +fetch_markdown_changelog() { + local url=$1 current=$2 latest=$3 + local data + data=$(curl -fsSL --max-time 10 --retry 2 "$url" 2>/dev/null) || { echo "(fetch failed)"; return 0; } + + python3 -c ' +import re, sys + +def parse_version(v): + parts = re.findall(r"\d+", v) + return tuple(int(p) for p in parts) if parts else (0,) + +current, latest = sys.argv[1], sys.argv[2] +text = sys.stdin.read() + +try: + cur_v, lat_v = parse_version(current), parse_version(latest) +except: + sys.exit(0) + +sections = re.split(r"(?=^## )", text, flags=re.M) +changes = [] + +for section in sections: + if not section.strip(): + continue + match = re.match(r"^##\s+.*?(\d+\.\d+\.\d+)", section, re.M) + if not match: + continue + try: + v = parse_version(match.group(1)) + if cur_v < v <= lat_v: + lines = section.strip().split("\n") + output = [lines[0].replace("## ", "")] + content_lines = [l for l in lines[1:] if l.strip()][:10] + output.extend(content_lines) + changes.append("\n".join(output)) + except: + continue + +for change in reversed(changes[-3:]): + print(change) + print() +' "$current" "$latest" <<< "$data" 2>/dev/null || true +} + +fetch_github_releases() { + local repo=$1 current=$2 latest=$3 + local json + json=$(gh api "repos/$repo/releases" 2>/dev/null) || { echo "(fetch failed)"; return 0; } + + python3 -c ' +import json, sys, re + +def parse_version(v): + match = re.search(r"(\d+)\.(\d+)\.(\d+)", v) + return tuple(int(x) for x in match.groups()) if match else (0, 0, 0) + +current, latest = sys.argv[1], sys.argv[2] +releases = json.load(sys.stdin) + +try: + cur_v, lat_v = parse_version(current), parse_version(latest) +except: + sys.exit(0) + +changes = [] +seen = set() + +for rel in releases: + if rel.get("prerelease"): + continue + tag = rel.get("tag_name", "") + if not tag: + continue + try: + v = parse_version(tag) + if v in seen or not (cur_v < v <= lat_v): + continue + seen.add(v) + ver = re.search(r"(\d+\.\d+\.\d+)", tag) + ver = ver.group(1) if ver else tag + body = (rel.get("body") or "").replace("\r\n", "\n").strip() + changes.append((v, ver, body)) + except: + continue + +for v, ver, body in sorted(changes, key=lambda x: x[0], reverse=True)[:3]: + print(f"{ver}") + if body: + for line in [l.rstrip() for l in body.split("\n") if l.strip()][:15]: + print(f" {line}") + print() +' "$current" "$latest" <<< "$json" 2>/dev/null || true +} + +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +# Display Helpers +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +section() { + echo -e "${CYAN}${BOLD}โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”${RESET}" + echo -e "${CYAN}${BOLD}$1${RESET}" + echo -e "${CYAN}${BOLD}โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”${RESET}" +} + +indent() { sed 's/^/ /'; } + +print_version_line() { + local tool=$1 current=$2 latest=$3 date=$4 + local name=$(get_display_name "$tool") + local url=$(get_tool_field "$tool" url) + local type=$(get_tool_field "$tool" type) + + local cur_fmt lat_fmt + if [[ $type == "github-commit" ]]; then + cur_fmt="${current:0:7}" + lat_fmt="${latest:0:7}" + [[ -z $current ]] || [[ $current == "-" ]] && cur_fmt="-" + else + cur_fmt=$(format_version "$current") + lat_fmt=$(format_version "$latest") + fi + + local date_str="" + [[ -n $date ]] && date_str=" ${DIM}($(format_datetime "$date"))${RESET}" + + local link_str="" + [[ -n $url ]] && link_str=" ${DIM}${url}${RESET}" + + local pad=$(printf "%-12s" "$name:") + + if [[ -z $current ]] || [[ $current == "-" ]]; then + echo -e " ${WHITE}${pad}${RESET} ${DIM}-${RESET} -> ${GREEN}${lat_fmt}${RESET}${date_str}${link_str} ${YELLOW}(not built)${RESET}" + return 1 + elif [[ $(normalize_version "$current") == $(normalize_version "$latest") ]] || [[ $current == "$latest" ]]; then + echo -e " ${DIM}${pad} ${lat_fmt}${RESET}${date_str}${link_str} ${GREEN}(up-to-date)${RESET}" + return 0 + else + echo -e " ${WHITE}${pad}${RESET} ${RED}${cur_fmt}${RESET} -> ${GREEN}${lat_fmt}${RESET}${date_str}${link_str}" + return 1 + fi +} + +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +# Version Storage (bash 3.2 compatible - no associative arrays) +# Uses naming convention: _VER_{CURRENT,LATEST,DATE}_{tool_key} +# โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +tool_key() { + echo "$1" | tr '-' '_' +} + +set_current() { eval "_VER_CURRENT_$(tool_key "$1")=\"$2\""; } +set_latest() { eval "_VER_LATEST_$(tool_key "$1")=\"$2\""; } +set_date() { eval "_VER_DATE_$(tool_key "$1")=\"$2\""; } + +get_current() { eval "echo \"\${_VER_CURRENT_$(tool_key "$1"):-}\""; } +get_latest() { eval "echo \"\${_VER_LATEST_$(tool_key "$1"):-}\""; } +get_date() { eval "echo \"\${_VER_DATE_$(tool_key "$1"):-}\""; } + +load_versions() { + local image=$1 + + echo -e "${DIM}Fetching versions...${RESET}" + + for tool in $(get_all_tools); do + local label=$(get_tool_field "$tool" label) + + # Current from image + if docker inspect "$image" >/dev/null 2>&1; then + set_current "$tool" "$(get_image_version "$image" "$label")" + else + set_current "$tool" "" + fi + + # Latest from source (respect env overrides) + local env_var latest_val + case $tool in + claude-code) env_var="CLAUDE_CODE_VERSION" ;; + codex) env_var="CODEX_VERSION" ;; + gemini-cli) env_var="GEMINI_CLI_VERSION" ;; + atlas-cli) env_var="ATLAS_CLI_VERSION" ;; + copilot-api) env_var="COPILOT_API_VERSION" ;; + esac + + eval "latest_val=\"\${$env_var:-}\"" + if [[ -n $latest_val ]]; then + set_latest "$tool" "$latest_val" + else + local fetched + fetched=$(fetch_latest_version "$tool") + if [[ -n $fetched ]]; then + set_latest "$tool" "$fetched" + else + # Network failure - fallback to current image version + local current=$(get_current "$tool") + if [[ -n $current ]]; then + echo -e "${YELLOW}Warning: Failed to fetch latest $tool, using current: $current${RESET}" >&2 + set_latest "$tool" "$current" + else + echo -e "${RED}Error: Cannot determine version for $tool${RESET}" >&2 + fi + fi + fi + + set_date "$tool" "$(fetch_version_date "$tool" "$(get_latest "$tool")")" + done +} + +print_version_summary() { + echo "" + echo -e "${YELLOW}${BOLD}Version Status${RESET}" + echo "" + + local needs_update=0 + for tool in $(get_all_tools); do + print_version_line "$tool" "$(get_current "$tool")" "$(get_latest "$tool")" "$(get_date "$tool")" || needs_update=1 + done + echo "" + + return $needs_update +} + +print_changelogs() { + for tool in $(get_all_tools); do + local current=$(get_current "$tool") + local latest=$(get_latest "$tool") + local changelog_source=$(get_tool_field "$tool" changelog) + + [[ -z $changelog_source ]] && continue + [[ -z $current ]] || [[ $current == "-" ]] && continue + + local cur_norm=$(normalize_version "$current") + local lat_norm=$(normalize_version "$latest") + [[ $cur_norm == "$lat_norm" ]] && continue + + local name=$(get_display_name "$tool") + section "$name Changelog" + local changes + changes=$(fetch_changelog "$tool" "$current" "$latest") + if [[ -n $changes ]]; then + echo "$changes" | indent + else + echo -e " ${DIM}(changelog unavailable)${RESET}" + fi + echo "" + done +} + +# Show recent changelogs for all tools (regardless of update status) +print_recent_changelogs() { + local depth=${CHANGELOG_DEPTH:-3} + + for tool in $(get_all_tools); do + local changelog_source=$(get_tool_field "$tool" changelog) + [[ -z $changelog_source ]] && continue + + local name=$(get_display_name "$tool") + local latest=$(get_latest "$tool") + + section "$name (latest: $latest)" + + local changes="" + if [[ $changelog_source == github:* ]]; then + changes=$(fetch_recent_github_releases "${changelog_source#github:}" "$depth") + else + changes=$(fetch_recent_markdown_changelog "$changelog_source" "$depth") + fi + + if [[ -n $changes ]]; then + echo "$changes" | indent + else + echo -e " ${DIM}(changelog unavailable)${RESET}" + fi + echo "" + done +} + +fetch_recent_markdown_changelog() { + local url=$1 count=${2:-3} + local data + data=$(curl -fsSL --max-time 10 --retry 2 "$url" 2>/dev/null) || { echo "(fetch failed)"; return 0; } + + python3 -c ' +import re, sys + +count = int(sys.argv[1]) +text = sys.stdin.read() +sections = re.split(r"(?=^## )", text, flags=re.M) +printed = 0 + +for section in sections: + if not section.strip() or printed >= count: + continue + match = re.match(r"^##\s+.*?(\d+\.\d+\.\d+)", section, re.M) + if not match: + continue + lines = section.strip().split("\n") + print(lines[0].replace("## ", "")) + for line in [l for l in lines[1:] if l.strip()][:8]: + print(line) + print() + printed += 1 +' "$count" <<< "$data" 2>/dev/null || true +} + +fetch_recent_github_releases() { + local repo=$1 count=${2:-3} + local json + json=$(gh api "repos/$repo/releases?per_page=$count" 2>/dev/null) || { echo "(fetch failed)"; return 0; } + + python3 -c ' +import json, sys, re + +count = int(sys.argv[1]) +releases = json.load(sys.stdin) +printed = 0 + +for rel in releases: + if rel.get("prerelease") or printed >= count: + continue + tag = rel.get("tag_name", "") + if not tag: + continue + ver = re.search(r"(\d+\.\d+\.\d+)", tag) + ver = ver.group(1) if ver else tag + body = (rel.get("body") or "").replace("\r\n", "\n").strip() + print(ver) + if body: + for line in [l.rstrip() for l in body.split("\n") if l.strip()][:12]: + print(f" {line}") + print() + printed += 1 +' "$count" <<< "$json" 2>/dev/null || true +} diff --git a/scripts/version-report.sh b/scripts/version-report.sh index dabb8f2..a1e5a84 100755 --- a/scripts/version-report.sh +++ b/scripts/version-report.sh @@ -1,255 +1,30 @@ #!/usr/bin/env bash -set -euo pipefail - -CLAUDE_CHANGELOG_URL=${CLAUDE_CHANGELOG_URL:-https://raw.githubusercontent.com/anthropics/claude-code/main/CHANGELOG.md} -CODEX_RELEASES_API=${CODEX_RELEASES_API:-https://api.github.com/repos/openai/codex/releases/latest} - -section() { - local title=$1 - echo "$title" - printf '%*s\n' "${#title}" '' | tr ' ' '-' -} - -indent() { sed 's/^/ /'; } - -normalize_version() { - local v=$1 - v=${v#v} - echo "$v" -} - -get_latest_npm_version() { - local pkg=$1 - npm view "$pkg" version 2>/dev/null || echo "" -} - -get_image_version() { - local image=$1 label=$2 - docker inspect "$image" 2>/dev/null | \ - jq -r --arg k "$label" '.[0].Config.Labels[$k] // ""' 2>/dev/null || echo "" -} - -compare_versions() { - local current=$1 latest=$2 name=$3 - current=$(normalize_version "$current") - latest=$(normalize_version "$latest") - - if [[ -z $current ]]; then - echo " $name: - -> v$latest (not built)" - return 1 - elif [[ $current == "$latest" ]]; then - echo " $name: v$current (up-to-date)" - return 0 - else - echo " $name: v$current -> v$latest (upgrade available)" - return 1 - fi -} - -fetch_changelog_between() { - local current=$1 latest=$2 - current=$(normalize_version "$current") - latest=$(normalize_version "$latest") - - if [[ -z $current ]] || [[ $current == "$latest" ]]; then - return - fi - - local data - if ! data=$(curl -fsSL --max-time 10 --retry 2 "$CLAUDE_CHANGELOG_URL" 2>/dev/null); then - return - fi - - python3 -c 'import re, sys - -def parse_version(v): - """Parse version string to tuple of ints for comparison""" - parts = re.findall(r"\d+", v) - return tuple(int(p) for p in parts) if parts else (0,) - -current = sys.argv[1] -latest = sys.argv[2] -text = sys.stdin.read() - -try: - cur_v = parse_version(current) - lat_v = parse_version(latest) -except: - sys.exit(0) - -# Match entire sections: ## heading followed by content until next ## or end -sections = re.split(r"(?=^## )", text, flags=re.M) -changes = [] - -for section in sections: - if not section.strip(): - continue - - # Extract version from heading - match = re.match(r"^##\s+.*?(\d+\.\d+\.\d+)", section, re.M) - if not match: - continue +# version-report.sh - Display version status and recent changelogs +# Used by: make versions - try: - v = parse_version(match.group(1)) - if cur_v < v <= lat_v: - # Clean up section: remove heading marker, limit lines - lines = section.strip().split("\n") - if lines: - # Keep version heading and first 10 content lines - output = [lines[0].replace("## ", "")] - content_lines = [l for l in lines[1:] if l.strip()][:10] - output.extend(content_lines) - changes.append("\n".join(output)) - except: - continue - -if changes: - for change in reversed(changes[-3:]): - print(change) - print() -' "$current" "$latest" <<< "$data" 2>/dev/null || true -} - -fetch_github_releases_between() { - local current=$1 latest=$2 - current=$(normalize_version "$current") - latest=$(normalize_version "$latest") - - if [[ -z $current ]] || [[ $current == "$latest" ]]; then - return - fi - - local json - if ! json=$(curl -fsSL --max-time 10 --retry 2 \ - "https://api.github.com/repos/openai/codex/releases" 2>/dev/null); then - return - fi - - python3 -c 'import json, sys, re - -def parse_version(v): - """Parse version string to tuple of ints for comparison""" - # Extract pure version numbers, ignore prefixes and suffixes - match = re.search(r"(\d+)\.(\d+)\.(\d+)", v) - if not match: - return (0, 0, 0) - return tuple(int(x) for x in match.groups()) - -current = sys.argv[1] -latest = sys.argv[2] -releases = json.load(sys.stdin) - -try: - cur_v = parse_version(current) - lat_v = parse_version(latest) -except: - sys.exit(0) - -changes = [] -seen_versions = set() - -for rel in releases: - # Skip prereleases (alpha, beta, rc) - if rel.get("prerelease", False): - continue - - tag = rel.get("tag_name") or "" - if not tag: - continue - - try: - v = parse_version(tag) - - # Skip duplicates and check version range - if v in seen_versions or not (cur_v < v <= lat_v): - continue - - seen_versions.add(v) - - # Extract clean version string - ver_match = re.search(r"(\d+\.\d+\.\d+)", tag) - ver = ver_match.group(1) if ver_match else tag - - name = rel.get("name") or ver - body = (rel.get("body") or "").strip() - # Convert \r\n to \n - body = body.replace("\r\n", "\n") +set -euo pipefail - changes.append((v, ver, name, body)) - except Exception: - continue +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +source "$SCRIPT_DIR/release-utils.sh" -# Sort by version tuple (newest first) and show up to 3 -for v, ver, name, body in sorted(changes, key=lambda x: x[0], reverse=True)[:3]: - print(f"{ver}") - if body: - lines = [l.rstrip() for l in body.split("\n") if l.strip()][:15] - for line in lines: - print(f" {line}") - print() -' "$current" "$latest" <<< "$json" 2>/dev/null || true -} +IMAGE=${MAIN_IMAGE:-ghcr.io/thevibeworks/deva:latest} +CHANGELOG_DEPTH=${CHANGELOG_DEPTH:-3} main() { - local image=${MAIN_IMAGE:-ghcr.io/thevibeworks/deva:latest} - section "Version Status" - echo "Image: $image" - - # Get current versions from built image - local cur_claude cur_codex image_exists=true - if docker inspect "$image" >/dev/null 2>&1; then - cur_claude=$(get_image_version "$image" "org.opencontainers.image.claude_code_version") - cur_codex=$(get_image_version "$image" "org.opencontainers.image.codex_version") - else - echo " (image not built locally)" - cur_claude="" - cur_codex="" - image_exists=false - fi + echo -e "${DIM}Time: $(date '+%Y-%m-%d %H:%M:%S')${RESET}" + echo -e "${DIM}Image: ${IMAGE}${RESET}" + echo "" - # Get latest versions - local lat_claude=${CLAUDE_CODE_VERSION:-$(get_latest_npm_version "@anthropic-ai/claude-code")} - local lat_codex=${CODEX_VERSION:-$(get_latest_npm_version "@openai/codex")} + load_versions "$IMAGE" local needs_update=0 - compare_versions "$cur_claude" "$lat_claude" "Claude Code" || needs_update=1 - compare_versions "$cur_codex" "$lat_codex" "Codex" || needs_update=1 - echo + print_version_summary || needs_update=1 - if [[ $needs_update -eq 1 ]]; then - if [[ -n $cur_claude ]] && [[ $(normalize_version "$cur_claude") != $(normalize_version "$lat_claude") ]]; then - section "Claude Code Changes" - local changes - changes=$(fetch_changelog_between "$cur_claude" "$lat_claude") - if [[ -n $changes ]]; then - echo "$changes" | indent - else - echo " (changelog unavailable)" - fi - echo - fi - - if [[ -n $cur_codex ]] && [[ $(normalize_version "$cur_codex") != $(normalize_version "$lat_codex") ]]; then - section "Codex Changes" - local changes - changes=$(fetch_github_releases_between "$cur_codex" "$lat_codex") - if [[ -n $changes ]]; then - echo "$changes" | indent - else - echo " (release notes unavailable)" - fi - echo - fi + print_recent_changelogs - if [[ $image_exists == true ]]; then - echo "Run 'make versions-up' to upgrade" - else - echo "Run 'make build' to build images" - fi - else - echo "All versions up-to-date" + if [[ $needs_update -eq 1 ]]; then + echo -e "${YELLOW}Run 'make versions-up' to upgrade${RESET}" fi } diff --git a/scripts/version-upgrade.sh b/scripts/version-upgrade.sh new file mode 100755 index 0000000..e3c18ee --- /dev/null +++ b/scripts/version-upgrade.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +# version-upgrade.sh - Upgrade all tools to latest versions +# Shows changelogs first, then builds after confirmation + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +source "$SCRIPT_DIR/release-utils.sh" + +# Defaults +CHECK_IMAGE=${MAIN_IMAGE:-ghcr.io/thevibeworks/deva:latest} +BUILD_IMAGE=${BUILD_IMAGE:-ghcr.io/thevibeworks/deva:latest} +RUST_IMAGE=${RUST_IMAGE:-ghcr.io/thevibeworks/deva:rust} +DOCKERFILE=${DOCKERFILE:-Dockerfile} +RUST_DOCKERFILE=${RUST_DOCKERFILE:-Dockerfile.rust} +COUNTDOWN=${COUNTDOWN:-5} +AUTO_YES=${AUTO_YES:-} + +usage() { + cat <