diff --git a/.github/ISSUE_TEMPLATE/pr_tracker.yml b/.github/ISSUE_TEMPLATE/pr_tracker.yml new file mode 100644 index 00000000..1dbe050a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/pr_tracker.yml @@ -0,0 +1,109 @@ +name: PR tracker (mirror) +description: Mirror an in-flight pull request as a tracking issue for Linear and Slack sync / 镜像一个进行中的 PR 作为追踪 issue +title: "[PR Track] #: " +labels: ["pr-mirror", "tracking"] +body: + - type: markdown + attributes: + value: | + Use this when you want a long-lived, auditable record of an upstream PR. + Linear and Slack subscribe to issues with the `pr-mirror` label. + 镜像一个 PR 用于 Linear / Slack 长期可审计追踪。带 `pr-mirror` 标签的 issue 会被订阅。 + - type: input + id: pr_number + attributes: + label: PR number + placeholder: "e.g. 196" + validations: + required: true + - type: input + id: pr_url + attributes: + label: PR URL + placeholder: https://github.com/EverMind-AI/EverOS/pull/ + validations: + required: true + - type: input + id: author + attributes: + label: Author handle + placeholder: "@github-login" + validations: + required: true + - type: dropdown + id: area + attributes: + label: Area + options: + - methods/EverCore + - methods/HyperMem + - benchmarks/EverMemBench + - benchmarks/EvoAgentBench + - use-cases + - documentation + - CI / build / release + - other + validations: + required: true + - type: dropdown + id: lane + attributes: + label: Review lane + description: How this PR should be triaged. / 该 PR 的优先级处理通道。 + options: + - hotfix (block release until merged) + - normal (standard review) + - docs-only (light review) + - exploratory (no merge intent) + validations: + required: true + - type: textarea + id: scope + attributes: + label: Scope summary + description: One paragraph. What does the PR change, and what is intentionally left out? + placeholder: | + Changes: + - ... + Out of scope: + - ... + validations: + required: true + - type: textarea + id: evidence + attributes: + label: Evidence snapshot + description: | + Required before this mirror can be closed. Paste the CI summary, test command output, + or the link to the run. "No mirror closes without evidence." + 关闭镜像 issue 前必填。粘贴 CI 摘要、测试命令输出或 run 链接。 + render: shell + validations: + required: true + - type: textarea + id: decisions + attributes: + label: Decision log + description: Notable review decisions (approvals, requested changes, deferrals). + placeholder: | + - 2026-05-13 @reviewer: requested change on tests/test_x.py + - 2026-05-13 @author: scoped follow-up to PR #... + - type: input + id: linear_issue + attributes: + label: Linear issue (optional) + placeholder: "EVE-123" + - type: input + id: slack_thread + attributes: + label: Slack thread (optional) + placeholder: "https://everminddash.slack.com/archives/.../p..." + - type: checkboxes + id: closure + attributes: + label: Closure criteria + description: Check all that apply before closing this mirror. + options: + - label: PR is merged, closed, or marked won't-fix upstream. + - label: Evidence snapshot above reflects the final state. + - label: Linear and Slack records have been updated (if linked). diff --git a/.github/ISSUE_TEMPLATE/security_tracker.yml b/.github/ISSUE_TEMPLATE/security_tracker.yml new file mode 100644 index 00000000..2f2c96c8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/security_tracker.yml @@ -0,0 +1,116 @@ +name: Security tracker (mirror) +description: Mirror a security PR or disclosure for Linear and Slack escalation / 镜像一个安全 PR 或披露 +title: "[Security Track] CWE-: " +labels: ["security", "pr-mirror", "tracking", "urgent"] +body: + - type: markdown + attributes: + value: | + Use this for any PR or disclosure that affects credentials, authn/authz, data exposure, + supply chain, or sandbox escape. The `urgent` label routes this to high-priority + notifications in Slack and Linear. + 用于凭证、认证授权、数据暴露、供应链、沙箱逃逸等安全 PR / 披露。`urgent` 标签会触发高优先级通知。 + Do NOT include exploit details that are not already public in the upstream PR. + 请勿写入未在 upstream PR 公开的利用细节。 + - type: input + id: cwe + attributes: + label: CWE id + placeholder: "CWE-798" + validations: + required: true + - type: input + id: pr_url + attributes: + label: Upstream PR or advisory URL + placeholder: https://github.com/EverMind-AI/EverOS/pull/ + validations: + required: true + - type: dropdown + id: severity + attributes: + label: Severity + options: + - Critical (full auth bypass / unauthenticated RCE / mass data loss) + - High (privileged data access / credential exposure / persistent compromise) + - Medium (limited data access / requires user interaction) + - Low (defense-in-depth / hardening) + validations: + required: true + - type: dropdown + id: exposure + attributes: + label: Reachability + description: How reachable is this in the documented quickstart / default config? + options: + - Default config (reproducible from a clean clone) + - Default config + network position + - Non-default config but documented + - Hypothetical / not yet reproducible + validations: + required: true + - type: textarea + id: affected + attributes: + label: Affected components + description: File paths, services, or versions impacted. + placeholder: | + - methods/EverCore/docker-compose.yaml (memsys-milvus-minio block) + - methods/EverCore/env.template + validations: + required: true + - type: textarea + id: fix_summary + attributes: + label: Proposed fix summary + description: One paragraph. What does the PR change? Cite the contract that makes it fail-closed. + validations: + required: true + - type: textarea + id: evidence + attributes: + label: Verification evidence + description: | + Required before closure. Show the commands and output that prove the fix works AND + that the unpatched state was exploitable. "No security mirror closes without evidence." + 关闭前必填。展示证明 fix 生效以及未修复状态可利用的命令与输出。 + render: shell + validations: + required: true + - type: textarea + id: residual + attributes: + label: Residual risk / follow-ups + description: Anything intentionally out of scope, plus follow-up issues that should be filed. + placeholder: | + - docs/installation/ still references the old default in examples; follow-up sweep needed. + - Consider adding a CI lint to catch hardcoded secrets in docker-compose files. + - type: input + id: linear_issue + attributes: + label: Linear issue (optional) + placeholder: "EVE-123" + - type: input + id: slack_thread + attributes: + label: Slack thread (optional) + placeholder: "https://everminddash.slack.com/archives/.../p..." + - type: checkboxes + id: disclosure + attributes: + label: Disclosure hygiene + description: Confirm before submitting. + options: + - label: This mirror contains no exploit details beyond what is already public in the upstream PR. + required: true + - label: The upstream PR or advisory link is correct and reachable. + required: true + - label: A maintainer has been pinged in Slack #p-evermind-dash or via Linear EVE if Severity is Critical or High. + - type: checkboxes + id: closure + attributes: + label: Closure criteria + options: + - label: Upstream PR merged, advisory published, or risk formally accepted. + - label: Verification evidence above reflects the merged state. + - label: Residual-risk follow-ups have issues filed (or explicitly waived). diff --git a/.github/scripts/overnight-watch.mjs b/.github/scripts/overnight-watch.mjs new file mode 100755 index 00000000..5fc2b1c4 --- /dev/null +++ b/.github/scripts/overnight-watch.mjs @@ -0,0 +1,371 @@ +#!/usr/bin/env node + +import { execFileSync } from "node:child_process"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +const config = { + repoOwner: process.env.REPO_OWNER || "Fearvox", + repoName: process.env.REPO_NAME || "EverOS", + upstreamRepo: process.env.UPSTREAM_REPO || "EverMind-AI/EverOS", + watchBranch: + process.env.WATCH_BRANCH || "codex-watch-overnight-2026-05-13", + ownerTimezone: process.env.OWNER_TIMEZONE || "America/Los_Angeles", + linearTeamId: + process.env.LINEAR_TEAM_ID || "233391d6-ec9e-4aa8-b534-16a221b8119a", + linearProjectId: + process.env.LINEAR_PROJECT_ID || "39aa3865-345c-4313-9dc0-ab3b509c5d21", + createTrackingIssue: + (process.env.CREATE_TRACKING_ISSUE || "false").toLowerCase() === "true", + githubToken: process.env.GITHUB_TOKEN || process.env.GH_TOKEN || "", + linearApiKey: process.env.LINEAR_API_KEY || "", +}; + +const repoSlug = `${config.repoOwner}/${config.repoName}`; +const recentWindowHours = Number(process.env.WATCH_WINDOW_HOURS || 24); +const since = new Date(Date.now() - recentWindowHours * 60 * 60 * 1000); +const now = new Date(); + +function run(command, args, options = {}) { + const result = execFileSync(command, args, { + encoding: "utf8", + stdio: ["ignore", "pipe", options.inheritStderr ? "inherit" : "pipe"], + env: { + ...process.env, + GH_TOKEN: config.githubToken || process.env.GH_TOKEN, + }, + }); + return result.trim(); +} + +function tryRun(command, args, fallback = "") { + try { + return run(command, args); + } catch (error) { + return fallback; + } +} + +function ghJson(endpoint) { + const output = run("gh", [ + "api", + "-H", + "Accept: application/vnd.github+json", + endpoint, + ]); + return JSON.parse(output); +} + +function ensureRemote(name, url) { + const remotes = tryRun("git", ["remote"]).split("\n").filter(Boolean); + if (!remotes.includes(name)) { + run("git", ["remote", "add", name, url]); + } +} + +function fetchGitState() { + ensureRemote("upstream", `https://github.com/${config.upstreamRepo}.git`); + tryRun("git", ["fetch", "origin", "--prune"], ""); + tryRun("git", ["fetch", "upstream", "main", "--prune"], ""); + + const forkHead = tryRun("git", ["rev-parse", "--short", "origin/main"], "unknown"); + const upstreamHead = tryRun( + "git", + ["rev-parse", "--short", "upstream/main"], + "unknown", + ); + const counts = tryRun( + "git", + ["rev-list", "--left-right", "--count", "origin/main...upstream/main"], + "0\t0", + ) + .split(/\s+/) + .map((value) => Number(value)); + + const watchBranchSha = tryRun( + "git", + ["ls-remote", "--heads", "origin", config.watchBranch], + "", + ) + .split(/\s+/)[0] + ?.slice(0, 12); + + return { + forkHead, + upstreamHead, + forkAhead: counts[0] || 0, + forkBehind: counts[1] || 0, + watchBranchSha: watchBranchSha || "", + }; +} + +function fetchGitHubState() { + const runs = ghJson( + `/repos/${repoSlug}/actions/runs?per_page=50&status=completed`, + ).workflow_runs || []; + const failedRuns = runs + .filter((runInfo) => new Date(runInfo.created_at) >= since) + .filter((runInfo) => + ["failure", "cancelled", "timed_out"].includes(runInfo.conclusion), + ) + .slice(0, 10); + + const upstreamPulls = ghJson( + `/repos/${config.upstreamRepo}/pulls?state=open&sort=updated&direction=desc&per_page=30`, + ) + .filter((pull) => new Date(pull.updated_at) >= since) + .slice(0, 10); + + const forkPulls = ghJson( + `/repos/${repoSlug}/pulls?state=open&sort=updated&direction=desc&per_page=30`, + ) + .filter((pull) => new Date(pull.updated_at) >= since) + .slice(0, 10); + + return { failedRuns, upstreamPulls, forkPulls }; +} + +function lineForPull(pull) { + return `- #${pull.number} ${pull.title} (${pull.user.login}, updated ${pull.updated_at})`; +} + +function lineForRun(runInfo) { + return `- ${runInfo.name}: ${runInfo.conclusion} (${runInfo.html_url})`; +} + +function renderReport(gitState, githubState) { + const findings = []; + + if (!gitState.watchBranchSha) { + findings.push(`Watch branch is not on origin: ${config.watchBranch}`); + } + if (gitState.forkBehind > 0) { + findings.push(`Fork main is behind upstream/main by ${gitState.forkBehind} commit(s).`); + } + if (githubState.failedRuns.length > 0) { + findings.push( + `${githubState.failedRuns.length} completed workflow run(s) failed in the last ${recentWindowHours}h.`, + ); + } + + const verdict = findings.length > 0 ? "FLAG" : "PASS"; + const localNow = now.toLocaleString("en-US", { + timeZone: config.ownerTimezone, + hour12: false, + }); + + return { + verdict, + body: [ + `# Overnight Fork Watch: ${verdict}`, + "", + `Generated: ${now.toISOString()} (${config.ownerTimezone}: ${localNow})`, + `Repository: ${repoSlug}`, + `Upstream: ${config.upstreamRepo}`, + `Watch branch: \`${config.watchBranch}\``, + "", + "## Drift", + "", + `- origin/main: \`${gitState.forkHead}\``, + `- upstream/main: \`${gitState.upstreamHead}\``, + `- fork ahead: ${gitState.forkAhead}`, + `- fork behind: ${gitState.forkBehind}`, + `- watch branch on origin: ${ + gitState.watchBranchSha ? `yes (\`${gitState.watchBranchSha}\`)` : "no" + }`, + "", + "## Findings", + "", + findings.length ? findings.map((item) => `- ${item}`).join("\n") : "- None.", + "", + `## Fork Workflow Failures (${recentWindowHours}h)`, + "", + githubState.failedRuns.length + ? githubState.failedRuns.map(lineForRun).join("\n") + : "- None.", + "", + `## Upstream PRs Updated (${recentWindowHours}h)`, + "", + githubState.upstreamPulls.length + ? githubState.upstreamPulls.map(lineForPull).join("\n") + : "- None.", + "", + `## Fork PRs Updated (${recentWindowHours}h)`, + "", + githubState.forkPulls.length + ? githubState.forkPulls.map(lineForPull).join("\n") + : "- None.", + "", + "## Operator Notes", + "", + "- This issue is safe for public tracking: no local paths, host/IP values, or secrets are included.", + "- A GitHub issue created by `GITHUB_TOKEN` does not trigger secondary workflows, so this watch mirrors to Linear directly when `LINEAR_API_KEY` is available.", + ].join("\n"), + }; +} + +function ensureLabel(name, color, description) { + tryRun("gh", [ + "label", + "create", + name, + "--repo", + repoSlug, + "--color", + color, + "--description", + description, + ]); +} + +function issueHasLinearMarker(issueNumber) { + const comments = ghJson(`/repos/${repoSlug}/issues/${issueNumber}/comments?per_page=100`); + return comments.some((comment) => comment.body.includes("Linear:")); +} + +async function mirrorToLinear(issueNumber, title, body) { + if (!config.linearApiKey || issueHasLinearMarker(issueNumber)) { + return; + } + + const mutation = ` + mutation IssueCreate($input: IssueCreateInput!) { + issueCreate(input: $input) { + success + issue { id identifier url } + } + } + `; + + const response = await fetch("https://api.linear.app/graphql", { + method: "POST", + headers: { + Authorization: config.linearApiKey, + "Content-Type": "application/json", + "x-apollo-operation-name": "IssueCreate", + }, + body: JSON.stringify({ + query: mutation, + variables: { + input: { + title, + description: [ + `**Source**: https://github.com/${repoSlug}/issues/${issueNumber}`, + "", + "---", + "", + body, + ].join("\n"), + teamId: config.linearTeamId, + projectId: config.linearProjectId, + priority: 3, + }, + }, + }), + }); + + const data = await response.json(); + if (!response.ok || data.errors || !data?.data?.issueCreate?.success) { + ensureLabel("sync-failed", "D93F0B", "Linear sync workflow failed for this issue"); + tryRun("gh", [ + "issue", + "edit", + String(issueNumber), + "--repo", + repoSlug, + "--add-label", + "sync-failed", + ]); + throw new Error(`Linear API error: ${JSON.stringify(data)}`); + } + + const linearIssue = data.data.issueCreate.issue; + const marker = `Linear: [${linearIssue.identifier}](${linearIssue.url})\n\n_Auto-created by overnight-watch._`; + run("gh", ["issue", "comment", String(issueNumber), "--repo", repoSlug, "--body", marker]); +} + +function findExistingIssue(title) { + const issues = JSON.parse( + run("gh", [ + "issue", + "list", + "--repo", + repoSlug, + "--state", + "open", + "--label", + "overnight-watch", + "--json", + "number,title", + "--limit", + "20", + ]), + ); + return issues.find((issue) => issue.title === title); +} + +async function createOrUpdateTrackingIssue(report) { + if (!config.createTrackingIssue || report.verdict === "PASS") { + return; + } + + ensureLabel("overnight-watch", "1D76DB", "Automated overnight fork watch"); + ensureLabel("tracking", "5319E7", "Long-lived tracking item"); + ensureLabel("pr-mirror", "0E8A16", "Mirrored into Linear or Slack tracking"); + + const date = now.toISOString().slice(0, 10); + const title = `[watch] Overnight fork patrol: ${date}`; + const existing = findExistingIssue(title); + const tempDir = mkdtempSync(join(tmpdir(), "everos-watch-")); + const bodyFile = join(tempDir, "body.md"); + writeFileSync(bodyFile, report.body); + + if (existing) { + run("gh", [ + "issue", + "comment", + String(existing.number), + "--repo", + repoSlug, + "--body-file", + bodyFile, + ]); + await mirrorToLinear(existing.number, title, report.body); + return; + } + + const created = run("gh", [ + "issue", + "create", + "--repo", + repoSlug, + "--title", + title, + "--body-file", + bodyFile, + "--label", + "overnight-watch", + "--label", + "tracking", + "--label", + "pr-mirror", + ]); + const match = created.match(/\/issues\/(\d+)/); + if (match) { + await mirrorToLinear(Number(match[1]), title, report.body); + } +} + +async function main() { + const gitState = fetchGitState(); + const githubState = fetchGitHubState(); + const report = renderReport(gitState, githubState); + console.log(report.body); + await createOrUpdateTrackingIssue(report); +} + +main().catch((error) => { + console.error(error.message); + process.exit(1); +}); diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index a8eec180..aa74ce5f 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -103,6 +103,9 @@ jobs: banner_match = re.search(r"\[!\[[^\]]*\]\([^)]+\)\]\(([^)]+)\)", cell) primary_match = re.search(r"^\[(?:Code|Plugin|Live Demo)\]\(([^)]+)\)", cell, flags=re.M) + if "Coming soon" in cell and not primary_match: + continue + if not banner_match: failures.append(f"{path}: {title}: missing linked banner") elif not primary_match: diff --git a/.github/workflows/linear-sync.yml b/.github/workflows/linear-sync.yml new file mode 100644 index 00000000..c2af07de --- /dev/null +++ b/.github/workflows/linear-sync.yml @@ -0,0 +1,124 @@ +name: Linear sync for tracking mirrors + +on: + issues: + types: [opened, labeled] + +permissions: + issues: write + contents: read + +concurrency: + group: linear-sync-issue-${{ github.event.issue.number }} + cancel-in-progress: false + +jobs: + sync: + if: | + contains(github.event.issue.labels.*.name, 'pr-mirror') && + (github.event.action == 'opened' || github.event.label.name == 'pr-mirror') + runs-on: ubuntu-latest + steps: + - name: Mirror GitHub issue to Linear + uses: actions/github-script@v7 + env: + LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }} + LINEAR_TEAM_ID: ${{ vars.LINEAR_TEAM_ID }} + LINEAR_PROJECT_ID: ${{ vars.LINEAR_PROJECT_ID }} + with: + script: | + const issue = context.payload.issue; + const repo = `${context.repo.owner}/${context.repo.repo}`; + + // Idempotency: skip if Linear marker comment already exists + const { data: comments } = await github.rest.issues.listComments({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + per_page: 100 + }); + if (comments.some(c => c.body.includes('🔗 Linear:'))) { + core.info('Already synced (marker comment present). Skipping.'); + return; + } + + const isUrgent = issue.labels.some(l => l.name === 'urgent'); + const priority = isUrgent ? 1 : 3; // 1=Urgent, 3=Medium + + const description = [ + `**Source**: [${repo}#${issue.number}](${issue.html_url})`, + '', + '---', + '', + issue.body || '_(no body provided)_', + '', + '---', + '', + `_Auto-synced from GitHub by [linear-sync workflow](https://github.com/${repo}/actions)._` + ].join('\n'); + + const mutation = ` + mutation IssueCreate($input: IssueCreateInput!) { + issueCreate(input: $input) { + success + issue { id identifier url } + } + } + `; + + const response = await fetch('https://api.linear.app/graphql', { + method: 'POST', + headers: { + 'Authorization': process.env.LINEAR_API_KEY, + 'Content-Type': 'application/json', + 'x-apollo-operation-name': 'IssueCreate' + }, + body: JSON.stringify({ + query: mutation, + variables: { + input: { + title: issue.title, + description: description, + teamId: process.env.LINEAR_TEAM_ID, + projectId: process.env.LINEAR_PROJECT_ID, + priority: priority + } + } + }) + }); + + const data = await response.json(); + + if (!response.ok || data.errors || !data?.data?.issueCreate?.success) { + const errMsg = 'Linear API error: ' + JSON.stringify(data, null, 2); + core.error(errMsg); + + try { + await github.rest.issues.createLabel({ + owner: context.repo.owner, + repo: context.repo.repo, + name: 'sync-failed', + color: 'D93F0B', + description: 'Linear sync workflow failed for this issue' + }); + } catch (e) { /* exists already */ } + + await github.rest.issues.addLabels({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + labels: ['sync-failed'] + }); + + throw new Error(errMsg); + } + + const linearIssue = data.data.issueCreate.issue; + core.info(`Created ${linearIssue.identifier} -> ${linearIssue.url}`); + + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + body: `🔗 Linear: [${linearIssue.identifier}](${linearIssue.url})\n\n_Auto-created by linear-sync workflow._` + }); diff --git a/.github/workflows/overnight-watch.yml b/.github/workflows/overnight-watch.yml new file mode 100644 index 00000000..3a0e8489 --- /dev/null +++ b/.github/workflows/overnight-watch.yml @@ -0,0 +1,56 @@ +name: Fork overnight watch + +on: + schedule: + # 02:13 America/Los_Angeles during daylight time. GitHub cron is UTC. + - cron: "13 9 * * *" + workflow_dispatch: + inputs: + create_tracking_issue: + description: "Open/update a tracking issue when the watch verdict is FLAG" + required: false + type: boolean + default: true + watch_branch: + description: "Fork playground branch to verify on origin" + required: false + type: string + default: "codex-watch-overnight-2026-05-13" + +permissions: + actions: read + contents: read + issues: write + pull-requests: read + +concurrency: + group: overnight-watch-${{ github.ref }} + cancel-in-progress: false + +jobs: + watch: + if: github.repository == 'Fearvox/EverOS' + runs-on: ubuntu-latest + env: + REPO_OWNER: ${{ vars.REPO_OWNER || 'Fearvox' }} + REPO_NAME: ${{ vars.REPO_NAME || 'EverOS' }} + UPSTREAM_REPO: EverMind-AI/EverOS + WATCH_BRANCH: ${{ inputs.watch_branch || vars.WATCH_BRANCH || 'codex-watch-overnight-2026-05-13' }} + OWNER_TIMEZONE: ${{ vars.OWNER_TIMEZONE || 'America/Los_Angeles' }} + LINEAR_TEAM_ID: ${{ vars.LINEAR_TEAM_ID || '233391d6-ec9e-4aa8-b534-16a221b8119a' }} + LINEAR_PROJECT_ID: ${{ vars.LINEAR_PROJECT_ID || '39aa3865-345c-4313-9dc0-ab3b509c5d21' }} + LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }} + GH_TOKEN: ${{ github.token }} + GITHUB_TOKEN: ${{ github.token }} + CREATE_TRACKING_ISSUE: ${{ inputs.create_tracking_issue || github.event_name == 'schedule' }} + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - uses: actions/setup-node@v4 + with: + node-version: 20 + + - name: Run overnight watch + run: node .github/scripts/overnight-watch.mjs diff --git a/.github/workflows/sync-upstream.yml b/.github/workflows/sync-upstream.yml new file mode 100644 index 00000000..bfd8d884 --- /dev/null +++ b/.github/workflows/sync-upstream.yml @@ -0,0 +1,61 @@ +name: Sync fork from upstream + +on: + schedule: + - cron: '17 */6 * * *' # 每 6 小时跑一次,分钟 17 错开高峰 + workflow_dispatch: # 也支持手动触发 + +permissions: + contents: write + issues: write + +jobs: + sync: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Configure git + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Add upstream remote + run: git remote add upstream https://github.com/EverMind-AI/EverOS.git + + - name: Fetch upstream + run: git fetch upstream + + - name: Rebase fork main onto upstream/main + id: rebase + run: | + git checkout main + set +e + git rebase upstream/main + rc=$? + if [ $rc -ne 0 ]; then + git rebase --abort + echo "conflict=true" >> "$GITHUB_OUTPUT" + exit $rc + fi + echo "conflict=false" >> "$GITHUB_OUTPUT" + + - name: Push to fork main + if: steps.rebase.outputs.conflict == 'false' + run: git push origin main --force-with-lease + + - name: Open issue on conflict + if: failure() && steps.rebase.outputs.conflict == 'true' + uses: actions/github-script@v7 + with: + script: | + await github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `[sync] Rebase conflict syncing fork from upstream (${new Date().toISOString().slice(0,10)})`, + body: `Auto-sync from \`upstream/main\` failed due to rebase conflicts.\n\nRun id: ${context.runId}\nWorkflow: ${context.workflow}\n\nResolve manually:\n\n\`\`\`\ncd ~/EverOS && git fetch upstream && git rebase upstream/main\n# resolve conflicts, then:\ngit push origin main --force-with-lease\n\`\`\``, + labels: ['tracking'] + }); diff --git a/.planning/may-agent/20-rust-runtime-scaffold.md b/.planning/may-agent/20-rust-runtime-scaffold.md new file mode 100644 index 00000000..f56375aa --- /dev/null +++ b/.planning/may-agent/20-rust-runtime-scaffold.md @@ -0,0 +1,305 @@ +# 20 — Rust Runtime Scaffold: Crate Structure, Traits, Prior Art + +**Status**: Draft +**Date**: 2026-05-13 +**Depends on**: 10-architecture.md, hermes-recon/architecture.md + +## TL;DR + +The Rust runtime replaces Hermes's Python CLI (615K LOC), TUI/Gateway layer, +and environment sandboxes with a tokio-based async runtime. It embeds the +Python agent core via PyO3 and provides a seccomp-bpf sandbox for tool +execution. Prior art: Tauri (desktop shell), tokio (async runtime), Burn +(ML framework), candle (inference). + +## Crate Map + +``` +may-agent/ # workspace root +├── Cargo.toml # workspace manifest +├── crates/ +│ ├── may-agent-cli/ # binary crate — entry point +│ │ ├── Cargo.toml +│ │ └── src/ +│ │ ├── main.rs # clap derive, subcommand dispatch +│ │ ├── commands/ +│ │ │ ├── run.rs # `may-agent run` +│ │ │ ├── gateway.rs # `may-agent gateway` +│ │ │ ├── mcp.rs # `may-agent mcp` +│ │ │ └── plugin.rs # `may-agent plugin` +│ │ └── config.rs # figment loader +│ │ +│ ├── may-agent-runtime/ # core runtime library +│ │ ├── Cargo.toml +│ │ └── src/ +│ │ ├── lib.rs # re-exports +│ │ ├── agent.rs # AgentHandle — Python subprocess mgmt +│ │ ├── plugin.rs # PluginRegistry — WASM + FFI loader +│ │ ├── sandbox.rs # Sandbox — seccomp/macos/fallback +│ │ ├── gateway.rs # GatewayService — tower service stack +│ │ ├── memory.rs # MemoryClient — Evercore HTTP client +│ │ └── telemetry.rs # tracing spans + metrics +│ │ +│ ├── may-agent-ffi/ # PyO3 bridge +│ │ ├── Cargo.toml +│ │ └── src/ +│ │ ├── lib.rs # #[pymodule] +│ │ ├── dispatch.rs # tool_dispatch() +│ │ ├── sandbox.rs # sandbox_run() +│ │ └── gateway.rs # gateway_deliver() +│ │ +│ ├── may-agent-desktop/ # Tauri shell (future) +│ │ ├── Cargo.toml +│ │ ├── tauri.conf.json +│ │ └── src/ +│ │ └── main.rs +│ │ +│ └── may-agent-sandbox/ # Standalone sandbox binary +│ ├── Cargo.toml +│ └── src/ +│ └── main.rs # seccomp worker process +│ +├── python/ # embedded Python (submodule: Hermes fork) +│ └── hermes-agent/ # git submodule at pinned SHA +│ +└── tests/ + ├── integration/ + └── sandbox-escape/ +``` + +## Core Traits + +### AgentHandle + +```rust +/// Manages the lifecycle of the embedded Python agent process. +#[async_trait] +pub trait AgentHandle: Send + Sync { + /// Start the Python agent subprocess. + async fn start(&self, config: AgentConfig) -> Result<(), AgentError>; + + /// Send a user message and await response. + async fn send(&self, msg: UserMessage) -> Result; + + /// Stop the agent subprocess gracefully. + async fn stop(&self) -> Result<(), AgentError>; + + /// Check if agent subprocess is alive. + fn is_alive(&self) -> bool; +} +``` + +**Implementation**: Spawns `python3 -m hermes_agent --mode headless` as a +tokio-managed subprocess. Communicates via JSON-RPC over stdin/stdout. + +### ToolDispatcher + +```rust +/// Dispatch a tool call to either the Python agent or a native Rust tool. +#[async_trait] +pub trait ToolDispatcher: Send + Sync { + /// Resolve which backend handles this tool. + fn resolve(&self, tool_name: &str) -> ToolBackend; + + /// Execute the tool and return result. + async fn execute(&self, call: ToolCall) -> Result; + + /// List available tools (merged Python + native). + fn list_tools(&self) -> Vec; +} + +pub enum ToolBackend { + /// Dispatch to Python agent (most tools). + Python, + /// Execute natively in Rust (performance-critical or sandbox-required). + Native, + /// Route through Evercore for memory operations. + Evercore, +} +``` + +### Sandbox + +```rust +/// Isolate tool execution from the host system. +#[async_trait] +pub trait Sandbox: Send + Sync { + /// Execute a command in the sandbox. + async fn execute(&self, cmd: SandboxCommand) -> Result; + + /// Check if the sandbox is intact (no escape detected). + fn health_check(&self) -> SandboxHealth; + + /// Get the sandbox backend name for logging. + fn backend(&self) -> &'static str; +} + +pub enum SandboxBackend { + /// seccomp-bpf (Linux) — syscall-level filtering. + Seccomp, + /// macOS Seatbelt — sandbox_init_with_parameters(). + Seatbelt, + /// Docker container — fallback when kernel sandbox unavailable. + Docker, + /// No sandbox — development mode only. + None, +} +``` + +### MemoryClient + +```rust +/// Client for Evercore's REST memory API. +#[async_trait] +pub trait MemoryClient: Send + Sync { + /// Retrieve relevant memories for a query. + async fn retrieve(&self, req: RetrieveRequest) -> Result, MemoryError>; + + /// Store a new memory entry. + async fn store(&self, req: StoreRequest) -> Result; + + /// Get session info. + async fn get_session(&self, session_id: &str) -> Result; + + /// Create a new session. + async fn create_session(&self, req: CreateSessionRequest) -> Result; +} + +pub struct RetrieveRequest { + pub session_id: String, + pub query: String, + pub top_k: usize, + pub tenant_id: String, +} +``` + +## Prior Art Analysis + +### Tauri — Desktop Shell Model + +**Repo**: `tauri-apps/tauri` (MIT, Rust) +**Relevance**: Desktop app shell for May Agent Desktop + +| Feature | Tauri approach | May Agent adoption | +|---------|---------------|-------------------| +| Window management | WRY (WebView) | Keep Hermes Web UI in Tauri window | +| IPC | `invoke` + `emit` (JSON) | Replace with agent JSON-RPC | +| Plugin system | Rust-side plugins | Reuse Tauri plugin model for agent plugins | +| Binary size | ~5MB base | Target: < 50MB with Python embed | +| Security | CSP + isolation pattern | Add seccomp sandbox to Tauri's CSP | + +### tokio — Async Runtime + +**Repo**: `tokio-rs/tokio` (MIT, Rust) +**Relevance**: Foundation for all async I/O in May Agent + +| Feature | tokio approach | May Agent adoption | +|---------|---------------|-------------------| +| I/O model | Mio + futures | Standard — all I/O is tokio | +| Task spawning | `tokio::spawn` | Agent sessions are tasks | +| Channels | `mpsc` / `broadcast` | Agent ↔ Gateway communication | +| Timers | `tokio::time` | Session timeouts, heartbeat | +| Signal handling | `tokio::signal` | Graceful shutdown | + +### Burn — ML Framework + +**Repo**: `tracel-ai/burn` (MIT/Apache 2.0, Rust) +**Relevance**: Optional — for local model inference + +| Feature | Burn approach | May Agent adoption | +|---------|---------------|-------------------| +| Model format | Burn native + ONNX import | If local inference needed | +| Backend | WGPU, Candle, NdArray | WGPU for cross-platform | +| Training | Autodiff + optimizers | Not needed (inference only) | +| **Verdict** | — | 🕐 Future — not in May 31 scope | + +### candle — Inference + +**Repo**: `huggingface/candle` (MIT/Apache 2.0, Rust) +**Relevance**: Optional — for local model inference + +| Feature | candle approach | May Agent adoption | +|---------|---------------|-------------------| +| Model loading | safetensors + GGUF | If local LLM needed | +| Quantization | GGML, GPTQ | For edge deployment | +| **Verdict** | — | 🕐 Future — not in May 31 scope | + +## Workspace Cargo.toml + +```toml +[workspace] +resolver = "2" +members = [ + "crates/may-agent-cli", + "crates/may-agent-runtime", + "crates/may-agent-ffi", + "crates/may-agent-desktop", + "crates/may-agent-sandbox", +] + +[workspace.dependencies] +tokio = { version = "1", features = ["full"] } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +tracing = "0.1" +tracing-subscriber = "0.3" +thiserror = "1" +async-trait = "0.1" + +[workspace.package] +version = "0.1.0" +edition = "2024" +license = "MIT" +repository = "https://github.com/Fearvox/may-agent" +``` + +## Build Pipeline + +```bash +# Development +cargo build --workspace + +# With Python embed +cargo build --workspace --features python-embed + +# Release (single binary with embedded Python) +cargo build --release --features python-embed,sandbox-seccomp + +# Test +cargo test --workspace +cargo test --workspace --features sandbox-seccomp -- --ignored # sandbox tests + +# Lint +cargo clippy --workspace -- -D warnings +cargo fmt --check --all +``` + +## Risk Assessment + +| Risk | Likelihood | Impact | Mitigation | +|------|-----------|--------|------------| +| PyO3 version conflicts | Medium | High | Pin PyO3 + Python minor version | +| seccomp too restrictive | Medium | Medium | Allowlist approach, test with Evil Agent Bench | +| Python GIL contention | High | Medium | Subprocess-per-session (no GIL sharing) | +| Cross-compilation pain | Medium | Low | CI matrix: linux-x86_64, macos-arm64 | +| Team Rust learning curve | High | Medium | Nolan owns Rust layer; team stays Python | + +## Files to Create (Sprint 1) + +1. `Cargo.toml` — workspace manifest +2. `crates/may-agent-cli/` — skeleton with clap +3. `crates/may-agent-runtime/` — AgentHandle + Sandbox + MemoryClient traits +4. `crates/may-agent-ffi/` — minimal PyO3 bridge +5. `crates/may-agent-sandbox/` — seccomp worker binary +6. `.github/workflows/rust-ci.yml` — build + test + lint + +## References + +- 10-architecture.md — system design with mermaid diagrams +- hermes-recon/architecture.md — Hermes internals +- Tauri: https://github.com/tauri-apps/tauri (MIT) +- tokio: https://github.com/tokio-rs/tokio (MIT) +- Burn: https://github.com/tracel-ai/burn (MIT/Apache 2.0) +- candle: https://github.com/huggingface/candle (MIT/Apache 2.0) +- PyO3: https://github.com/PyO3/pyo3 (MIT/Apache 2.0) +- `CLAUDE_DESKTOP_SANDBOX_SOURCE_TRUTH.md` — macOS Seatbelt reference diff --git a/docs/fork-playground/overnight-watch.md b/docs/fork-playground/overnight-watch.md new file mode 100644 index 00000000..379fd21b --- /dev/null +++ b/docs/fork-playground/overnight-watch.md @@ -0,0 +1,54 @@ +# Overnight Fork Watch + +This fork can move fast, but the upstream feed should stay boring and auditable. +The overnight watch is a small GitHub Actions patrol for `Fearvox/EverOS`. + +## What It Checks + +- `origin/main` drift against `EverMind-AI/EverOS` `upstream/main`. +- Whether the active playground branch exists on the fork: + `codex-watch-overnight-2026-05-13`. +- Failed, cancelled, or timed-out fork workflow runs in the last 24 hours. +- Upstream and fork pull requests updated in the last 24 hours. + +## Tracking Behavior + +The workflow prints a public-safe report on every run. If the verdict is `FLAG`, +it opens or updates a GitHub issue labeled: + +- `overnight-watch` +- `tracking` +- `pr-mirror` + +Issues created by `GITHUB_TOKEN` do not trigger secondary workflows. Because of +that, the watch mirrors the tracking issue to Linear directly when +`LINEAR_API_KEY` is available. The target Linear team/project are: + +- Team: `233391d6-ec9e-4aa8-b534-16a221b8119a` +- Project: `39aa3865-345c-4313-9dc0-ab3b509c5d21` + +A `FLAG` verdict does not fail the watch workflow by itself. Runtime errors +still fail the workflow, but expected drift or downstream failures are reported +through the tracking issue so the watch does not poison its own next run. + +## Manual Run + +```bash +REPO_OWNER=Fearvox \ +REPO_NAME=EverOS \ +WATCH_BRANCH=codex-watch-overnight-2026-05-13 \ +OWNER_TIMEZONE=America/Los_Angeles \ +LINEAR_TEAM_ID=233391d6-ec9e-4aa8-b534-16a221b8119a \ +LINEAR_PROJECT_ID=39aa3865-345c-4313-9dc0-ab3b509c5d21 \ +CREATE_TRACKING_ISSUE=false \ +node .github/scripts/overnight-watch.mjs +``` + +Set `CREATE_TRACKING_ISSUE=true` only when you want the local run to mutate +GitHub issues. + +## Public-Surface Hygiene + +Reports intentionally avoid local absolute paths, host/IP values, token names +beyond the required GitHub secret names, and operator-only commands. They should +be safe to show in Discord or a screen share.