diff --git a/.github/ISSUE_TEMPLATE/pr_tracker.yml b/.github/ISSUE_TEMPLATE/pr_tracker.yml new file mode 100644 index 00000000..1dbe050a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/pr_tracker.yml @@ -0,0 +1,109 @@ +name: PR tracker (mirror) +description: Mirror an in-flight pull request as a tracking issue for Linear and Slack sync / 镜像一个进行中的 PR 作为追踪 issue +title: "[PR Track] #: " +labels: ["pr-mirror", "tracking"] +body: + - type: markdown + attributes: + value: | + Use this when you want a long-lived, auditable record of an upstream PR. + Linear and Slack subscribe to issues with the `pr-mirror` label. + 镜像一个 PR 用于 Linear / Slack 长期可审计追踪。带 `pr-mirror` 标签的 issue 会被订阅。 + - type: input + id: pr_number + attributes: + label: PR number + placeholder: "e.g. 196" + validations: + required: true + - type: input + id: pr_url + attributes: + label: PR URL + placeholder: https://github.com/EverMind-AI/EverOS/pull/ + validations: + required: true + - type: input + id: author + attributes: + label: Author handle + placeholder: "@github-login" + validations: + required: true + - type: dropdown + id: area + attributes: + label: Area + options: + - methods/EverCore + - methods/HyperMem + - benchmarks/EverMemBench + - benchmarks/EvoAgentBench + - use-cases + - documentation + - CI / build / release + - other + validations: + required: true + - type: dropdown + id: lane + attributes: + label: Review lane + description: How this PR should be triaged. / 该 PR 的优先级处理通道。 + options: + - hotfix (block release until merged) + - normal (standard review) + - docs-only (light review) + - exploratory (no merge intent) + validations: + required: true + - type: textarea + id: scope + attributes: + label: Scope summary + description: One paragraph. What does the PR change, and what is intentionally left out? + placeholder: | + Changes: + - ... + Out of scope: + - ... + validations: + required: true + - type: textarea + id: evidence + attributes: + label: Evidence snapshot + description: | + Required before this mirror can be closed. Paste the CI summary, test command output, + or the link to the run. "No mirror closes without evidence." + 关闭镜像 issue 前必填。粘贴 CI 摘要、测试命令输出或 run 链接。 + render: shell + validations: + required: true + - type: textarea + id: decisions + attributes: + label: Decision log + description: Notable review decisions (approvals, requested changes, deferrals). + placeholder: | + - 2026-05-13 @reviewer: requested change on tests/test_x.py + - 2026-05-13 @author: scoped follow-up to PR #... + - type: input + id: linear_issue + attributes: + label: Linear issue (optional) + placeholder: "EVE-123" + - type: input + id: slack_thread + attributes: + label: Slack thread (optional) + placeholder: "https://everminddash.slack.com/archives/.../p..." + - type: checkboxes + id: closure + attributes: + label: Closure criteria + description: Check all that apply before closing this mirror. + options: + - label: PR is merged, closed, or marked won't-fix upstream. + - label: Evidence snapshot above reflects the final state. + - label: Linear and Slack records have been updated (if linked). diff --git a/.github/ISSUE_TEMPLATE/security_tracker.yml b/.github/ISSUE_TEMPLATE/security_tracker.yml new file mode 100644 index 00000000..2f2c96c8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/security_tracker.yml @@ -0,0 +1,116 @@ +name: Security tracker (mirror) +description: Mirror a security PR or disclosure for Linear and Slack escalation / 镜像一个安全 PR 或披露 +title: "[Security Track] CWE-: " +labels: ["security", "pr-mirror", "tracking", "urgent"] +body: + - type: markdown + attributes: + value: | + Use this for any PR or disclosure that affects credentials, authn/authz, data exposure, + supply chain, or sandbox escape. The `urgent` label routes this to high-priority + notifications in Slack and Linear. + 用于凭证、认证授权、数据暴露、供应链、沙箱逃逸等安全 PR / 披露。`urgent` 标签会触发高优先级通知。 + Do NOT include exploit details that are not already public in the upstream PR. + 请勿写入未在 upstream PR 公开的利用细节。 + - type: input + id: cwe + attributes: + label: CWE id + placeholder: "CWE-798" + validations: + required: true + - type: input + id: pr_url + attributes: + label: Upstream PR or advisory URL + placeholder: https://github.com/EverMind-AI/EverOS/pull/ + validations: + required: true + - type: dropdown + id: severity + attributes: + label: Severity + options: + - Critical (full auth bypass / unauthenticated RCE / mass data loss) + - High (privileged data access / credential exposure / persistent compromise) + - Medium (limited data access / requires user interaction) + - Low (defense-in-depth / hardening) + validations: + required: true + - type: dropdown + id: exposure + attributes: + label: Reachability + description: How reachable is this in the documented quickstart / default config? + options: + - Default config (reproducible from a clean clone) + - Default config + network position + - Non-default config but documented + - Hypothetical / not yet reproducible + validations: + required: true + - type: textarea + id: affected + attributes: + label: Affected components + description: File paths, services, or versions impacted. + placeholder: | + - methods/EverCore/docker-compose.yaml (memsys-milvus-minio block) + - methods/EverCore/env.template + validations: + required: true + - type: textarea + id: fix_summary + attributes: + label: Proposed fix summary + description: One paragraph. What does the PR change? Cite the contract that makes it fail-closed. + validations: + required: true + - type: textarea + id: evidence + attributes: + label: Verification evidence + description: | + Required before closure. Show the commands and output that prove the fix works AND + that the unpatched state was exploitable. "No security mirror closes without evidence." + 关闭前必填。展示证明 fix 生效以及未修复状态可利用的命令与输出。 + render: shell + validations: + required: true + - type: textarea + id: residual + attributes: + label: Residual risk / follow-ups + description: Anything intentionally out of scope, plus follow-up issues that should be filed. + placeholder: | + - docs/installation/ still references the old default in examples; follow-up sweep needed. + - Consider adding a CI lint to catch hardcoded secrets in docker-compose files. + - type: input + id: linear_issue + attributes: + label: Linear issue (optional) + placeholder: "EVE-123" + - type: input + id: slack_thread + attributes: + label: Slack thread (optional) + placeholder: "https://everminddash.slack.com/archives/.../p..." + - type: checkboxes + id: disclosure + attributes: + label: Disclosure hygiene + description: Confirm before submitting. + options: + - label: This mirror contains no exploit details beyond what is already public in the upstream PR. + required: true + - label: The upstream PR or advisory link is correct and reachable. + required: true + - label: A maintainer has been pinged in Slack #p-evermind-dash or via Linear EVE if Severity is Critical or High. + - type: checkboxes + id: closure + attributes: + label: Closure criteria + options: + - label: Upstream PR merged, advisory published, or risk formally accepted. + - label: Verification evidence above reflects the merged state. + - label: Residual-risk follow-ups have issues filed (or explicitly waived). diff --git a/.github/scripts/overnight-watch.mjs b/.github/scripts/overnight-watch.mjs new file mode 100755 index 00000000..5fc2b1c4 --- /dev/null +++ b/.github/scripts/overnight-watch.mjs @@ -0,0 +1,371 @@ +#!/usr/bin/env node + +import { execFileSync } from "node:child_process"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +const config = { + repoOwner: process.env.REPO_OWNER || "Fearvox", + repoName: process.env.REPO_NAME || "EverOS", + upstreamRepo: process.env.UPSTREAM_REPO || "EverMind-AI/EverOS", + watchBranch: + process.env.WATCH_BRANCH || "codex-watch-overnight-2026-05-13", + ownerTimezone: process.env.OWNER_TIMEZONE || "America/Los_Angeles", + linearTeamId: + process.env.LINEAR_TEAM_ID || "233391d6-ec9e-4aa8-b534-16a221b8119a", + linearProjectId: + process.env.LINEAR_PROJECT_ID || "39aa3865-345c-4313-9dc0-ab3b509c5d21", + createTrackingIssue: + (process.env.CREATE_TRACKING_ISSUE || "false").toLowerCase() === "true", + githubToken: process.env.GITHUB_TOKEN || process.env.GH_TOKEN || "", + linearApiKey: process.env.LINEAR_API_KEY || "", +}; + +const repoSlug = `${config.repoOwner}/${config.repoName}`; +const recentWindowHours = Number(process.env.WATCH_WINDOW_HOURS || 24); +const since = new Date(Date.now() - recentWindowHours * 60 * 60 * 1000); +const now = new Date(); + +function run(command, args, options = {}) { + const result = execFileSync(command, args, { + encoding: "utf8", + stdio: ["ignore", "pipe", options.inheritStderr ? "inherit" : "pipe"], + env: { + ...process.env, + GH_TOKEN: config.githubToken || process.env.GH_TOKEN, + }, + }); + return result.trim(); +} + +function tryRun(command, args, fallback = "") { + try { + return run(command, args); + } catch (error) { + return fallback; + } +} + +function ghJson(endpoint) { + const output = run("gh", [ + "api", + "-H", + "Accept: application/vnd.github+json", + endpoint, + ]); + return JSON.parse(output); +} + +function ensureRemote(name, url) { + const remotes = tryRun("git", ["remote"]).split("\n").filter(Boolean); + if (!remotes.includes(name)) { + run("git", ["remote", "add", name, url]); + } +} + +function fetchGitState() { + ensureRemote("upstream", `https://github.com/${config.upstreamRepo}.git`); + tryRun("git", ["fetch", "origin", "--prune"], ""); + tryRun("git", ["fetch", "upstream", "main", "--prune"], ""); + + const forkHead = tryRun("git", ["rev-parse", "--short", "origin/main"], "unknown"); + const upstreamHead = tryRun( + "git", + ["rev-parse", "--short", "upstream/main"], + "unknown", + ); + const counts = tryRun( + "git", + ["rev-list", "--left-right", "--count", "origin/main...upstream/main"], + "0\t0", + ) + .split(/\s+/) + .map((value) => Number(value)); + + const watchBranchSha = tryRun( + "git", + ["ls-remote", "--heads", "origin", config.watchBranch], + "", + ) + .split(/\s+/)[0] + ?.slice(0, 12); + + return { + forkHead, + upstreamHead, + forkAhead: counts[0] || 0, + forkBehind: counts[1] || 0, + watchBranchSha: watchBranchSha || "", + }; +} + +function fetchGitHubState() { + const runs = ghJson( + `/repos/${repoSlug}/actions/runs?per_page=50&status=completed`, + ).workflow_runs || []; + const failedRuns = runs + .filter((runInfo) => new Date(runInfo.created_at) >= since) + .filter((runInfo) => + ["failure", "cancelled", "timed_out"].includes(runInfo.conclusion), + ) + .slice(0, 10); + + const upstreamPulls = ghJson( + `/repos/${config.upstreamRepo}/pulls?state=open&sort=updated&direction=desc&per_page=30`, + ) + .filter((pull) => new Date(pull.updated_at) >= since) + .slice(0, 10); + + const forkPulls = ghJson( + `/repos/${repoSlug}/pulls?state=open&sort=updated&direction=desc&per_page=30`, + ) + .filter((pull) => new Date(pull.updated_at) >= since) + .slice(0, 10); + + return { failedRuns, upstreamPulls, forkPulls }; +} + +function lineForPull(pull) { + return `- #${pull.number} ${pull.title} (${pull.user.login}, updated ${pull.updated_at})`; +} + +function lineForRun(runInfo) { + return `- ${runInfo.name}: ${runInfo.conclusion} (${runInfo.html_url})`; +} + +function renderReport(gitState, githubState) { + const findings = []; + + if (!gitState.watchBranchSha) { + findings.push(`Watch branch is not on origin: ${config.watchBranch}`); + } + if (gitState.forkBehind > 0) { + findings.push(`Fork main is behind upstream/main by ${gitState.forkBehind} commit(s).`); + } + if (githubState.failedRuns.length > 0) { + findings.push( + `${githubState.failedRuns.length} completed workflow run(s) failed in the last ${recentWindowHours}h.`, + ); + } + + const verdict = findings.length > 0 ? "FLAG" : "PASS"; + const localNow = now.toLocaleString("en-US", { + timeZone: config.ownerTimezone, + hour12: false, + }); + + return { + verdict, + body: [ + `# Overnight Fork Watch: ${verdict}`, + "", + `Generated: ${now.toISOString()} (${config.ownerTimezone}: ${localNow})`, + `Repository: ${repoSlug}`, + `Upstream: ${config.upstreamRepo}`, + `Watch branch: \`${config.watchBranch}\``, + "", + "## Drift", + "", + `- origin/main: \`${gitState.forkHead}\``, + `- upstream/main: \`${gitState.upstreamHead}\``, + `- fork ahead: ${gitState.forkAhead}`, + `- fork behind: ${gitState.forkBehind}`, + `- watch branch on origin: ${ + gitState.watchBranchSha ? `yes (\`${gitState.watchBranchSha}\`)` : "no" + }`, + "", + "## Findings", + "", + findings.length ? findings.map((item) => `- ${item}`).join("\n") : "- None.", + "", + `## Fork Workflow Failures (${recentWindowHours}h)`, + "", + githubState.failedRuns.length + ? githubState.failedRuns.map(lineForRun).join("\n") + : "- None.", + "", + `## Upstream PRs Updated (${recentWindowHours}h)`, + "", + githubState.upstreamPulls.length + ? githubState.upstreamPulls.map(lineForPull).join("\n") + : "- None.", + "", + `## Fork PRs Updated (${recentWindowHours}h)`, + "", + githubState.forkPulls.length + ? githubState.forkPulls.map(lineForPull).join("\n") + : "- None.", + "", + "## Operator Notes", + "", + "- This issue is safe for public tracking: no local paths, host/IP values, or secrets are included.", + "- A GitHub issue created by `GITHUB_TOKEN` does not trigger secondary workflows, so this watch mirrors to Linear directly when `LINEAR_API_KEY` is available.", + ].join("\n"), + }; +} + +function ensureLabel(name, color, description) { + tryRun("gh", [ + "label", + "create", + name, + "--repo", + repoSlug, + "--color", + color, + "--description", + description, + ]); +} + +function issueHasLinearMarker(issueNumber) { + const comments = ghJson(`/repos/${repoSlug}/issues/${issueNumber}/comments?per_page=100`); + return comments.some((comment) => comment.body.includes("Linear:")); +} + +async function mirrorToLinear(issueNumber, title, body) { + if (!config.linearApiKey || issueHasLinearMarker(issueNumber)) { + return; + } + + const mutation = ` + mutation IssueCreate($input: IssueCreateInput!) { + issueCreate(input: $input) { + success + issue { id identifier url } + } + } + `; + + const response = await fetch("https://api.linear.app/graphql", { + method: "POST", + headers: { + Authorization: config.linearApiKey, + "Content-Type": "application/json", + "x-apollo-operation-name": "IssueCreate", + }, + body: JSON.stringify({ + query: mutation, + variables: { + input: { + title, + description: [ + `**Source**: https://github.com/${repoSlug}/issues/${issueNumber}`, + "", + "---", + "", + body, + ].join("\n"), + teamId: config.linearTeamId, + projectId: config.linearProjectId, + priority: 3, + }, + }, + }), + }); + + const data = await response.json(); + if (!response.ok || data.errors || !data?.data?.issueCreate?.success) { + ensureLabel("sync-failed", "D93F0B", "Linear sync workflow failed for this issue"); + tryRun("gh", [ + "issue", + "edit", + String(issueNumber), + "--repo", + repoSlug, + "--add-label", + "sync-failed", + ]); + throw new Error(`Linear API error: ${JSON.stringify(data)}`); + } + + const linearIssue = data.data.issueCreate.issue; + const marker = `Linear: [${linearIssue.identifier}](${linearIssue.url})\n\n_Auto-created by overnight-watch._`; + run("gh", ["issue", "comment", String(issueNumber), "--repo", repoSlug, "--body", marker]); +} + +function findExistingIssue(title) { + const issues = JSON.parse( + run("gh", [ + "issue", + "list", + "--repo", + repoSlug, + "--state", + "open", + "--label", + "overnight-watch", + "--json", + "number,title", + "--limit", + "20", + ]), + ); + return issues.find((issue) => issue.title === title); +} + +async function createOrUpdateTrackingIssue(report) { + if (!config.createTrackingIssue || report.verdict === "PASS") { + return; + } + + ensureLabel("overnight-watch", "1D76DB", "Automated overnight fork watch"); + ensureLabel("tracking", "5319E7", "Long-lived tracking item"); + ensureLabel("pr-mirror", "0E8A16", "Mirrored into Linear or Slack tracking"); + + const date = now.toISOString().slice(0, 10); + const title = `[watch] Overnight fork patrol: ${date}`; + const existing = findExistingIssue(title); + const tempDir = mkdtempSync(join(tmpdir(), "everos-watch-")); + const bodyFile = join(tempDir, "body.md"); + writeFileSync(bodyFile, report.body); + + if (existing) { + run("gh", [ + "issue", + "comment", + String(existing.number), + "--repo", + repoSlug, + "--body-file", + bodyFile, + ]); + await mirrorToLinear(existing.number, title, report.body); + return; + } + + const created = run("gh", [ + "issue", + "create", + "--repo", + repoSlug, + "--title", + title, + "--body-file", + bodyFile, + "--label", + "overnight-watch", + "--label", + "tracking", + "--label", + "pr-mirror", + ]); + const match = created.match(/\/issues\/(\d+)/); + if (match) { + await mirrorToLinear(Number(match[1]), title, report.body); + } +} + +async function main() { + const gitState = fetchGitState(); + const githubState = fetchGitHubState(); + const report = renderReport(gitState, githubState); + console.log(report.body); + await createOrUpdateTrackingIssue(report); +} + +main().catch((error) => { + console.error(error.message); + process.exit(1); +}); diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index a8eec180..aa74ce5f 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -103,6 +103,9 @@ jobs: banner_match = re.search(r"\[!\[[^\]]*\]\([^)]+\)\]\(([^)]+)\)", cell) primary_match = re.search(r"^\[(?:Code|Plugin|Live Demo)\]\(([^)]+)\)", cell, flags=re.M) + if "Coming soon" in cell and not primary_match: + continue + if not banner_match: failures.append(f"{path}: {title}: missing linked banner") elif not primary_match: diff --git a/.github/workflows/linear-sync.yml b/.github/workflows/linear-sync.yml new file mode 100644 index 00000000..c2af07de --- /dev/null +++ b/.github/workflows/linear-sync.yml @@ -0,0 +1,124 @@ +name: Linear sync for tracking mirrors + +on: + issues: + types: [opened, labeled] + +permissions: + issues: write + contents: read + +concurrency: + group: linear-sync-issue-${{ github.event.issue.number }} + cancel-in-progress: false + +jobs: + sync: + if: | + contains(github.event.issue.labels.*.name, 'pr-mirror') && + (github.event.action == 'opened' || github.event.label.name == 'pr-mirror') + runs-on: ubuntu-latest + steps: + - name: Mirror GitHub issue to Linear + uses: actions/github-script@v7 + env: + LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }} + LINEAR_TEAM_ID: ${{ vars.LINEAR_TEAM_ID }} + LINEAR_PROJECT_ID: ${{ vars.LINEAR_PROJECT_ID }} + with: + script: | + const issue = context.payload.issue; + const repo = `${context.repo.owner}/${context.repo.repo}`; + + // Idempotency: skip if Linear marker comment already exists + const { data: comments } = await github.rest.issues.listComments({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + per_page: 100 + }); + if (comments.some(c => c.body.includes('🔗 Linear:'))) { + core.info('Already synced (marker comment present). Skipping.'); + return; + } + + const isUrgent = issue.labels.some(l => l.name === 'urgent'); + const priority = isUrgent ? 1 : 3; // 1=Urgent, 3=Medium + + const description = [ + `**Source**: [${repo}#${issue.number}](${issue.html_url})`, + '', + '---', + '', + issue.body || '_(no body provided)_', + '', + '---', + '', + `_Auto-synced from GitHub by [linear-sync workflow](https://github.com/${repo}/actions)._` + ].join('\n'); + + const mutation = ` + mutation IssueCreate($input: IssueCreateInput!) { + issueCreate(input: $input) { + success + issue { id identifier url } + } + } + `; + + const response = await fetch('https://api.linear.app/graphql', { + method: 'POST', + headers: { + 'Authorization': process.env.LINEAR_API_KEY, + 'Content-Type': 'application/json', + 'x-apollo-operation-name': 'IssueCreate' + }, + body: JSON.stringify({ + query: mutation, + variables: { + input: { + title: issue.title, + description: description, + teamId: process.env.LINEAR_TEAM_ID, + projectId: process.env.LINEAR_PROJECT_ID, + priority: priority + } + } + }) + }); + + const data = await response.json(); + + if (!response.ok || data.errors || !data?.data?.issueCreate?.success) { + const errMsg = 'Linear API error: ' + JSON.stringify(data, null, 2); + core.error(errMsg); + + try { + await github.rest.issues.createLabel({ + owner: context.repo.owner, + repo: context.repo.repo, + name: 'sync-failed', + color: 'D93F0B', + description: 'Linear sync workflow failed for this issue' + }); + } catch (e) { /* exists already */ } + + await github.rest.issues.addLabels({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + labels: ['sync-failed'] + }); + + throw new Error(errMsg); + } + + const linearIssue = data.data.issueCreate.issue; + core.info(`Created ${linearIssue.identifier} -> ${linearIssue.url}`); + + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + body: `🔗 Linear: [${linearIssue.identifier}](${linearIssue.url})\n\n_Auto-created by linear-sync workflow._` + }); diff --git a/.github/workflows/overnight-watch.yml b/.github/workflows/overnight-watch.yml new file mode 100644 index 00000000..3a0e8489 --- /dev/null +++ b/.github/workflows/overnight-watch.yml @@ -0,0 +1,56 @@ +name: Fork overnight watch + +on: + schedule: + # 02:13 America/Los_Angeles during daylight time. GitHub cron is UTC. + - cron: "13 9 * * *" + workflow_dispatch: + inputs: + create_tracking_issue: + description: "Open/update a tracking issue when the watch verdict is FLAG" + required: false + type: boolean + default: true + watch_branch: + description: "Fork playground branch to verify on origin" + required: false + type: string + default: "codex-watch-overnight-2026-05-13" + +permissions: + actions: read + contents: read + issues: write + pull-requests: read + +concurrency: + group: overnight-watch-${{ github.ref }} + cancel-in-progress: false + +jobs: + watch: + if: github.repository == 'Fearvox/EverOS' + runs-on: ubuntu-latest + env: + REPO_OWNER: ${{ vars.REPO_OWNER || 'Fearvox' }} + REPO_NAME: ${{ vars.REPO_NAME || 'EverOS' }} + UPSTREAM_REPO: EverMind-AI/EverOS + WATCH_BRANCH: ${{ inputs.watch_branch || vars.WATCH_BRANCH || 'codex-watch-overnight-2026-05-13' }} + OWNER_TIMEZONE: ${{ vars.OWNER_TIMEZONE || 'America/Los_Angeles' }} + LINEAR_TEAM_ID: ${{ vars.LINEAR_TEAM_ID || '233391d6-ec9e-4aa8-b534-16a221b8119a' }} + LINEAR_PROJECT_ID: ${{ vars.LINEAR_PROJECT_ID || '39aa3865-345c-4313-9dc0-ab3b509c5d21' }} + LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }} + GH_TOKEN: ${{ github.token }} + GITHUB_TOKEN: ${{ github.token }} + CREATE_TRACKING_ISSUE: ${{ inputs.create_tracking_issue || github.event_name == 'schedule' }} + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - uses: actions/setup-node@v4 + with: + node-version: 20 + + - name: Run overnight watch + run: node .github/scripts/overnight-watch.mjs diff --git a/.github/workflows/sync-upstream.yml b/.github/workflows/sync-upstream.yml new file mode 100644 index 00000000..bfd8d884 --- /dev/null +++ b/.github/workflows/sync-upstream.yml @@ -0,0 +1,61 @@ +name: Sync fork from upstream + +on: + schedule: + - cron: '17 */6 * * *' # 每 6 小时跑一次,分钟 17 错开高峰 + workflow_dispatch: # 也支持手动触发 + +permissions: + contents: write + issues: write + +jobs: + sync: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Configure git + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Add upstream remote + run: git remote add upstream https://github.com/EverMind-AI/EverOS.git + + - name: Fetch upstream + run: git fetch upstream + + - name: Rebase fork main onto upstream/main + id: rebase + run: | + git checkout main + set +e + git rebase upstream/main + rc=$? + if [ $rc -ne 0 ]; then + git rebase --abort + echo "conflict=true" >> "$GITHUB_OUTPUT" + exit $rc + fi + echo "conflict=false" >> "$GITHUB_OUTPUT" + + - name: Push to fork main + if: steps.rebase.outputs.conflict == 'false' + run: git push origin main --force-with-lease + + - name: Open issue on conflict + if: failure() && steps.rebase.outputs.conflict == 'true' + uses: actions/github-script@v7 + with: + script: | + await github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `[sync] Rebase conflict syncing fork from upstream (${new Date().toISOString().slice(0,10)})`, + body: `Auto-sync from \`upstream/main\` failed due to rebase conflicts.\n\nRun id: ${context.runId}\nWorkflow: ${context.workflow}\n\nResolve manually:\n\n\`\`\`\ncd ~/EverOS && git fetch upstream && git rebase upstream/main\n# resolve conflicts, then:\ngit push origin main --force-with-lease\n\`\`\``, + labels: ['tracking'] + }); diff --git a/.planning/may-agent/10-architecture.md b/.planning/may-agent/10-architecture.md new file mode 100644 index 00000000..15aa191e --- /dev/null +++ b/.planning/may-agent/10-architecture.md @@ -0,0 +1,264 @@ +# 10 — Architecture: May Agent System Design + +**Status**: Draft +**Date**: 2026-05-13 +**Depends on**: 00-vision.md, hermes-recon/architecture.md + +## System Overview + +```mermaid +graph TB + subgraph "User Surface" + WEB[Web UI
Hermes Workspace] + TUI[TUI
Ink/React] + MSG[Messaging
WeCom/Feishu/Slack] + end + + subgraph "Rust Runtime (new)" + CLI[Rust CLI
clap + tokio] + PLUGIN[Plugin Loader
WASM + FFI] + SANDBOX[Sandbox
tokio + seccomp] + GATEWAY[Messaging Gateway
tower + hyper] + end + + subgraph "Python Agent Core (forked)" + AGENT[AIAgent
run_agent.py] + TOOLS[Tool Registry
80+ tools] + MODEL[Model Providers
OpenRouter/Anthropic/GMI] + end + + subgraph "Evercore Memory Kernel" + MEMGR[Memory Manager] + API[REST API
FastAPI] + STORE[Tenant Store
PostgreSQL + pgvector] + PROMPT[Prompt Engine
EN/ZH variants] + end + + subgraph "Evaluation" + EMB[EverMem Bench] + EAB[EvoAgent Bench] + EAB2[Evil Agent Bench] + end + + WEB --> CLI + TUI --> CLI + MSG --> GATEWAY + CLI --> PLUGIN + CLI --> SANDBOX + GATEWAY --> AGENT + PLUGIN --> AGENT + AGENT --> TOOLS + AGENT --> MODEL + AGENT --> MEMGR + MEMGR --> API + API --> STORE + PROMPT --> MEMGR + MEMGR --> EMB + AGENT --> EAB + SANDBOX --> EAB2 +``` + +## Component Architecture + +### Layer 1: Rust Runtime + +``` +may-agent/ +├── may-agent-cli/ # CLI binary (clap) +├── may-agent-runtime/ # Core runtime (tokio) +│ ├── plugin-loader/ # WASM + FFI plugin host +│ ├── sandbox/ # Process sandbox (seccomp-bpf on Linux) +│ ├── gateway/ # Messaging gateway (tower services) +│ └── config/ # Config loading (figment/serde) +├── may-agent-ffi/ # Python ↔ Rust FFI (PyO3) +├── may-agent-desktop/ # Tauri desktop shell +└── may-agent-mobile/ # Mobile FFI stubs (future) +``` + +### Layer 2: Python Agent Core (forked from Hermes) + +- `run_agent.py` — AIAgent class (forked, ~12K LOC) +- `model_tools.py` — tool orchestration (forked) +- `tools/` — 80+ tool modules (forked, auto-discovered via registry) +- `plugins/model-providers/` — inference backends (forked) +- `plugins/context_engine/` — context augmentation (extended with Evercore) + +### Layer 3: Evercore Memory Kernel + +- `memory_manager.py` — core memory operations (tenant-scoped) +- REST API controllers — CRUD for memory entries (`src/infra_layer/adapters/input/api/`) +- Prompt engine — bilingual EN/ZH prompt variants (`src/memory_layer/prompts/`) +- Tenant storage — PostgreSQL + pgvector (EverCore infrastructure) + +## Data Flow + +```mermaid +sequenceDiagram + participant U as User (WeCom/Feishu) + participant GW as Rust Gateway + participant AG as AIAgent (Python) + participant TR as Tool Registry + participant MM as Memory Manager + participant DB as Tenant Store + + U->>GW: 帮我回顾上次关于 Rust runtime 的讨论 + GW->>AG: dispatch(user_message, session_id) + AG->>MM: retrieve(session_id, query, top_k=5) + MM->>DB: vector search query + DB-->>MM: [memory_entry, ...] + MM-->>AG: [relevant_contexts] + AG->>AG: construct_prompt(user_message, contexts) + AG->>TR: execute_tool(tool_call) + TR-->>AG: tool_result + AG-->>GW: response_text + GW-->>U: 根据上次讨论,我们选择了 tokio... + AG->>MM: store(session_id, memory_entry) + MM->>DB: insert memory entry +``` + +## Decision Matrix + +### Runtime Language + +| Criterion | Python (fork only) | Rust + Python (hybrid) | Pure Rust | +|-----------|-------------------|----------------------|-----------| +| Dev speed | Fast (existing code) | Medium (FFI glue) | Slow (rewrite all) | +| Memory safety | GIL only | Seccomp + type safety | Type safety everywhere | +| Binary size | ~200MB (venv) | ~50MB (Rust + Python embed) | ~20MB | +| Team familiarity | High (Python-first team) | Mixed (Nolan + 七仔) | Low (team Python-native) | +| Sandbox quality | Docker-based | Seccomp-bpf (kernel) | Full control | +| May 31 viability | Ships | Ships (MVP) | Too slow | +| **Verdict** | — | ✅ **Chosen** | — | + +### Memory Backend + +| Criterion | Honcho (Hermes default) | mem0 | Evercore | +|-----------|------------------------|------|----------| +| Multi-tenant | No | No | Yes (built-in) | +| Long-term recall | Basic | Vector only | Prompt-optimized | +| EN/ZH prompts | No | No | Yes (bilingual) | +| Benchmarked | No | No | Yes (EverMem + EvoAgent) | +| Integration effort | 0 (already works) | Low (plugin exists) | Medium (FFI + API) | +| **Verdict** | — | — | ✅ **Chosen** | + +### Sandbox Strategy + +| Criterion | Docker | Seccomp-bpf (Rust) | macOS Seatbelt | +|-----------|--------|-------------------|----------------| +| Isolation quality | Good (container) | Best (syscall filter) | Good (kernel) | +| Cross-platform | Yes | Linux only | macOS only | +| Performance | ~200ms cold | ~5ms cold | ~5ms cold | +| Implementation | Existing (Hermes env) | New (tokio + libseccomp) | New (native sandbox_init) | +| **Verdict** | Fallback | ✅ **Primary** | Desktop only | + +### Messaging Gateway + +| Platform | Effort | Priority | Rationale | +|----------|--------|----------|-----------| +| WeCom (企业微信) | Medium | P0 | Chinese enterprise #1 | +| Feishu (飞书) | Medium | P0 | Chinese enterprise #2 | +| Slack | Low | P1 | International + internal use | +| Discord | Low | P1 | Community + Nolan's inter-PR | +| Tanka | High | P2 | New platform, less mature API | +| Telegram | Low | P3 | International fallback | + +## Crate Dependencies (Rust) + +```toml +# may-agent-runtime/Cargo.toml (proposed) +[dependencies] +tokio = { version = "1", features = ["full"] } +pyo3 = { version = "0.22", features = ["extension-module"] } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +clap = { version = "4", features = ["derive"] } +tower = "0.4" +hyper = "1" +figment = { version = "0.10", features = ["toml", "env"] } +tracing = "0.1" +tracing-subscriber = "0.3" + +[target.'cfg(target_os = "linux")'.dependencies] +libseccomp = "0.3" + +[target.'cfg(target_os = "macos")'.dependencies] +# macOS Seatbelt via sandbox_init (libsystem) +``` + +## Component Specification + +### Rust CLI + +```rust +// Proposed CLI structure using clap derive +#[derive(Parser)] +enum Command { + /// Start interactive agent session + Run { config: PathBuf }, + /// Start messaging gateway + Gateway { config: PathBuf, platform: String }, + /// Start MCP server + Mcp { config: PathBuf }, + /// Manage plugins + Plugin { subcommand: PluginCmd }, +} +``` + +### Python-Rust FFI Bridge + +Key function signatures for PyO3 bridge: + +```rust +// Rust side: Python-callable functions +#[pyfunction] +fn tool_dispatch(tool_name: &str, params: &str) -> PyResult; + +#[pyfunction] +fn sandbox_run(command: &str, env_vars: HashMap) -> PyResult; + +#[pyfunction] +fn gateway_deliver(platform: &str, chat_id: &str, message: &str) -> PyResult<()>; +``` + +### Memory Bridge + +The memory bridge connects the Rust runtime to Evercore's Python API: + +``` +Rust Gateway → POST /api/v1/memory/retrieve + POST /api/v1/memory/store + GET /api/v1/memory/session/{id} +``` + +Wire format: JSON, same schema as EverCore REST API controllers. +See `30-evercore-integration-contract.md` for full API contract. + +## Prior Art Analysis + +| Project | Language | Sandbox | Memory | License | Stars | +|---------|----------|---------|--------|---------|-------| +| Hermes Agent | Python | Docker/SSH | Pluggable (8 backends) | MIT | 147k | +| OpenClaw | TypeScript | Docker | Built-in | Proprietary | — | +| gbrain | TypeScript | Docker | Opinionated Hermes | MIT | 15k | +| Claude Code | TS + Native | macOS Seatbelt | Built-in | Proprietary | — | +| **May Agent** | **Rust + Python** | **Seccomp + Docker** | **Evercore** | **MIT (fork)** | **TBD** | + +## Integration Contract Outline + +See `30-evercore-integration-contract.md` for full specification. Key interfaces: + +1. **Memory API**: `retrieve(session_id, query, top_k) → [MemoryEntry]`, `store(session_id, entry) → MemoryId` +2. **Session API**: `create_session(tenant_id, config) → SessionId`, `get_session(session_id) → Session` +3. **Tool Proxy API**: `dispatch_tool(tool_name, params) → ToolResult` (optional — for sandboxed tool execution) +4. **Health API**: `GET /health → {status, version, tenants_active}` + +## References + +- 00-vision.md — strategy and success criteria +- hermes-recon/architecture.md — Hermes agent internals +- `methods/EverCore/src/agentic_layer/memory_manager.py` — core memory manager +- `methods/EverCore/src/infra_layer/adapters/input/api/` — REST API controllers +- Claude Desktop sandbox forensics: `CLAUDE_DESKTOP_SANDBOX_SOURCE_TRUTH.md` +- Tauri: https://tauri.app (desktop shell reference) +- tokio: https://tokio.rs (async runtime reference) +- Hermes upstream: `NousResearch/hermes-agent` (SHA: HEAD 2026-05-13) diff --git a/docs/fork-playground/overnight-watch.md b/docs/fork-playground/overnight-watch.md new file mode 100644 index 00000000..379fd21b --- /dev/null +++ b/docs/fork-playground/overnight-watch.md @@ -0,0 +1,54 @@ +# Overnight Fork Watch + +This fork can move fast, but the upstream feed should stay boring and auditable. +The overnight watch is a small GitHub Actions patrol for `Fearvox/EverOS`. + +## What It Checks + +- `origin/main` drift against `EverMind-AI/EverOS` `upstream/main`. +- Whether the active playground branch exists on the fork: + `codex-watch-overnight-2026-05-13`. +- Failed, cancelled, or timed-out fork workflow runs in the last 24 hours. +- Upstream and fork pull requests updated in the last 24 hours. + +## Tracking Behavior + +The workflow prints a public-safe report on every run. If the verdict is `FLAG`, +it opens or updates a GitHub issue labeled: + +- `overnight-watch` +- `tracking` +- `pr-mirror` + +Issues created by `GITHUB_TOKEN` do not trigger secondary workflows. Because of +that, the watch mirrors the tracking issue to Linear directly when +`LINEAR_API_KEY` is available. The target Linear team/project are: + +- Team: `233391d6-ec9e-4aa8-b534-16a221b8119a` +- Project: `39aa3865-345c-4313-9dc0-ab3b509c5d21` + +A `FLAG` verdict does not fail the watch workflow by itself. Runtime errors +still fail the workflow, but expected drift or downstream failures are reported +through the tracking issue so the watch does not poison its own next run. + +## Manual Run + +```bash +REPO_OWNER=Fearvox \ +REPO_NAME=EverOS \ +WATCH_BRANCH=codex-watch-overnight-2026-05-13 \ +OWNER_TIMEZONE=America/Los_Angeles \ +LINEAR_TEAM_ID=233391d6-ec9e-4aa8-b534-16a221b8119a \ +LINEAR_PROJECT_ID=39aa3865-345c-4313-9dc0-ab3b509c5d21 \ +CREATE_TRACKING_ISSUE=false \ +node .github/scripts/overnight-watch.mjs +``` + +Set `CREATE_TRACKING_ISSUE=true` only when you want the local run to mutate +GitHub issues. + +## Public-Surface Hygiene + +Reports intentionally avoid local absolute paths, host/IP values, token names +beyond the required GitHub secret names, and operator-only commands. They should +be safe to show in Discord or a screen share.