fix(inspect): harden Docker discovery with batched inspect, fix find, add tests #103
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| #!/usr/bin/env bash | ||
| # Pre-commit hook: fast gate (build + unit tests + race detection). | ||
| # Full CI (integration, e2e, coverage thresholds) runs in CI pipeline. | ||
| # Per DORA research: pre-commit should be <10s for fast feedback loops. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| echo "==> Pre-commit: build check" | ||
| go build -o /dev/null ./cmd/ | ||
|
|
||
| echo "==> Pre-commit: unit tests with race detection" | ||
| go test -race ./pkg/chatarchive/... ./internal/chatarchivecmd/... | ||
|
|
||
| echo "Pre-commit checks passed." |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,78 @@ | |||||||||||||||||||||||||||||
| name: Chat Archive Quality | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| on: | |||||||||||||||||||||||||||||
| pull_request: | |||||||||||||||||||||||||||||
| paths: | |||||||||||||||||||||||||||||
| - 'pkg/chatarchive/**' | |||||||||||||||||||||||||||||
| - 'internal/chatarchivecmd/**' | |||||||||||||||||||||||||||||
| - 'cmd/create/chat_archive.go' | |||||||||||||||||||||||||||||
| - 'cmd/backup/chats.go' | |||||||||||||||||||||||||||||
| - 'test/e2e/**' | |||||||||||||||||||||||||||||
| - 'scripts/chatarchive-ci.sh' | |||||||||||||||||||||||||||||
| - 'package.json' | |||||||||||||||||||||||||||||
| - '.github/hooks/pre-commit' | |||||||||||||||||||||||||||||
| - '.github/hooks/setup-hooks.sh' | |||||||||||||||||||||||||||||
| - 'scripts/install-git-hooks.sh' | |||||||||||||||||||||||||||||
| - '.github/workflows/chatarchive-quality.yml' | |||||||||||||||||||||||||||||
| push: | |||||||||||||||||||||||||||||
| branches: | |||||||||||||||||||||||||||||
| - main | |||||||||||||||||||||||||||||
| - develop | |||||||||||||||||||||||||||||
| paths: | |||||||||||||||||||||||||||||
| - 'pkg/chatarchive/**' | |||||||||||||||||||||||||||||
| - 'internal/chatarchivecmd/**' | |||||||||||||||||||||||||||||
| - 'cmd/create/chat_archive.go' | |||||||||||||||||||||||||||||
| - 'cmd/backup/chats.go' | |||||||||||||||||||||||||||||
| - 'test/e2e/**' | |||||||||||||||||||||||||||||
| - 'scripts/chatarchive-ci.sh' | |||||||||||||||||||||||||||||
| - 'package.json' | |||||||||||||||||||||||||||||
| - '.github/hooks/pre-commit' | |||||||||||||||||||||||||||||
| - '.github/hooks/setup-hooks.sh' | |||||||||||||||||||||||||||||
| - 'scripts/install-git-hooks.sh' | |||||||||||||||||||||||||||||
| - '.github/workflows/chatarchive-quality.yml' | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| jobs: | |||||||||||||||||||||||||||||
| chatarchive-ci: | |||||||||||||||||||||||||||||
| name: Chat Archive CI (${{ matrix.os }}) | |||||||||||||||||||||||||||||
| runs-on: ${{ matrix.os }} | |||||||||||||||||||||||||||||
| strategy: | |||||||||||||||||||||||||||||
| fail-fast: false | |||||||||||||||||||||||||||||
| matrix: | |||||||||||||||||||||||||||||
| os: [ubuntu-latest, macos-latest, windows-latest] | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| steps: | |||||||||||||||||||||||||||||
| - name: Checkout | |||||||||||||||||||||||||||||
| uses: actions/checkout@v4 | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| - name: Set up Go | |||||||||||||||||||||||||||||
| uses: actions/setup-go@v5 | |||||||||||||||||||||||||||||
| with: | |||||||||||||||||||||||||||||
| go-version-file: 'go.mod' | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| - name: Set up Node | |||||||||||||||||||||||||||||
| uses: actions/setup-node@v4 | |||||||||||||||||||||||||||||
| with: | |||||||||||||||||||||||||||||
| node-version: '20' | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| - name: Download Go modules | |||||||||||||||||||||||||||||
| run: go mod download | |||||||||||||||||||||||||||||
| shell: bash | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| - name: Run chat archive CI | |||||||||||||||||||||||||||||
| run: npm run ci | |||||||||||||||||||||||||||||
| shell: bash | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| - name: Upload verification summary | |||||||||||||||||||||||||||||
| if: always() | |||||||||||||||||||||||||||||
| uses: actions/upload-artifact@v4 | |||||||||||||||||||||||||||||
| with: | |||||||||||||||||||||||||||||
| name: chatarchive-summary-${{ matrix.os }} | |||||||||||||||||||||||||||||
| path: outputs/chatarchive-ci/ | |||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| - name: Publish job summary | |||||||||||||||||||||||||||||
| if: always() | |||||||||||||||||||||||||||||
| shell: bash | |||||||||||||||||||||||||||||
| run: | | |||||||||||||||||||||||||||||
| if [[ -f outputs/chatarchive-ci/summary.txt ]]; then | |||||||||||||||||||||||||||||
| cat outputs/chatarchive-ci/summary.txt >> "$GITHUB_STEP_SUMMARY" | |||||||||||||||||||||||||||||
| fi | |||||||||||||||||||||||||||||
|
Comment on lines
+36
to
+78
Check warningCode scanning / CodeQL Workflow does not contain permissions Medium
Copilot AutofixAI 25 days ago In general, to fix this class of issue you explicitly declare a For this specific workflow, the simplest and safest fix without changing existing functionality is to add a workflow-level Concretely:
Suggested changeset
1
.github/workflows/chatarchive-quality.yml
Copilot is powered by AI and may make mistakes. Always verify output.
|
|||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| // cmd/create/chat_archive.go | ||
| // | ||
| // Thin orchestration layer for chat-archive. Business logic lives in | ||
| // pkg/chatarchive/ per the cmd/ vs pkg/ enforcement rule. | ||
|
|
||
| package create | ||
|
|
||
| import ( | ||
| "github.com/CodeMonkeyCybersecurity/eos/internal/chatarchivecmd" | ||
| eos "github.com/CodeMonkeyCybersecurity/eos/pkg/eos_cli" | ||
| "github.com/CodeMonkeyCybersecurity/eos/pkg/eos_io" | ||
| "github.com/spf13/cobra" | ||
| ) | ||
|
|
||
| // CreateChatArchiveCmd copies and deduplicates chat transcripts. | ||
| var CreateChatArchiveCmd = &cobra.Command{ | ||
| Use: "chat-archive", | ||
| Short: "Copy and deduplicate chat transcripts into a local archive", | ||
| Long: `Find transcript-like files (jsonl/json/html), copy unique files into one archive, | ||
| and write an index manifest with duplicate mappings. | ||
|
|
||
| Examples: | ||
| eos create chat-archive | ||
| eos create chat-archive --source ~/.claude --source ~/dev | ||
| eos create chat-archive --exclude conversation-api --exclude .cache | ||
| eos create chat-archive --dry-run`, | ||
| RunE: eos.Wrap(runCreateChatArchive), | ||
| } | ||
|
|
||
| func init() { | ||
| CreateCmd.AddCommand(CreateChatArchiveCmd) | ||
| chatarchivecmd.BindFlags(CreateChatArchiveCmd) | ||
| } | ||
|
|
||
| func runCreateChatArchive(rc *eos_io.RuntimeContext, cmd *cobra.Command, _ []string) error { | ||
| return chatarchivecmd.Run(rc, cmd) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| *Last Updated: 2026-03-21* | ||
|
|
||
| # pkg/inspect Follow-Up Issues | ||
|
|
||
| Issues discovered during adversarial review of `pkg/inspect/docker.go`. | ||
|
|
||
| ## Issue 1: output.go / terraform_modular.go — 37 staticcheck warnings (P2) | ||
|
|
||
| **Problem**: `WriteString(fmt.Sprintf(...))` should be `fmt.Fprintf(...)` throughout output.go and terraform_modular.go. | ||
| **Impact**: Performance (unnecessary string allocation) and lint noise. | ||
| **Fix**: Replace all `tf.WriteString(fmt.Sprintf(...))` with `fmt.Fprintf(tf, ...)`. | ||
| **Files**: `pkg/inspect/output.go`, `pkg/inspect/terraform_modular.go` | ||
| **Effort**: ~30 min mechanical refactor | ||
|
|
||
| ## Issue 2: services.go — unchecked filepath.Glob error (P2) | ||
|
|
||
| **Problem**: `pkg/inspect/services.go:381` ignores `filepath.Glob` error. | ||
| **Impact**: Silent failure when glob patterns are invalid. | ||
| **Fix**: Check and log the error. | ||
| **Effort**: 5 min | ||
|
|
||
| ## Issue 3: kvm.go — goconst violations (P3) | ||
|
|
||
| **Problem**: String constants `"active"`, `"UUID"` repeated without named constants. | ||
| **Impact**: Violates P0 Rule #12 (no hardcoded values). | ||
| **Fix**: Extract to constants in `kvm.go` or a `constants.go` file. | ||
| **Effort**: 15 min | ||
|
|
||
| ## Issue 4: Pre-existing lint issues across 30+ files on this branch (P1) | ||
|
|
||
| **Problem**: `npm run ci` fails due to 165 lint issues across the branch. | ||
| **Impact**: Cannot merge until resolved. | ||
| **Root cause**: Accumulated tech debt from many feature PRs merged without lint cleanup. | ||
| **Fix**: Dedicated lint cleanup pass before PR merge. | ||
| **Effort**: 2-4 hours | ||
|
|
||
| ## Issue 5: Inspector lacks Docker SDK integration (P3) | ||
|
|
||
| **Problem**: All Docker operations use shell commands instead of the Docker SDK. | ||
| **Impact**: Fragile parsing, no type safety, extra process spawns. | ||
| **Fix**: Migrate to `github.com/docker/docker/client` SDK for container/image/network/volume operations. | ||
| **Rationale**: CLAUDE.md P1 states "ALWAYS use Docker SDK" for container operations. | ||
| **Effort**: 1-2 days | ||
|
|
||
| ## Issue 6: Compose file search does not guard against TOCTOU (P3) | ||
|
|
||
| **Problem**: Between `os.Stat` size check and `os.ReadFile`, the file could be replaced. | ||
| **Impact**: Theoretical DoS via race condition on symlink swap. | ||
| **Fix**: Read file first, then check size of bytes read (simpler and race-free). | ||
| **Effort**: 15 min |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| package chatarchivecmd | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "io" | ||
| "strings" | ||
| "time" | ||
|
|
||
| "github.com/CodeMonkeyCybersecurity/eos/pkg/chatarchive" | ||
| "github.com/CodeMonkeyCybersecurity/eos/pkg/eos_io" | ||
| "github.com/spf13/cobra" | ||
| "github.com/uptrace/opentelemetry-go-extra/otelzap" | ||
| "go.uber.org/zap" | ||
| ) | ||
|
|
||
| func BindFlags(cmd *cobra.Command) { | ||
| cmd.Flags().StringSlice("source", chatarchive.DefaultSources(), "Source directories to scan") | ||
| cmd.Flags().String("dest", chatarchive.DefaultDest(), "Destination archive directory") | ||
| cmd.Flags().StringSlice("exclude", nil, "Path substrings to exclude from discovery (e.g. --exclude conversation-api)") | ||
| cmd.Flags().Bool("dry-run", false, "Show what would be archived without copying files") | ||
| } | ||
|
|
||
| func Run(rc *eos_io.RuntimeContext, cmd *cobra.Command) error { | ||
| sources, _ := cmd.Flags().GetStringSlice("source") | ||
| dest, _ := cmd.Flags().GetString("dest") | ||
| excludes, _ := cmd.Flags().GetStringSlice("exclude") | ||
| dryRun, _ := cmd.Flags().GetBool("dry-run") | ||
|
|
||
| result, err := chatarchive.Archive(rc, chatarchive.Options{ | ||
| Sources: sources, | ||
| Dest: dest, | ||
| Excludes: excludes, | ||
| DryRun: dryRun, | ||
| }) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| logger := otelzap.Ctx(rc.Ctx) | ||
| writeSummary(cmd.OutOrStdout(), result, dryRun, logger) | ||
| logger.Info("Chat archive summary", | ||
| zap.Int("sources_requested", result.SourcesRequested), | ||
| zap.Int("sources_scanned", result.SourcesScanned), | ||
| zap.Int("sources_missing", len(result.MissingSources)), | ||
| zap.Int("skipped_symlinks", result.SkippedSymlinks), | ||
| zap.Int("unreadable_entries", result.UnreadableEntries), | ||
| zap.Int("unique_files", result.UniqueFiles), | ||
| zap.Int("duplicates", result.Duplicates), | ||
| zap.Int("already_archived", result.Skipped), | ||
| zap.Int("empty_files", result.EmptyFiles), | ||
| zap.Int("failures", result.FailureCount), | ||
| zap.Duration("duration", result.Duration), | ||
| zap.Bool("dry_run", dryRun)) | ||
| for _, failure := range result.Failures { | ||
| logger.Warn("Chat archive file failure", | ||
| zap.String("path", failure.Path), | ||
| zap.String("stage", failure.Stage), | ||
| zap.String("reason", failure.Reason)) | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
|
|
||
| func formatSummary(result *chatarchive.Result, dryRun bool) string { | ||
| lines := []string{ | ||
| statusLine(dryRun), | ||
| fmt.Sprintf("Sources scanned: %d/%d", result.SourcesScanned, result.SourcesRequested), | ||
| fmt.Sprintf("Unique files: %d", result.UniqueFiles), | ||
| fmt.Sprintf("Duplicates in this run: %d", result.Duplicates), | ||
| fmt.Sprintf("Already archived: %d", result.Skipped), | ||
| fmt.Sprintf("Empty files ignored: %d", result.EmptyFiles), | ||
| fmt.Sprintf("File failures: %d", result.FailureCount), | ||
| fmt.Sprintf("Unreadable entries skipped: %d", result.UnreadableEntries), | ||
| fmt.Sprintf("Symlinks skipped: %d", result.SkippedSymlinks), | ||
| fmt.Sprintf("Duration: %s", result.Duration.Round(10*time.Millisecond)), | ||
| } | ||
|
|
||
| if result.ManifestPath != "" { | ||
| lines = append(lines, fmt.Sprintf("Manifest: %s", result.ManifestPath)) | ||
| } | ||
| if result.RecoveredManifestPath != "" { | ||
| lines = append(lines, fmt.Sprintf("Recovered corrupt manifest: %s", result.RecoveredManifestPath)) | ||
| } | ||
| if len(result.MissingSources) > 0 { | ||
| lines = append(lines, fmt.Sprintf("Unavailable sources: %s", strings.Join(result.MissingSources, ", "))) | ||
| } | ||
| if result.FailureCount > len(result.Failures) { | ||
| lines = append(lines, fmt.Sprintf("Additional failures not shown: %d", result.FailureCount-len(result.Failures))) | ||
| } | ||
|
|
||
| return strings.Join(lines, "\n") | ||
| } | ||
|
|
||
| func statusLine(dryRun bool) string { | ||
| if dryRun { | ||
| return "Dry run complete." | ||
| } | ||
| return "Archive complete." | ||
| } | ||
|
|
||
| func writeSummary(w io.Writer, result *chatarchive.Result, dryRun bool, logger otelzap.LoggerWithCtx) { | ||
| if _, err := fmt.Fprintln(w, formatSummary(result, dryRun)); err != nil { | ||
| logger.Warn("Failed to write chat archive summary", zap.Error(err)) | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This job never runs the new chatarchive checks. I checked
package.json, andnpm run cistill maps tonpm run ci:debug --silent, soscripts/chatarchive-ci.shis never executed here. As a result the dedicated unit/integration/e2e/coverage gates forpkg/chatarchive/**are skipped, and the workflow usually will not produce theoutputs/chatarchive-ci/artifacts it tries to upload.Useful? React with 👍 / 👎.