Add directive-driven improvement and prompt surface optimization#46
Open
Add directive-driven improvement and prompt surface optimization#46
Conversation
… surface optimization Three AutoAgent-inspired concepts integrated into the evidence-centric improve loop: 1. Continuous mode (--continuous / --max-iterations=N): Re-baselines after each accepted improvement and iterates, compounding small wins. Stops when an iteration produces no accepted candidates. 2. Directive-driven improvement (improve-directive.md): Externalizes improvement strategy into a human-editable Markdown file. Operators can specify priority gates, focus mode (false positives vs negatives), edit style preferences, and custom instructions — all injected into LLM diagnosis/fix prompts. 3. Prompt surface optimization (--prompt-surface): Extends the bounded surface to include LLM prompts within gates (vision.ts prompt, triangulation weights). The fix generator gets context about which regions are prompts vs logic, preferring prompt edits for prompt-related failures. https://claude.ai/code/session_01SJkfKmU2V83UrCvgyH2JAD
Owner
Author
|
Deferring until dirty count reaches 0 and the basic improve loop is stable. What's merge-ready:
What needs more work:
Good research. Just not the priority right now. The priority is clearing the last 7 dirty scenarios and publishing v0.8.0 with clean sensors. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces two major enhancements to the autonomous improvement engine:
Directive-Driven Improvement — Operators can now guide the improvement loop through a
improve-directive.mdfile instead of modifying TypeScript. This follows AutoAgent's "program the meta-agent" pattern.Prompt Surface Optimization — Extends the improvement loop to recognize and optimize LLM prompts and tunable thresholds within gate files (e.g., vision.ts, triangulation.ts), allowing the LLM to prefer prompt edits over logic changes when appropriate.
Continuous Mode — Implements hill-climbing iteration support, allowing the improvement engine to re-baseline and iterate after each accepted improvement.
Key Changes
improve-directive.ts(new)loadDirective()— Loads and parsesimprove-directive.mdwith structured fields (priority gates, focus mode, edit style) and custom instructionsformatDirectiveForPrompt()— Injects directive context into LLM promptsapplyDirectiveToBundles()— Prioritizes evidence bundles based on directive's priority gatesimprove-prompt-surface.ts(new)extractPromptRegion()— Extracts actual prompt text from source filesformatPromptSurfaceContext()— Provides LLM with prompt region metadata and tuning adviceisPromptRegion()— Checks if a file/function is a tunable prompt surfaceimprove.ts(modified)runImproveLoop()intorunSingleIteration()to support continuous modeself-test.ts(modified)--continuous,--max-iterations=N,--directive=PATH,--prompt-surfacetypes.ts(modified)ImproveConfigwithmaxIterations,directivePath,promptSurfacefieldsimprove-directive.md(new)improve-directive.test.ts(new).gitignore(modified).verify/directory (created by test runs)Notable Implementation Details
https://claude.ai/code/session_01SJkfKmU2V83UrCvgyH2JAD