Implement in-place seed for assembly pipeline#323
Open
Conversation
Introduce AssemblePlan that separates planning from execution in file assembly. The plan pre-computes all chunk placements (self-seed, file seeds, store fetches, skip-in-place) into a DAG of steps with explicit dependencies, replacing the interleaved sequencer approach. This lays the groundwork for #312 (destination-as-seed) by making assembly sources composable and the planning phase extensible. Key changes: - New AssemblePlan with functional options and step-based execution - Split assembly sources into separate files (fileseed, selfseed, store, skip) - Self-seed matching now uses longestMatchFrom for longer sequences - Plan validation detects stale file seeds before execution - Comprehensive tests for plan generation and in-place detection - Remove sequencer.go, selfseed.go in favor of new plan types Closes #312
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactors the assembly pipeline into a plan-based architecture and adds in-place seed support, enabling efficient file reconstruction when the target file already contains chunk data at different offsets.
AssemblePlanthat pre-computes all chunk placements into a DAG of steps with explicit dependencies, replacing the interleaved sequencer approachInPlaceSeedwhich rearranges chunks already present in the target file using Tarjan's SCC algorithm for cycle detection and buffer-break resolutionskipInPlace,inPlaceCopy,fileSeedSource,selfSeedSegment,copyFromStorelongestMatchFromfor longer contiguous sequencesCloses #312