Frrist/feat/continue client upgrade#11
Draft
frrist wants to merge 17 commits into
Draft
Conversation
The client-upgrade branch had deleted pkg/agentstore and migrated the core
(client, tokenstore, presets, verification, locator, dagservice, blobs/model)
to ucantone/libforge, but the migration was incomplete across the cmd layer,
the upload persistence layer, and the test suite, so the module did not build.
Build (now green via `make build`):
- Makefile: rename the ldflags var GOFLAGS→LDFLAGS. It collided with the
special go env var; when GOFLAGS is exported in the shell, make re-exported
the ldflags string into `go build`, which failed to parse it.
- internal/cmdutil: construct the client with the new
client.New(signer, serviceID, serviceURL, …) signature, the PEM signer it
already loads, tokenstore.NewFsStore, env-key override via ucantone
ed25519.Parse, and a traced UCAN HTTP client. Add client.ServiceID().
- pkg/preparation/sqlrepo/util: DID column bytes→string (did.String/Parse) and
invocation column → ucantone invocation.Encode/Decode. Propagate ucantone
ucan.Invocation through sqlrepo/{indexes,shards} and pkg/bus/events.
- Port login (RequestAccess/Unpack/AddProofs), reset, account list, space info,
space provision to the new APIs.
- Disable, with clear TODO(forrest) errors, the commands that build delegations
with go-ucanto or use removed client APIs: retrieve, verify, gateway serve,
unixfs ls, delegation create, proof add, space generate, blob ls, ls, and the
internal upload demo. Stub client.Spaces() and cmdutil.ResolveDIDWebAndWrap.
Tests (cascade reduced 10→6 failing packages; see TODO(forrest)):
- Repoint go-libstoracha/testutil → libforge/testutil and drop obsolete
.(cidlink.Link).Cid assertions now that RandomCID returns cid.Cid.
- Remaining failures are blocked on unported test infrastructure (the ctestutil
go-ucanto server harness, the recording mockclient, a ucantone sharded-dag
index helper) — addressed in follow-up commits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rebuild the in-process test retrieval server (pkg/client/testutil/retrievalclient.go) on libforge/ucan/retrieval + the content/retrieve command instead of the deleted go-ucanto/server/retrieval, and drop the last go-ucanto import from blobadd.go (helpers.Must → libforge testutil.Must). The pkg/client/testutil package now compiles against ucantone, unblocking the test packages that depend on it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add internal/testutil.RandomShardedDagIndexView (libforge/blobindex based), replacing go-libstoracha/testutil's version which has no libforge equivalent. - Port pkg/verification/cache_test.go to libforge/ucantone: libforge/blobindex, ucantone ed25519 signers, and location commitments built with assertcmds.Location.Invoke (ucan.Invocation) instead of go-ucanto delegations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Position{Offset,Length} -> libforge blobindex.Range{Start,End}, and build the
test location commitment with assertcmds.Location.Invoke (ucantone) instead of
go-ucanto assert delegations. pkg/verification tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Port dagservice_test.go and exchange_test.go to the new locator.Commitment struct (was a go-ucanto capability) and libforge blobindex.Range, with a content- keyed mock retriever. Range.End is inclusive (consistent with the verification retrieval path and the coalesced-block extraction that reads End-Start+1 bytes), so fix withinGap's gap computation, which was off by one (b.Start - a.End instead of b.Start - a.End - 1) and rejected exactly-maxGap-sized gaps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
storacha_test asserted space/blob/replicate, filecoin/offer, and rich space/index/add behavior that is commented out / stubbed in the production API (PostProcessUploadedShards is a no-op). Reduce the test to the surviving flow: space/blob/add, space/index/add, upload/add. Add internal/testutil.RandomLocationInvocation. Also fix the DID column storage: did.String() is TEXT but the columns are BLOB, which SQLite strict typing rejects — store the DID string as []byte instead. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
client_test.go tested the removed Proofs(CapabilityQuery) wildcard model and the removed WithStore/WithPrincipal/WithAdditionalProofs options; reduce it to the new surface (New + AddProofs + ProofChain + Reset). Port retrieve_test.go to the new locator.Commitment struct + libforge retrieval client, dropping the obsolete invalid-DID-string case (Commitment.Node is now a typed did.DID). NOTE: pkg/client's test package does not yet compile — 8 other test files (accessdelegate, bloblist, claimaccess, indexadd, pollclaim, requestaccess, spaceblobreplicate, uploadshardlist) still build fake servers with the go-ucanto server.Provide harness and need porting to ucantone server routes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Establishes the porting recipe for the pkg/client fake-server tests: replace the go-ucanto ctestutil.NewTestServer(server.WithServiceMethod(server.Provide(...))) + client.WithConnection pattern with ctestutil.Client(t, WithServerRoutes(func(deps) server.NewRoute(<libforge cmd>, func(req *binding.Request[...], res *binding.Response[...]) ...))). Proofs/delegations built via <cmd>.Delegate(issuer, audience, subject). NOTE: pkg/client tests still don't compile — 7 more files follow this same recipe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Port accessdelegate, requestaccess, claimaccess, pollclaim, bloblist, uploadshardlist, indexadd, blobadd to ctestutil.Client(t, WithServerRoutes(...)) with libforge command bindings. Reduce indexadd (SpaceIndexAdd->IndexAdd dropped size/root). Skip spaceblobreplicate_test (production still go-ucanto) and spaces_test (Client.Spaces() is stubbed) with TODO(forrest). Also fix the ctestutil blob/add harness: the accept receipt must carry the pdp/accept invocation in its metadata (BlobAdd reads it from there). pkg/client tests now pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fix nodevalue_test for libforge testutil.RandomCID returning cid.Cid (wrap as a cidlink.Link to match NodeValue's link return). Skip-stub the two largest remaining test files with handoff notes (originals in git history): - pkg/client/locator (indexlocator_test): ~1000-line mock-indexer port - pkg/preparation (preparation_test): end-to-end fake-service port make build and make test are now green; locator + preparation deep tests are skipped pending a dedicated port (production for both is already migrated). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite indexlocator_test.go (TestLocator + TestLocateMany) against the ucantone model: claims are assert/location invocations (decoded via invocation.Decode), the mock QueryResult returns []types.Block / []cid.Cid, indexes use libforge blobindex (NewShardedDagIndex/Archive), blobindex.Position -> Range, and Commitment.Nb().Location -> Commitment.Location. pkg/client/locator tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite preparation_test.go: the fake service uses ctestutil.WithServerRoutes (index/add + upload/add) plus the blob/add harness; index/add and upload/add are recorded via libforge command bindings. Reduce assertions to the surviving flow (no replicate/offer, which are commented out in production storacha) and drop the perf benchmark. The index is still produced by go-libstoracha blobindex in the production pipeline, so the test's blockstore extracts it with that package. pkg/preparation and pkg/client/locator now pass — make build and make test are fully green with no skipped packages. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add tokenstore enumeration (Store.Delegations) and re-derive spaces from the held delegations: each distinct subject of a valid, non-attestation delegation addressed to the agent is a space, with the granting delegations as access proofs. Space names move from go-ucanto facts to delegation metadata (SpaceNameMetaKey); add SpaceNameMetadata for grants to set it. Restore the real spaces_test coverage against this model. Also fix a latent bug: FsStore.path was never set in NewFsStore, so save()/load() operated on an empty path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- proof add: decode a UCAN delegation container and AddProofs. - delegation create: build delegation(s) for the requested commands plus the authorizing proof chain, encoded as a container (loadable via proof add). - blob ls / ls: list via the ported BlobList/UploadList/UploadShardList; the --proof flag now loads a container into the token store. - cmdutil: add AddProofsFromFile, remove the dead go-ucanto MustGetProof and ResolveDIDWebAndWrap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- space generate: provision via ProviderAdd(serviceID) and grant full access (command.Top()) to the agent and account, recording the name in delegation metadata; account grant registered via AccessDelegate. - retrieve: authorize the indexer with a content/retrieve delegation and stream via the ported locator + dagservice + dagfs. - verify: drive verification.VerifyDAGRetrieval with content/retrieve auth + ProofChain-based proof getter; spaces come from the new Client.Spaces(). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both authorize the indexing service with a content/retrieve delegation and serve via the ported locator + dagservice. Gateway derives served spaces from Client.Spaces() and derives the libp2p peer ID from the agent's ed25519 key (ucan.Signer no longer exposes a Verifier). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The demo exercises the scan->DAG->shard->upload pipeline; replace its go-ucanto ctestutil fake service with an inline mock storacha.Client (BlobAdd returns a generated location commitment; IndexAdd/UploadAdd succeed), so it runs without a real service or test harness. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A continuation of #10.
Completes the mid-refactor handoff: the go-ucanto → ucantone / libforge UCAN
library swap (and
pkg/agentstore→pkg/tokenstore).make build,go vet ./...,and
make testare all green, with zeroagentstorereferences remaining.Done
GOFLAGS→LDFLAGSMakefile collision; portedclient construction in
internal/cmdutiltoclient.New(signer, serviceID, serviceURL, …)with
WithTokenStore/WithReceiptsClient.Client.Spaces()reimplemented on the ucantone model (enumerates token-storedelegations by subject; names from delegation metadata).
proof add,delegation create,blob ls,ls,space generate,retrieve,verify,gateway serve,unixfs ls,and the upload
demo(now driven by an inline mockstoracha.Client, no test harness).pkg/clientserver-harness tests,accessdelegate,retrieve,dagservice(incl. an off-by-one fix inwithinGap),verification(retrieval + cache), the
ctestutilretrieval harness, theIndexLocatortests, and thepreparation end-to-end test. Fixed a DID-stored-as-BLOB SQLite typing bug and a latent
FsStore.pathinitialization bug along the way.What remains (deliberately not done — needs product intent)
All remaining
TODO(forrest)markers cluster on one disabled feature plus twoinformational notes. None are wireable stubs:
space/blob/replicate+filecoin/offerare commented out in production(
pkg/preparation/storacha/storacha.go);PostProcessUploadedShardsis a no-op.Re-enabling is a product decision @ash deferred — and it can't be fully completed today:
blob/replicateexists in libforge, butfilecoin/offerhas no command in libforge(there is no
filecoinpackage underlibforge/commands). Needs design intent beforere-enabling.
pkg/client/spaceblobreplicate.go—Client.SpaceBlobReplicateis still go-ucanto andfully commented out; its test (
spaceblobreplicate_test.go) is skipped.pkg/preparation/storacha/storacha_test.go,pkg/preparation/preparation_test.goassertonly the surviving flow (
blob/add,index/add,upload/add); replicate/offer assertionsare noted to restore when re-enabled.
indexadd_test.go—SpaceIndexAdd(index,size,root,space)legitimately shrank toIndexAdd(index,space); test asserts the reduced surface.client_test.go— the wildcardProofs(CapabilityQuery)API was removed by design;proof selection now lives in libforge's
ProofChain. The oldTestProofs/TestWithAdditionalProofscoverage would need re-homing againstProofChainif thatselection logic ever returns to guppy.
Grep the breadcrumbs:
grep -rn "TODO(forrest)" --include="*.go" .