Skip to content

Frrist/fix/deps mess pt2#7

Open
frrist wants to merge 18 commits into
mainfrom
frrist/fix/deps-mess-pt2
Open

Frrist/fix/deps mess pt2#7
frrist wants to merge 18 commits into
mainfrom
frrist/fix/deps-mess-pt2

Conversation

@frrist
Copy link
Copy Markdown
Member

@frrist frrist commented May 16, 2026

Summary

Stacks on #6. Reshapes the persistence config surface so the two persistence axes — database and object-store — are peers, each modeled as a discriminated union over backends. app.StorageConfig is deleted; cfg.Database and cfg.ObjectStore are top-level peers under AppConfig.

Each step is a small focused commit.

Commits

1. d213c48 — move SQLite paths into DatabaseConfig; dedupe DB providers

SQLite database paths (replicator.db, aggregator/jobqueue.db, …) were derived inside each fx provider from cfg.Storage.DataDir. Now they're populated once in RepoConfig.ToAppConfig and live on DatabaseConfig.SQLite. The five DB providers stop deriving paths and just read the field.

2. 69de904 — flag DatabaseConfig as single-backend-per-deployment

Reshape DatabaseConfig into a discriminated union: explicit Type selector (sqlite | postgres) plus per-backend nested sub-configs (SQLite, Postgres). Cross-field validation rejects configs that mix backends. Documented as all-or-nothing-per-deployment.

3. 3c9c635 — object-store config as discriminated union; delete dead types

Same pattern applied to the object-store layer. ObjectStoreConfig becomes Type (memory | filesystem | s3) plus Local, Filesystem, and S3 sub-configs. Four stores (KeyStore, Aggregator, Publisher, RetrievalJournal) always live on local disk and stay under Local; the bulk stores pick a backend. Drops three dead types in the process: BlobStorageConfig (extracted but never consumed), StashStoreConfig (no provider exists), and s3.ProvideConfigs (all outputs unused). Renames EgressTrackerStorageConfigRetrievalJournalConfig to match what it actually configures.

4. b3bfb5f — lift persistence to AppConfig top-level; delete StorageConfig

Promote Database, ObjectStore, DataDir, and TempDir from AppConfig.Storage to top-level AppConfig peers. app.StorageConfig is replaced by app.PersistenceConfig — same field set, but now a transient loader-output bundle from RepoConfig.ToAppConfig, not an AppConfig section. Five fx Params structs narrow from app.StorageConfig to app.DatabaseConfig (they were only reading Database.IsPostgres()).

5. 774df49 — chore: gofmt

What this enables

  • Adding a future database backend (Yugabyte, MySQL) is mechanical: one sibling sub-struct, one Type constant, one helper branch.
  • Same shape for a future object-store backend (GCS, Azure).
  • The "what kind of config is this?" question has obvious answers — cfg.Database is database config, cfg.ObjectStore is object-store config. No more reaching through a Storage umbrella that grouped unrelated concerns.

Out of scope

  • TOML/YAML user-facing config format. The on-disk layout (top-level [database], [object_store]) is unchanged.
  • Adding new backends. The shape exists; only sqlite, memory, filesystem, and s3 are wired.

Test plan

frrist and others added 12 commits May 15, 2026 12:14
Adds pdpfake — in-memory types.PieceAPI and commp.Calculator fakes plus
an fx module that exposes them as pdp.PDP — and switches the three
existing fxtest suites to wire them in alongside app.UCANModule.

Today the tests are the only consumers of the non-PDP code path: every
fxtest setup omits app.PDPModule, leaving pdp.PDP nil in the handler
graph, and tests seed bytes via svc.Blobs().Store().Put. Subsequent
commits will delete that path entirely; routing the tests through PDP
first lets the deletion remain a pure-removal commit.

No production code changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deletes the bimodal seams: the six `if s.PDP() == nil` branches in
handlers (allocate, accept, replica/transfer, pdp_info), the
`if params.API != nil` adapter switch in retrieval wiring, and the
`optional:"true"` tags on PDP fields in the storage and replicator fx
providers.

With the non-PDP path gone, the byte-storage surface on the blobs
service is no longer reachable: removes Store, Presigner and Access from
the blobs.Blobs interface; deletes pkg/service/blobs/server.go (the
GET/PUT /blob/:blob handler), pkg/access and pkg/presigner packages,
and pkg/fx/presigner. blobs.Blobs now exposes only Allocations and
Acceptances. Also retires the deprecated blobstore.PDPStore alias.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the fat view interfaces (AllocateService, AcceptService,
TransferService, BlobAllocateService, BlobAcceptService,
ReplicaAllocateService, PDPInfoService, AccessGrantService) with
per-handler `*Deps` structs populated by fx, each declaring its own
narrow consumer-defined interfaces next to the handler:

  - AllocationStore, PieceAllocator alongside Allocate
  - AcceptanceStore, PieceReader alongside Accept
  - PieceResolver alongside pdp/info

The Deps structs use fx.In, so wiring stays declarative — pkg/fx/storage
gains a handful of identity providers that bind concrete production
types (allocationstore.AllocationStore, types.PieceAPI) to the narrow
handler-side interfaces, and a NewUploadConnection provider that
extracts the upload client.Connection from AppConfig as a top-level
bean.

The replicator's adapter struct goes away; the Service now constructs a
TransferDeps value once and reuses it for the queue handler and the
failure receipt. transferBlobFromSource builds an AcceptDeps subset
from the TransferDeps to call into blobhandler.Accept.

Tests stop populating `var svc storage.Service` and instead populate
the narrow types they actually assert against. The umbrella interfaces
still exist as composition (storageServiceWrapper) but no longer
satisfy any view interface — commit 4 deletes them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes the umbrellas that the per-handler Deps refactor made obsolete:

  - storage.Service (and the storageServiceWrapper that satisfied it)
  - pdp.PDP (and TODO_PDP_Impl, ProvideTODOPDPImplInterface)
  - blobs.Blobs (and the entire pkg/service/blobs + pkg/fx/blobs)

The replicator constructor now takes the narrow concrete deps
(types.PieceAPI, commp.Calculator, acceptancestore.AcceptanceStore)
directly instead of the pdp.PDP and blobs.Blobs umbrellas. The PDP fx
module provides types.PieceReaderAPI via fx.As so retrieval's
non-optional API field resolves without going through TODO_PDP_Impl.

pdpfake's fx module no longer manufactures a pdp.PDP wrapper — it
provides types.PieceAPI / types.PieceReaderAPI / commp.Calculator
directly, mirroring the production wiring.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds table-driven unit tests against Allocate and Accept that operate on
the per-handler Deps structs directly, with in-memory implementations
(allocationstore + acceptancestore backed by go-datastore MapDatastore,
plus pdpfake). The full suite for pkg/service/storage/handlers/blob
runs in ~17ms versus ~7s for the fxtest-based integration tests in
pkg/service/storage, with coverage at 72.6%.

Allocate coverage focuses on the decision tree:
  - no prior allocation, blob not received → size = requested, address set
  - allocation exists, blob not received → size = 0, address set
  - allocation exists, blob received → size = 0, address nil
  - allocation in another space, blob received → size = requested, address nil
  - unsupported hash → error

Accept coverage:
  - piece absent → "piece not found" error
  - piece held → returns Claim + PDP invocation, persists acceptance with
    pdp/accept promise, persists location claim, calls publisher exactly once

Transfer unit tests are intentionally deferred — the handler is mostly
HTTP/UCAN glue best covered by the existing fxtest integration tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Applies the same Deps-struct pattern to retrieval that we used for
storage. The shape was identical: a fat retrieval.Service umbrella
(ID + Allocations + Blobs() blobstore.BlobGetter), a RetrievalService
wrapper, and per-handler view interfaces (BlobRetrievalService,
SpaceContentRetrievalService) that handlers reached through to call
.Blobs().Get(). With those gone, the only thing keeping
blobstore.BlobGetter alive was a BlobGetterAdapter wrapping
types.PieceReaderAPI to satisfy it.

  - spacecontent.Retrieve now takes types.PieceReaderAPI directly and
    calls pieces.Read returning *types.PieceReader. Range errors still
    propagate as blobstore.RangeNotSatisfiableError through the
    production piece.StoreReader chain, so the test assertions remain
    valid.
  - WithBlobRetrieveMethod and WithSpaceContentRetrieveMethod take new
    BlobRetrieveDeps / SpaceContentRetrieveDeps structs (fx.In).
  - pkg/service/retrieval/{interface,service}.go deleted (umbrella + wrapper).
  - pkg/pdp/store/adapter deleted (the BlobGetterAdapter was its only purpose).
  - pkg/fx/retrieval/provider.go deleted — the retrieval module had no
    remaining providers; pkg/fx/retrieval/ucan.Module still wires the
    UCAN handlers.
  - blobstore.BlobGetter removed; Get is folded directly into Blobstore.
  - fx.As(new(blobstore.BlobGetter)) annotations dropped from the
    memory/s3/filesystem store providers.
  - Retrieval tests construct piece.NewStoreReader(blobstore...) to
    produce a PieceReaderAPI from the existing in-memory blobstore —
    using the production read path keeps RangeNotSatisfiableError
    assertions accurate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes the last bag-style interfaces and a couple of dead scaffolds
that were leftover after the storage/retrieval refactor:

  - claims.Claims (2-getter umbrella over ClaimStore + Publisher). Handlers
    that used to chain `deps.Claims.Store().Put(...)` and
    `deps.Claims.Publisher().Publish(...)` now take the two narrow deps
    directly in their Deps structs.
  - The whole pkg/service/claims/{interface.go,service.go} pair goes
    with it (only server.go survives — the /claim/:claim route registrar).
  - publisher.Publisher.Store() — the interface method had zero callers
    outside a nil-returning test stub. The PublisherStore is already a
    top-level fx bean. Removed from the interface, the impl method, and
    the corresponding field on PublisherService.
  - pkg/service/storage/server.go `Server` struct and
    pkg/service/retrieval/server.go `Server` struct were never
    instantiated. Only their NewHandler peers are used by fx.
  - pkg/store/claimstore deleted — the package was a single file
    aliasing delegationstore.DelegationStore with an author-acknowledged
    "// TODO a glorified type alias, remove this" comment. All callers
    switched to delegationstore.DelegationStore directly.

replicator.New's constructor and the matching fx Params struct now take
ClaimStore + Publisher fields instead of the umbrella.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Moves the parallel pkg/fx/ tree's service-mirror modules into the
packages they describe. Each impl package now owns its own fx wiring as
fx.go, eliminating the maintenance tax of keeping two parallel trees in
sync (which already failed once — pkg/fx/blobs lingered after the impl
side was deleted, and we caught pkg/fx/retrieval in the BlobGetter
removal). The fx framework is no longer hidden behind a parallel
directory; the colocation makes deletions and renames atomic.

Moves:
  - pkg/fx/claims          → pkg/service/claims/fx.go
  - pkg/fx/replicator      → pkg/service/replicator/fx.go
  - pkg/fx/publisher       → pkg/service/publisher/fx.go
  - pkg/fx/proofs          → pkg/service/proofs/fx.go
  - pkg/fx/principalresolver → pkg/principalresolver/fx.go
  - pkg/fx/wallet          → pkg/wallet/fx.go
  - pkg/fx/storage/        → pkg/service/storage/fx.go
  - pkg/fx/storage/ucan/{,handlers} → merged into pkg/service/storage/ucan/fx.go
  - pkg/fx/retrieval/ucan/{,handlers} → merged into pkg/service/retrieval/ucan/fx.go

Stays in pkg/fx/ (legitimately framework-only — no impl-package home):
  app, echo, root, database, store, pdp, scheduler, identity,
  claimvalidation.

The storage/ucan and retrieval/ucan merges fold the former "outer"
provider (Handler + Module) and the "inner" handlers Module (per-ability
fx.Annotate calls) into a single Module per package. To avoid a
parent→child cycle in the storage tree (where pkg/service/storage/fx.go
would need pkg/service/storage/ucan.PieceResolver while
pkg/service/storage/ucan/fx.go needs storage.NewHandler), the echo
handler that wraps the UCAN server moved into the ucan subpackage as an
unexported newEchoHandler, and pkg/service/{storage,retrieval}/server.go
were deleted.

No behavior change. The fx graph composition is identical; the only
visible difference is import paths in pkg/fx/app/{ucan,common,pdp}.go.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ructor

Treats the AppConfig-as-context-bag pattern as the same smell that bag
interfaces were. Services no longer take app.AppConfig and reach inside
for the one or two fields they need — instead, CommonModules supplies
each meaningful sub-config struct via fx.Supply, and consumers request
just what they actually use.

CommonModules additions (pkg/fx/app/common.go):
  fx.Supply(cfg.UCANService.Services)            // ExternalServicesConfig
  fx.Supply(cfg.UCANService.Services.Upload)     // UploadServiceConfig
  fx.Supply(cfg.UCANService.Services.Indexer)    // IndexingServiceConfig
  fx.Supply(cfg.UCANService.Services.Publisher)  // PublisherServiceConfig
  fx.Supply(cfg.UCANService.Services.EgressTracker)

Consumer narrowing:
  - identity.ProvideIdentity      AppConfig             → IdentityConfig
  - principalresolver.NewFx       AppConfig             → UCANServiceConfig
  - publisher.NewFx               AppConfig             → PublisherServiceConfig + IndexingServiceConfig
  - retrieval/ucan.NewServerHandler Params.Config       → Upload UploadServiceConfig
  - storage handler Deps types (AccessGrantDeps,
    TransferDeps) replace `UploadConn client.Connection`
    with `Upload app.UploadServiceConfig`; handler bodies
    read `.Connection` from the struct.

The top-level `client.Connection` provider (storage.NewUploadConnection)
is deleted — it existed only to lift one config field into the fx graph
as a bare primitive. With the source struct supplied directly, the lift
isn't needed.

With that done, replicator's two-step constructor (positional New,
fx-wrapper NewFx) has no remaining justification. Its only callers were
inside the fx wrapper, and its config dep is now a single
UploadServiceConfig field. Collapsed into a single fx.In Params-based
constructor.

Publisher keeps its two-constructor shape: publisher.New(id, store,
addr, opts...) is the variadic-options API that publisher_test.go
exercises directly with custom options like WithLogLevel. publisher.NewFx
wraps it for fx callers — that wrapping is the real config-translation
layer and earns its keep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cleanup pass after the audit. Two categories of cuts.

Dead nil checks. After the claims/publisher umbrella refactor, both
blob.Accept and replica.createLocationAssertion unconditionally set
their pdp/accept invocation field — the nil checks at three call sites
were unreachable:
  - pkg/service/storage/ucan/blob_accept.go:49 (resp.PDP)
  - pkg/service/storage/handlers/replica/transfer.go:222 (acceptResp.PDP)
  - pkg/service/storage/handlers/replica/transfer.go:241 (pdpAcceptInv)

Dead provider. pkg/service/egresstracker/fx.go had a ProvideConsolidationStore
function that was commented out of fx.Provide (line 35: //ProvideConsolidationStore).
Nothing referenced it; consolidation stores come from pkg/fx/store/{memory,
filesystem,s3} providers instead. Deleted the function and its leveldb
imports.

Five remaining AppConfig consumers narrowed to their actual config slice:
  - ProvideEthClient(cfg AppConfig)         → ProvideEthClient(cfg PDPServiceConfig)
  - ProvideLotusClient(cfg AppConfig)       → ProvideLotusClient(cfg PDPServiceConfig)
  - StartEchoServer(cfg AppConfig, ...)     → StartEchoServer(cfg ServerConfig, ...)
  - ProvideReceiptsClient(cfg AppConfig)    → ProvideReceiptsClient(cfg EgressTrackerServiceConfig)
  - NewEgressTrackerService(... cfg AppConfig) → ServerConfig + EgressTrackerServiceConfig

Only CommonModules still takes app.AppConfig — and that's correct, it's
the assembly point that explodes the whole thing into the supply tree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@frrist frrist requested a review from alanshaw as a code owner May 16, 2026 00:13
frrist and others added 6 commits May 15, 2026 17:19
CI's go-check job flagged three files as not gofmt-ed and required the
aws-sdk-go-v2 dependencies (no longer used after the s3 module cleanup
in this branch) to be removed from go.mod via go mod tidy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two intertwined cleanups in pkg/fx/database/provider.go: copy-paste
duplication across the three job-queue providers, and SQLite paths
derived inside provider code instead of declared in config.

Config changes (pkg/config/app/storage.go):
  - Add SQLiteConfig as a sibling of PostgresConfig under DatabaseConfig.
    It holds per-logical-database SQLite file paths.
  - Delete ReplicatorStorageConfig and SchedulerConfig — both were
    explicitly-empty placeholders with comments noting the gap that
    SQLiteConfig now fills.
  - Drop the matching fields from StorageConfig.
  - Drop the "sqlite doesn't have a config" comment on DatabaseConfig.

Config loader (pkg/config/repo.go):
  - Populate SQLite paths in RepoConfig.ToAppConfig via filepath.Join
    on r.DataDir — same pattern already used for the object-store Dir
    fields. Paths match what the deleted sqliteDBPath helper produced,
    so existing deployments find their databases at the same spot.

fx wiring:
  - CommonModules now supplies cfg.Storage.Database so providers can
    take app.DatabaseConfig directly without reaching through
    StorageConfig.

Database providers (pkg/fx/database/provider.go):
  - Three Provide*DB functions for the job queues become one-liners
    delegating to a shared newJobQueueDB helper. The helper takes
    DatabaseConfig + schema name + SQLite path; branches on backend;
    sets up the lifecycle hook once. Reduces the file from 294 lines
    to 169.
  - ProvideTaskEngineDB stays separate (different return type, different
    SQLite option profile) but also narrows to DatabaseConfig.
  - sqliteDBPath helper deleted; path derivation lives in the config
    loader now.

This completes the discriminated-union pattern that DatabaseConfig was
already half-using (Type + Postgres but no SQLite peer). Adding a future
backend (e.g. Yugabyte) is now mechanical: add a sibling sub-config, a
new DatabaseType constant, an IsYugabyte method, and one branch in
newJobQueueDB. No callers change.

The object-store side (app.StorageConfig with implicit S3-or-filesystem
selection and per-store Dir fields) needs the same treatment eventually
but is its own substantial change and stays out of this commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mixed-backend deployments are not supported — every logical database in
a deployment uses the backend selected by DatabaseConfig.Type. The
previous code populated SQLite paths unconditionally in the config
loader, even for Postgres deployments, which made the type's runtime
state look like both backends were in play.

Three changes flag the constraint:

  - app.DatabaseConfig and its sub-config fields get doc comments
    explicitly stating the all-or-nothing rule and that only the
    selected backend's sub-config is meaningful.
  - repo.go's ToAppConfig populates SQLite paths only when
    IsSQLite() — so a Postgres deployment shows an empty SQLiteConfig,
    matching what the doc says.
  - DatabaseConfig.ToAppConfig (the repo-level converter) now errors
    when type is sqlite (or empty) but a postgres section has been
    configured. Previously the postgres section was silently discarded;
    now the conflict surfaces at load time. Two new test cases cover
    the conflict and the empty-type-with-postgres variant.

Type-level enforcement (e.g. a tagged-union interface) would be the
strongest form but doesn't play well with mapstructure-based config
loading. Doc + conditional population + load-time validation is the
practical compromise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the database-side restructure from commit 261ccec. The object-store
layer now has the same shape: an explicit Type selector + per-backend
nested sub-configs, all-or-nothing-per-deployment, no implicit selection.
Both user-facing TOML and internal types reorganize symmetrically.

Config types (pkg/config/app/storage.go):
  - New ObjectStoreType (memory | filesystem | s3) and ObjectStoreConfig
    discriminated union with Local / Filesystem / S3 sub-fields.
  - Local always populated for filesystem and s3 backends; holds paths
    for the four stores that must remain on disk regardless of backend
    (KeyStore, AggregatorDatastore, PublisherStore, RetrievalJournal).
  - Only the bulk sub-config matching Type (Filesystem or S3) is
    meaningful; others zero-valued.
  - StorageConfig flattens — all per-store fields move under ObjectStore.
  - IsMemory/IsFilesystem/IsS3 helpers mirror DatabaseConfig.

Dead deletions:
  - app.BlobStorageConfig — populated and extracted into ProvideConfigs
    in both filesystem and s3 modules but never consumed by any NewX
    provider. The PDP/blob store uses PDPStoreConfig.
  - app.StashStoreConfig — populated and extracted, but no NewStashStore
    provider exists anywhere in the codebase.
  - s3.ProvideConfigs function + s3.Configs struct — every output
    unused; bulk-store providers in s3 mode consume *Stores directly.
  - Top-level StorageConfig.S3 *S3Config field — moves under ObjectStore.S3
    as a value (presence indicated by Type).

Renames:
  - app.EgressTrackerStorageConfig -> app.RetrievalJournalConfig. The
    config names what's actually being configured: an append-only file
    journal that the egress-tracker service consumes. On-disk path
    moves from <DataDir>/egress_tracker/journal to
    <DataDir>/retrieval_journal.

TOML breaking changes (no shim — piri has no deployed users):
  - Top-level `[s3]` block moves to `[object_store.s3]`.
  - Explicit `[object_store] type = "memory" | "filesystem" | "s3"`
    replaces the implicit "S3 != nil means S3" selector.
  - Conflict validation: configuring `[object_store.s3]` with a non-s3
    type now errors at load time, mirroring the database-side flag.

Loader (pkg/config/repo.go):
  - RepoConfig.S3 *S3Config field removed; ObjectStoreConfig field added.
  - ToAppConfig fully populates the new structure: Type selection,
    Local paths for filesystem+s3, Filesystem bulk paths only for
    filesystem mode, S3 only for s3 mode.
  - Cross-field validation:
    - object_store.type "s3" requires object_store.s3 section.
    - object_store.s3 set with non-s3 type returns error.
    - object_store.type "filesystem" requires non-empty data_dir.
  - Seven new test cases in repo_test.go cover the validation.

Backend providers:
  - filesystem/provider.go: ProvideConfigs now extracts from
    ObjectStoreConfig.Local + .Filesystem instead of flat StorageConfig
    fields. LocalOnlyConfigs takes ObjectStoreConfig and extracts only
    the Local subset. NewRetrievalJournal takes RetrievalJournalConfig.
  - s3/provider.go: ProvideConfigs/Configs deleted (entire decomposition
    was unused). NewStores takes app.S3Config directly. The nil-check
    on cfg.S3 is gone — the discriminated union makes it unreachable.
  - memory/provider.go: unchanged.

Fx selector (pkg/fx/store/provider.go): StorageModule switches on the
explicit ObjectStoreConfig.Type instead of the implicit "S3 != nil"
pattern.

CLI (cmd/cli/setup/register.go): storageConfig and baseRepoConfig
mirror the new TOML shape. Test cases updated to match.

Test config (pkg/internal/testutil/config.go): NewTestConfig now
explicitly sets ObjectStore.Type = memory so tests run in the in-memory
backend (previously the test config defaulted to filesystem by
omission, which broke after Type became explicit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promote DataDir, TempDir, Database, and ObjectStore from AppConfig.Storage
to top-level AppConfig peers. The two persistence axes are now visibly
siblings rather than hiding behind a "storage" umbrella that grouped
unrelated concerns.

app.StorageConfig is replaced by app.PersistenceConfig — the same field
set, but now a transient loader-output bundle returned by RepoConfig
.ToAppConfig and never surfaced as an AppConfig section.

Five fx Params structs narrow from app.StorageConfig to app.DatabaseConfig:
they only ever read Database.IsPostgres(), so the bag-for-one-boolean
smell goes away too.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI's go-check job flagged the file (introduced in 69de904 / extended in
3c9c635) as not gofmt-ed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@frrist frrist force-pushed the frrist/fix/deps-mess-pt2 branch from 778cfd6 to 774df49 Compare May 16, 2026 00:21
Base automatically changed from frrist/fix/deps-mess to main May 20, 2026 01:00
@frrist frrist self-assigned this May 20, 2026
Comment thread pkg/config/repo.go
if !objCfg.IsMemory() {
objCfg.Local = app.LocalStorePaths{
Aggregator: app.AggregatorStorageConfig{Dir: filepath.Join(r.DataDir, "aggregator", "datastore")},
Publisher: app.PublisherStorageConfig{Dir: filepath.Join(r.DataDir, "publisher")},
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for the IPNI chain? This can (and usually does) live in s3 if configured.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ermm, I don't think there is an S3 verion of the Publisher store... iirc its interface, which is quite large, didn't quite fit - I can't remember wht though. Bascially there are a set of stores which are always local to the Piri node: aggregator store, publisher store, retrival journal, and key store iirc.

Copy link
Copy Markdown
Member

@alanshaw alanshaw May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was an S3 store we can use/adapt here https://github.com/fil-forge/piri/blob/b94c81177c8c88a3e07bee79427a9f45cc8943a5/pkg/aws/s3store.go but also requires dynamo tables which we'd probably have to port to postgres.

ipniStore := aws.NewS3Store(cfg.Config, cfg.IPNIStoreBucket, cfg.IPNIStorePrefix, cfg.S3Options...)
chunkLinksTable := aws.NewDynamoProviderContextTable(cfg.Config, cfg.ChunkLinksTableName, cfg.DynamoOptions...)
metadataTable := aws.NewDynamoProviderContextTable(cfg.Config, cfg.MetadataTableName, cfg.DynamoOptions...)
publisherStore := store.NewPublisherStore(ipniStore, chunkLinksTable, metadataTable, store.WithMetadataContext(metadata.MetadataContext))

Opened an issue here #16

Comment thread pkg/service/storage/fx.go
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't fx.Annotate/fx.As for doing this?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think those could be used as an alterntive, but we'd need to call them where we provide the store, wrapping its constructor, to provide the returned value as the set of interfaces we need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants