Dependency Resolution Tracking Issue

[nrdxp]

Description

This issue tracks the effort to implement robust dependency management in eka. The work has spanned multiple pull requests and includes several architectural decisions. This serves as a central point to understand the history and current status of dependency resolution.

Completed Work

• Initial Dependency Schema and Lockfile Format

  • Defined the initial dependency schema (d3912708, 8eb732aa)
  • Specified and implemented the initial lockfile format and parser (ee5d6f63, a8147f25, 5122828f)

• Remote Repository Interaction

  • Implemented efficient querying of remote git repositories using gix::Url (98735641)
  • Allowed fetching refs without pulling objects to improve performance (13878fdd)
  • Reused network connections for subsequent queries (20f3b1b1)

• Core Dependency Resolution Logic

  • Implemented atom version matching against a VersionReq (c0fff6eb)
  • Initial implementation of the eka add command (a83b5bdc, 9da27206)
  • Foundational implementation of dependency resolution (8448e659)
  • Implemented legacy git repo version resolution to find the best matching tag (c79a5226)

• Manifest and Lockfile Synchronization

  • Initial direct pin manifest writer (5b2bd534)
  • Encapsulated logic in ManifestWriter to ensure the manifest and lockfile remain in sync (1788c438)
  • Implemented initial lock synchronization to ensure the manifest is the source of truth (8507aaa9)
  • Completed lock synchronization logic to handle all cases (c1b24d0e)

• URI Handling

  • Implemented parsing for aliased URLs and URLs with git refs (249abe62, b4f1b5c8)

Related PRs

• feat: dependency resolution #19
• feat: pin dependency fetching #21
• feat: finalize dependency management #23

Outstanding Todos

via PR: #23

• pre-publish manifest/lock sanity checks

• path based dependency handing

• indirect pins (from deps)

• a finalized version of atom-nix's lock parsing evaluation code aligned with eka's lock format

  • as an alternative to the previous complexity of from dependencies, said lock parser will have a generic mechanism to modify import behavior

• ADR#7 Stable Schema implementation

  • Implement core types for new schema
  • Implement mirrored package set resolution
  • Use the new format to rework the lock sychronization code
    • re-work atom sychronization
    • re-work manifest writing
    • re-work direct nix dependency synchronization
    • implement version constraints on direct dependency urls
      • parse version constraints to build an Aliased url with a possible version_from field
      • use said field during resolution to determine the proper locked version

• ADR#9 Named Roots & Identity (relevant parts implemented, changes to ADR design incoming)

  • Configuration:
    • Implement ekala.toml parser.
    • Enforce strict parsing to reject unknown keys.
  • Discovery:
    • Implement namespaced Git ref for discovery (refs/tags/ekala/root/v1/<project-tag>).
    • Update eka init to create and push the tag.
  • Identity:
    • Modify the atom ID hashing algorithm to include the project-tag.
  • eka add Command:
    • Implement conflict detection.
    • Add --set <alias> flag for conflict resolution.
  • Manifest & Lockfile:
    • Integrate with [package.sets] in atom.toml.

Relevant ADRs

• ADR #1: Lock Generation
• ADR #2: eka add command
• ADR #3: Pure Rust Pin Dependencies
• ADR #6: Snix HTTP Caching
• ADR #7: Refined Manifest Format
• ADR #9: Named Roots MVP

[nrdxp]

the bulk of the work is done, fitting time to close