Merge pull request #135 from diffo-dev/120-refactor-assigner

120 refactor assigner

Author: matt-beanland

AGENTS.md

@@ -28,6 +28,7 @@ lib/diffo/provider/
     info.ex                     # Runtime introspection via Spark.InfoGenerator
     characteristic.ex           # Characteristic build helpers
     feature.ex                  # Feature build helpers
+    pool.ex                     # Pool struct + create_pools/2 + update_pools/3
     instance_role.ex            # InstanceRole struct
     party_declaration.ex        # PartyDeclaration struct
     place_declaration.ex        # PlaceDeclaration struct
@@ -36,13 +37,19 @@ lib/diffo/provider/
     persisters/                 # Spark transformers — bake DSL state into module
     transformers/               # TransformBehaviour — action argument injection
     verifiers/                  # Compile-time DSL correctness checks
+  assigner/
+    assigner.ex                 # Diffo.Provider.Assigner — assign/3 (pools do) and assign/4
+    assignable_characteristic.ex # AssignableCharacteristic — pool bounds + algorithm
+    assigned_to_relationship.ex # AssignedToRelationship — assignedTo edges (pool/thing/assigned)
   base_instance.ex              # Ash Fragment for Instance resources
   base_party.ex                 # Ash Fragment for Party resources
   base_place.ex                 # Ash Fragment for Place resources
   components/
     base_characteristic.ex      # Ash Fragment for typed characteristic resources
+    base_relationship.ex          # Ash Fragment for shared Relationship structure
     calculations/
       characteristic_value.ex   # Calculation: builds .Value TypedStruct from record fields
+      assigned_values.ex        # Calculation: returns list of assigned integers for a pool+thing
     instance/extension.ex       # Thin marker (sections: []) — kind identification
     party/extension.ex          # Thin marker
     place/extension.ex          # Thin marker
@@ -89,6 +96,11 @@ provider do
     characteristic :ports, {:array, MyApp.PortCharacteristic}
   end
 
+  pools do
+    pool :cores, :core   # assignable pool; thing name is :core
+    pool :vlans, :vlan
+  end
+
   features do
     feature :advanced_routing, is_enabled?: false do
       characteristic :policy, MyApp.RoutingPolicy
@@ -143,11 +155,31 @@ mix test path/to/test.exs:LINE    # single test
 mix test --max-failures 5         # stop early
 ```
 
+## Module naming and Neo4j labels
+
+AshNeo4j derives a node label from the **last segment** of the module name. Two resources
+whose names end in the same word get the same label, which causes read collisions.
+
+**Rule:** suffix every resource module with its kind so the last segment is unique:
+- Instance resources: `MyApp.Instance.WidgetInstance` (not `MyApp.Instance.Widget`)
+- Characteristic resources: `MyApp.Characteristic.WidgetCharacteristic` (not `MyApp.Characteristic.Widget`)
+- Party/Place resources: follow the same convention if ambiguity is possible.
+
+E.g. `Diffo.Test.Instance.CardInstance` → label `:CardInstance`,
+and `Diffo.Test.Characteristic.CardCharacteristic` → label `:CardCharacteristic` — no collision.
+
 ## Common agent mistakes
 
 - Using old `structure do` / top-level `instances do` — use `provider do` only.
 - Using `party :role, Type, reference: true` — use `party_ref :role, Type`.
 - Using a plain `Ash.TypedStruct` as a `characteristic` DSL target — use a `BaseCharacteristic`-derived resource instead; the TypedStruct belongs in `<Module>.Value`.
+- Using `characteristic :name, Diffo.Provider.AssignableCharacteristic` for pools — use `pools do / pool :name, :thing / end` instead.
+- Using the removed `AssignableValue` TypedStruct — it no longer exists; use `pools do`.
+- Calling `Assigner.assign/4` when a `pools do` declaration exists — prefer `Assigner.assign/3` which looks up the thing automatically.
+- Forgetting to call `Pool.update_pools/3` in `:define` actions when the resource has `pools do` — pool bounds (`first`, `last`, `algorithm`) are set here.
+- Using `characteristic :pool_name, Diffo.Provider.AssignedToRelationship` — `AssignedToRelationship` is not a characteristic; use `pools do / pool :name, :thing / end` instead.
+- Querying `Diffo.Provider.Relationship` for assignment records — assignment relationships are on `Diffo.Provider.AssignedToRelationship`; access them via `instance.assignments`.
+- Filtering `instance.forward_relationships` for `type == :assignedTo` — those records no longer exist there; use `instance.assignments` directly.
 - Calling `build_before/1` or `build_after/2` in actions — these run automatically.
 - Declaring `:specified_by`, `:features`, `:characteristics` as action arguments.
 - Editing `documentation/dsls/DSL-Diffo.Provider.Extension.md` — it is Spark-generated;

documentation/dsls/DSL-Diffo.Provider.Extension.md

@@ -30,6 +30,10 @@ the sections relevant to it, and verifiers enforce correct usage.
         end
       end
 
+      pools do
+        pool :ports, :port
+      end
+
       parties do
         party :provider, MyApp.Provider
         party_ref :owner, MyApp.InfrastructureCo
@@ -92,6 +96,8 @@ Provider DSL — structure, roles, and behaviour for this resource kind
  * [features](#provider-features)
    * feature
      * characteristic
+ * [pools](#provider-pools)
+   * pool
  * [parties](#provider-parties)
    * party
    * parties
@@ -276,6 +282,50 @@ Adds a Characteristic
 
 
 
+### provider.pools
+Assignable pools on this Instance — each pool maps to an AssignableCharacteristic
+
+### Nested DSLs
+ * [pool](#provider-pools-pool)
+
+
+### Examples
+```
+pools do
+  pool :ports, :port
+end
+
+```
+
+
+
+
+### provider.pools.pool
+```elixir
+pool name, thing
+```
+
+
+Declares an assignable pool — a named range of values for auto-assignment
+
+
+
+
+
+### Arguments
+
+| Name | Type | Default | Docs |
+|------|------|---------|------|
+| [`name`](#provider-pools-pool-name){: #provider-pools-pool-name .spark-required} | `atom` |  | The pool name (matches the AssignableCharacteristic name). |
+| [`thing`](#provider-pools-pool-thing){: #provider-pools-pool-thing .spark-required} | `atom` |  | The name of the thing being assigned within the pool (e.g. :port). |
+
+
+
+
+
+
+
+
 ### provider.parties
 Party roles on this resource — `party`/`parties`/`party_ref` for Instance kinds; `role` for Party and Place kinds
 

documentation/how_to/use_diffo_provider_extension.livemd

@@ -141,6 +141,8 @@ The id is a stable UUID4, the same in every environment for this Instance kind.
 
 **`features do`** — optional capabilities with their own typed characteristic payload.
 
+**`pools do`** — assignable pools for partial resource allocation. Each `pool :name, :thing` declaration creates an `AssignableCharacteristic` node during `build` and generates `pools/0` / `pool/1` on the module. Pool bounds (`first`, `last`, `algorithm`, `assignable_type`) are set in a `:define` action via `Pool.update_pools/3`. Assignment actions use `Assigner.assign/3` — the thing name is looked up from the pool declaration.
+
 **`parties do`** — party roles: `party` (singular), `parties` (plural), `party_ref` (reference, no direct edge).
 
 **`places do`** — place roles: `place` (singular), `places` (plural), `place_ref` (reference).
@@ -151,7 +153,7 @@ arguments automatically onto that action.
 
 Each characteristic is a dedicated Ash resource using the `Diffo.Provider.BaseCharacteristic` fragment. It carries direct typed attributes and a `:value` calculation that builds a companion `<Module>.Value` TypedStruct for ordered JSON encoding. The TypedStruct uses [AshJason.TypedStruct](https://hexdocs.pm/ash_jason/) to control field order in the JSON output.
 
-For partial resource allocation and assignment we've created Diffo.Provider.Assigner. It is used by the host resource, which declares a characteristic with a `Diffo.Provider.AssignableValue` TypedStruct. Allocation is managed within the Provider domain using this characteristic. Assignment to Services or Resources is via 'reverse' type: "assignedTo" relationships enriched by relationship characteristics.
+For partial resource allocation and assignment we've created `Diffo.Provider.Assigner`. The host resource declares a `pools do` section listing each assignable pool by name and the kind of thing being assigned. Pool bounds (first/last value, algorithm) are set via a `:define` action. Each assignment is stored as a `Diffo.Provider.AssignedToRelationship` node (Neo4j label `:AssignmentRelationship`) carrying `pool`, `thing`, and the `assigned` value. These are distinct from regular TMF `Diffo.Provider.Relationship` nodes and are accessible on an instance via `instance.assignments`.
 
 Let's imagine a Compute domain which operates GPU and NPU resources. We want to expose a Cluster composite resource which can be dynamically composed of a number of GPU and NPU cores.
 
@@ -381,7 +383,7 @@ defmodule Diffo.Compute.GpuCharacteristic.Value do
 end
 ```
 
-The GPU resource declares `GpuCharacteristic` for the typed `:gpu` slot and keeps `AssignableValue` for the `:cores` allocation pool (the assigner still uses the dynamic characteristic pattern). The `update :define` action now only needs to handle the dynamic `:cores` update — the typed `:gpu` characteristic is updated directly on the characteristic resource:
+The GPU resource declares `GpuCharacteristic` for the typed `:gpu` slot and uses `pools do` to declare the `:cores` assignable pool. The `update :define` action updates both the typed characteristic and the pool bounds. The `update :assign_core` action uses `Assigner.assign/3` — the thing name (`:core`) is looked up from the pool declaration automatically:
 
 ```elixir
 defmodule Diffo.Compute.GPU do
@@ -391,10 +393,10 @@ defmodule Diffo.Compute.GPU do
 
   alias Diffo.Provider.BaseInstance
   alias Diffo.Provider.Instance.Relationship
-  alias Diffo.Provider.Instance.Characteristic
+  alias Diffo.Provider.Extension.Characteristic
+  alias Diffo.Provider.Extension.Pool
   alias Diffo.Provider.Assigner
   alias Diffo.Provider.Assignment
-  alias Diffo.Provider.AssignableValue
   alias Diffo.Compute
   alias Diffo.Compute.GpuCharacteristic
 
@@ -418,7 +420,10 @@ defmodule Diffo.Compute.GPU do
 
     characteristics do
       characteristic :gpu, GpuCharacteristic
-      characteristic :cores, AssignableValue
+    end
+
+    pools do
+      pool :cores, :core
     end
 
     behaviour do
@@ -442,11 +447,13 @@ defmodule Diffo.Compute.GPU do
     end
 
     update :define do
-      description "allocates the GPU cores (AssignableValue)"
+      description "sets GPU identity and allocates the cores pool"
       argument :characteristic_value_updates, {:array, :term}
 
       change after_action(fn changeset, result, _context ->
-               with {:ok, result} <- Characteristic.update_values(result, changeset),
+               with {:ok, result} <-
+                      Characteristic.update_all(result, changeset, characteristics()),
+                    {:ok, result} <- Pool.update_pools(result, changeset, pools()),
                     {:ok, result} <- Compute.get_gpu_by_id(result.id),
                     do: {:ok, result}
              end)
@@ -464,19 +471,18 @@ defmodule Diffo.Compute.GPU do
     end
 
     update :assign_core do
-      description "relates the GPU with an instance by assigning a core"
+      description "assigns a core from this GPU to another instance"
       argument :assignment, :struct, constraints: [instance_of: Assignment]
 
       change after_action(fn changeset, result, _context ->
-               with {:ok, result} <- Assigner.assign(result, changeset, :cores, :core),
+               with {:ok, result} <- Assigner.assign(result, changeset, :cores),
                     {:ok, result} <- Compute.get_gpu_by_id(result.id),
                     do: {:ok, result}
              end)
     end
   end
 end
 ```
-```
 
 ## Party Extension
 
@@ -745,27 +751,19 @@ gpu_1 = Compute.build_gpu!(%{name: "GPU 1"})
 gpu_2 = Compute.build_gpu!(%{name: "GPU 2"})
 ```
 
-We set the typed `:gpu` characteristic directly on the characteristic resource, then allocate the `:cores` AssignableValue via `update :define`:
+We define each GPU: setting its typed `:gpu` characteristic fields and allocating the `:cores` pool bounds. Both are passed via `characteristic_value_updates` to the `:define` action — `Characteristic.update_all` handles the typed `:gpu` update and `Pool.update_pools` handles the `:cores` pool bounds:
 
 ```elixir
-# Update the typed GpuCharacteristic on each GPU
-[gpu_char_1] = Enum.filter(gpu_1.characteristics, fn c -> c.name == :gpu end)
-[gpu_char_2] = Enum.filter(gpu_2.characteristics, fn c -> c.name == :gpu end)
-
-gpu_attrs = %{family: :nvidia, model: "GeForce RTX5090", technology: :blackwell}
-Compute.update_gpu_characteristic!(gpu_char_1, gpu_attrs)
-Compute.update_gpu_characteristic!(gpu_char_2, gpu_attrs)
-```
-
-```elixir
-# Allocate the cores pool (AssignableValue — dynamic characteristic)
-core_updates = [cores: [first: 1, last: 680, free: 680, assignable_type: "tensor"]]
+gpu_attrs = [
+  gpu: [family: :nvidia, model: "GeForce RTX5090", technology: :blackwell],
+  cores: [first: 1, last: 680, assignable_type: "tensor"]
+]
 
-gpu_1 = Compute.define_gpu!(gpu_1, %{characteristic_value_updates: core_updates})
-gpu_2 = Compute.define_gpu!(gpu_2, %{characteristic_value_updates: core_updates})
+gpu_1 = Compute.define_gpu!(gpu_1, %{characteristic_value_updates: gpu_attrs})
+gpu_2 = Compute.define_gpu!(gpu_2, %{characteristic_value_updates: gpu_attrs})
 ```
 
-The GPU's `:cores` characteristic is an AssignableValue that tracks how many cores are free (unassigned). We can render one as json:
+The `:cores` pool is backed by an `AssignableCharacteristic` node that records the range bounds and algorithm. Free cores are computed at assignment time from the count of existing `AssignmentRelationship` records — there is no stored `free` counter. We can render one as json:
 
 ```elixir
 Jason.encode!(gpu_1, pretty: true) |> IO.puts
@@ -785,16 +783,16 @@ gpu_1 = Compute.assign_gpu_core!(gpu_1, assignment)
 gpu_2 = Compute.assign_gpu_core!(gpu_2, assignment)
 ```
 
-Now our cluster should have a core from each gpu. Check in the neo4j browser for the type: :assignedTo Relationship from the gpu_1 and gpu_2 to the clusters. There should be four, each with a Relationship Characteristic of core, with a value of the assigned core, e.g. 1, 2.
+Now our cluster should have a core from each GPU. Check in the Neo4j browser for `:AssignmentRelationship` nodes from `gpu_1` and `gpu_2` to `cluster_1`. There should be four — each carries `pool: :cores`, `thing: :core`, and the `assigned` integer value (e.g. 1, 2, 3 from gpu_1 and 1 from gpu_2).
 
-Also the gpu will show each assignedTo relationship, since these are forward relationships. These should also show the relationship characteristic:
+The GPU's `assignments` hold each assignment, showing the assigned core number in the JSON encoding as a `resourceRelationshipCharacteristic`:
 
 ```elixir
 Jason.encode!(gpu_1, pretty: true) |> IO.puts
 ```
 
-Make sure you have a look at it in the neo4j browser. There should be Relationship nodes with a role of :assignedTo from each GPU resource instance to the cluster_1 resource instance. Each Relationship should be defined by a Characteristic with the assigned core number.
-There is no central assignment table, rather the relationships ARE the assignments.
+Make sure you have a look at it in the neo4j browser. There should be `:AssignmentRelationship` nodes from each GPU resource instance to the `cluster_1` resource instance, each carrying the assigned core number.
+There is no central assignment table — the `AssignedToRelationship` nodes ARE the assignments. They are separate from the regular `:Relationship` nodes used for TMF service/resource relationships, and are accessible in Elixir via `instance.assignments`.
 
 As an exercise, clone the GPU resource to create an NPU resource and assign some NPU cores from it to your cluster. Check that the assigned NPU cores are unique.
 
@@ -805,7 +803,9 @@ What happens when I request a specific assignment from an instance to which the
 
 In this tutorial you've used Diffo's unified `provider do` extension to define a Compute domain with:
 
-- A composite Cluster resource with GPU core assignment via `Diffo.Provider.Assigner`
+- A composite Cluster resource that receives GPU cores via `Diffo.Provider.Assigner`
+- A GPU resource using `pools do` to declare the `:cores` assignable pool — `pool :cores, :core` replaces the old `characteristic :cores, AssignableValue` pattern
+- Assignments stored on `Diffo.Provider.AssignedToRelationship` nodes (Neo4j label `:AssignmentRelationship`, distinct from TMF `:Relationship` nodes); accessible via `instance.assignments`
 - `Tenant` and `Engineer` Party kinds declared with `provider do` that express which instances they operate and manage
 - A `DataCentre` Place kind that declares the instances located at it
 

lib/diffo/provider.ex

@@ -77,6 +77,19 @@ defmodule Diffo.Provider do
       define :delete_relationship, action: :destroy
     end
 
+    resource Diffo.Provider.AssignedToRelationship do
+      define :create_assigned_to_relationship, action: :create_assignment
+      define :get_assigned_to_relationship_by_id, action: :read, get_by: :id
+      define :delete_assigned_to_relationship, action: :destroy
+    end
+
+    resource Diffo.Provider.AssignableCharacteristic do
+      define :create_assignable_characteristic, action: :create
+      define :get_assignable_characteristic_by_id, action: :read, get_by: :id
+      define :update_assignable_characteristic, action: :update
+      define :delete_assignable_characteristic, action: :destroy
+    end
+
     resource Diffo.Provider.Characteristic do
       define :create_characteristic, action: :create
       define :get_characteristic_by_id, action: :read, get_by: :id