Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 38 additions & 33 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ is the Diffo project; upstream bugs found while working in Diffo belong here.

1. Read `usage-rules.md` — the canonical rules for using AshNeo4j, including naming conventions,
relationship semantics, aggregate kinds, and the test sandbox.
2. Understand the label system (see **Label system** below) — the three-level label concept is
2. Understand the label system (see **Label system** below) — the label concept is
a frequent source of bugs and the most important thing to get right.
3. Run `mix test` before and after your change to confirm nothing regressed.

Expand All @@ -40,10 +40,12 @@ lib/
functions for every query shape used by the data layer
query_helper.ex — Translates Ash.Query (filter, sort, offset, limit) into
a Cypher.Query; entry point is query_nodes/1
resource/info.ex — All DSL introspection: label/1, module_label/1, labels/1,
resource/info.ex — All DSL introspection: label/1, module_label/1, domain_label/1,
domain_fragment_label/1, all_labels/1, label_pair/1,
mapping/1, relate/1, translations/1, and relationship helpers
resource_mapping.ex — %ResourceMapping{} struct (module, label, module_label,
labels, properties, edges, guards, skip)
domain_fragment_label, all_labels, label_pair,
properties, edges, guards, skip)
edge_descriptor.ex — %EdgeDescriptor{} struct (relationship, label, direction,
destination_label)
neo4j_helper.ex — Low-level node/edge operations via Bolty
Expand All @@ -53,7 +55,8 @@ lib/
sandbox.ex — AshNeo4j.Sandbox: per-test transaction isolation
util.ex — short_name/1, to_camel_case/1, reverse/1
persisters/
persist_labels.ex — Computes and persists domain_label, module_label, label, labels
persist_labels.ex — Computes and persists domain_label, module_label, label,
domain_fragment_label, all_labels, label_pair
persist_translations.ex — Builds attribute → property name keyword list; excludes
belongs_to source attributes and skip attributes
persist_relate.ex — Merges explicit relate DSL with default auto-generated edges
Expand All @@ -78,26 +81,27 @@ test/

## Label system

Every node has three distinct label concepts. Getting them confused is the most common
Every node has several distinct label concepts. Getting them confused is the most common
source of bugs:

| Name | Persisted as | Example | When used |
|---|---|---|---|
| `domain_label` | `:domain_label` | `:Servo` | Written on CREATE only — never used to match |
| `module_label` | `:module_label` | `:ShelfInstance` | Written on CREATE; should be part of MATCH |
| `label` | `:label` | `:Instance` | May differ from module_label when a fragment declares a base type label; used as the MATCH label |
| `labels` | `:labels` | `[:Servo, :ShelfInstance, :Instance]` | Full CREATE label list — `[domain_label | [module_label, label] |> Enum.uniq()]` |
| `domain_label` | `:domain_label` | `:Servo` | Written on CREATE; also part of MATCH via `label_pair` |
| `module_label` | `:module_label` | `:ShelfInstance` | Written on CREATE; also part of MATCH via `label_pair` |
| `label` | `:label` | `:Instance` | May differ from `module_label` when a resource fragment declares a base type; written on CREATE only |
| `domain_fragment_label` | `:domain_fragment_label` | `:Telco` | Written on CREATE only — from a domain fragment using `AshNeo4j.DataLayer.Domain`; `nil` when none declared |
| `all_labels` | `:all_labels` | `[:Servo, :ShelfInstance, :Instance, :Telco]` | Full CREATE label list — `[domain_label, module_label, label, domain_fragment_label]` deduped |
| `label_pair` | `:label_pair` | `[:Servo, :ShelfInstance]` | MATCH label list — always `[domain_label, module_label]`; uniquely identifies this resource type |

**Key invariant:** `labels` (all three) are written on `CREATE`. For `MATCH` / `UPDATE` /
`DELETE`, the domain label is never used. When the resource uses a fragment that contributes a
different `label` (e.g. `:Instance` from `BaseInstance`), reading with only that label matches
nodes from all resources that extend the same fragment — a correctness bug. Use
`[module_label, label]` (deduped) for MATCH so reads are scoped to the exact resource.
**Key invariant:** `all_labels` are written on `CREATE`. For `MATCH` / `UPDATE` / `DELETE`,
use `mapping.label_pair` — always `[domain_label, module_label]`. This two-label combination
uniquely identifies the exact resource type and prevents cross-fragment contamination.

`Cypher.node(:s, [module_label, label])` produces `"(s:ShelfInstance:Instance)"` — correct.
`Cypher.node(:s, [label])` produces `"(s:Instance)"` — scans the whole fragment family.
`Cypher.node(:s, [:Servo, :ShelfInstance])` produces `"(s:Servo:ShelfInstance)"` — correct.
`Cypher.node(:s, [:Instance])` produces `"(s:Instance)"` — scans every resource extending the same fragment.
`Cypher.node(:s, [:ShelfInstance])` produces `"(s:ShelfInstance)"` — scopes to module but not domain (avoid).

`ResourceInfo.module_label/1` and `mapping.module_label` always hold the resource-specific label.
`mapping.label_pair` always holds `[domain_label, module_label]`. Use it for all MATCH patterns.

## Translations (attribute ↔ property name mapping)

Expand Down Expand Up @@ -174,13 +178,14 @@ a `{cypher_string, params}` tuple for `Cypher.run/1`.
`Cypher.node(variable, labels)` takes a list of label atoms and produces `"(var:L1:L2)"`.
`Cypher.parameterized_node/3` does the same with a property map for parameterized MATCH patterns.

When adding a new builder or modifying an existing one, keep `label` parameters as `atom()`
for single-label callers. If a builder needs to support multi-label MATCH (e.g. after the
#257 fix), update the typespec to `atom() | [atom()]` and handle both.
All MATCH/UPDATE/DELETE builders accept `atom() | [atom()]` for source label parameters — pass
`mapping.label_pair` (a list) for all resource operations. Single-atom callers still work for
destination labels (which remain a single label in most patterns).

The aggregate builders (`aggregate_per_record`, `aggregate_total`, `related_nodes`) use direct
string interpolation for the source node pattern — `"(s:#{source_label})"`. To support
multi-label source MATCH these must be updated alongside the read builders.
The aggregate builders (`aggregate_per_record`, `aggregate_total`, `related_nodes`) use a
`labels_string/1` private helper to render `[domain, module]` as `"Domain:Module"` inside
string-interpolated Cypher patterns — `"(s:#{labels_string(label_pair)})"`. When modifying
aggregate builders, use `labels_string/1` for the source pattern, not direct atom interpolation.

## Running tests

Expand Down Expand Up @@ -214,10 +219,10 @@ as a follow-up comment, then leave it with the upstream maintainers.

## Common agent mistakes

- **Matching on `mapping.label` alone** when the resource uses a fragment with a different base
type label (e.g. `:Instance`). MATCH must use `[mapping.module_label, mapping.label]` so
reads are scoped to the exact resource module. `mapping.label` alone matches every resource
that extends the same fragment.
- **Not using `mapping.label_pair` for MATCH.** All read, update, delete, and aggregate queries
must use `mapping.label_pair` (`[domain_label, module_label]`) as the source node pattern.
Using `mapping.label` alone matches every resource that extends the same fragment. Using
`mapping.module_label` alone (without domain) risks collisions across domains.

- **Re-adding `belongs_to` source attributes to translations.** They are intentionally excluded
by `PersistTranslations`. Their values come from enrichments (the OPTIONAL MATCH result).
Expand All @@ -230,9 +235,9 @@ as a follow-up comment, then leave it with the upstream maintainers.
visible, `PersistRelate` generates default edges with wrong labels (e.g. `:BELONGS_TO`
instead of `:SPECIFIED_BY`), causing enrichments to silently fail.

- **Using `mapping.label` in aggregate Cypher builders** (`aggregate_per_record`,
`aggregate_total`, `related_nodes`). These use `"(s:#{source_label})"` directly and have the
same fragment-scoping bug as the read builders. Fix them alongside the read path.
- **Using a single label in aggregate Cypher builders** (`aggregate_per_record`,
`aggregate_total`, `related_nodes`). These use `"(s:#{labels_string(source_label)})"` with a
`labels_string/1` helper. Always pass `mapping.label_pair` as the source label here too.

- **Registering a transformer under `persisters:`** and expecting `before?`/`after?` ordering
relative to other transformers to be honoured. Persisters always run after ALL transformers.
Expand All @@ -243,9 +248,9 @@ as a follow-up comment, then leave it with the upstream maintainers.
label, only one instance is removed. Prefer `List.delete_at` or label filtering by explicit
set membership when precision matters.

- **Treating `domain_label` as a MATCH label.** The domain label is written on CREATE so that
domain-scoped traversals work, but it is never used for reading. Matching on it would return
every node in the domain, not just the target resource.
- **Treating `domain_label` alone as a MATCH label.** The domain label is part of `label_pair`
and is used in MATCH, but always paired with `module_label`. Matching on domain label alone
would return every node in the domain, not just the target resource.

- **Forgetting to update `relation_read` in `Cypher.Query`** when changing MATCH label logic.
The `relationship_read/7` builder emits a separate `MATCH (s:SrcLabel)-[r:EdgeLabel]-(d:DestLabel)`
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,14 @@ See [Conventional Commits](Https://conventionalcommits.org) for commit guideline

* **Aggregates** — full support for `:count`, `:exists`, `:sum`, `:avg`, `:min`, `:max`, `:first`, `:list` aggregate kinds, declared in the standard Ash `aggregates` block. Aggregates are executed as Cypher `OPTIONAL MATCH` traversals; single-hop and multi-hop relationship paths are both supported.
* **Aggregates on embedded/JSON-type fields** — when `field:` points to an attribute stored as JSON (`Ash.TypedStruct`, `Ash.Type.NewType`, embedded resources, `Ash.Type.Map`, etc.) AshNeo4j collects raw JSON from Neo4j and deserializes in Elixir. `:list` and `:first` return fully-typed structs; `:sum`/`:avg`/`:min`/`:max` work on directly comparable values.
* **Expression aggregates (`expr:`)** — programmatic aggregate API (`Ash.aggregate/3`) accepts `expr:` to aggregate over a sub-field of an embedded struct or any Ash expression, without needing to elevate the field. Uses `Ash.Expr.eval_hydrated/2` on full destination records.
* **Expression aggregates (`expr:`)** — programmatic aggregate API (`Ash.aggregate/3`) accepts `expr:` to aggregate over a sub-field of an embedded struct or any Ash expression, without needing to elevate the field. Fetches full destination records and evaluates expressions in Elixir.
* **Expression calculations** — `calculate :name, :type, expr(...)` declarations are now evaluated in Elixir after records are loaded. Supports load (`Ash.load!`), filter (`Ash.Query.filter`), and sort (`Ash.Query.sort`). Embedded struct fields work directly via `get_path` — no elevation needed.

### Improvements

* Cypher query struct family extended; `Neo4jHelper` refactored to use it
* Calculation-based filter predicates are excluded from Cypher WHERE and evaluated in-memory via `Ash.Filter.Runtime`
* Calculation-based sort terms are applied post-load via `Ash.Actions.Sort.runtime_sort/3`
* Calculation-based sort terms are applied in Elixir after records are loaded

## [v0.4.1](https://github.com/diffo-dev/ash_neo4j/compare/v0.4.0...v0.4.1) (2026-05-06)

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@ Ash.aggregate(MyResource, {:total_bandwidth, :sum, [
]})
```

For `expr:` aggregates, AshNeo4j fetches full destination records, evaluates the Ash expression on each via `Ash.Expr.eval_hydrated/2`, and aggregates in Elixir. Any valid Ash expression works — `get_path` for nested struct navigation, arithmetic, etc. Note: `expr:` is a programmatic API and is not available in the resource-level `aggregates do` DSL block.
For `expr:` aggregates, AshNeo4j fetches full destination records, evaluates the Ash expression on each in Elixir, and aggregates the results. Any valid Ash expression works — `get_path` for nested struct navigation, arithmetic, etc. Note: `expr:` is a programmatic API and is not available in the resource-level `aggregates do` DSL block.

## Calculations

Expand All @@ -327,7 +327,7 @@ Calculations can be:

- **Loaded** — `Ash.load!(records, [:score_doubled])`
- **Filtered on** — `Ash.Query.filter(score_doubled > 10)` — AshNeo4j loads all matching nodes then evaluates the filter in Elixir
- **Sorted on** — `Ash.Query.sort(score_doubled: :asc)` — applied in Elixir after records are loaded via `Ash.Actions.Sort.runtime_sort/3`
- **Sorted on** — `Ash.Query.sort(score_doubled: :asc)` — applied in Elixir after records are loaded

**Embedded struct fields work without elevation.** `get_path(dog, [:age])` navigates into a `DogTypedStruct` directly — records arrive with embedded types fully deserialized, so any Ash expression that works in-memory works in a calculation.

Expand Down
11 changes: 7 additions & 4 deletions lib/data_layer.ex
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ defmodule AshNeo4j.DataLayer do
"""
neo4j do
label :Comment
relate [{:post, :BELONGS_TO, :outgoing}]
relate [{:post, :BELONGS_TO, :outgoing, :Post}]
end
"""
],
Expand All @@ -83,17 +83,20 @@ defmodule AshNeo4j.DataLayer do
type: {:list, {:tuple, [:atom, :atom, :atom, :atom]}},
doc:
"Optional list of relationships, as tuples of {relationship_name, edge_label, edge_direction, destination_label}",
required: false
required: false,
default: []
],
guard: [
type: {:list, {:tuple, [:atom, :atom, :atom]}},
doc: "Optional list of node relationships, as tuples of {edge_label, edge_direction, destination_label}",
required: false
required: false,
default: []
],
skip: [
type: {:list, :atom},
doc: "Optional list of attributes not to be stored directly as node properties",
required: false
required: false,
default: []
]
]
}
Expand Down
18 changes: 1 addition & 17 deletions lib/data_layer/domain/info.ex
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,5 @@

defmodule AshNeo4j.DataLayer.Domain.Info do
@moduledoc "Introspection helpers for AshNeo4j.DataLayer.Domain"

alias Spark.Dsl.Extension

@doc """
Returns the label declared in the domain's `neo4j do` block.

The label is written on CREATE for every node whose resource belongs
to this domain. It provides an additional axis for graph traversal
independent of the specific resource type.

Returns `nil` if the domain does not use `AshNeo4j.DataLayer.Domain` or
declares no label.
"""
@spec label(Ash.Domain.t()) :: atom() | nil
def label(domain) do
Extension.get_opt(domain, [:neo4j], :label, nil, true)
end
use Spark.InfoGenerator, extension: AshNeo4j.DataLayer.Domain, sections: [:neo4j]
end
38 changes: 1 addition & 37 deletions lib/data_layer/info.ex
Original file line number Diff line number Diff line change
Expand Up @@ -4,41 +4,5 @@

defmodule AshNeo4j.DataLayer.Info do
@moduledoc "Introspection helpers for AshNeo4j.DataLayer"

alias Spark.Dsl.Extension

@doc """
Returns the label DSL of the resource.
The label is the PascalCase short name of the resource's Elixir Module name by default, but can be overridden by setting the :label option in the DSL. It is used as a Neo4j label for all nodes of the resource.
"""
@spec label(Ash.Resource.t()) :: atom() | nil
def label(resource) do
Extension.get_opt(resource, [:neo4j], :label, nil, true)
end

@doc """
Returns the relate DSL of the resource
"""
@spec relate(Ash.Resource.t()) :: list(tuple()) | nil
def relate(resource) do
Extension.get_opt(resource, [:neo4j], :relate, [], true)
end

@doc """
Returns the guard DSL of the resource
"""
@spec guard(Ash.Resource.t()) :: list(tuple()) | nil
def guard(resource) do
Extension.get_opt(resource, [:neo4j], :guard, [], true)
end

@doc """
Returns the skip DSL of the resource.
The skip DSL is a list of attribute names which are not translated to node properties, either because they are transient or because they will be stored as relationships rather than properties.
By default, all attributes which are the source of a 1:1 belongs_to relationship are skipped, but additional attributes can be skipped by setting the :skip option in the DSL.
"""
@spec skip(Ash.Resource.t()) :: list() | nil
def skip(resource) do
Extension.get_opt(resource, [:neo4j], :skip, [], true)
end
use Spark.InfoGenerator, extension: AshNeo4j.DataLayer, sections: [:neo4j]
end
5 changes: 4 additions & 1 deletion lib/persisters/persist_labels.ex
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,10 @@ defmodule AshNeo4j.Persisters.PersistLabels do
domain_fragment_label =
if Code.ensure_loaded?(domain_module) and
function_exported?(domain_module, :spark_dsl_config, 0) do
AshNeo4j.DataLayer.Domain.Info.label(domain_module)
case AshNeo4j.DataLayer.Domain.Info.neo4j_label(domain_module) do
{:ok, label} -> label
:error -> nil
end
end

# module_label is always the short name of the resource module itself (:Shelf).
Expand Down
15 changes: 10 additions & 5 deletions lib/resource/info.ex
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,12 @@ defmodule AshNeo4j.Resource.Info do
case Extension.get_persisted(resource, :domain_fragment_label, nil) do
nil ->
domain = Extension.get_persisted(resource, :domain, nil)
if domain, do: AshNeo4j.DataLayer.Domain.Info.label(domain), else: nil
if domain do
case AshNeo4j.DataLayer.Domain.Info.neo4j_label(domain) do
{:ok, label} -> label
:error -> nil
end
end

val ->
val
Expand Down Expand Up @@ -97,8 +102,8 @@ defmodule AshNeo4j.Resource.Info do
properties: translations(resource),
edges: Enum.map(relate(resource), &EdgeDescriptor.from_relate/1),
relationship_attributes: relationship_attributes(resource),
guards: AshNeo4j.DataLayer.Info.guard(resource),
skip: AshNeo4j.DataLayer.Info.skip(resource)
guards: AshNeo4j.DataLayer.Info.neo4j_guard!(resource),
skip: AshNeo4j.DataLayer.Info.neo4j_skip!(resource)
}
end

Expand Down Expand Up @@ -296,7 +301,7 @@ defmodule AshNeo4j.Resource.Info do
@doc """
Converts an attribute name to a node property name string, translating if necessary
"""
@spec convert_to_property_name(Ash.Resource.t(), Ash.Query.Ref.t()) :: String.t() | nil
@spec convert_to_property_name(Ash.Resource.t(), struct()) :: String.t() | nil
def convert_to_property_name(resource, ash_query_ref)
when is_atom(resource) and is_struct(ash_query_ref, Ash.Query.Ref) do
attribute_name = Ash.Query.Ref.name(ash_query_ref)
Expand Down Expand Up @@ -347,7 +352,7 @@ defmodule AshNeo4j.Resource.Info do
"""
@spec preserve_node_relationships(Ash.Resource.t()) :: list(tuple())
def preserve_node_relationships(resource) when is_atom(resource) do
Enum.reduce(relate(resource), AshNeo4j.DataLayer.Info.guard(resource), fn {name, edge_label, edge_direction,
Enum.reduce(relate(resource), AshNeo4j.DataLayer.Info.neo4j_guard!(resource), fn {name, edge_label, edge_direction,
destination_label},
acc ->
relationship = Ash.Resource.Info.relationship(resource, name)
Expand Down
Loading
Loading