Skip to content

Specification node write contention — SPECIFIED_BY edge causes lock contention at scale #159

@matt-beanland

Description

@matt-beanland

Description

When a new instance is created, AshNeo4j writes a SPECIFIED_BY edge from the Instance node to the Specification node. Neo4j locks both nodes involved in any edge write to update their degree counts. With a single Specification backing a large number of instances — potentially millions in production — every concurrent instance creation competes for a lock on the same Specification node.

This is distinct from the loading-side density concern closed in #102. Ash never loads from the Specification end, so read performance is not the issue. The hazard is at write time: under concurrent load, instance creation transactions will queue behind each other waiting for the Specification node lock. At sufficient scale this produces transaction timeouts or deadlocks, and the behaviour under those conditions in AshNeo4j is not fully verified.

What we need

The SPECIFIED_BY edge should be eliminated. specification_id is already stored as an attribute on the Instance node — the edge is redundant. Loading the Specification becomes a point lookup by ID rather than edge traversal, which is actually faster than following a dense edge and avoids the write lock entirely.

This requires a change to AshNeo4j to support an indirect relationship mode: the belongs_to :specification relationship stores and reads via the attribute only, with no corresponding graph edge. The Instance node retains specification_id as a queryable property; the SPECIFIED_BY relationship label disappears from the graph.

Why it matters

Any production deployment building instances at scale — bulk provisioning, automated fulfilment — will hit this. It is a silent hazard: performance degrades gradually as instance count grows, and the failure mode under peak load is not obvious. The fix is clean and makes the graph less cluttered, consistent with the indirect-relationship direction discussed in #102.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions