Skip to content

Ansible ND 4.X | Resource Manager Module with Pydantic Models + Smart Endpoints#249

Open
jeetugangwar11 wants to merge 26 commits intoCiscoDevNet:developfrom
jeetugangwar11:resource_manager_1104026
Open

Ansible ND 4.X | Resource Manager Module with Pydantic Models + Smart Endpoints#249
jeetugangwar11 wants to merge 26 commits intoCiscoDevNet:developfrom
jeetugangwar11:resource_manager_1104026

Conversation

@jeetugangwar11
Copy link
Copy Markdown
Collaborator

@jeetugangwar11 jeetugangwar11 commented Apr 10, 2026

Description:
This PR adds the nd_manage_resource_manager module for resource allocation management in the cisco.nd collection, including the core resource handler, endpoint wrappers, Pydantic validation/response models, and shared constants.

What's Included:
Resource handler — [nd_manage_resource_manager_resources.py] supports gathered, merged and deleted states.
The module pre-fetches all existing resources and switches at init time, translates playbook switch management IPs to switchId serial numbers automatically, and uses a single-batch POST for creates and a POST to actions/remove for deletes.
A ResourceManagerDiffEngine stateless helper class handles entity-name normalisation (tilde-order-insensitive), IPv4/IPv6-aware value comparison, to_add/to_update/to_delete/idempotent diff classification, and partial-match mismatch diagnostics (same entity_name, differing pool_name/scope_type/switch) logged to the debugs output bucket.

Endpoint Definitions:
Three endpoint classes in manage_fabrics_resources.py:
EpManageFabricResourcesGet — GET /api/v1/manage/fabrics/{fabricName}/resources — supports switchId, poolName query params and Lucene-style filter/max/offset/sort.
EpManageFabricResourcesPost — POST /api/v1/manage/fabrics/{fabricName}/resources — batch resource allocation; optional tenantName query param.
EpManageFabricResourcesActionsRemovePost — POST /api/v1/manage/fabrics/{fabricName}/resources/actions/remove — batch release of allocated resource IDs.

One additional endpoint in manage_fabrics_switches.py:
EpManageFabricSwitchesGet — GET /api/v1/manage/fabrics/{fabricName}/switches — used at init to build the fabricManagementIp to switchId translation map.

Pydantic Model Layer:
Model files in plugins/module_utils/models/manage_resource_manager/:
resource_manager_config_model.py — ResourceManagerConfigModel: validates a single playbook config entry including per-field normalisation (pool_type uppercase, scope_type underscore form, IPv4/v6/CIDR resource value), cross-field rules (pre-allocated requires resource, switch required for non-fabric scopes, pool_name/scope_type compatibility via POOL_SCOPE_MAP), and state-aware context validation.

resource_manager_request_model.py — ResourceManagerRequest / ResourceManagerBatchRequest: POST body models; includes five discriminated scope models (FabricScope, DeviceScope, DeviceInterfaceScope, DevicePairScope, LinkScope).

resource_manager_response_model.py — ResourceManagerResponse / ResourcesManagerBatchResponse: parse GET and batch POST responses; reuse the same scope union models.

remove_resource_by_id_request_model.py — RemoveResourcesByIdsRequest: {"resourceIds": [...]} body for the actions/remove endpoint.

remove_resource_by_id_response_model.py — RemoveResourcesByIdsResponse / RemoveResourcesByIdResponse: parse the per-item removal result list.

switchs_response_model.py — GetAllSwitchesResponse / SwitchRecord: parse the switches GET response used for IP-to-serial translation.

resource_validators.py — ResourceValidators: shared static helpers for IP address, CIDR, and range validation reused across models.

constants.py — POOL_SCOPE_MAP, SCOPE_TYPE_TO_API, API_SCOPE_TYPE_TO_PLAYBOOK, PoolType, ScopeType, VlanType enums.

Input aliasing:
switch_id is aliased as switch_ip in the argument spec, allowing users to specify either name interchangeably. Pydantic validators normalize both to the field expected by the API.

Merged State:
Merged state — manage_merged uses ResourceManagerDiffEngine.compute_changes to classify each proposed resource as to_add (not in fabric) or to_update (value differs).
Idempotent resources (already matching) are skipped. All pending creates — both new and value-changed — are sent in a single batch POST to resources.
Each item in the batch response is then validated against the corresponding playbook config via validate_resource_api_fields, which performs tilde-order-insensitive entity name comparison and IPv4/v6-aware resource value comparison.
Resources not listed in config are left untouched. Check mode records what would be created without issuing any API calls.

Phase 1 — Delete: orphan resources (present in fabric but absent from config, to_delete bucket) and old values of changed resources (to_update bucket) are collected by integer resourceId and removed in a single batch POST to resources/actions/remove.

Phase 2 — Create: new resources (to_add) and reissued resources with new values (to_update) are created in a single batch POST to resources. The API response is validated per-item via validate_resource_api_fields. Resources whose value already matches the desired config (idempotent) are left untouched. Check mode logs what would be deleted and created without issuing any API calls.

Delete State:
Delete state — manage_deleted uses ResourceManagerDiffEngine.compute_changes to identify which playbook-listed resources are present in the fabric (idempotent or to_update buckets).
Only resources explicitly listed in config are removed; unrelated existing resources are left untouched, matching the ND nd_rm_get_diff_deleted() behaviour.
Deletion is a single batch POST to resources/actions/remove with the collected integer resourceId list. Resources not found in the fabric are silently skipped — the operation is idempotent.

Gathered State:
Gathered state — manage_gathered returns resources in the playbook config format (entity_name, pool_type, pool_name, scope_type, resource, and optionally switch).
When config is omitted, all fabric resources are returned. When config is provided, each entry acts as a filter on entity_name, pool_name, and switch (matched by switchId); a resource must satisfy every non-None criterion to be included, and deduplication by resourceId prevents a resource from appearing twice when matched by multiple filter entries.
The pool_type field (ID/IP/SUBNET) is inferred from the raw resource value by attempting ip_network then ip_address parsing. Multi-switch resources sharing the same (entity_name, pool_name, pool_type, scope_type, resource) key are merged into a single entry with a consolidated switch list.
The gathered output can be fed back directly into state: merged without modification.

Work In Progress:
ND output format structure

Copy link
Copy Markdown
Collaborator

@allenrobel allenrobel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding comments.


# pylint: disable=invalid-name
__metaclass__ = type
# pylint: enable=invalid-name
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

__metaclass__ = type is no longer needed and can be removed. Same comment applies throughout.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have addressed it in all file

```
"""

switch_id: Optional[str] = Field(default=None, min_length=1, description="Serial Number or Id of the switch/leaf")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already a switch_sn for params which expect a switch serial number. Does switch_id require something beyond this?

In module_utils/endpoints/mixins.py:

class SwitchSerialNumberMixin(BaseModel):
    """Mixin for endpoints that require switch_sn parameter."""

    switch_sn: Optional[str] = Field(default=None, min_length=1, description="Switch serial number")

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I need property switchId in query param key and for that reason I added it.


switch_id: Optional[str] = Field(default=None, min_length=1, description="Serial Number or Id of the switch/leaf")
pool_name: Optional[str] = Field(default=None, min_length=1, description="Name of the Pool")
tenant_name: Optional[str] = Field(default=None, min_length=1, description="Name of the tenant")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tenant_name parameter would be useful for the following endpoints within the manage schema:

.../interfaceFlowRules/l3OutEncaps     (required)
.../interfaceFlowRules/l3OutInterfaces (required)
.../interfaceFlowRules/l3Outs          (required)
.../switches/{switchId}/pendingConfig  (optional)
.../vrfFlowRules/vrfs                  (required)

This would imply that tenant_name would be better moved to module_utils/endpoints/mixins.py.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have moved it to mixins


model_config = COMMON_CONFIG

fabric_name: str = Field(min_length=1, max_length=64, description="Name of the fabric")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be better as a shared field. Matt is looking into creating this, so perhaps we can look into it when he's finished. Same comment applies for the other models that contain fabric_name.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have made the change

@@ -0,0 +1,123 @@
# -*- coding: utf-8 -*-

# Copyright: (c) 2026, Allen Robel (@arobel) <arobel@cisco.com>
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change copyright to yourself.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel, Done for all files created by me

default="default",
alias="vrfName",
description="VRF name for the resource",
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as previous comment. We should share common model parameters.

Copy link
Copy Markdown
Collaborator Author

@jeetugangwar11 jeetugangwar11 Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , Ill change it once I get shared fields file from Matt

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @jeetugangwar11 . Can you add a TODO comment to the code so this doesn't slip through the cracks? It might take Matt some time to implement.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , Added TODO as mentioned

@@ -0,0 +1,281 @@
# -*- coding: utf-8 -*-
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We no longer need the coding boilerplate since we're using python v3.

Also, the filename switchs_response_model.py should be switches_response_model.py assuming you want to indicate plurality for switch.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes @allenrobel, I wanted to indicate plurality. it was typo i have renamed the file


__metaclass__ = type
__copyright__ = "Copyright (c) 2026 Cisco and/or its affiliates."
__author__ = "Jeet Ram (@jeeram) <jeeram@cisco.com>"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we are still using __copyright__ and __author__. Definitely __metaclass__ can be removed.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , Changes has been made

log_config = Log()
log_config.config = "/Users/jeeram/ansible/collections/ansible_collections/cisco/nd/plugins/module_utils/logging_config.json"
log_config.commit()
log = logging.getLogger("nd.nd_manage_resource_manager")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why a separate logger?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have made code change to use log.py file

@jeetugangwar11 jeetugangwar11 changed the title Ansible ND 4.X | WIP | Resource Manager Module with Pydantic Models + Smart Endpoints Ansible ND 4.X | Resource Manager Module with Pydantic Models + Smart Endpoints Apr 21, 2026
Copy link
Copy Markdown
Collaborator

@allenrobel allenrobel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. There are a few more cases where legacy annotations are used.
  2. Add a couple TODO commens to the code to track items that are waiting on others.

Returns:
Tilde-separated string with parts sorted alphabetically.
"""
normalis_entity_name = "~".join(sorted(entity_name.split("~")))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo? Should be normalize_entity_name

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have renamed it


from __future__ import annotations

from typing import Any, ClassVar, Dict, List
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove Dict, List imports and convert all to dict, list

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have made the changes.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have made changes


from __future__ import annotations

from typing import Any, ClassVar, List, Optional
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove List import and convert type hints to list

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , Made the changes


import re
from ipaddress import ip_address, ip_network
from typing import Any, ClassVar, Dict, List, Optional
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove legacy Dict, List, Optional and use modern annotations.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , Made the changes

vrf_name: Optional[str] = Field(
default=None,
description=("VRF name associated with the resource allocation. Use 'default' for the global default VRF. When omitted, the default VRF is assumed."),
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a TODO code comment so this is tracked in the code.

from __future__ import annotations

from ipaddress import ip_address
from typing import Any, ClassVar, Dict, List, Literal, Optional, Union
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove legacy Dict, List, Optional, Union and use modern annotations e.g.:

  • Dict -> dict
  • List -> list
  • Optional -> str | None
  • Union -> str | int

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have made the chages for the same


from __future__ import annotations

from typing import Any, ClassVar, Dict, List, Optional, Union
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove legacy Dict, List, Optional, Union and use modern annotations e.g.:

Dict -> dict
List -> list
Optional -> str | None
Union -> str | int

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have made the chages for the same

default="default",
alias="vrfName",
description="VRF name for the resource",
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @jeetugangwar11 . Can you add a TODO comment to the code so this doesn't slip through the cracks? It might take Matt some time to implement.

from __future__ import annotations

from ipaddress import ip_address, ip_network
from typing import Optional
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove legacy Optional and use modern annotations e.g.:

Optional -> str | None

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have made the chages for the same


from __future__ import annotations

from typing import Any, ClassVar, List, Optional
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove legacy List, Optional and use modern annotations e.g.:

List -> list
Optional -> str | None

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenrobel , I have made the chages for the same

@jeetugangwar11 jeetugangwar11 force-pushed the resource_manager_1104026 branch from d8ff149 to 4da8c07 Compare April 24, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants