Doc for network extension

Author: weizhouapache

source/adminguide/extensions.rst

@@ -17,7 +17,7 @@
 Extensions
 ==========
 
-Extensions are a new mechanism introduced in Apache CloudStack to allow administrators to extend the platform's functionality by integrating external systems or custom workflows. Currently, CloudStack supports a single extension type called Orchestrator.
+Extensions are a new mechanism introduced in Apache CloudStack to allow administrators to extend the platform's functionality by integrating external systems or custom workflows. Currently, CloudStack supports two extension types: Orchestrator and NetworkOrchestrator.
 
 In the UI, extensions can be managed under *Extensions* menu.
 
@@ -26,7 +26,7 @@ In the UI, extensions can be managed under *Extensions* menu.
 Overview
 ^^^^^^^^
 
-An extension in CloudStack is defined as an external binary (written in any programming language) that implements specific actions CloudStack can invoke. This allows operators to manage resource lifecycle operations outside CloudStack, such as provisioning VMs in third-party systems or triggering external automation pipelines.
+An extension in CloudStack is defined as an external binary (written in any programming language) that implements specific actions CloudStack can invoke. This allows operators to manage resource lifecycle operations outside CloudStack, such as provisioning VMs in third-party systems, orchestrating network and VPC services on external devices, or triggering external automation pipelines.
 
 Extensions are managed through the API and UI, with support for configuration, resource mappings, and action execution.
 
@@ -41,7 +41,7 @@ Administrators can define and manage the following components of an extension:
 
    - Configuration Details: Key-value properties used by the extension at runtime.
 
-   - Resource Mappings: Association between extensions and CloudStack resources such as clusters, etc.
+   - Resource Mappings: Association between extensions and CloudStack resources such as clusters and physical networks.
 
 Path and Availabilty
 ^^^^^^^^^^^^^^^^^^^^
@@ -80,6 +80,25 @@ An Orchestrator extension enables CloudStack to delegate VM orchestration to an
 
    |extension.png|
 
+NetworkOrchestrator Extension
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A NetworkOrchestrator extension enables CloudStack to delegate guest network and VPC service orchestration to an external network system. Key features include:
+
+    - Physical Network Mapping: NetworkOrchestrator extensions are registered with a CloudStack physical network instead of a cluster.
+
+    - Provider-based Integration: When a NetworkOrchestrator extension is registered with a physical network, CloudStack creates an external network service provider using the extension name. Network and VPC offerings can then use that provider.
+
+    - Capability-driven Services: Supported services are declared through the extension details ``network.services`` and optional per-service capabilities in ``network.service.capabilities``. CloudStack uses these declarations when exposing supported services and validating offering capabilities.
+
+    - Network and VPC Lifecycle: Depending on the declared services, the extension can handle operations for guest networks, VPCs, public IPs, NAT, load balancing, DHCP, DNS, userdata, network ACLs, and related restart or reapply flows.
+
+    - Registration Details: Resource-specific details such as device endpoints, credentials, host lists, or interface mappings can be stored on the physical-network registration and updated later through the UI or the ``updateRegisteredExtension`` API.
+
+    - Network and VPC Custom Actions: Admins can define custom actions for ``Network`` and ``Vpc`` resources when the extension advertises the ``CustomAction`` service.
+
+    - Reference Implementation: A Linux network namespace based implementation is available in `cloudstack-extensions <https://github.com/apache/cloudstack-extensions/tree/network-namespace/Network-Namespace>`_. This reference backend has been validated with KVM-based smoke tests.
+
 
 CloudStack provides built-in Orchestrator Extensions for Proxmox, Hyper-V, and MaaS, which work with their respective environments out of the box.
 

source/adminguide/extensions/custom_actions.rst

@@ -17,7 +17,9 @@
 Custom Actions
 ^^^^^^^^^^^^^^
 
-In addition to standard instance operations, extensions support custom actions. These can be configured via UI in the extension details view or the addCustomAction API. The extension binary or script must implement handlers for these action names and process any provided parameters.
+In addition to standard lifecycle operations, extensions support custom actions. These can be configured via UI in the extension details view or the addCustomAction API. The extension binary or script must implement handlers for these action names and process any provided parameters.
+
+For Orchestrator extensions, custom actions typically target ``VirtualMachine`` resources. For NetworkOrchestrator extensions, custom actions can target ``Network`` and ``Vpc`` resources when the extension advertises the ``CustomAction`` network service.
 
    |add-custom-action.png|
 
@@ -43,12 +45,27 @@ A single parameter can have the following details:
 
    - **valueoptions**: Options for the value of the parameter. This is allowed only for NUMBER and STRING type.
 
+Supported Resource Types
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Custom actions can be attached to the following resource types:
+
+   - ``VirtualMachine`` for Orchestrator extensions.
+
+   - ``Network`` for NetworkOrchestrator extensions.
+
+   - ``Vpc`` for NetworkOrchestrator extensions.
+
+For network and VPC custom actions, CloudStack dispatches the action to the external provider that serves the ``CustomAction`` service for the selected resource.
+
 
 Running Custom Action
 ~~~~~~~~~~~~~~~~~~~~~
 
 All enabled custom actions can then be triggered for a resource of the type the action is defined for or provided while running, using the **Run Action** view or runCustomAction API.
 
+The same API is used for VM, network, and VPC custom actions. The available actions shown in the UI depend on the selected resource type and the extension bound to that resource.
+
    |run-custom-action-instance.png|
 
    |run-custom-action.png|

source/adminguide/extensions/developer.rst

@@ -16,7 +16,18 @@
 Writing Extensions for CloudStack
 =================================
 
-The CloudStack Extensions Framework allows developers and operators to write extensions using any programming language or script. From CloudStack’s perspective, an extension is simply an executable capable of handling specific actions and processing input payloads. CloudStack invokes the executable by passing the action name and the path to a JSON-formatted payload file as command-line arguments. The extension processes the payload, performs the required operations on an external system, and returns the result as a JSON response written to `stdout`.
+The CloudStack Extensions Framework allows developers and operators to write extensions using any programming language or script. From CloudStack’s perspective, an extension is an executable capable of handling CloudStack actions and integrating with an external system.
+
+Extension Types
+^^^^^^^^^^^^^^^
+
+CloudStack currently supports two extension types:
+
+- ``Orchestrator`` for external instance lifecycle management. These extensions are registered with ``Cluster`` resources.
+
+- ``NetworkOrchestrator`` for external guest network and VPC service orchestration. These extensions are registered with ``PhysicalNetwork`` resources.
+
+Both types share the same extension lifecycle in the UI and API, but their invocation model and supported resource mappings differ.
 
 
 Create a New Extension
@@ -26,11 +37,12 @@ You must first register a new extension using the API or UI:
 
 .. code-block:: bash
 
-   cloudmonkey createExtension name=myext path=myext-executable
+  cloudmonkey createExtension name=myext type=Orchestrator path=myext-executable
 
 Arguments:
 
 - ``name``: Unique name
+- ``type``: ``Orchestrator`` or ``NetworkOrchestrator``
 - ``path``: Relative path to the executable. Root path will be `/usr/share/cloudstack-management/extensions/<extension_name>`
 
 The path must be:
@@ -43,10 +55,28 @@ If no explicit path is provided during extension creation, CloudStack will scaff
 
 CloudStack checks extension readiness periodically and shows its state in the UI/API.
 
-Extension Structure
-^^^^^^^^^^^^^^^^^^^
+For ``NetworkOrchestrator`` extensions, define supported services in the extension detail ``network.services`` and optionally declare per-service capabilities in ``network.service.capabilities``. CloudStack uses these values when exposing supported providers and validating network and VPC offerings.
+
+Register Extension With a Resource
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+After creating an extension, register it with the CloudStack resource it serves:
+
+- ``Orchestrator`` extensions are registered with ``Cluster`` resources.
+
+- ``NetworkOrchestrator`` extensions are registered with ``PhysicalNetwork`` resources.
+
+Resource-level details supplied during registration are useful for endpoints, credentials, host lists, interface mappings, or other device-specific settings. These registration details can later be changed with the ``updateRegisteredExtension`` API without removing the existing mapping.
+
+When a ``NetworkOrchestrator`` extension is registered with a physical network, CloudStack creates a network service provider using the extension name. Network and VPC offerings can then use that provider name.
+
+Invocation Model
+^^^^^^^^^^^^^^^^
+
+Orchestrator Invocation
+~~~~~~~~~~~~~~~~~~~~~~~
 
-Your extension must support the following invocation structure:
+Orchestrator extensions must support the following invocation structure:
 
 .. code-block:: bash
 
@@ -64,10 +94,23 @@ Sample Invocation:
 
    /usr/share/cloudstack-management/extensions/myext/myext.py deploy /var/lib/cloudstack/management/extensions/myext/162345.json 60
 
+NetworkOrchestrator Invocation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+NetworkOrchestrator extensions use service-specific command names and named CLI arguments instead of the payload-file format above. Depending on the declared services, CloudStack can invoke commands such as ``ensure-network-device``, ``implement-network``, ``shutdown-network``, ``destroy-network``, ``assign-ip``, ``add-static-nat``, ``add-port-forward``, ``apply-fw-rules``, ``apply-lb-rules``, ``apply-network-acl``, ``implement-vpc``, ``shutdown-vpc``, or ``custom-action``.
+
+CloudStack always passes registration and runtime state using named arguments such as:
+
+- ``--physical-network-extension-details`` for details stored on the physical-network registration.
+
+- ``--network-extension-details`` for per-network or per-VPC state maintained by the extension.
+
+Some commands may also receive temporary file paths for larger payloads.
+
 Input Format (Payload)
 ^^^^^^^^^^^^^^^^^^^^^^
 
-CloudStack provides input via a JSON file, which your executable must read and parse.
+For Orchestrator extensions, CloudStack provides input via a JSON file, which your executable must read and parse.
 
 Example:
 
@@ -107,10 +150,12 @@ Example:
 
 The schema varies depending on the resource and action. Use this to perform context-specific logic.
 
+For NetworkOrchestrator extensions, CloudStack passes command-specific CLI arguments such as network IDs, VPC IDs, VLAN tags, IP addresses, action parameters, and JSON strings in named options. The exact input depends on the command being invoked and the services declared in ``network.services``.
+
 Output Format
 ^^^^^^^^^^^^^
 
-Your extension should write a response JSON to ``stdout``. Example:
+Orchestrator extensions should write a response JSON to ``stdout``. Example:
 
 .. code-block:: json
 
@@ -122,11 +167,20 @@ Your extension should write a response JSON to ``stdout``. Example:
 For custom actions, CloudStack will display the ``message`` in the UI if the output JSON includes ``"printmessage": "true"``.
 The ``message`` field can be a string, a JSON object or a JSON array.
 
+For NetworkOrchestrator extensions, most commands signal success or failure by exit code. Some commands return command-specific data on ``stdout``, such as the JSON returned by ``ensure-network-device`` or the output of a ``custom-action``. Keep the output aligned with the command contract implemented by the extension.
+
+Declaring Network Services and Capabilities
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+``NetworkOrchestrator`` extensions should declare their supported services in the extension detail ``network.services`` as a comma-separated list. Example values include ``SourceNat``, ``StaticNat``, ``PortForwarding``, ``Firewall``, ``Lb``, ``Dhcp``, ``Dns``, ``UserData``, ``NetworkACL``, ``Gateway``, ``Vpn``, and ``CustomAction``.
+
+Use ``network.service.capabilities`` to provide a JSON object describing capability values for the declared services. CloudStack uses these capabilities when listing supported providers and validating network or VPC offerings.
+
 Action Lifecycle
 ^^^^^^^^^^^^^^^^
 
 1. A CloudStack action (e.g., deploy VM) triggers a corresponding extension action.
-2. CloudStack invokes the extension’s executable with appropriate parameters.
+2. CloudStack invokes the extension’s executable using the protocol defined by the extension type.
 3. The extension processes the input and responds within the timeout.
 4. CloudStack continues action workflow based on the result.
 
@@ -182,9 +236,12 @@ Custom Actions
 You can define new custom actions for users or admin-triggered workflows.
 
 - Register via UI or ``addCustomAction`` API
+- Choose a resource type that matches the extension type: ``VirtualMachine`` for Orchestrator, ``Network`` or ``Vpc`` for NetworkOrchestrator
 - Define input parameters (name, type, required)
 - Implement the handler for the custom action in your executable.
 
+For NetworkOrchestrator extensions, advertise the ``CustomAction`` service in ``network.services`` if the extension should receive custom actions for network or VPC resources.
+
 CloudStack UI will render forms dynamically based on these definitions.
 
 Best Practices
@@ -195,6 +252,8 @@ Best Practices
 - Avoid hard dependencies on CloudStack internals
 - Implement logging for troubleshooting
 - Use exit code and ``stdout`` for signaling success/failure
+- Keep ``network.services`` and ``network.service.capabilities`` aligned with the services your NetworkOrchestrator implementation actually handles
+- Use resource registration details for environment-specific settings and rotate them with ``updateRegisteredExtension`` instead of recreating mappings when possible
 
 Extension Examples
 ^^^^^^^^^^^^^^^^^^
@@ -231,10 +290,12 @@ Extension Examples
    else:
        print(json.dumps({"success": False, "result": {"message": "Unknown action"}}))
 
-For a clearer understanding of how to implement an extension, developers can refer to the base shell script scaffolded by CloudStack for orchestrator-type extensions. This script is located at:
+For a clearer understanding of how to implement an Orchestrator extension, developers can refer to the base shell script scaffolded by CloudStack for orchestrator-type extensions. This script is located at:
 
 /usr/share/cloudstack-common/scripts/vm/hypervisor/external/provisioner/provisioner.sh
 
 It serves as a template with minimal required action handlers, making it a useful starting point for building new extensions.
 
+For NetworkOrchestrator extensions, refer to the Network Extension Script Protocol in the CloudStack source tree and the reference implementation in `cloudstack-extensions <https://github.com/apache/cloudstack-extensions/tree/network-namespace/Network-Namespace>`_.
+
 Additionally, CloudStack includes in-built extensions for Proxmox and Hyper-V that demonstrate how to implement extensions in different languages - Bash and Python.

source/adminguide/extensions/troubleshooting.rst

@@ -28,6 +28,8 @@ Validate the Extension Path
 
    - Ensure files are stored at: `/usr/share/cloudstack-management/extensions/<extension_name>`
 
+   - For NetworkOrchestrator extensions, ensure the configured path resolves either to an executable file or to a directory that contains an executable named `<extension_name>.sh`.
+
    - CloudStack runs a background task at regular intervals to verify path readiness. If the path is not ready, its state will appear as Not Ready in the UI or API responses.
 
    - Alerts are generated if the extension path is not ready.
@@ -45,13 +47,37 @@ Verify Payload Handling
 
    - Improper parsing of the payload is a common cause of failure—log any parsing errors in your extension binary for debugging.
 
+   - NetworkOrchestrator extensions use command-specific CLI arguments in addition to JSON strings such as ``--physical-network-extension-details`` and ``--network-extension-details``. Verify that the script accepts the expected command names and argument format for the services it implements.
+
+Check Resource Registration and Provider State
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+   - Orchestrator extensions must be registered with a ``Cluster`` resource. NetworkOrchestrator extensions must be registered with a ``PhysicalNetwork`` resource.
+
+   - If resource-specific configuration changes are needed after registration, use the UI or the ``updateRegisteredExtension`` API instead of unregistering and recreating the mapping.
+
+   - For NetworkOrchestrator extensions, confirm that a network service provider with the same name as the extension exists on the target physical network and is in the expected state before creating network or VPC offerings.
+
+Verify Declared Network Services
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+   - NetworkOrchestrator extensions should declare supported services in the extension detail ``network.services``.
+
+   - Per-service capabilities should be declared in ``network.service.capabilities``.
+
+   - If the expected provider services do not appear while creating a network or VPC offering, verify that these details were saved correctly and that any JSON value was quoted correctly when creating or updating the extension.
+
+   - Common declared services include ``SourceNat``, ``StaticNat``, ``PortForwarding``, ``Firewall``, ``Lb``, ``Dhcp``, ``Dns``, ``UserData``, ``NetworkACL``, and ``CustomAction``. Additional services such as ``Gateway`` or ``Vpn`` can also be declared when supported by the implementation.
+
 Refer to Base Extension Scripts
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
    - For guidance on implementing supported actions, refer to the base scripts present for each extension type.
 
    - For Orchestrator-type extensions, see: `/usr/share/cloudstack-common/scripts/vm/hypervisor/external/provisioner/provisioner.sh`
 
+   - For NetworkOrchestrator-type extensions, refer to the Network Extension Script Protocol in the CloudStack source tree and the reference implementation in `cloudstack-extensions <https://github.com/apache/cloudstack-extensions/tree/network-namespace/Network-Namespace>`_.
+
    - These scripts provide examples of how to handle standard actions like start, stop, status, etc.
 
 Check Logs for Errors
@@ -67,4 +93,6 @@ Check Logs for Errors
 
         3. Output parsing
 
+         4. Provider and service resolution for network and VPC operations
+
    - Any exceptions or exit code issues.