Update doc: use JSON payload

Author: weizhouapache

source/adminguide/extensions/custom_actions.rst

@@ -58,13 +58,21 @@ Custom actions can be attached to the following resource types:
 
 For network and VPC custom actions, CloudStack dispatches the action to the external provider that serves the ``CustomAction`` service for the selected resource.
 
+For ``NetworkOrchestrator`` extensions, the action is executed as ``custom-action`` using the standard payload-file invocation model:
+
+.. code-block:: bash
+
+   /path/to/<extension_name>.sh custom-action <payload_file> <timeout_seconds>
+
+The payload file contains top-level keys such as ``action``, ``action-params``, ``physical-network-extension-details``, and ``network-extension-details``. Unlike other network extension commands, the custom action request does not wrap its command-specific values inside a nested ``payload`` object.
+
 
 Running Custom Action
 ~~~~~~~~~~~~~~~~~~~~~
 
-All enabled custom actions can then be triggered for a resource of the type the action is defined for or provided while running, using the **Run Action** view or runCustomAction API.
+All enabled custom actions can then be triggered for a resource of the type the action is defined for or provided while running, using the **Run Action** view or the relevant custom action API.
 
-The same API is used for VM, network, and VPC custom actions. The available actions shown in the UI depend on the selected resource type and the extension bound to that resource.
+For network and VPC custom actions, CloudStack passes the full request in the payload file and returns the script's ``stdout`` to the caller. The available actions shown in the UI depend on the selected resource type and the extension bound to that resource.
 
    |run-custom-action-instance.png|
 

source/adminguide/extensions/developer.rst

@@ -97,15 +97,21 @@ Sample Invocation:
 NetworkOrchestrator Invocation
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-NetworkOrchestrator extensions use service-specific command names and named CLI arguments instead of the payload-file format above. Depending on the declared services, CloudStack can invoke commands such as ``ensure-network-device``, ``implement-network``, ``shutdown-network``, ``destroy-network``, ``assign-ip``, ``add-static-nat``, ``add-port-forward``, ``apply-fw-rules``, ``apply-lb-rules``, ``apply-network-acl``, ``implement-vpc``, ``shutdown-vpc``, or ``custom-action``.
+NetworkOrchestrator extensions also use the payload-file invocation model:
 
-CloudStack always passes registration and runtime state using named arguments such as:
+.. code-block:: bash
+
+  /path/to/<extension_name>.sh <command> <payload_file> <timeout_seconds>
+
+CloudStack resolves the executable in this order:
 
-- ``--physical-network-extension-details`` for details stored on the physical-network registration.
+- ``<extensionPath>/<extensionName>.sh``
 
-- ``--network-extension-details`` for per-network or per-VPC state maintained by the extension.
+- ``<extensionPath>`` itself, if it is an executable file
 
-Some commands may also receive temporary file paths for larger payloads.
+Depending on the declared services, CloudStack can invoke commands such as ``ensure-network-device``, ``implement-network``, ``shutdown-network``, ``destroy-network``, ``restore-network``, ``implement-vpc``, ``shutdown-vpc``, ``update-vpc-source-nat-ip``, ``assign-ip``, ``release-ip``, ``add-static-nat``, ``delete-static-nat``, ``add-port-forward``, ``delete-port-forward``, ``apply-fw-rules``, ``apply-network-acl``, ``add-dhcp-entry``, ``remove-dhcp-entry``, ``config-dhcp-subnet``, ``remove-dhcp-subnet``, ``set-dhcp-options``, ``add-dns-entry``, ``config-dns-subnet``, ``remove-dns-subnet``, ``save-vm-data``, ``save-password``, ``save-userdata``, ``save-sshkey``, ``save-hypervisor-hostname``, ``apply-lb-rules``, and ``custom-action``.
+
+The command name determines the operation. The payload file carries registration details, stored network or VPC state, and command-specific fields.
 
 Input Format (Payload)
 ^^^^^^^^^^^^^^^^^^^^^^
@@ -150,7 +156,21 @@ Example:
 
 The schema varies depending on the resource and action. Use this to perform context-specific logic.
 
-For NetworkOrchestrator extensions, CloudStack passes command-specific CLI arguments such as network IDs, VPC IDs, VLAN tags, IP addresses, action parameters, and JSON strings in named options. The exact input depends on the command being invoked and the services declared in ``network.services``.
+For NetworkOrchestrator extensions, CloudStack writes a JSON payload file and passes its path as the second argument. For standard network and VPC commands, the payload has this envelope:
+
+.. code-block:: json
+
+   {
+     "physical-network-extension-details": {},
+     "network-extension-details": {},
+     "payload": {}
+   }
+
+``physical-network-extension-details`` contains the registration details stored against the physical network, enriched with values such as the physical network name. ``network-extension-details`` contains the per-network or per-VPC state saved by the extension, including the output previously returned by ``ensure-network-device``. ``payload`` contains the command-specific fields.
+
+Frequently used payload fields include ``network_id``, ``vpc_id``, ``vlan``, ``gateway``, ``cidr``, ``extension_ip``, ``public_ip``, ``public_cidr``, ``public_vlan``, ``public_gateway``, ``private_ip``, ``nic_uuid``, ``dns``, and ``domain``.
+
+For ``custom-action``, CloudStack still uses a payload file, but the command-specific values are placed at the top level instead of under ``payload``. The request includes keys such as ``action``, ``action-params``, ``physical-network-extension-details``, and ``network-extension-details``.
 
 Output Format
 ^^^^^^^^^^^^^
@@ -167,7 +187,7 @@ Orchestrator extensions should write a response JSON to ``stdout``. Example:
 For custom actions, CloudStack will display the ``message`` in the UI if the output JSON includes ``"printmessage": "true"``.
 The ``message`` field can be a string, a JSON object or a JSON array.
 
-For NetworkOrchestrator extensions, most commands signal success or failure by exit code. Some commands return command-specific data on ``stdout``, such as the JSON returned by ``ensure-network-device`` or the output of a ``custom-action``. Keep the output aligned with the command contract implemented by the extension.
+For NetworkOrchestrator extensions, all commands must exit with code ``0`` on success and a non-zero code on failure. ``ensure-network-device`` is special: it must write a single-line JSON object to ``stdout`` and CloudStack persists that JSON for later calls. ``custom-action`` may also return output on ``stdout``. Other commands should not rely on ``stdout`` for normal operation because CloudStack ignores it apart from debug logging.
 
 Declaring Network Services and Capabilities
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -176,6 +196,44 @@ Declaring Network Services and Capabilities
 
 Use ``network.service.capabilities`` to provide a JSON object describing capability values for the declared services. CloudStack uses these capabilities when listing supported providers and validating network or VPC offerings.
 
+For example, ``Firewall`` capabilities can declare values such as ``SupportedProtocols``, ``SupportedEgressProtocols``, and ``SupportedTrafficDirection``; ``Lb`` can declare ``SupportedLBAlgorithms`` and ``SupportedProtocols``; and ``SourceNat`` can declare ``SupportedSourceNatTypes``.
+
+CloudStack Setup for NetworkOrchestrator
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+After creating a ``NetworkOrchestrator`` extension, the usual setup flow is:
+
+1. Deploy the executable to the path returned by ``listExtensions`` on every management server.
+2. Register the extension with a ``PhysicalNetwork`` using ``registerExtension`` and pass any device-specific details required by the script.
+3. Enable the generated network service provider on that physical network.
+4. Create network or VPC offerings that map supported services to the extension name.
+5. When registration details need to change, update them with ``updateRegisteredExtension`` instead of deleting and recreating the mapping.
+
+Service-to-Command Mapping
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+NetworkOrchestrator scripts only need to implement the commands for the services they advertise in ``network.services``.
+
+- ``SourceNat`` and ``Gateway`` trigger ``assign-ip`` and ``release-ip``.
+- ``StaticNat`` triggers ``add-static-nat`` and ``delete-static-nat``.
+- ``PortForwarding`` triggers ``add-port-forward`` and ``delete-port-forward``.
+- ``Firewall`` triggers ``apply-fw-rules``.
+- ``Lb`` triggers ``apply-lb-rules``.
+- ``NetworkACL`` triggers ``apply-network-acl``.
+- ``Dhcp`` triggers ``add-dhcp-entry``, ``remove-dhcp-entry``, ``config-dhcp-subnet``, ``remove-dhcp-subnet``, and ``set-dhcp-options``.
+- ``Dns`` triggers ``add-dns-entry``, ``config-dns-subnet``, and ``remove-dns-subnet``.
+- ``UserData`` triggers ``save-vm-data``, ``save-password``, ``save-userdata``, ``save-sshkey``, and ``save-hypervisor-hostname``.
+- Network lifecycle operations always include ``ensure-network-device``, ``implement-network``, ``shutdown-network``, ``destroy-network``, and ``restore-network``.
+- VPC lifecycle operations include ``ensure-network-device``, ``implement-vpc``, ``shutdown-vpc``, and ``update-vpc-source-nat-ip``.
+- Operator-triggered actions use ``custom-action``.
+
+VPC and Extension IP Notes
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For VPC-backed networks, CloudStack includes ``vpc_id`` in tier-level payloads and invokes VPC-level lifecycle commands without a ``network_id``. Use that value when the implementation needs to keep all tiers of a VPC on the same device.
+
+``extension_ip`` is the IP address the external device presents on the guest side. When the extension provides ``SourceNat`` or ``Gateway``, it typically matches the network gateway. When the extension only provides services such as ``Dhcp``, ``Dns``, or ``UserData``, CloudStack can allocate a dedicated guest-side IP for the device and pass it as ``extension_ip``.
+
 Action Lifecycle
 ^^^^^^^^^^^^^^^^
 
@@ -242,6 +300,8 @@ You can define new custom actions for users or admin-triggered workflows.
 
 For NetworkOrchestrator extensions, advertise the ``CustomAction`` service in ``network.services`` if the extension should receive custom actions for network or VPC resources.
 
+For network extension custom actions, the script should read the payload file passed on the command line and parse the top-level keys rather than looking for a nested ``payload`` object.
+
 CloudStack UI will render forms dynamically based on these definitions.
 
 Best Practices
@@ -254,6 +314,7 @@ Best Practices
 - Use exit code and ``stdout`` for signaling success/failure
 - Keep ``network.services`` and ``network.service.capabilities`` aligned with the services your NetworkOrchestrator implementation actually handles
 - Use resource registration details for environment-specific settings and rotate them with ``updateRegisteredExtension`` instead of recreating mappings when possible
+- Keep non-``ensure-network-device`` commands quiet on ``stdout`` unless the command contract explicitly returns output, such as ``custom-action``
 
 Extension Examples
 ^^^^^^^^^^^^^^^^^^

source/adminguide/extensions/troubleshooting.rst

@@ -47,7 +47,13 @@ Verify Payload Handling
 
    - Improper parsing of the payload is a common cause of failure—log any parsing errors in your extension binary for debugging.
 
-   - NetworkOrchestrator extensions use command-specific CLI arguments in addition to JSON strings such as ``--physical-network-extension-details`` and ``--network-extension-details``. Verify that the script accepts the expected command names and argument format for the services it implements.
+   - NetworkOrchestrator extensions receive ``<command> <payload_file> <timeout_seconds>``. Verify that the script accepts the command name and reads the JSON payload file rather than expecting named CLI options.
+
+   - For standard network and VPC commands, confirm that the payload file contains the expected top-level keys: ``physical-network-extension-details``, ``network-extension-details``, and ``payload``.
+
+   - For ``custom-action``, confirm that the request uses top-level keys such as ``action`` and ``action-params`` instead of a nested ``payload`` object.
+
+   - For ``ensure-network-device``, confirm that the script prints a single-line JSON object to ``stdout``. For other commands, unexpected ``stdout`` output is usually a sign that the script is not following the current command contract.
 
 Check Resource Registration and Provider State
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -58,6 +64,8 @@ Check Resource Registration and Provider State
 
    - For NetworkOrchestrator extensions, confirm that a network service provider with the same name as the extension exists on the target physical network and is in the expected state before creating network or VPC offerings.
 
+   - If the provider exists but operations still fail, verify that the provider was enabled after registration and that the intended offering maps each supported service to the extension name.
+
 Verify Declared Network Services
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -95,4 +103,6 @@ Check Logs for Errors
 
          4. Provider and service resolution for network and VPC operations
 
+         5. Failures while decoding nested payload values such as firewall rules, ACL rules, restore data, or VM metadata blobs
+
    - Any exceptions or exit code issues.