Skip to content

Use network as connector_name for Gateway executors#8150

Merged
nikspz merged 62 commits intodevelopmentfrom
feature/connector-level-retry
Apr 17, 2026
Merged

Use network as connector_name for Gateway executors#8150
nikspz merged 62 commits intodevelopmentfrom
feature/connector-level-retry

Conversation

@fengtality
Copy link
Copy Markdown
Contributor

@fengtality fengtality commented Mar 29, 2026

Summary

Refactors Gateway executors to use network identifier as connector_name (e.g., solana-mainnet-beta) instead of DEX format. Updates OrderExecutor and LPExecutor to work with this architecture, and updates lp_rebalancer controller to use both executors with autoswap support.

Key Changes

Gateway Connector Architecture

  • connector_name: Now uses network identifier (e.g., solana-mainnet-beta)
  • lp_provider: New field for LP executor in format dex/trading_type (e.g., meteora/clmm)
  • swap_provider: Auto-resolved from network config during start_network()

OrderExecutor

  • Works with Gateway connectors using network-based connector_name
  • Gateway _create_order() falls back to swap_provider when dex_name not provided
  • Add executed_amount_base, average_executed_price, filled_amount_quote properties

LPExecutor

  • Use lp_provider field instead of separate dex_name + trading_type
  • Add close-out swap support via SWAPPING state (when keep_position=False)
  • Add connector-level retry logic in gateway_base.py
  • Parse lp_provider to get dex_name and trading_type for Gateway API calls

lp_rebalancer Controller

  • Uses LPExecutor for liquidity positions
  • Uses OrderExecutor for autoswap (automatic token swapping when balance insufficient)
  • Auto-close via executor limit prices (replaces timer-based rebalancing)
  • Remove swap_provider config (uses network config automatically)

Removed

  • gateway_lp.py, gateway_swap.py classes

Related PRs

Repo PR Description
hummingbot #8150 Use network as connector_name (this PR)
hummingbot-api #142 Use network as connector_name for API
gateway #620 Fix fee calculation, add error parsing

Test plan

  • All existing tests pass
  • 100% diff coverage for LP executor tests
  • Run lp_rebalancer controller on a TOKEN-USDC pair on Meteora
  • Run lp_rebalancer controller on a TOKEN-SOL pair on Orca
  • Run xemm_multiple_levels controller btw SOL-USDC on Solana and Binance
  • Run arbitrage controller btw SOL-USDC on Solana and Binance
  • Test gateway lp command
  • Test gateway swap command

fengtality and others added 5 commits April 1, 2026 13:43
- Add retry logic with error code classification in gateway_base.py
- Retryable: TRANSACTION_TIMEOUT (network issues)
- Non-retryable: SIMULATION_FAILED, INSUFFICIENT_BALANCE, SLIPPAGE_EXCEEDED, etc.
- Move retry handling from executors to connectors for consistency
- Add gateway_utils.py for shared connector validation
- Update gateway_lp.py and gateway_swap.py to return event objects

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- New executor for single swap operations on Gateway AMM connectors
- Uses GatewaySwap connector's place_order for proper order tracking
- Handles retry logic at connector level
- Stores executed orders for position tracking (keep_position support)
- State flow: NOT_STARTED -> EXECUTING -> COMPLETED/FAILED

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add position tracking via held_position_orders (like GridExecutor)
- Store ADD events as SELL, REMOVE events as BUY for standard P&L
- Use tx_hash as deduplication key (unique per transaction)
- Add connector validation with auto-normalization (/clmm suffix)
- Support keep_position for position hold aggregation
- Fix P&L calculation for failed executors
- Use asdict(event) pattern for consistent serialization

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add autoswap feature for automatic token swapping when balance insufficient
- Add negative offset support for in-range positions
- Fix _initial_position_created flag to only set when position is active
- Add swap_provider and swap_buffer_pct config options
- Improve status display with position tracking info
- Ensure rebalance always uses side 1 or 2, never side 0

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add SwapExecutor to executor orchestrator registry
- Fix orchestrator exit hang (break vs continue in stop loop)
- Fix position hold edge cases (division by zero, NaN handling)
- Add LP history JSON API via MQTT endpoint
- Add lphistory JSON export for API consumption
- Add simple_xemm_gateway example script

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@fengtality fengtality force-pushed the feature/connector-level-retry branch from 44ae7f1 to a7119fe Compare April 1, 2026 20:44
fengtality and others added 13 commits April 2, 2026 14:52
- Change connector_name from DEX format (orca/clmm) to network format (solana-mainnet-beta)
- Add dex_name field for DEX protocol name (e.g., "orca", "meteora")
- Rename dex parameter to dex_name throughout for clarity
- Combined GatewaySwap and GatewayLp into unified Gateway class
- Update gateway swap command to use network format and auto-fetch default swap provider
- Update gateway LP and pool commands for new architecture
- Update all tests for new parameter naming

Architecture:
- connector_name: Network identifier (e.g., "solana-mainnet-beta")
- dex_name: DEX protocol name (e.g., "orca", "jupiter")
- trading_type: Pool type (e.g., "clmm", "amm", "router")

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Missed renaming dex field in the controller config.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add get_default_swap_provider(network) method to GatewayHttpClient
  that auto-detects the default swap provider (router) for a network
- Update lp_rebalancer to use the helper for autoswap feature
- Make swap_provider config optional - uses network default if not set
- Fix Gateway connector response parsing (networks/trading_types arrays)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Rename parameter from 'connector' to 'dex_type' throughout gateway LP
command to prevent confusion with network connector. The dex_type
parameter represents the DEX type in format 'dex_name/trading_type'
(e.g., 'orca/clmm'), not the network connector.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ectors

feat: refactor gateway connector architecture to use network format
GatewayHttpClient methods (quote_swap, execute_swap, execute_quote) expect
dex and trading_type as separate parameters, not connector.

- Add _parse_dex_name() helper to split "jupiter/router" into ("jupiter", "router")
- Update get_quote_price() to use dex and trading_type params
- Update _create_order() to use dex and trading_type params
- Require dex_name for swap operations on unified Gateway connector

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update test_gateway_lp_command.py error message to match new "DEX type" wording
- Rewrite test_gateway_lp.py to use new network-based connector architecture
- Fix test_lp_executor.py to pass dex_name and trading_type to get_pool_info_by_address
- Update test_swap_executor.py for new SwapExecutorConfig schema (connector_name as network, swap_provider field)
- Fix test_executor_orchestrator.py incorrect NaN assertion for buy_amount_quote
- Fix swap_executor.py to use config.connector_name instead of removed config.network

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix swap executor stuck in RUNNING state due to missing dex_name
  in place_order() and get_quote_price() calls
- Refactor provider config: use lp_provider (format: "dex/trading_type")
  instead of separate dex_name + trading_type fields
- Make swap_provider optional - resolves network default if not provided
- Add parse_provider() utility to gateway_utils.py
- Update lp_rebalancer controller to use new lp_provider field

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When an order fails quickly (e.g., insufficient balance), the order tracker
emits MarketOrderFailureEvent and removes the order. If the executor misses
the event, it would get stuck in EXECUTING state forever.

Now if the order is not found for 3 consecutive checks, the executor
assumes the order failed and transitions to FAILED state.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When connector lookup fails in _create_position() or _close_position(),
the executor would return early without changing state, causing infinite
retries. Now properly transitions to FAILED state.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When keep_position=False, LP executor now executes a swap to return to
the original position after closing the LP position, similar to how grid
executor handles spot positions.

Changes:
- Add SWAPPING state to LPExecutorStates
- Add active_swap_order tracking to LPExecutorState
- Add _execute_closeout_swap() method to sell excess base or buy back base
- Include fees in base diff calculation (consistent with position_hold)
- Handle swap order status and failures

The calculation matches position_hold tracking:
- ADD stores: base_amount, quote_amount (initial deposit)
- REMOVE stores: base_amount + base_fee, quote_amount + quote_fee (received)
- Swap amount: (received_base) - initial_base

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@rapcmia rapcmia moved this from Backlog to Under Review in Pull Request Board Apr 15, 2026
fengtality and others added 3 commits April 15, 2026 15:43
Rate Oracle fixes:
- Return None instead of Decimal(0) in _safe_get_last_traded_price on error
- Filter out None values in _safe_get_last_traded_prices before setting prices
- Add defensive checks in find_rate to skip division when price is zero

Gateway LP command fixes:
- Rename parser argument from 'connector' to 'dex_type' to match function signature
- Add get_pool_info method to Gateway class that takes trading_pair, dex_name, trading_type
- Update fetch_and_display_pool_info to accept dex_name and trading_type parameters
- Derive is_clmm from trading_type instead of passing as separate parameter

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The test now correctly expects {} instead of {"BTC-USDT": Decimal("0")}
when price fetch fails, matching the fix to avoid division by zero in rate oracle.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@fengtality
Copy link
Copy Markdown
Contributor Author

@nikspz both issues you found should be fixed now

@rapcmia
Copy link
Copy Markdown
Contributor

rapcmia commented Apr 16, 2026

Commit 78777bc

  • Clean installation of PR and setup with gateway (development branch) ✅
  • Retest Nikita's reported issues for:
  • The gateway lp <connector> add-liquidity command
  • generic.lp_rebalancer test

Test gateway lp <connector> add-liquidity

After setting up gateway lp <connector> add-liquidity, got an error unexpected keyword argument 'connector'
image

2026-04-16 13:56:39,813 - 1319256 - hummingbot.client.hummingbot_application - ERROR - Error in add liquidity: GatewayHttpClient.clmm_quote_position() got an unexpected keyword argument 'connector'
Traceback (most recent call last):
  File "/home/eddga/hummingbot/hummingbot/8150/hummingbot/client/command/gateway_lp_command.py", line 619, in _add_liquidity
    quote_result = await self._get_gateway_instance().clmm_quote_position(
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        connector=dex_type,
        ^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
        slippage_pct=slippage_pct
        ^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
TypeError: GatewayHttpClient.clmm_quote_position() got an unexpected keyword argument 'connector'

Steps to reproduce:

  1. Use gateway lp meteora/clmm add-liquidity
  2. Fill the required inputs until it tries to calculating optimal token amounts
  3. Observed returned error

Test generic.lp_rebalancer

  • When starting script config using start --v2 command, returned Failed to start strategy v2_with_controllers: (sqlite3.OperationalError) unable to open database file
    (Background on this error at: https://sqlalche.me/e/20/e3q8) ❌
    image
    2026-04-16 14:28:01,766 - 1394579 - hummingbot.core.gateway.gateway_http_client - INFO - Gateway Service is ONLINE.
    2026-04-16 14:28:50,128 - 1394579 - hummingbot.client.hummingbot_application - ERROR - MQTT is already stopped!
    2026-04-16 14:29:06,048 - 1396573 - hummingbot.core.trading_core - ERROR - Failed to start strategy v2_with_controllers: (sqlite3.OperationalError) unable to open database file
    (Background on this error at: https://sqlalche.me/e/20/e3q8)
    
  • Tried to run the same script config using bin/hummingbot_quickstart.py -p a --v2 conf_v2_with_controllers_1.yml, it returned Failed to stawrt v2_with_controllers: 'solana-mainnet-beta' ❌
    image
    **2026-04-16 14:16:27,133 - 1374054 - hummingbot.core.trading_core - ERROR - Failed to start strategy v2_with_controllers: 'solana-mainnet-beta'
    2026-04-16 14:16:27,152 - 1374054 - hummingbot.core.gateway.gateway_http_client - INFO - Gateway Service is ONLINE.**
    

Steps to reproduce:

  • Setup a generic.lp_rebalancer controller config (default)
    id: BuFw4sStQ5DFv9cF31bzrKWmqK2pLznDfVbiRySyGHNv
    controller_name: lp_rebalancer
    controller_type: generic
    total_amount_quote: '6'
    manual_kill_switch: false
    initial_positions: []
    candles_config: []
    connector_name: 'solana-mainnet-beta'
    lp_provider: meteora/clmm
    trading_pair: SOL-USDC
    pool_address: BGm1tav58oGcsQJehL9WXBFXF7D27vZsKefj4xJKD5Y
    side: 3
    position_width_pct: '0.5'
    position_offset_pct: '0.01'
    rebalance_threshold_pct: '1'
    sell_price_max: null
    sell_price_min: null
    buy_price_max: null
    buy_price_min: null
    strategy_type: null
    autoswap: false
    swap_buffer_pct: '0.01'
    
  • Start v2 controller using start --v2 or quickstart
  • Observed error when trying to start the v2 controller

@@ -84,41 +132,54 @@ async def control_task(self):
await self._create_position()

case LPExecutorStates.OPENING:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the future we might want to remove the extra state management and eval conditions when is running and when is shutting down as the rest of the executors

@fengtality
Copy link
Copy Markdown
Contributor Author

@rapcmia I think the lp_rebalancer error you're seeing is due to an outdated DB for the script. Delete your data/conf_v2_with_controllers.sqlite and start with a fresh DB.

I will check the gateway_lp command issue now.

fengtality and others added 5 commits April 16, 2026 11:28
Gateway LP command fixes:
- Change connector=dex_type to dex=dex_name with trading_type in:
  - clmm_quote_position
  - amm_quote_liquidity
  - clmm_collect_fees

Market data provider fixes:
- Simplify gateway connector detection using chain-network format
- Parse dex and trading_type from gateway_price_provider_by_chain
- Fix get_price call to use dex/trading_type instead of connector

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Gateway LP command fixes:
- Change connector=dex_type to dex=dex_name with trading_type in:
  - clmm_quote_position
  - amm_quote_liquidity
  - clmm_collect_fees

Market data provider fixes:
- Simplify gateway connector detection using chain-network format
- Parse dex and trading_type from gateway_price_provider_by_chain
- Fix get_price call to use dex/trading_type instead of connector

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Gateway connector now uses its _swap_provider (from network config)
when dex_name is not explicitly provided. This allows arbitrage and
other executors to call get_quote_price without passing dex_name.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove dex_name parameter from get_quote_price and get_order_price.
Gateway connector now always uses its _swap_provider from network config.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add get_dex_info() for DEX-format connectors (e.g., "orca/clmm")
  Returns (dex_name, trading_type, chain, network, error)
- Simplify get_connector_chain_network() for network-format only (e.g., "solana-mainnet-beta")
- Fix _parse_network() to use hardcoded chains instead of GATEWAY_CHAINS
  which may be empty before gateway connects
- Update gateway commands to use appropriate helper based on connector format
- Update tests to mock get_dex_info instead of get_connector_chain_network

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@nikspz nikspz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • latest commit ef065e0
    • gateway lp issue fixed ✅
      • added position successfully ✅
    • gateway swap: ok
    • gateway lp meteora/clmm position-info ❌ issue: No liquidity positions found for SOL-USDC (I'll create separate issue)
    • gateway lp meteora/clmm remove-liquidity ❌ issue: No liquidity positions found for SOL-USDC (I'll create separate issue)

- Convert markPx from string to float in hyperliquid _get_last_traded_price
- Remove invalid pool_address argument from clmm_positions_owned call

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@rapcmia
Copy link
Copy Markdown
Contributor

rapcmia commented Apr 17, 2026

Commit 118fd92

  • Test gateway lp commands (meteora and orca) ✅
  • Test generic.lp_rebalancer ✅
    • Successfully opened LP position
    • Relaunch hummingbot and start lp-rebalancer config again, found existing lp position
    • On stop command, Lp position closed. Status terminated closed by POSITION_HOLD
  • Test generic.arbitrage_controller (kucoin and jupiter/router) ✅
  • Test generic.xemm_multiple_order_levels (kucoin and jupiter/router) ✅
  • Create local .whl and integrate with hummingbot-api/142 ✅
    • Start with gateway development branch ok
    • Run lp_executor on meteora and orca ok
    • Check executor status RUNNING
    • Stopped executor successfully
    • DB status Terminated and closed POSITION_HOLD

Logs, config and sqlite 17042026a.zip

rapcmia
rapcmia previously approved these changes Apr 17, 2026
@rapcmia
Copy link
Copy Markdown
Contributor

rapcmia commented Apr 17, 2026

While wrapping up the PR, i also observed that quickstart issue still exist for both source and docker ❌
image

 Affected connectors:
  - solana-mainnet-beta
  - solana-devnet
  - ethereum-mainnet
  - ethereum-arbitrum
  - ethereum-avalanche
  - ethereum-base
  - ethereum-bsc
  - ethereum-celo
  - ethereum-optimism
  - ethereum-polygon
  - ethereum-sepolia
  • solana-mainnet-beta and ethereum-* are also Gateway connectors, but startup only waits for /-style connectors to be ready. This can cause issues with chain-network formatting.

Previous comment #8150

Tried to run the same script config using bin/hummingbot_quickstart.py -p a --v2 conf_v2_with_controllers_1.yml, it returned Failed to stawrt v2_with_controllers: 'solana-mainnet-beta' ❌

Steps to reproduce:

  1. Create any v2 controller that uses connector_name as solana-mainnet-beta or ethereum-<network e.g base, arbitrum, etc..>
  2. Launch hummingbot using quickstart on either source or docker

fengtality and others added 5 commits April 17, 2026 07:37
- trading_core: Wait for gateway and register connectors before creating
  network-format connectors (e.g., "solana-mainnet-beta") in initialize_markets
- start_command: Make _strategy_uses_gateway_connector async to register
  connectors before checking if they're gateway connectors

This fixes the KeyError when using network-format connectors in quickstart
because connectors weren't registered in AllConnectorSettings yet.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes:
- Remove dead code: duplicate self._name assignment in GatewayBase.__init__
- Fix mutable default argument: trading_pairs List[str]=[] -> Optional[List[str]]=None
- Fix NO_ROUTE_FOUND comment contradiction (remove "can retry" mention)
- Fix LPExecutor using stale pool_info.price, use self._current_price instead
- Make SOL buffer configurable: add native_currency and native_currency_buffer config fields
- Fix object.__setattr__ -> use Pydantic model_copy(update=...) for config updates
- Fix type mismatch: _pending_swap_side typed as Optional[TradeType] not Optional[int]
- Initialize _swap_not_found_count in __init__ instead of using getattr pattern
- Fix Dict[Any] -> Dict[str, Any] type hint in get_gas_estimates
- Extract hardcoded 'SOL' fallback to DEFAULT_NATIVE_CURRENCY constant
- Fix _check_autoswap_needed parameter type: side should be TradeType not int

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…traction

After changing trading_pairs default to None, the token extraction loop
would crash since it iterates over the raw parameter. Must use
self._trading_pairs which has already been converted to empty list.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Instead of config fields in lp_rebalancer, add get_native_currency_buffer()
method to GatewayBase that returns chain-specific buffer values:
- SOL: 0.1 (~$15, covers rent + multiple txs)
- ETH: 0.01 (~$25)
- Others: 0.01 (conservative default)

lp_rebalancer now fetches native_currency and buffer from the connector.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- SOL: 0.1 (~$15)
- ETH: 0.005 (~$12)
- Others: 0.005

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@nikspz nikspz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • commit e89837e + gw-dev
    • Test gateway lp commands
      • gateway lp meteora/clmm add-liquidity SOL-USDC ✅
      • gateway lp meteora/clmm position-info ✅
      • gateway lp meteora/clmm remove-liquidity ✅
    • Test generic.lp_rebalancer
      • quickstart: review issue fixed ✅
        • docker started successfully ✅
        • source: started successfully with bin/hummingbot_quickstart.py -p a --v2 conf_v2_with_controllers_1.yml ✅

@nikspz nikspz merged commit adfe75d into development Apr 17, 2026
2 checks passed
@nikspz nikspz deleted the feature/connector-level-retry branch April 17, 2026 21:03
@rapcmia rapcmia moved this from Under Review to Development v2.14.0 in Pull Request Board Apr 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Development v2.14.0

Development

Successfully merging this pull request may close these issues.

4 participants