[feat][CGS][application-manager] add intelligent queue selection based on yarn resource usage

[v-kkhuang]

What is the purpose of the change

Background/Problem:
Currently, Linkis uses a fixed queue configuration for Yarn jobs. This can lead to inefficient resource utilization when some queues are heavily loaded while others have available capacity. The system needs the ability to automatically select the optimal queue based on real-time resource usage.

Purpose of Change:
This PR adds intelligent queue selection functionality that monitors Yarn queue resource usage in real-time and automatically selects the optimal queue based on configurable thresholds. When the secondary queue has available capacity (usage below threshold), jobs are directed there; otherwise, they use the primary queue.

Value/Impact:
After the change, Linkis can optimize resource utilization across multiple queues, reduce job wait times, and improve overall cluster efficiency. The system provides configurable thresholds, engine/creator filtering, and automatic fallback to ensure stability.

Related issues/PRs

Related issues: close apache#5415
Related pr:none

Brief change log

• Add performSmartQueueSelection method in DefaultEngineCreateService to execute queue selection before YarnResource creation
• Add configuration parameters for secondary queue feature (enable, threshold, supported engines, supported creators)
• Add intelligent queue selection logic with three-dimensional resource usage evaluation (memory, CPU, instances)
• Update RMConfiguration with secondary queue settings

Checklist

• I have read the Contributing Guidelines on pull requests.
• I have explained the need for this PR and the problem it solves
• I have explained the changes or the new features added to this PR
• I have added tests corresponding to this change
• I have updated the documentation to reflect this change
• I have verified that this change is backward compatible
• If this is a code change: I have written unit tests to fully verify the new behavior.

[casionone]

LGTM.