Skip to content

Commit 0670f97

Browse files
authored
Merge pull request #267 from microsoft/vyokky/dev
update bitex and shield for paper
2 parents f419e63 + 19385d3 commit 0670f97

13 files changed

Lines changed: 216 additions & 255 deletions

File tree

README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818

1919
<br/>
2020

21+
[![arxiv](https://img.shields.io/badge/Paper-arXiv:2511.11332-b31b1b.svg)](https://arxiv.org/abs/2511.11332)&ensp;
2122
[![arxiv](https://img.shields.io/badge/Paper-arXiv:2504.14603-b31b1b.svg)](https://arxiv.org/abs/2504.14603)&ensp;
2223
![Python Version](https://img.shields.io/badge/Python-3776AB?&logo=python&logoColor=white-blue&label=3.10%20%7C%203.11)&ensp;
2324
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)&ensp;
@@ -446,7 +447,7 @@ API_DEPLOYMENT_ID: "your-deployment-id"
446447
- 🎯 **Heterogeneous Orchestration**: Safe, asynchronous execution with capability-based device matching
447448
- 🔌 **Unified AIP Protocol**: WebSocket-based secure agent coordination with fault tolerance
448449
- 🛠️ **MCP-Empowered Agent Framework**: Template-driven toolkit for rapid device agent development
449-
- 📄 **Research Paper**: [UFO³: Weaving the Digital Agent Galaxy](https://arxiv.org/abs/[TBD])
450+
- 📄 **Research Paper**: [UFO³: Weaving the Digital Agent Galaxy](https://arxiv.org/abs/2511.11332)
450451

451452
**Key Features:**
452453
- First multi-device orchestration framework for GUI agents
@@ -474,11 +475,10 @@ If you use UFO³ Galaxy or UFO² in your research, please cite the relevant pape
474475
### UFO³ Galaxy Framework (2025)
475476
```bibtex
476477
@article{zhang2025ufo3,
477-
title = {{UFO³: Weaving the Digital Agent Galaxy}},
478-
author = {Zhang, Chaoyun and [Authors TBD]},
479-
journal = {arXiv preprint arXiv:[TBD]},
478+
title={UFO$^3$: Weaving the Digital Agent Galaxy},
479+
author = {Zhang, Chaoyun and Li, Liqun and Huang, He and Ni, Chiming and Qiao, Bo and Qin, Si and Kang, Yu and Ma, Minghua and Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei},
480+
journal = {arXiv preprint arXiv:2511.11332},
480481
year = {2025},
481-
note = {Multi-device orchestration framework with Constellation-based planning}
482482
}
483483
```
484484

README_ZH.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
<a href="https://trendshift.io/repositories/7874" target="_blank"><img src="https://trendshift.io/api/badge/repositories/7874" alt="microsoft%2FUFO | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
1818
<br/>
1919

20+
[![arxiv](https://img.shields.io/badge/Paper-arXiv:2511.11332-b31b1b.svg)](https://arxiv.org/abs/2511.11332)&ensp;
2021
[![arxiv](https://img.shields.io/badge/Paper-arXiv:2504.14603-b31b1b.svg)](https://arxiv.org/abs/2504.14603)&ensp;
2122
![Python Version](https://img.shields.io/badge/Python-3776AB?&logo=python&logoColor=white-blue&label=3.10%20%7C%203.11)&ensp;
2223
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)&ensp;
@@ -439,7 +440,7 @@ API_DEPLOYMENT_ID: "your-deployment-id"
439440
- 🎯 **异构编排**:基于能力的设备匹配实现安全的异步执行
440441
- 🔌 **统一 AIP 协议**:基于 WebSocket 的安全智能体协调,具有容错能力
441442
- 🛠️ **支持 MCP 的智能体框架**:用于快速设备智能体开发的模板驱动工具包
442-
- 📄 **研究论文**:[UFO³: Weaving the Digital Agent Galaxy](https://arxiv.org/abs/[TBD])
443+
- 📄 **研究论文**:[UFO³: Weaving the Digital Agent Galaxy](https://arxiv.org/abs/2511.11332)
443444

444445
**核心特性:**
445446
- 首个用于 GUI 智能体的多设备编排框架
@@ -467,11 +468,10 @@ API_DEPLOYMENT_ID: "your-deployment-id"
467468
### UFO³ Galaxy 框架(2025)
468469
```bibtex
469470
@article{zhang2025ufo3,
470-
title = {{UFO³: Weaving the Digital Agent Galaxy}},
471-
author = {Zhang, Chaoyun and [Authors TBD]},
472-
journal = {arXiv preprint arXiv:[TBD]},
471+
title={UFO$^3$: Weaving the Digital Agent Galaxy},
472+
author = {Zhang, Chaoyun and Li, Liqun and Huang, He and Ni, Chiming and Qiao, Bo and Qin, Si and Kang, Yu and Ma, Minghua and Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei},
473+
journal = {arXiv preprint arXiv:2511.11332},
473474
year = {2025},
474-
note = {Multi-device orchestration framework with Constellation-based planning}
475475
}
476476
```
477477

documents/docs/choose_path.md

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -200,11 +200,14 @@ You don't have to choose just one! Here are common hybrid patterns:
200200

201201
### Pattern 1: UFO² as Galaxy Device
202202

203-
**Setup:** Run UFO² in agent-server mode as a Galaxy device
203+
**Setup:** Run UFO² as a Galaxy device (requires both server and client)
204204

205205
```bash
206-
# On Windows desktop
207-
python -m ufo --mode agent-server --port 5005
206+
# Terminal 1: Start UFO² Server on Windows desktop
207+
python -m ufo.server.app --port 5000
208+
209+
# Terminal 2: Start UFO² Client (connect to server)
210+
python -m ufo.client.client --ws --ws-server ws://localhost:5000/ws --client-id my_windows_device --platform windows
208211
```
209212

210213
**Benefits:**
@@ -230,13 +233,20 @@ python -m ufo --task "Your current task"
230233
# config/galaxy/devices.yaml (prepare in advance)
231234
devices:
232235
- device_id: "my_windows"
233-
server_url: "ws://localhost:5005/ws"
236+
server_url: "ws://localhost:5000/ws" # Where UFO client connects to UFO server
237+
os: "windows"
234238
capabilities: ["office", "web"]
235239
```
236240
237-
**Phase 3:** Add Galaxy when needed
241+
**Phase 3:** Start UFO device agent and connect to Galaxy
238242
```bash
239-
# Start using Galaxy for multi-device tasks
243+
# Terminal 1: Start UFO Server on your Windows machine
244+
python -m ufo.server.app --port 5000
245+
246+
# Terminal 2: Start UFO Client (connects to UFO server above)
247+
python -m ufo.client.client --ws --ws-server ws://localhost:5000/ws --client-id my_windows --platform windows
248+
249+
# Terminal 3: Start Galaxy (on control machine, can be same or different)
240250
python -m galaxy --request "Cross-device workflow"
241251
```
242252

@@ -274,10 +284,13 @@ python -m galaxy --request "Cross-device workflow"
274284

275285
### Misconception 2: "I need to rewrite everything to migrate to Galaxy"
276286

277-
**Reality:** UFO² can run as a Galaxy device with zero code changes:
287+
**Reality:** UFO² can run as a Galaxy device with minimal changes:
278288
```bash
279-
# Existing UFO² instance becomes Galaxy device
280-
python -m ufo --mode agent-server --port 5005
289+
# Terminal 1: Start UFO Server
290+
python -m ufo.server.app --port 5000
291+
292+
# Terminal 2: Start UFO Client in WebSocket mode
293+
python -m ufo.client.client --ws --ws-server ws://localhost:5000/ws --client-id my_device --platform windows
281294
```
282295

283296
**Learn More:** [Migration Guide](./getting_started/migration_ufo2_to_galaxy.md#option-2-convert-ufo2-instance-to-galaxy-device)

documents/docs/client/computer_manager.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ graph TB
6868

6969
---
7070

71-
## 🏗�?Computer Manager Architecture
71+
## 🏗Computer Manager Architecture
7272

7373
### Computer Instance Management
7474

@@ -110,7 +110,7 @@ Data collection tools are designed for non-invasive information gathering, while
110110

111111
## Computer Manager Architecture
112112

113-
## 🖥�?Computer (Instance) Architecture
113+
## 🖥Computer (Instance) Architecture
114114

115115
### Internal Structure
116116

@@ -157,9 +157,9 @@ graph TB
157157
|-----------|------|---------|
158158
| `_name` | `str` | Computer name (identifier) |
159159
| `_process_name` | `str` | Associated process (e.g., "notepad.exe") |
160-
| `_data_collection_servers` | `Dict[str, BaseMCPServer]` | Namespace �?MCP server mapping (data collection) |
161-
| `_action_servers` | `Dict[str, BaseMCPServer]` | Namespace �?MCP server mapping (actions) |
162-
| `_tools_registry` | `Dict[str, MCPToolCall]` | Tool key �?tool info mapping |
160+
| `_data_collection_servers` | `Dict[str, BaseMCPServer]` | Namespace MCP server mapping (data collection) |
161+
| `_action_servers` | `Dict[str, BaseMCPServer]` | Namespace MCP server mapping (actions) |
162+
| `_tools_registry` | `Dict[str, MCPToolCall]` | Tool key tool info mapping |
163163
| `_meta_tools` | `Dict[str, Callable]` | Built-in meta tools |
164164
| `_executor` | `ThreadPoolExecutor` | Thread pool for tool execution (10 workers) |
165165
| `_tool_timeout` | `int` | Tool execution timeout: **6000 seconds (100 minutes)** |
@@ -396,7 +396,7 @@ result = await asyncio.wait_for(
396396

397397
---
398398

399-
## 🛠�?Tool Registry
399+
## 🛠Tool Registry
400400

401401
### Tool Registration
402402

documents/docs/configuration/system/galaxy_devices.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,10 @@ graph LR
4444
```
4545
UFO2/
4646
├── config/
47-
�? └── galaxy/
48-
�? ├── devices.yaml # �?Device definitions (this file)
49-
�? ├── constellation.yaml # �?Runtime settings
50-
�? └── agent.yaml.template # �?Agent LLM configuration template
47+
└── galaxy/
48+
├── devices.yaml # 📄 Device definitions (this file)
49+
├── constellation.yaml # ⚙️ Runtime settings
50+
└── agent.yaml.template # 🤖 Agent LLM configuration template
5151
```
5252

5353
**Loading in Code:**
@@ -545,34 +545,34 @@ def validate_device_config(device: dict) -> bool:
545545

546546
**1. Use Meaningful device_id**
547547
```yaml
548-
# �?Good: Descriptive and unique
548+
# Good: Descriptive and unique
549549
device_id: "windows_office_pc_01"
550550
device_id: "linux_prod_server_us_west_01"
551551
device_id: "gpu_ml_workstation_lab_a"
552552
553-
# �?Bad: Generic or ambiguous
553+
# Bad: Generic or ambiguous
554554
device_id: "device1"
555555
device_id: "test"
556556
device_id: "agent"
557557
```
558558

559559
**2. Specify Granular Capabilities**
560560
```yaml
561-
# �?Good: Specific capabilities
561+
# Good: Specific capabilities
562562
capabilities:
563563
- "web_browsing_chrome"
564564
- "office_excel_automation"
565565
- "email_outlook"
566566
567-
# �?Bad: Vague capabilities
567+
# Bad: Vague capabilities
568568
capabilities:
569569
- "office"
570570
- "internet"
571571
```
572572

573573
**3. Include Rich Metadata**
574574
```yaml
575-
# �?Good: Comprehensive metadata
575+
# Good: Comprehensive metadata
576576
metadata:
577577
location: "datacenter_us_west_rack_a42"
578578
performance: "very_high"
@@ -582,7 +582,7 @@ def validate_device_config(device: dict) -> bool:
582582
gpu_type: "NVIDIA A100"
583583
gpu_count: 4
584584
585-
# �?Bad: Minimal metadata
585+
# Bad: Minimal metadata
586586
metadata:
587587
location: "server room"
588588
```

documents/docs/galaxy/overview.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -456,7 +456,6 @@ UFO³ Galaxy consists of several integrated components working together:
456456
| **TaskConstellationOrchestrator** | `galaxy/constellation/orchestrator/` | Asynchronous execution, event coordination, safety enforcement |
457457
| **TaskConstellation** | `galaxy/constellation/task_constellation.py` | DAG data structure, validation, and modification APIs |
458458
| **DeviceManager** | `galaxy/client/device_manager.py` | WebSocket connections, heartbeat monitoring, message routing |
459-
| **Agent Server** | `ufo/mode/agent_server.py` | Device-side WebSocket server, AIP protocol handler |
460459

461460
### Supporting Infrastructure
462461

@@ -579,7 +578,7 @@ print(f"🔀 Parallelism Ratio: {const_stats['parallelism_ratio']:.2f}")
579578

580579
---
581580

582-
## 📚 Learn More
581+
## 📚 Learn More
583582

584583
- **Research Paper**: [UFO³: Weaving the Digital Agent Galaxy](https://arxiv.org/) *(Coming Soon)*
585584
- **UFO² (Desktop AgentOS)**: [Documentation](../ufo2/overview.md)

documents/docs/getting_started/quick_start_mobile.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -999,7 +999,7 @@ python -m galaxy --interactive
999999

10001000
---
10011001

1002-
## Understanding Mobile Agent Internals
1002+
## 🔍 Understanding Mobile Agent Internals
10031003

10041004
Now that you have Mobile Agent running, you may want to understand how it works under the hood:
10051005

@@ -1035,7 +1035,7 @@ Learn more: [MCP Commands Reference](../mobile/commands.md)
10351035

10361036
---
10371037

1038-
## 🐛 Common Issues & Troubleshooting
1038+
## 🐛 Common Issues & Troubleshooting
10391039

10401040
### Issue 1: ADB Device Not Found
10411041

documents/docs/index.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
<p><em>A Multi-Device Orchestration Framework for Cross-Platform Intelligent Automation</em></p>
88
</div>
99

10-
10+
[![arxiv](https://img.shields.io/badge/Paper-arXiv:2511.11332-b31b1b.svg)](https://arxiv.org/abs/2511.11332)
1111
[![arxiv](https://img.shields.io/badge/Paper-arXiv:2504.14603-b31b1b.svg)](https://arxiv.org/abs/2504.14603)
1212
![Python Version](https://img.shields.io/badge/Python-3776AB?&logo=python&logoColor=white-blue&label=3.10%20%7C%203.11)
1313
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
@@ -438,10 +438,10 @@ If you use UFO³ in your research, please cite:
438438
**UFO³ Galaxy Framework (2025)**
439439
```bibtex
440440
@article{zhang2025ufo3,
441-
title = {{UFO³: Weaving the Digital Agent Galaxy}},
442-
author = {Zhang, Chaoyun and [Authors TBD]},
443-
journal = {arXiv preprint arXiv:[TBD]},
444-
year = {2025}
441+
title={UFO$^3$: Weaving the Digital Agent Galaxy},
442+
author = {Zhang, Chaoyun and Li, Liqun and Huang, He and Ni, Chiming and Qiao, Bo and Qin, Si and Kang, Yu and Ma, Minghua and Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei},
443+
journal = {arXiv preprint arXiv:2511.11332},
444+
year = {2025},
445445
}
446446
```
447447

0 commit comments

Comments
 (0)