Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: Test Suite

on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]

steps:
- uses: actions/checkout@v3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt

- name: Run tests
run: |
pytest -v --tb=short

- name: Run tests with merged scripts
run: |
pytest -v --merged --tb=short

perf-smoke:
runs-on: ubuntu-latest
name: Performance Smoke Test

steps:
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.9"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
pip install pytest-timeout

- name: Run performance smoke test
run: |
# 运行性能测试,确保合并大型代码库的性能
pytest tests/test_perf_hash_lookup.py::test_large_codebase_performance -v --tb=short
timeout-minutes: 2 # 整体超时2分钟

- name: Report performance
if: always()
run: |
echo "✅ Performance smoke test completed"
25 changes: 24 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,14 @@ PySymphony/
│ │ ├── order_test.py # Test cases for dependency ordering
│ │ └── complex_deps.py # Complex multi-layer dependency tests
│ ├── test_regression.py # Regression tests
│ └── test_advanced_merger_fixes.py # Tests for advanced merger fixes
│ ├── test_advanced_merger_fixes.py # Tests for advanced merger fixes
│ ├── test_perf_hash_lookup.py # Performance tests for B1 fix (O(N²) optimization)
│ ├── test_runtime_alias_conflict.py # Tests for B2 fix (import alias conflicts)
│ ├── test_attr_reference_validation.py # Tests for B3 fix (attribute validation)
│ └── test_class_method_order_multi_inherit.py # Tests for B4 fix (class-method ordering)
├── .github/ # GitHub Actions CI/CD
│ └── workflows/
│ └── test.yml # Test suite workflow with perf-smoke job
├── conftest.py # Pytest configuration with AST auditor integration
├── pytest.ini # Pytest settings
├── requirements-dev.txt # Development dependencies
Expand All @@ -63,17 +70,22 @@ PySymphony/
- **`pysymphony/auditor/auditor.py`**: Industrial-grade multi-stage AST analysis system:
- **SymbolTableBuilder**: Builds comprehensive symbol tables with scope tracking
- **ReferenceValidator**: Validates all symbol references with LEGB scope resolution
- **B3 Enhancement**: Now validates attribute existence on objects (e.g., detects `obj.non_existent_method()`)
- **PatternChecker**: Detects specific patterns (e.g., multiple main blocks)
- **ASTAuditor**: Coordinates all analysis stages and provides detailed error reports

### 🚀 Code Merger Tool
- **`scripts/advanced_merge.py`**: The comprehensive implementation with advanced AST analysis:
- **Advanced scope analysis**: Full LEGB (Local, Enclosing, Global, Built-in) scope resolution
- **B1 Optimization**: O(1) scope lookup using `defnode_to_scope` hash mapping
- **Symbol tracking**: Comprehensive tracking of all Python symbols (functions, classes, variables)
- **Enhanced attribute resolution**: Supports nested attribute chains (e.g., `a.b.c.d`)
- **Correct nonlocal/global handling**: Properly tracks and preserves scope declarations
- **Import alias mapping**: Complete support for all import patterns and aliases
- **B2 Enhancement**: Adds `__mod` suffix to prevent runtime conflicts
- **Main block deduplication**: Correctly handles module initialization statements
- **Topological sorting enhancements**:
- **B4 Fix**: Ensures classes are always output before their methods

### Example Code
- **`examples/demo_packages/a_pkg/a.py`**: Contains `global_same()`, `hello()`, `hello2()` - demonstrates internal dependencies
Expand Down Expand Up @@ -206,10 +218,15 @@ python scripts/advanced_merge.py examples/example_complex_deps.py .
- **Topological sorting**: Ensures correct function definition order using graph algorithms
- Fixed algorithm to properly handle dependency chains
- Reverses final order to ensure dependencies are defined first
- **B4 Fix**: Ensures classes are always defined before their methods
- **Conflict detection**: Analyzes symbol frequency to determine renaming necessity
- **Import alias resolution**: Correctly handles `import X as Y` patterns
- Maps aliases to their corresponding renamed functions
- Preserves original alias relationships in merged code
- **B2 Fix**: Adds `__mod` suffix to all import aliases to prevent runtime conflicts
- **Performance optimizations**:
- **B1 Fix**: O(1) scope lookup using `defnode_to_scope` hash mapping
- Efficient symbol resolution avoiding O(N²) complexity

## Demo Dependency Patterns

Expand Down Expand Up @@ -346,6 +363,12 @@ The project uses a multi-stage AST auditor (`pysymphony.auditor.ASTAuditor`) tha

## Recent Improvements

### Issue #34: Core Stability Sprint
1. **B1 - Performance Optimization**: Fixed O(N²) scope lookup by implementing `defnode_to_scope` hash mapping
2. **B2 - Runtime Alias Conflicts**: Added `__mod` suffix to all import aliases to prevent conflicts with local definitions
3. **B3 - Attribute Reference Validation**: Enhanced `ReferenceValidator` to check attribute existence on objects
4. **B4 - Class-Method Topology**: Fixed topological sorting to ensure classes are always defined before their methods

### Issue #18: Industrial-Grade Testing System
1. **AST Auditor**: Implemented multi-stage static analysis system
2. **Test Architecture**: Created layered test structure (unit/integration/e2e)
Expand Down
7 changes: 7 additions & 0 deletions STATIC_ANALYSIS_IMPLEMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,10 @@ class ScopeInfo:
- 在作用域链中查找符号定义
- 排除内置名称(如 `len`、`print` 等)
- 记录所有未定义的引用
- **B3 增强**:验证属性引用的有效性
- 递归解析属性链(如 `a.b.c.d`)
- 检查类成员的存在性
- 排除外部模块属性(如 `os.path`)

**作用域解析算法**:
1. 从当前作用域开始查找
Expand Down Expand Up @@ -152,6 +156,7 @@ tests/
1. **单次遍历**:每个阶段只遍历 AST 一次
2. **内存效率**:使用引用而非复制 AST 节点
3. **快速查找**:使用字典进行 O(1) 符号查找
4. **B1 优化**:引入 `defnode_to_scope` 哈希映射,避免 O(N²) 的作用域查找

## 未来改进方向

Expand All @@ -160,6 +165,8 @@ tests/
3. **并行处理**:对大型文件进行并行分析
4. **自定义规则**:支持用户定义的检查规则
5. **IDE 集成**:提供 LSP 支持
6. **B2 完善**:处理更复杂的动态导入场景(如条件导入链)
7. **更深层的属性验证**:支持多层属性链的完整验证

## 结论

Expand Down
40 changes: 39 additions & 1 deletion pysymphony/auditor/auditor.py
Original file line number Diff line number Diff line change
Expand Up @@ -215,9 +215,47 @@ def visit_Name(self, node: ast.Name):

def visit_Attribute(self, node: ast.Attribute):
"""访问属性引用"""
# 只检查基础对象,不检查属性名
# B3 修复:实现属性引用验证
# 首先检查基础对象
self.visit(node.value)

# 递归解析属性链,获取根符号
root_obj = node.value
attr_chain = [node.attr]

while isinstance(root_obj, ast.Attribute):
attr_chain.insert(0, root_obj.attr)
root_obj = root_obj.value

# 如果根对象是名称,尝试解析它
if isinstance(root_obj, ast.Name):
root_symbol = self.find_symbol(root_obj.id)

if root_symbol:
# 检查是否是外部模块(如 os, sys 等)
# 对于外部模块,我们不检查属性
if root_symbol.type == 'import' and root_obj.id in ['os', 'sys', 'json', 're',
'math', 'datetime', 'pathlib',
'collections', 'itertools']:
return

# 对于本地符号,检查第一层属性是否存在
# 注意:这里只做基础检查,不做深层次的属性验证
first_attr = attr_chain[0]

# 如果是类符号,检查类的成员
if root_symbol.type == 'class':
# 查找类中定义的方法和属性
class_members = set()
for child_scope in self.current_scope.children:
if child_scope.parent == root_symbol.scope:
class_members.update(child_scope.symbols.keys())

# 如果属性不在类成员中,记录未定义引用
if first_attr not in class_members and first_attr not in ['__init__', '__call__',
'__str__', '__repr__']:
self.undefined_names.append((f"{root_obj.id}.{first_attr}", node.lineno))

def visit_FunctionDef(self, node: ast.FunctionDef):
"""访问函数定义"""
self.enter_scope(node.name)
Expand Down
Loading
Loading