Skip to content

[MAINTENANCE] Python CommonLib: add package structure, fix imports, and expand Python test suite #130

@yunlishao-vibe

Description

@yunlishao-vibe

Task Summary

The Python CommonLib (ConfigHelper.py, MsgHelper.py, SocketHelper.py, VehDataMsgDefs.py) lacks a proper package structure, has inconsistent import patterns, and the Python test suite under tests/Python/ needs expansion to cover core message and socket functionality.

Why is this needed?

Currently:

  • No __init__.py in CommonLib/ — it's not a proper Python package, imports rely on sys.path.insert() hacks
  • Both tests/Python/SimpleEchoClient/simple_echo_client.py and tests/Python/SimpleTrafficLight/echo_client.py duplicate the sys.path.insert pattern manually
  • SocketHelper.py has a # TODO: add traffic light and detector data comment
  • Python tests only validate socket echo behavior — no unit tests for message packing/unpacking
  • MsgHelper.py at 28K lines is likely the largest file in the Python codebase and has never been unit tested
  • requirements.txt references outdated SUMO version numbers in comments and is inconsistent with environment.yml
  • Two separate conda environment definitions (environment.yml and requirements.txt) with different package lists

Scope of Work

1. Add Python package structure

  • Add CommonLib/__init__.py exposing key classes
  • Update all test files to use clean from CommonLib.X import Y imports
  • Remove sys.path hacks from test scripts

2. Fix requirements files

  • Reconcile requirements.txt and environment.yml — one should be authoritative
  • Remove outdated version comments in requirements.txt (e.g., vissim==11.0, carmaker==10.0.1)
  • Ensure environment.yml includes all packages needed to run Python tests without SUMO/VISSIM/CarMaker installed

3. Implement SocketHelper.py TODO

  • The # TODO: add traffic light and detector data in SocketHelper.py — implement or convert to proper issue

4. Expand Python unit tests

Add tests/Python/unit/ directory with:

  • test_msg_helper.py — test message packing/unpacking with known byte sequences
  • test_config_helper.py — test YAML config parsing with sample config files
  • test_socket_helper.py — test message header parsing (no live socket needed, use mock data)

5. Add CI-runnable test runner

  • Add tests/Python/run_unit_tests.py (or pytest config) that runs all unit tests without requiring any simulator
  • These should be runnable with: python -m pytest tests/Python/unit/

Files Involved

  • CommonLib/__init__.py (new)
  • CommonLib/SocketHelper.py
  • CommonLib/MsgHelper.py
  • CommonLib/ConfigHelper.py
  • CommonLib/VehDataMsgDefs.py
  • requirements.txt
  • environment.yml
  • tests/Python/ (all)
  • tests/Python/unit/ (new)

Acceptance Criteria

  • CommonLib/__init__.py exists and package imports work cleanly
  • No sys.path.insert hacks in test files
  • requirements.txt and environment.yml are consistent
  • tests/Python/unit/test_msg_helper.py passes with pytest
  • tests/Python/unit/test_config_helper.py passes with pytest
  • SocketHelper.py TODO resolved or tracked as issue
  • All unit tests runnable without any simulator installed

Branch and PR Target

  • All PRs for this issue should target the dev branch (not main)
  • First ensure main is merged into dev before starting work

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions