Skip to content

Contributing.md

Latest commit

 

History

History
86 lines (61 loc) · 7.16 KB

Contributing.md

File metadata and controls

86 lines (61 loc) · 7.16 KB

Contributing to Boost Data Collector

This document describes how to contribute to the project, with emphasis on the service layer and data-write rules.

Service layer: single place for writes

Each Django app that has models provides a services.py module. This is the only place where code should create, update, or delete rows for that app’s models.

Rule

  • All inserts/updates/deletes for an app’s models must go through functions in that app’s services.py.
  • Do not call Model.objects.create(), model.save(), or model.delete() from outside services.py (e.g. from management commands, views, other apps, or tests that are not testing the service layer itself).

Why

  • Single place for write logic: Validation, defaults, and side effects live in one module.
  • Easier to change: Schema or business rules can be updated in one place.
  • Clear API: Contributors know where to look and what to call.

Which apps have a service layer

App File Notes
cppa_user_tracker cppa_user_tracker/services.py Identity, profiles, emails, staging.
github_activity_tracker github_activity_tracker/services.py Repos, languages, licenses, issues, PRs.
boost_library_tracker boost_library_tracker/services.py Boost libraries, versions, dependencies, categories, roles.
boost_library_docs_tracker boost_library_docs_tracker/services.py BoostDocContent and BoostLibraryDocumentation (doc scrape and sync status).
boost_usage_tracker boost_usage_tracker/services.py External repos, Boost usage, missing-header tmp.
cppa_pinecone_sync cppa_pinecone_sync/services.py Pinecone fail list and sync status writes.
discord_activity_tracker discord_activity_tracker/services.py Servers, channels, messages, reactions (Discord user profiles in cppa_user_tracker).
cppa_youtube_script_tracker cppa_youtube_script_tracker/services.py YouTube channels, videos, tags, transcript state, speaker links.
clang_github_tracker clang_github_tracker/services.py Clang/llvm GitHub issue, PR, and commit upserts; fetch watermarks.
boost_mailing_list_tracker boost_mailing_list_tracker/services.py Mailing list messages and names.
cppa_slack_tracker cppa_slack_tracker/services.py Slack teams, channels, messages, membership.
wg21_paper_tracker wg21_paper_tracker/services.py WG21 papers, authors, mailings.

For a full list of functions, parameter/return types, and validation (e.g. empty name raises ValueError), see Service_API.md and the per-app docs in service_api/ (index: service_api/README.md). DTO protocols shared across trackers are documented in service_api/core_protocols.md (generated from core/protocols.py).

Regenerating service API docs

Reference tables in docs/service_api/*.md are produced by scripts/generate_service_docs.py from each app’s services.py and from core/protocols.py.

  • Markers: Each file contains <!-- SERVICE_API:GENERATED:START --> … <!-- SERVICE_API:GENERATED:END -->. The script replaces only that region. Put hand-written notes (usage, cross-app warnings, command help) below the END marker.
  • Regenerate locally: python scripts/generate_service_docs.py (optional: --app <django_app_label> for one module).
  • Check only: python scripts/generate_service_docs.py --check exits non-zero if committed markdown would change.
  • CI / pre-commit: The lint job runs pre-commit, which includes this check. Pull requests that change only ignored paths (**.md, docs/** per .github/workflows/actions.yml) do not run CI; any PR that touches **/services.py or core/protocols.py still runs the check—regenerate docs before pushing.

How to use

  1. From management commands or other apps: Import and call the service functions.

    from cppa_user_tracker.services import create_identity, add_email
from github_activity_tracker.services import get_or_create_language, add_pull_request_label

identity = create_identity(display_name="Jane")
add_email(identity.profiles.first(), "jane@example.com")
lang, _ = get_or_create_language("Python")
add_pull_request_label(pr, "bug")

  2. Adding new write behavior: Add a new function in the app’s services.py (and optionally a helper in the same module). Do not add new writes by calling the model or manager directly from outside services.py.

  3. Reading data: No restriction. Use the ORM as usual: Model.objects.filter(...), model.related_set.all(), etc.

Testing

  • Running tests: From the project root, install dev deps (pip install -r requirements-dev.lock or uv pip install -r requirements-dev.lock), start the test database (docker compose -f docker-compose.test.yml up -d), set DATABASE_URL (and SECRET_KEY for the process) as in README.md, then run python -m pytest. Tests always use PostgreSQL (config.test_settings); there is no SQLite fallback.
  • See README.md and Development_guideline.md for full commands and options.
  • Unit tests for services.py: Call the service functions and assert on the database (or mocks) as needed.
  • Other tests: Prefer service functions when setting up data. If you must create models directly for tests, keep it in test code (e.g. fixtures or test helpers) and avoid doing the same in production code.

Other guidelines

  • Branching: Create feature branches from develop. Open pull requests against develop. See Development_guideline.md.
  • Code style: Use Python 3.11+ and follow Django and project conventions. Use the project’s logging (logging.getLogger(__name__)). Before pushing, run uv run pyright (with dev deps) for the paths covered by pyrightconfig.json, and ensure CI’s lint / pyright / test jobs would pass.
  • Database: Use the Django ORM and migrations. Writes only through the service layer as above.
  • Docs: Update this doc (and app services.py docstrings) when adding new apps or changing the write rules. After changing services.py or core/protocols.py, run python scripts/generate_service_docs.py and commit the updated docs/service_api/ files.

Related documentation