README.coders

This file contains notes to coders who want to contribute to the project.

#
# Development Setup
#
# Install development dependencies
uv pip install -e ".[dev]"

# Install pre-commit hooks (required for all contributors)
pre-commit install

#
# Code Style
#
The code must run under Python 3.10+

Follow the PEP 8 style guide. Formatting and linting are enforced by ruff
via pre-commit hooks. You can also run them manually:

# Format code
ruff format src/compiletools/

# Lint code
ruff check src/compiletools/

# Auto-fix lint issues
ruff check --fix src/compiletools/

Ruff configuration is in pyproject.toml under [tool.ruff].

#
# Pre-commit Hooks
#
Pre-commit hooks run automatically on every commit. They enforce:

- Code formatting (ruff format)
- Linting (ruff check)
- Trailing whitespace removal
- YAML/TOML validation
- Merge conflict detection
- Secret detection (detect-secrets)

To run all hooks against the entire codebase:

pre-commit run --all-files

To skip hooks for a specific commit (not recommended):

git commit --no-verify

#
# Running tests
#
The project uses pytest as its testing framework:

# Run all tests
pytest src/compiletools

# Run tests in parallel (faster)
pytest src/compiletools -n auto

# Run specific test file
pytest src/compiletools/test_utils.py

# Run with verbose output
pytest src/compiletools -v

#
# Caches/Performance
#
When you are doing performance testing there are 2 caches you need to be aware of

1) ccache:  This can be cleaned by ccache -C
2) cake:    Object files go to objdir (default: bin/<variant>/obj/).
            Executables go to bindir (default: bin/<variant>/).
            To clear: rm -rf bin
            With --file-locking enabled, objdir may be a shared location
            (e.g., <gitroot>/obj or /some/path/to/obj).

#
# Profiling
#
Three methods for profiling ct-cake and other ct-* commands:

# 1. Inline cProfile
source <compiletoolsrepo>/.venv/bin/activate && python -m cProfile -s cumulative $(which ct-cake) --auto

# 2. Project profiling script (supports any ct-* command, temp output dirs, metrics summary)
scripts/profile-ct ct-cake -d /path/to/project
scripts/profile-ct ct-compilation-database -d /path/to/project -- --include "dir1 dir2"

# 3. System-level profiling with py-spy (generates flame graph SVG)
py-spy record -o profile.svg -- python $(which ct-cake) --auto

#
# Build Timing
#
ct-cake supports --timing to collect per-phase build timing data.
See src/compiletools/README.ct-timing-report.rst for full documentation
of ct-timing-report and its output modes (TUI, summary, compare, chrome-trace).

#
# Gotchas
#
If you encounter any weird build issues (say due to a currently unknown bug in compiletools), then first try:
rm -rf bin

#
# Package Structure
#
src/compiletools/     - Main package code
  *_backend.py        - Pluggable build backends (make, ninja, cmake, bazel, shake, tup)
  build_graph.py      - Backend-agnostic IR (BuildRule, BuildGraph)
  build_backend.py    - BuildBackend ABC and backend registry
scripts/              - Shell script wrappers (ct-build, ct-release, etc.)
ct-*                  - Command-line entry points
pyproject.toml        - Modern Python packaging configuration

#
# Worktree-Based Development
#
The repository uses a worktree layout: the `master` directory is the main
worktree under a parent directory (e.g., compiletools/master/).

Feature branches should be checked out as sibling worktrees:

# Create a worktree for a feature branch
git worktree add ../feature-x feature-x

# List worktrees
git worktree list

# Remove a worktree when done
git worktree remove ../feature-x

The profiling script `profile-ct-cake-worktree` creates temporary profiling
worktrees as siblings (e.g., compiletools/profile_worktree_master/).

#
# Making a release
#
The ct-release script automates the release process.  Read that script if you want to do each step manually.