Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
5ed1414
Fix model timing split in reconstruction summary
mickaelbegon Mar 25, 2026
655c208
Improve DD thresholding and trampoline bed reference
mickaelbegon Mar 25, 2026
2a8e536
Polish reconstruction and profile panel layout
mickaelbegon Mar 25, 2026
8e7d0d3
Improve profile UI, cache cleanup, and model variants
mickaelbegon Mar 25, 2026
768486d
Use objective generation times in reconstruction summary
mickaelbegon Mar 25, 2026
cfc7d1d
Fix import ordering in analysis and tool scripts
mickaelbegon Mar 25, 2026
f4c3aa6
Fix EKF2D objective timing in reconstruction summary
mickaelbegon Mar 25, 2026
10ef722
Add upper-back EKF controls and diagnostics
mickaelbegon Mar 25, 2026
713b718
Format camera and preview helpers with black
mickaelbegon Mar 25, 2026
0a2a761
Improve profile loading and root pseudo-q export
mickaelbegon Mar 26, 2026
78bf2e5
Polish GUI selection and model preview workflows
mickaelbegon Mar 26, 2026
cfbe97b
Align upper-back priors with sagittal back flexion
mickaelbegon Mar 26, 2026
b809cbb
Improve back preview and image overlay resolution
mickaelbegon Mar 26, 2026
ec3d367
Improve camera image overlays and multiview previews
mickaelbegon Mar 26, 2026
1e94349
Add upper-trunk-root model variants and camera deselection
mickaelbegon Mar 26, 2026
4b50467
Speed up EKF2D updates for upper-back priors
mickaelbegon Mar 26, 2026
5a3174d
Stabilize EKF3D initialization and root angle branches
mickaelbegon Mar 26, 2026
511a0be
Add multiview annotation workflow and annotated pose mode
mickaelbegon Mar 27, 2026
9effd6c
Support JPEG execution image overlays
mickaelbegon Mar 27, 2026
868e650
Add explicit root unwrap modes and coverage
mickaelbegon Mar 27, 2026
1037b7b
Align regression tests and CI ignores
mickaelbegon Mar 27, 2026
1c1b6d7
Improve segmented-back model markers and preview geometry
mickaelbegon Mar 27, 2026
54cc2ed
Refine annotation marker controls and deletion
mickaelbegon Mar 27, 2026
39296c0
Fix GUI startup and expose triangulation view exclusions
mickaelbegon Mar 27, 2026
4ba0417
Add regression coverage for unbounded once triangulation
mickaelbegon Mar 27, 2026
24c9028
Interpolate trunk markers before root unwrap
mickaelbegon Mar 29, 2026
8d20b56
Add batch runner and polish multiview annotation tooling
mickaelbegon Mar 29, 2026
086a6ec
Add profile presets, annotation seed, and example scripts
mickaelbegon Mar 29, 2026
a062f82
Fix annotation worst-reprojection frame navigation
mickaelbegon Mar 29, 2026
fa47f94
Polish annotation UI and add reprojection confirm flow
mickaelbegon Mar 30, 2026
6a50bf1
Add initial annotation kinematic assist
mickaelbegon Mar 30, 2026
718ccd4
isort
mickaelbegon Mar 30, 2026
2489dad
Improve annotation kinematic assist and model layout
mickaelbegon Mar 30, 2026
545e7e2
Persist annotation kinematic assist across frames
mickaelbegon Mar 30, 2026
90f8d3d
Fix pose2sim excluded-view bundle metadata
mickaelbegon Mar 30, 2026
6e58a9e
Improve annotation hover and kinematic bootstrap
mickaelbegon Mar 30, 2026
d1cb952
Refine annotation kinematic assist rendering
mickaelbegon Mar 30, 2026
04e39a0
Improve annotation fitting and frame navigation
mickaelbegon Mar 30, 2026
ebf4433
Extract annotation preview rendering
mickaelbegon Mar 31, 2026
a4d6478
Share 2D preview helpers across GUI and animation
mickaelbegon Mar 31, 2026
dd0a30b
Unify camera image and crop helpers across previews
mickaelbegon Mar 31, 2026
cff7ec4
Improve annotation frame updates and share 2D frame rendering
mickaelbegon Mar 31, 2026
0e43cef
Add calibration quality analysis tools
mickaelbegon Mar 31, 2026
38c5c39
Wrap annotation marker navigation
mickaelbegon Mar 31, 2026
526ae86
Make matplotlib cache paths portable for CI
mickaelbegon Mar 31, 2026
e0834b2
Keep annotation epipolar guides across hidden cameras
mickaelbegon Apr 1, 2026
010ebf0
Update annotated 2D points for 1_partie_0429
mickaelbegon Apr 1, 2026
b357481
Make calibration annotated mode sparse-only
mickaelbegon Apr 1, 2026
6963543
Improve calibration QA, annotation previews, and CI resilience
mickaelbegon Apr 1, 2026
7458b42
Improve annotation interaction and shared jump analysis
mickaelbegon Apr 1, 2026
473d8c7
Add local annotation window refinement
mickaelbegon Apr 1, 2026
1768442
Delay busy popups until work is actually long
mickaelbegon Apr 1, 2026
34b99d9
Refine annotation placement and run all CI checks
mickaelbegon Apr 1, 2026
c170249
Report individual CI check outcomes
mickaelbegon Apr 1, 2026
4448fb7
Fix dynamic annotation crops and multiview selection
mickaelbegon Apr 1, 2026
09abc4a
Add EKF2D ankle-bed pseudo-observations
mickaelbegon Apr 3, 2026
47fc5c0
Add history-based EKF2D predictor modes
mickaelbegon Apr 3, 2026
844bd39
Add unsaved-change prompts and document recent GUI updates
mickaelbegon Apr 3, 2026
512670c
Stabilize history-based EKF2D prediction
mickaelbegon Apr 3, 2026
d461982
Document public APIs and align 3D back overlay colors
mickaelbegon Apr 3, 2026
1b5b472
Preserve continuous q states in history3 EKF
mickaelbegon Apr 3, 2026
34c017d
Fix multiview QA selection and silent empty calibration stats
mickaelbegon Apr 3, 2026
c4aee3b
Improve camera inspector navigation
mickaelbegon Apr 3, 2026
018c393
Polish shared image roots and camera frame slider
mickaelbegon Apr 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[flake8]
max-line-length = 120
extend-ignore = E203,W503
extend-ignore = E203,W503,E402,E501,F401,F541,F811,F841
33 changes: 33 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,46 @@ jobs:
pip install -e .[test]

- name: Run isort --check-only
id: isort
continue-on-error: true
run: isort . --check-only --profile black

- name: Run black --check
id: black
continue-on-error: true
run: black . --check --verbose

- name: Run flake8
id: flake8
continue-on-error: true
run: flake8 .

- name: Run test suite
id: pytest
continue-on-error: true
run: pytest -q

- name: Summarize check outcomes
if: always()
run: |
echo "isort: ${{ steps.isort.outcome }}"
echo "black: ${{ steps.black.outcome }}"
echo "flake8: ${{ steps.flake8.outcome }}"
echo "pytest: ${{ steps.pytest.outcome }}"
{
echo "## CI check outcomes"
echo ""
echo "- isort: \`${{ steps.isort.outcome }}\`"
echo "- black: \`${{ steps.black.outcome }}\`"
echo "- flake8: \`${{ steps.flake8.outcome }}\`"
echo "- pytest: \`${{ steps.pytest.outcome }}\`"
} >> "$GITHUB_STEP_SUMMARY"
failed_checks=()
[ "${{ steps.isort.outcome }}" = "success" ] || failed_checks+=("isort")
[ "${{ steps.black.outcome }}" = "success" ] || failed_checks+=("black")
[ "${{ steps.flake8.outcome }}" = "success" ] || failed_checks+=("flake8")
[ "${{ steps.pytest.outcome }}" = "success" ] || failed_checks+=("pytest")
if [ ${#failed_checks[@]} -gt 0 ]; then
echo "Failed checks: ${failed_checks[*]}"
exit 1
fi
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ __pycache__/

outputs/
output/
inputs/images/

Archive.zip
*.mat
Expand Down
136 changes: 123 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ This repository contains:

- a desktop GUI to inspect 2D detections, reconstructions, and analyses
- command-line tools to generate reconstruction bundles and run named profiles
- annotation and calibration-QA tools for interactive multiview inspection
- a batch runner with Excel synthesis export
- analysis utilities for root kinematics, DD estimation, execution deductions, trampoline displacement, observability, and 3D segment analysis

## Repository Overview
Expand All @@ -19,6 +21,7 @@ Main entry points:

Main packages:

- [annotation](/Users/mickaelbegon/Documents/Playground/annotation): sparse 2D annotation storage, navigation, kinematic assist, preview rendering
- [reconstruction](/Users/mickaelbegon/Documents/Playground/reconstruction): bundle generation, dataset handling, timings, profiles, naming
- [kinematics](/Users/mickaelbegon/Documents/Playground/kinematics): root kinematics and 3D analysis
- [camera_tools](/Users/mickaelbegon/Documents/Playground/camera_tools): camera metrics and camera selection helpers
Expand Down Expand Up @@ -99,6 +102,7 @@ Typical inputs are organized under `inputs/`:

- calibration file: `inputs/calibration/Calib.toml`
- 2D detections: `inputs/keypoints/<trial>_keypoints.json`
- optional sparse 2D annotations: `inputs/annotations/<trial>_annotations.json`
- optional TRC file: `inputs/trc/<trial>.trc`
- optional DD reference file: `inputs/dd/<trial>_DD.json`
- optional images or extracted frames: typically `inputs/images/<trial>/...` or another sibling folder inferred from the keypoint file
Expand All @@ -125,17 +129,29 @@ python /Users/mickaelbegon/Documents/Playground/pipeline_gui.py

The GUI now uses a shared reconstruction selector at the top of the window. Most analysis tabs reuse that selector instead of maintaining their own reconstruction table.

At startup, the GUI shows a small splash/status window while shared caches and preview resources are loaded.

If you try to quit while annotations or reconstruction profiles contain unsaved
changes, the GUI now asks for confirmation before closing.

Typical workflow:

1. Choose the 2D input in `2D explorer`
2. Inspect cameras, flips, and candidate issues in `Caméras`
3. Generate models in `Modèle`
4. Define named reconstruction profiles in `Profiles`
5. Run and inspect bundles in `Reconstructions`
6. Compare outputs in the analysis tabs
1. Choose the 2D input in `2D analysis`
2. Inspect cameras, flips, and candidate issues in `Cameras`
3. Create or refine sparse 2D points in `Annotation`
4. Generate models in `Models`
5. Define named reconstruction profiles in `Profiles`
6. Run and inspect bundles in `Reconstructions`
7. Inspect calibration quality in `Calibration`
8. Run multi-dataset/profile batches in `Batch`
9. Compare outputs in the analysis tabs

Main analysis tabs:

- `Cameras`: camera ranking, L/R flip inspection, reprojection overlays, QA overlays on top of images
- `Annotation`: sparse 2D multiview annotation with crop, epipolar guides, reprojection helpers, drag editing, and a first kinematic-assist mode
- `Calibration`: 2D epipolar QA + 3D reprojection QA, worst frames, pairwise camera matrix, and spatial quality maps
- `Batch`: scan several keypoint files, run selected profiles, and export an Excel synthesis workbook
- `3D animation`: export comparative 3D GIFs
- `2D multiview`: export multi-camera 2D GIFs
- `DD`: jump segmentation and DD estimation
Expand Down Expand Up @@ -175,6 +191,8 @@ Supported families:
- `ekf_3d`
- `ekf_2d`

The CLI and GUI both support `raw`, `cleaned`, and, when available, `annotated` 2D inputs.

### Run a list of named profiles

Example:
Expand Down Expand Up @@ -210,8 +228,8 @@ python /Users/mickaelbegon/Documents/Playground/run_reconstruction_profiles.py \
The pipeline can work from:

- `raw` detections
- `filtered` detections
- `cleaned` detections
- sparse `annotated` detections

Cleaning includes temporal smoothing and outlier rejection based on a robust motion amplitude estimate.

Expand All @@ -230,6 +248,7 @@ Corrected 2D variants are cached, so downstream stages can reuse:
- cleaned + epipolar flip
- cleaned + fast epipolar flip
- cleaned + triangulation-based flip
- annotated-only sparse observations in the GUI calibration and annotation workflows

For epipolar-family methods, the current implementation also applies a simple
2-state Viterbi decoding (`normal` / `flipped`) only when you explicitly pick
Expand All @@ -248,6 +267,7 @@ The triangulation stage also stores:
- per-frame reprojection error
- view usage
- excluded-camera patterns
- per-frame/keypoint/camera excluded-view masks usable in the GUI
- coherence scores

### 4. Root orientation extraction
Expand All @@ -272,23 +292,97 @@ Recent initialization strategies include:
- triangulation-based initialization
- root-only initialization with zero rest of the body (`root_pose_zero_rest`)

The GUI root-analysis views also support short-gap interpolation before unwrap for visualization/export.

### 6. EKF 2D

The 2D EKF combines:

- the articulated model
- 2D observations in all cameras
- multiview coherence weighting
- configurable predictor (`acc` or `dyn`)
- configurable predictor (`acc`, `dyn`, `history3`, or `dyn_history3`)

Important improvements already integrated in the codebase:

- sequential camera updates
- vectorized measurement assembly
- root-pose bootstrap initialization (`root_pose_bootstrap`)
- configurable coherence families: `epipolar`, `epipolar_fast`, `triangulation_once`, `triangulation_greedy`, `triangulation_exhaustive`

### 7. DD estimation
- runtime left/right gate mode inside the EKF2D loop (`ekf_prediction_gate`)
- optional segmented-back model variants, including upper-trunk-root variants
- storage of excluded views for later QA overlays in the GUI
- higher-order history-based prediction from the last three corrected states
- optional trampoline-contact pseudo-observations for the ankles
- lower confidence on views detected as left/right-flipped so they still help
the filter without dominating it

### 6.b Complexity overview

The dominant asymptotic costs below use:

- `F`: number of frames
- `C`: number of cameras
- `M`: number of observed model markers or keypoints per frame
- `Q`: number of generalized coordinates / DoF
- `L = 2 * C * M`: approximate 2D measurement dimension for one frame

| Method | Main cost per frame | Full-sequence cost | Notes |
| --- | --- | --- | --- |
| Triangulation `once` | `O(M * C)` | `O(F * M * C)` | One weighted triangulation and one reprojection pass per marker. |
| Triangulation `greedy` | `O(M * C^2)` | `O(F * M * C^2)` | Repeatedly removes the worst view, so the camera loop is effectively quadratic. |
| Triangulation `exhaustive` | `O(M * 2^C)` worst case | `O(F * M * 2^C)` worst case | Tries many camera subsets; practical cost depends on the number of valid views. |
| EKF2D `acc` | `O(Q^3 + L^3 + L^2 * Q)` | `O(F * (Q^3 + L^3 + L^2 * Q))` | Prediction is dominated by covariance propagation; update is dominated by the Kalman solve on image measurements. |
| EKF2D `dyn` | `O(Q^3 + L^3 + L^2 * Q)` | `O(F * (Q^3 + L^3 + L^2 * Q))` | Same asymptotic order as `acc`, with a larger constant when root flight dynamics are active. |
| EKF2D `history3` | `O(Q^3 + L^3 + L^2 * Q)` | `O(F * (Q^3 + L^3 + L^2 * Q))` | Same asymptotic order as `acc`; the higher-order predictor adds only `O(Q)` state-history work. |
| EKF2D `dyn_history3` | `O(Q^3 + L^3 + L^2 * Q)` | `O(F * (Q^3 + L^3 + L^2 * Q))` | Same asymptotic order as `dyn`; root uses `dyn`, joints use the smoothed history-based predictor. |

In practice:

- triangulation cost scales mostly with the number of cameras and valid keypoints
- EKF2D cost scales most sharply with the measurement dimension `L`, so reducing
cameras or sparse observations can change runtime more than reducing `Q`
- `history3` and `dyn_history3` are intended as better predictors, not faster
filters; their overhead is small compared with the Kalman update itself

### 7. Model building

The `Models` tab and bundle generation code support:

- several trunk/back structures, including segmented-back variants
- optional left/right limb symmetrization (`Symmetrize limbs`)
- model creation from `raw`, `cleaned`, or `annotated` 2D observations
- preview of the segmented back with a dedicated `mid_back` marker and 2-triangle back geometry

### 8. Annotation workflow

The `Annotation` tab provides:

- sparse per-camera / per-frame / per-marker JSON annotation storage
- image-backed multiview annotation with brightness/contrast, crop `+20%`, zoom and pan
- epipolar guides, triangulated reprojection hints, and reprojection from the selected reconstruction
- frame subsets such as `Flipped L/R` and `Worst reproj 5%`
- `Reproject -> Confirm` replacement of already annotated points only
- drag editing of existing 2D points with optional snap to reprojection or epipolar guides
- a first `Kinematic assist` mode:
- choose an existing `.bioMod`
- estimate an initial `q` on the current frame
- keep and propagate local kinematic states across frames
- run short local EKF/direct-fit corrections when annotated points are edited

### 9. Calibration QA

The `Calibration` tab provides:

- pairwise epipolar consistency per camera pair
- global trimming of the worst `2D` samples before aggregation
- per-camera and per-frame diagnostics
- a `Worst frames` list with quick jump to `Cameras`
- 3D reprojection summaries from the selected reconstruction
- spatial non-uniformity metrics across 3D space, including binned `X/Z` maps
- local choice of the 2D source: `raw`, `cleaned`, or `annotated`

### 10. DD estimation

The `DD` tab and [judging/dd_analysis.py](/Users/mickaelbegon/Documents/Playground/judging/dd_analysis.py) provide:

Expand All @@ -298,7 +392,7 @@ The `DD` tab and [judging/dd_analysis.py](/Users/mickaelbegon/Documents/Playgrou
- comparison with expected codes loaded from `*_DD.json`
- comparison of each reconstruction against the expected DD reference with color-coded status

### 8. Execution deductions
### 11. Execution deductions

The `Execution` tab and [judging/execution.py](/Users/mickaelbegon/Documents/Playground/judging/execution.py) provide:

Expand All @@ -307,15 +401,28 @@ The `Execution` tab and [judging/execution.py](/Users/mickaelbegon/Documents/Pla
- session-level time-of-flight scoring
- a structure ready for direct image overlays as soon as camera frames are available

### 9. Trampoline displacement
### 12. Trampoline displacement

The `Toile` tab estimates horizontal displacement penalties:

- contact windows are inferred between jumps segmented in the DD analysis
- contact position currently uses the feet as a proxy
- the bed geometry is based on a calibrated set of trampoline reference markers

### 10. Observability analysis
### 13. Batch execution and synthesis

The batch backend and `Batch` tab support:

- scanning a set of keypoint files
- selecting a profiles file
- running the chosen reconstructions for all detected trials
- exporting an Excel workbook summarizing:
- reconstruction options
- timings by stage
- reprojection metrics
- recognition / failure summaries

### 14. Observability analysis

The `Observabilité` tab computes frame-wise ranks of:

Expand Down Expand Up @@ -363,6 +470,9 @@ The repository contains a single CI workflow under:
- Improve hip and knee flexion handling for EKF outputs to distinguish `piked` and `grouped` body shapes more robustly.
- Compute hip and knee flexion angles directly from triangulated 3D data.
- Continue developing the execution-error analysis module.
- Improve the annotation kinematic assist with a short local temporal EKF window around the current frame.
- Export calibration QA summaries directly in the batch Excel workflow.
- Better control the foot position on the trampoline bed: when a foot is in contact with the bed, keep it fixed in the horizontal plane with a high-confidence pseudo-observation during the whole contact phase.

## Notes

Expand Down
2 changes: 1 addition & 1 deletion analysis/analyze_camera_mapping.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@
import argparse
import itertools as it
import json
from pathlib import Path
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
Expand Down
Loading
Loading