Skip to content

document_draft.md

Latest commit

 

History

History
192 lines (144 loc) · 6.78 KB

document_draft.md

File metadata and controls

192 lines (144 loc) · 6.78 KB

Application Overview

Name: CPS Physical Education Livestream Analysis PlatformPurpose: An AI-powered educational tool for real-time physical education performance assessment and student skill developmentPrimary Use Case: Teachers conduct PE sessions where students perform physical activities, record themselves (or are ), and receive immediate AI-powered feedback on their technique and performance

Core Functionality & Gemini API Usage

  1. Real-Time Video Analysis During PE Sessions
  • What happens: Students record their physical activity performances (max 59 seconds)
  • Gemini API role: Analyzes video frames to provide real-time technical feedback on:
    • Body positioning and posture
    • Coordination and movement fluidity
    • Timing and rhythm
    • Balance and control
    • Power/force application
    • Accuracy and precision
  • Educational value: Immediate, actionable coaching feedback addressing specific biomechanical improvements
  1. Performance-Based Scoring System
  • Gemini API analyzes: Technical execution quality (40%), skill demonstration level (35%), technique accuracy (25%)
  • Scoring scale: 1-10 performance ratings mapped to 0-100 scores
    • 1-2: Unable to perform skill (15-25 points)
    • 5-6: Developing skills (55-65 points)
    • 9-10: Mastery level (90-96 points)
  • Child-friendly feedback: All responses converted to second-person ("you") with age-appropriate language for 6-12 year olds
  1. Timestamped Feedback Generation
  • Process: Every few seconds during recording, captures video frame → sends to Gemini API
  • Response constraint: EXACTLY 20 words or less for each timestamp
  • Categorization: Feedback automatically tagged as:
    • 💪 GREAT JOB (positive reinforcement)
    • 💡 TIP (improvement suggestions)
    • 👀 FOCUS (attention/engagement reminders)
  1. Post-Recording Comprehensive Analysis
  • Full video analysis: After recording stops, entire video sent to Gemini API
  • Provides:
    • Overall technical performance rating
    • Specific strengths identified
    • Misconceptions/errors detected
    • Technique-specific feedback
    • Gameplay concept feedback (where applicable)
    • Overall performance score (0-100)

Educational Safeguards & Responsible AI Implementation

Positive-Only Feedback System

  • All feedback filtered to remove negativity/discouragement
  • Off-task language converted to encouraging alternatives ("distracted" → "observing surroundings")
  • Default assumption: students are engaged and trying their best
  • Explicit prompt instructions: "BE EXTREMELY POSITIVE AND ENCOURAGING"

Age-Appropriate Language Processing

  • Complex vocabulary automatically simplified:
    • "demonstrate" → "show"
    • "utilize" → "use"
    • "analyze" → "look at"
  • Second-person addressing for personal connection
  • Maximum word limits enforced for attention spans

Performance-Based Assessment (Not Effort-Based)

  • Scoring focuses ONLY on observable technical ability
  • Removes subjective "engagement" metrics
  • Objective biomechanical evaluation
  • Transparent rubric aligned with PE learning standards

Class Management & Data Flow

Teacher Workflow

  1. Creates class with auto-generated unique code
  2. Uploads CSV namelist (optional) for student validation
  3. Creates session with learning outcomes (e.g., "proper volleyball serve form")
  4. Starts session → students join via unique link

Student Workflow

  1. Joins session via class code URL
  2. Selects/enters name (validated against namelist if exists)
  3. Records performance (camera toggle for front/back camera)
  4. Receives real-time feedback during recording
  5. Views comprehensive analysis after recording
  6. Writes personal reflection on areas to improve

Data Storage (Supabase)

  • teacher_sessions: Session metadata and learning outcomes
  • student_livestream_analysis: AI analysis results, scores, feedback
  • class_namelists: Pre-defined student rosters
  • class_enrollments: Student participation tracking

Technical Architecture

Frontend: React 18 + TypeScript + Vite

  • MediaRecorder API for video capture
  • Platform-unified video processing (same code iOS/desktop)
  • Base64 video encoding for API transmission

Backend: Supabase Edge Functions (Deno)

  • gemini-analysis: Image/video analysis endpoint
  • analyze-student-livestream: Full video comprehensive analysis
  • Uses Gemini 2.5 Flash model

Video Processing

  • Supported formats: webm (vp9/vp8), mp4 fallback
  • 59-second recording limit (age-appropriate attention spans)
  • Automatic camera stop at time limit
  • iOS/Safari optimized with hardware acceleration

Privacy & Security Considerations

Student Privacy

  • No authentication required for students (frictionless access)
  • Video data base64-encoded in transit
  • No permanent video storage (blob URLs revoked after analysis)
  • Optional namelist validation prevents impersonation

Data Handling

  • Videos analyzed immediately then discarded
  • Only analysis results and scores persisted
  • Student reflections stored separately
  • Teacher-controlled session activation/deactivation

API Request Patterns

Request Type 1: Real-Time Frame Analysis

{ imageData: "base64_jpeg", prompt: "PE teacher analyzing movement...", learningOutcome: "proper basketball dribbling", requestType: "live-analysis" } Response: ~20 word technical feedback

Request Type 2: Post-Recording Full Analysis

{ sessionId: "uuid", studentName: "Student Name", learningOutcomes: ["outcome1", "outcome2"], videoBlob: "base64_video_data", mimeType: "video/webm" } Response: Complete performance assessment with scoring

Request Type 3: Activity Scoring

{ prompt: "activity analysis text...", requestType: "scoring" } Response: 1-4 star rating with brief explanation

Key Differentiators for Educational Use

  1. Immediate formative assessment - Students see feedback during performance
  2. Self-reflection integration - Students write improvement goals
  3. Teacher dashboard - Class-wide analytics, trends, misconception tracking
  4. Age-appropriate design - Simple language, visual indicators, short feedback
  5. Inclusive feedback - Assumes best effort, focuses on technical growth
  6. Cross-platform compatibility - Works on iPads (common in schools)

Gemini API Value Proposition

  • Educational impact: Enables 1:1 personalized coaching at scale (teacher can monitor entire class)
  • Technical requirement: Vision API essential for biomechanical movement analysis
  • Real-time capability: Fast inference needed for in-session feedback
  • Context understanding: Gemini's multimodal capabilities excel at nuanced PE skill assessment
  • Child safety: Text output easily filtered/sanitized for appropriate feedback