Lula GPU test - DO NOT CLOSE until 25th march

[Kwame Yeboah (Yeboahmedia)]

PR Type

Enhancement

---

Description

• Replace device model-based exclusion with GPU renderer detection

• Use WebGL debug info as primary GPU detection method

• Add UA-CH architecture hints as secondary fallback

• Debug alert() calls left in for testing purposes

---

Diagram Walkthrough

flowchart LR
  A["WebGL Debug Info"] -- "getGpuRenderer()" --> B["isExcludedGpuFromWebGL()"]
  C["UA-CH Hints API"] -- "getSystemArchitectureHints()" --> D["Check hints for excluded GPU"]
  B -- "excluded" --> E["CPU Delegate"]
  B -- "not excluded" --> D
  D -- "excluded" --> E
  D -- "not excluded" --> F["GPU Delegate"]
  E --> G["FaceLandmarker.createFromOptions"]
  F --> G

Loading

File Walkthrough

Relevant files

Enhancement

mediapipeManager.ts GPU-based exclusion detection replacing device model checks packages/web-components/lib/components/selfie/src/smartselfie-capture/utils/mediapipeManager.ts Replaced EXCLUDED_DEVICES (device model strings) with EXCLUDED_GPUS (Adreno GPU identifiers like adreno-830, adreno-8xx, adreno-9xx) Added getGpuRenderer() to query GPU renderer via WebGL WEBGL_debug_renderer_info extension Added isExcludedGpuFromWebGL() to check if the detected GPU matches exclusion list, including regex for Adreno 8xx series Replaced isExcludedDeviceUsingHints() with getSystemArchitectureHints() that fetches broader UA-CH hints (architecture, model, platform, etc.) Added getDelegateFromGpuDetection() combining WebGL (primary) and UA-CH (secondary) checks to determine CPU vs GPU delegate Contains debug alert() calls that should be removed before merging Refactored getMediapipeInstance to use new getDelegateFromGpuDetection() for delegate selection +98/-30

---

---

Need help?

Type /help how to ... in the comments thread for any questions about PR-Agent usage.

Check out the documentation for more information.

[prfectionist]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🏅 Score: 45

🧪 No relevant tests

🔒 Security concerns Sensitive information exposure: The alert() calls on lines 87, 103-104, and 111-112 display GPU renderer strings and UA-CH hint data (which may include device model, platform version, and full browser version list) directly to the user. While primarily a UX issue, if this code reaches production it exposes device fingerprinting information. Additionally, getHighEntropyValues requests more data than needed (architecture, model, platform, platformVersion, fullVersionList) which could be considered over-collection of user data.

🔀 No multiple PR themes

⚡ Recommended focus areas for review Debug alerts in production Multiple alert() calls are left in the code (lines 87, 103-104, 111-112). The PR description acknowledges these are for testing, but they will block the UI thread and disrupt user experience if merged. These must be removed or replaced with non-blocking logging before merge. alert(`[SmileID] Excluded GPU via WebGL: ${renderer}. Using CPU.`); return 'CPU'; } // Secondary check: UA-CH hints (may contain GPU info in some browsers) const hintString = await getSystemArchitectureHints(); if (hintString) { const normalizedHintString = hintString.replace(/[\s_-]/g, ''); const hasExcludedGpuInHints = EXCLUDED_GPUS.some((gpu) => normalizedHintString.includes(gpu.toLowerCase().replace(/[\s_-]/g, '')), ) || /adreno8\d{2}/.test(normalizedHintString); if (hasExcludedGpuInHints) { alert( `[SmileID] Excluded GPU via UA-CH hints: ${hintString}. Using CPU.`, ); return 'CPU'; } } // Default to GPU when no exclusion is detected alert( `[SmileID] No excluded GPU detected. WebGL renderer: ${renderer ?? 'unavailable'}, UA-CH: ${hintString ?? 'unavailable'}. Using GPU.`, ); Incomplete exclusion list EXCLUDED_GPUS contains 'adreno-8xx' and 'adreno-9xx' as literal strings, but these are wildcard-style patterns that won't match actual renderer strings (e.g., "Adreno (TM) 830"). The includes() check treats them literally. The separate regex /adreno8\d{2}/ partially compensates for the 8xx case but there's no equivalent regex for 9xx. Also, the regex won't match "Adreno (TM) 830" after normalization to "adreno(tm)830" since the regex expects "adreno8" immediately followed by two digits, but "adreno(tm)830" has "(tm)" in between. const EXCLUDED_GPUS = ['adreno-830', 'adreno-8xx', 'adreno-9xx']; /** * @description Gets the GPU renderer string using WebGL debug info extension. * @returns {string | null} The GPU renderer string or null if unavailable. */ const getGpuRenderer = (): string | null => { try { const canvas = document.createElement('canvas'); const gl = canvas.getContext('webgl') || canvas.getContext('experimental-webgl'); if (!gl || !(gl instanceof WebGLRenderingContext)) return null; const ext = gl.getExtension('WEBGL_debug_renderer_info'); if (!ext) return null; return gl.getParameter(ext.UNMASKED_RENDERER_WEBGL) as string | null; } catch { return null; } }; /** * @description Checks if the GPU renderer matches any excluded GPU. * @returns {boolean} True if the GPU is excluded. */ const isExcludedGpuFromWebGL = (): boolean => { const renderer = getGpuRenderer()?.toLowerCase() ?? ''; if (!renderer) return false; const normalizedRenderer = renderer.replace(/[\s_-]/g, ''); return ( EXCLUDED_GPUS.some((gpu) => normalizedRenderer.includes(gpu.toLowerCase().replace(/[\s_-]/g, '')), ) || /adreno8\d{2}/.test(normalizedRenderer) ); WebGL resource leak getGpuRenderer() creates a canvas and obtains a WebGL context but never calls gl.getExtension('WEBGL_lose_context')?.loseContext() or otherwise releases the context. On repeated calls or resource-constrained mobile devices, this could exhaust WebGL context slots (browsers typically limit to ~8-16 active contexts). const getGpuRenderer = (): string | null => { try { const canvas = document.createElement('canvas'); const gl = canvas.getContext('webgl') || canvas.getContext('experimental-webgl'); if (!gl || !(gl instanceof WebGLRenderingContext)) return null; const ext = gl.getExtension('WEBGL_debug_renderer_info'); if (!ext) return null; return gl.getParameter(ext.UNMASKED_RENDERER_WEBGL) as string | null; } catch { return null; } }; Duplicate normalization logic The GPU exclusion normalization and matching logic (normalize string, check EXCLUDED_GPUS with some/includes, plus regex test) is duplicated between isExcludedGpuFromWebGL() and the UA-CH hints check in getDelegateFromGpuDetection(). This should be extracted into a shared helper to avoid inconsistencies and reduce maintenance burden. const isExcludedGpuFromWebGL = (): boolean => { const renderer = getGpuRenderer()?.toLowerCase() ?? ''; if (!renderer) return false; const normalizedRenderer = renderer.replace(/[\s_-]/g, ''); return ( EXCLUDED_GPUS.some((gpu) => normalizedRenderer.includes(gpu.toLowerCase().replace(/[\s_-]/g, '')), ) || /adreno8\d{2}/.test(normalizedRenderer) ); }; declare global { interface Window { __smileIdentityMediapipe?: { instance: FaceLandmarker | null; loading: Promise<FaceLandmarker> | null; loaded: boolean; }; } } /** * @description Reads system architecture hints from User-Agent Client Hints. * @returns {Promise<string | null>} Lower-cased hint string or null when hints are unavailable. */ const getSystemArchitectureHints = async (): Promise<string | null> => { if (typeof navigator === 'undefined' || !navigator.userAgentData) { return null; } try { const hints = await navigator.userAgentData.getHighEntropyValues([ 'architecture', 'model', 'platform', 'platformVersion', 'fullVersionList', ]); return JSON.stringify(hints).toLowerCase(); } catch (error) { console.warn('UA-CH architecture hints fetch failed.', error); return null; } }; /** * @description Determines the MediaPipe delegate based on WebGL renderer info and UA-CH hints. * Uses WebGL renderer as primary detection, UA-CH hints as secondary. * @returns {Promise<'CPU' | 'GPU'>} CPU when excluded GPU is detected; otherwise GPU. */ const getDelegateFromGpuDetection = async (): Promise<'CPU' | 'GPU'> => { const renderer = getGpuRenderer(); // Primary check: WebGL renderer info (most reliable for GPU detection) if (isExcludedGpuFromWebGL()) { alert(`[SmileID] Excluded GPU via WebGL: ${renderer}. Using CPU.`); return 'CPU'; } // Secondary check: UA-CH hints (may contain GPU info in some browsers) const hintString = await getSystemArchitectureHints(); if (hintString) { const normalizedHintString = hintString.replace(/[\s_-]/g, ''); const hasExcludedGpuInHints = EXCLUDED_GPUS.some((gpu) => normalizedHintString.includes(gpu.toLowerCase().replace(/[\s_-]/g, '')), ) || /adreno8\d{2}/.test(normalizedHintString); Privacy/permission concern getHighEntropyValues requests architecture, model, platform, platformVersion, and fullVersionList, but only the GPU-related info is needed. Requesting more high-entropy values than necessary may trigger browser permission prompts or be blocked by Permissions-Policy headers, and raises unnecessary privacy concerns. const hints = await navigator.userAgentData.getHighEntropyValues([ 'architecture', 'model', 'platform', 'platformVersion', 'fullVersionList', ]);

[Copilot]

Pull request overview

This PR updates the MediaPipe initialization logic in the web-components selfie capture flow to choose a CPU vs GPU delegate based on detected GPU characteristics (targeting Adreno devices) rather than device-model checks.

Changes:

• Added WebGL-based GPU renderer detection and exclusion matching.
• Added UA-CH high-entropy hint collection as a secondary detection signal.
• Wired the new delegate-selection logic into FaceLandmarker.createFromOptions.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[github-actions]

This branch has been deployed to s3 / cloudfront.

✅ Preview URL for Smart Camera Web:

https://cdn.smileidentity.com/js/preview-anothergputestbranch/smart-camera-web.js

✅ Preview URL for Embed:

https://cdn.smileidentity.com/inline/preview-anothergputestbranch/js/script.min.js

✅ Preview URL for Web Client (Sandbox):

https://d5bzocs2ogm9j.cloudfront.net

✅ Preview URL for Web Client (Production):

https://dt3n0uhj0x1bw.cloudfront.net

[github-actions]

This PR is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

This PR was closed because it has been stalled for 7 days with no activity.