Skip to content

feat(streaming): integrate native external streamer MVP#180

Closed
zortos293 wants to merge 30 commits intodevfrom
capy/external-streamer-mvp
Closed

feat(streaming): integrate native external streamer MVP#180
zortos293 wants to merge 30 commits intodevfrom
capy/external-streamer-mvp

Conversation

@zortos293
Copy link
Copy Markdown
Collaborator

@zortos293 zortos293 commented Apr 1, 2026

Description

This PR introduces a native Rust external streamer application (opennow-streamer) and integrates it into OpenNOW as an alternative streaming backend behind a feature flag. The native app handles WebRTC transport, video/audio decoding via FFmpeg, rendering via SDL2, and local input capture, while the Electron main process retains responsibility for CloudMatch session lifecycle and signaling orchestration.

Architecture

  • Electron Main (opennow-stable/src/main/services/streamerManager.ts): Spawns and manages the external opennow-streamer process via a local control socket. Reuses existing GfnSignalingClient and CloudMatch logic, forwarding session info, offers, and ICE candidates to the native process.
  • Native Streamer (opennow-streamer/): A Rust application managing the native peer connection, SDP manipulation (NVST, codec preference, server IP fix), media decode/render pipeline (FFmpeg/Sdl2), and input encoding (mouse/keyboard/controller).
  • Renderer (opennow-stable/src/renderer/src/App.tsx): Branches between the legacy Chromium WebRTC path and the external streamer path based on the new useExternalStreamer setting.

What’s included

  • opennow-streamer/ native project

    • WebRTC peer connection, ICE, and data channel handling using webrtc-rs.
    • SDP helpers (sdp.rs) for NVST SDP generation, codec filtering, and server IP fixing.
    • Native input capture (window.rs, input.rs) for keyboard (with modifiers), mouse, and controller, mapped to GFN binary protocol.
    • Media pipeline (media.rs) depacketizing H.264/H.265 RTP and decoding via bundled FFmpeg; Opus audio decoded via libopus.
    • Loopback socket IPC (control.rs) for Electron ↔ native control.
  • Electron integration

    • opennow-stable/src/main/services/streamerManager.ts: Service to spawn the binary, establish the control channel, and proxy signaling.
    • opennow-stable/src/main/index.ts: Register streamer manager IPC handlers and lifecycle hooks.
    • opennow-stable/src/renderer/src/App.tsx: Updated handlePlayGame to select the external path when enabled.
    • opennow-stable/src/renderer/src/components/SettingsPage.tsx: Added "Use External Streamer" toggle with Beta badge.
    • opennow-stable/src/shared/gfn.ts: Added StreamerState and useExternalStreamer setting definitions.
    • opennow-stable/src/preload/index.ts: Exposed streamer control and state APIs to the renderer.
  • Build and packaging

    • opennow-streamer/Cargo.toml: Rust project manifest with dependencies (webrtc, sdl2, opus, bytes).
    • opennow-stable/scripts/bundle-native-runtime.mjs: Build script to compile the Rust binary and bundle it alongside a sidecar FFmpeg binary into resources/bin/.
    • opennow-stable/package.json: Updated build script to run the bundler and extraResources configuration to include resources/bin/**.

Limitations

  • MVP targets H.264/H.265 video and Opus audio. AV1 native decode is not yet implemented.
  • Recording and screenshot workflows are not migrated to the external path in this phase.
  • Microphone uplink is not implemented in the external path.

Open in Capy OPE-25 · 5.4

@zortos293 zortos293 added the capy Generated by capy.ai label Apr 1, 2026 — with Capy AI
Copy link
Copy Markdown

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 2 comments

zortos293 and others added 27 commits April 1, 2026 13:24
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@zortos293 zortos293 closed this Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capy Generated by capy.ai

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant