A software suite designed to simplify the complexities in streaming and receiving audio and video.
rosVid makes use of both RTSP and ROS2 as transport and viewing mechanisms for video streams.
The supported types of input are:
- MIPI
- RTSP
- UDP
- USB
The supported types of output are:
- ROS2 image viewing
- RTSP
In the SRC folder you will find libStreamer and mavPool. These modules are designed to keep the code base slim by abstracting out things where it makes sense.
For ease of use both of these libraries have been pre-compiled under Releases.
While libStreamer is a requirement, mavPool has now been made optional and is invoked by the mavPool parameter.
For the ROS2 code both ROS2 Humble and ROS2 Kilted are both supported.
The server and client are both stand-alone Python and no ROS2 installation is required.
The RTSP server will transmit audio when the --audio flag is set. To receive the audio with the RTSP client --audio must also be set.
Available parameters for server:
--abitrate Opus bitrate [Default is 64000]
--acard ALSA card index [Default is 0]
--achannels Audio channels [Default is 2]
--adevice ALSA deviceindex [Default is 0]
--arate Audio sampel rate [Default is 48000]
--audio Enable audio [Default is False]
--bitrate Desired bitrate [Default is 2500]
--codec Desired codec [Default is h264, can also be h265]
--device Device to stream from [Default is /dev/video0]
--format Desired pixel format [Default is I420]
--fps Desired frames per second [Default is 30]
--height Desired video height [Default is 480]
--mipi MIPI camera usage [Default is none]
--port Desired port to stream on [Default is 8554]
--endpoint Desired endpoint for the stream [Default is /video]
--width Desired video width [Default is 640]
Available parameters for client:
--audio Enable audio [Default is False]
--audio-max-buffers Audio queue max buffers [Default is 5]
--audio-leaky Leaky audio queue for low latency [Default is True]
--audio-sync Enable audio sink sync [Default is True]
--codec Video codec [Default is h264]
--drop-on-latency Drop late buffers [Default is True]
--latency RTSP source latency in ms [Default is 0]
--video-leaky Leaky video queue [Default is False]
--video-max-buffers Video queue max buffers [Default is 30]
--video-sync Enable video sink sync [Default is True]
--url RTSP URL [Default is rtsp://127.0.0.1:8554/video]'
MIPI input is gathered by the libStreamer.transports.rtsp module.
Example syntax:
python3 ./rtspServer.py --mipiThe above will connect to the MIPI camera on /dev/video0 by default. If /dev/video0 is not the assignment for the MIPI camera, select the camera with --device.
The associated RTSP stream is now ready to view using more traditional RTSP viewing methods as well as the ROS2 viewing technique further down in this file.
The captures module for RTSP allows connecting to an RTSP video stream and them transmitting the contents over ROS2. If you are using a camera which already has RTSP capabilities the only thing to add is the rtspUrl. If you are using the provided rtspServer then no parameters need to be added other than if you are doing this on two different computers, if so simply add the rtspUrl accordingly.
Example syntax:
ros2 run captures rtspThe above will connect to an RTSP stream running on rtsp://127.0.0.1:8554/video. To change this default behavior as well as other options you can choose from the available parameters below:
codec h264 or h265 [Default is h264]
height Height of video to output [Default is 480]
jpegQuality Compression quality [Default is 50]
pubSpeed Rate of published frames per second [Default is 30]
rtspUrl The associated RTSP URL to capture [Default is rtsp://127.0.0.1:8554/video]
topicId Defines a unique identifier for the topic [Optional, for multiple instances]
width Width of video to output [Default is 640]
When running multiple video cameras at the same time the topicId parameter will modify the topics on the backend.
If using the h265 codec in conjunction with the rtsp module, that module must also invoke h265 via the codec parameter.
UDP input listens on port 5600 on all interfaces by default.
udp usage:
ros2 run captures udpThe available parameters for udp are as follows:
clockRate RTP clock rate for the stream [Default is 90000]
codec Video codec (h264 or h265) [Default is h264]
height Expected / scaled video height [Default is 480]
jpegQuality JPEG compression quality [Default is 50]
mavPool Enable MAVLink metadata integration [Default is False]
pubSpeed Published frames per second [Default is 30]
topicId Defines a unique identifier for the topic [Optional, for multiple instances]
udpHost UDP source address to bind to [Default is 0.0.0.0 (all interfaces)]
udpPort UDP port to listen on [Default is 5600]
width Expected / scaled video width [Default is 640]
USB input is gathered using either the phys module or the rtspServer.
Connection using the phys module allows for transmission of the video over ROS2.
Connection using the rtspServer allows for transmission of the video over RTSP as well as being able to be transmitted over ROS2 via the rtsp module.
phys usage:
ros2 run captures physThe available parameters for phys are as follows:
device The device to use [Default is /dev/video0]
fps Transmitted frames per second [Default is 30]
height The transmitted video height [Default is 480]
jpegQuality Compression quality [Default is 50]
pubSpeed Published frames per second [Default is 30]
topicId Defines a unique identifier for the topic [Optional, for multiple instances]
width The transmitted video width [Default is 640]
rtspServer usage:
python3 ./rtspServer.pyWhen running multiple video cameras at the same time the topicId parameter will modify the topics on the backend.
The output produced by rtspServer allows for a user to stream the video to an RTSP viewer of their choice.
The output produced by the phys or rtsp captures allows for a user to watch the video using the watcher module. The available parameters are as such:
capture Defines which module to use [Default is phys, other option is rtsp]
compressed Defines whether to use the compressed options [Compression quality is determined via the phys or rtsp parameters]
pickled Defines whether to use the pickled approach [Lowest latency]
topicId Defines a unique identifier for the topic [Optional, for multiple instances]
If capture is not invoked the expected input is the phys module.
If neither compressed or pickled is invoked, then the Image message is using for receiving the message.
If topicId is used for either phys or rtsp, it must be declared at launch for the watcher to capture the stream.
In terms of which approach to use, "it depends":
- The pickled approach is the smoothest in terms of quality and latency, very close to an FPV type feel:
ros2 run captures watcher --ros-args -p pickled:=True- The compressed viewing is better in terms of quality and latency than
Image:
ros2 run captures watcher --ros-args -p compressed:=True -p capture:=rtsp- Image viewing:
ros2 run captures watcherThe phys, rtsp and udp executables create a ROS2 Subscriber which allows for user-defined changes while the module is running. Detailed instructions for doing so are found here.
In the next phase of development, rosVid will evolve from a streaming and viewing platform into a reactive system. The idea is to monitor the pixel-level changes from the video feed, detect motion, identify patterns, and trigger automated responses in real time (hence ROS2 from the beginning). That capability will open the door for applications ranging from surveillance and robotics to autonomous vehicles, where instantaneous visual feedback is critical. The focus will be on creating a modular framework that allows users to define custom reactions to visual events, transforming raw video streams into actionable intelligence.