Slow FPS and no effect by transcode on Raspberry Pi

[yakir12]

I'm measuring FPS of Raspberry Pi Camera Module 2 NoIR on a Raspberry Pi 4. Due to issue #346 I'm using VideoIO.jl v0.9.5. This is Julia v1.7.2.

I'm seeing surprisingly low FPS (~7 FPS) as well as no effect by setting transcode = false.

Since the frame pipeline has a buffer which can return a buffered frame much faster than retrieving a new one from the CCD, it is necessary to either "flush" the buffer before timing (any ideas on how to do that?), or simply filter out the unrealistically high FPS (which is what I did here).
I think I managed to achieve this with:

using VideoIO, UnicodePlots

function benchmark(cam)
    img = read(cam)
    t = zeros(100)
    for i in eachindex(t)
        read!(cam, img)
        t[i] = Base.time()
    end
    fps = 1 ./ diff(t)
    filter!(<(50), fps) # filter out all the unrealisticly high FPSs
    println(histogram(fps))
end

for transcode in (true, false)
    cam = opencamera(; transcode)
    println("Transcode: ", transcode)
    benchmark(cam)
    close(cam)
end

The results are:

Transcode: true
                ┌                                        ┐ 
   [ 6.0,  7.0) ┤████████████████████████████████████  94  
   [ 7.0,  8.0) ┤█▎ 3                                      
   [ 8.0,  9.0) ┤  0                                       
   [ 9.0, 10.0) ┤  0                                       
   [10.0, 11.0) ┤  0                                       
   [11.0, 12.0) ┤▍ 1                                       
                └                                        ┘ 
                                 Frequency                 
Transcode: false
                ┌                                        ┐ 
   [ 5.0, 10.0) ┤████████████████████████████████████  98  
   [10.0, 15.0) ┤  0                                       
   [15.0, 20.0) ┤  0                                       
   [20.0, 25.0) ┤  0                                       
   [25.0, 30.0) ┤  0                                       
   [30.0, 35.0) ┤  0                                       
   [35.0, 40.0) ┤▍ 1                                       
                └                                        ┘ 
                                 Frequency                 

[yakir12]

I took the liberty to test this with all the available VideoIO.jl versions, to see if it was always this slow, and indeed, it was:

Row │ version  min      median   max
─────┼────────────────────────────────────
   1 │ 0.4.0    missing  missing  missing 
   2 │ 0.5.0    missing  missing  missing 
   3 │ 0.5.1    missing  missing  missing 
   4 │ 0.5.2    missing  missing  missing 
   5 │ 0.5.3    missing  missing  missing 
   6 │ 0.5.4    missing  missing  missing 
   7 │ 0.5.5    missing  missing  missing 
   8 │ 0.5.6    missing  missing  missing 
   9 │ 0.6.0    missing  missing  missing 
  10 │ 0.6.1    missing  missing  missing 
  11 │ 0.6.10         7        7       31
  12 │ 0.6.11         7        7       28
  13 │ 0.6.2    missing  missing  missing 
  14 │ 0.6.3    missing  missing  missing 
  15 │ 0.6.4          7        7       22
  16 │ 0.6.5          7        7       22
  17 │ 0.6.6          7        7       22
  18 │ 0.6.7          7        7       22
  19 │ 0.6.8          6        7       22
  20 │ 0.6.9          7        7       22
  21 │ 0.7.0          7        7       22
  22 │ 0.7.1          7        7       41
  23 │ 0.7.2          7        7       25
  24 │ 0.7.3          7        7       22
  25 │ 0.7.4          6        7       22
  26 │ 0.8.0          7        7       21
  27 │ 0.8.1          7        7       21
  28 │ 0.8.2          7        7       22
  29 │ 0.8.3          7        7       22
  30 │ 0.8.4          6        7       22
  31 │ 0.9.0          7        7       27
  32 │ 0.9.1          6        7       27
  33 │ 0.9.2          7        7       21
  34 │ 0.9.3          7        7       22
  35 │ 0.9.4          6        7       48
  36 │ 0.9.5          7        7       21
  37 │ 0.9.6    missing  missing  missing 

Here's the code that generated this table:

import Pkg
using DataFrames
df = DataFrame(version = String[], min = Int[], median = Int[], max = Int[])
allowmissing!(df, [:min, :median, :max])

function available_versions(pkgname::AbstractString)
    registry = only(filter(r -> r.name == "General", Pkg.Registry.reachable_registries()))
    pkg = only(filter(pkg -> pkg.name == pkgname, collect(values(registry.pkgs))))
    return keys(Pkg.Registry.registry_info(pkg).version_info)
end

for version in string.(available_versions("VideoIO"))
  try 
    txt = read(`julia -e """
               import Pkg
               Pkg.activate(temp = true)
               Pkg.add(name = \"VideoIO\", version = \"$version\")
               using VideoIO, Statistics
               cam = opencamera()
               img = read(cam)
               t = zeros(100)
               for i in eachindex(t)
               read!(cam, img)
               t[i] = Base.time()
               end
               fps = round.(Int, 1 ./ diff(t))
               filter!(<(50), fps)
               m, M = extrema(fps)
               μ = round(Int, median(fps))
               print(m, \",\", μ, \",\", M)
               """`, String)
    min, median, max = Base.parse.(Int, split(txt, ','))
    push!(df, (; version, min, median, max))
  catch ex
    min = median = max = missing
    push!(df, (; version, min, median, max))
  end
end

sort!(df, :version)

I should add that I tried the same thing on my own PC (not an RPI) and the result was that all the versions had a median of 10 FPS -- surprisingly low rates as well.