InexactError when loading x265 video saved with lossy `monochrome8` profile

[BioTurboNick]

Video saved with

imgstack_low = reinterpret(Gray{N0f8}, UInt8.(imgstack .% 0xFF))
VideoIO.save(imgpath, eachslice(imgstack_low, dims = 3), framerate = 5, codec_name = "libx265", encoder_options = (var"x265-params" = "profile=monochrome8", preset = "medium"))

Trying to re-load:

ERROR: InexactError: trunc(UInt32, -1)
Stacktrace:
  [1] throw_inexacterror(f::Symbol, #unused#::Type{UInt32}, val::Int64)
    @ Core .\boot.jl:614
  [2] checked_trunc_uint
    @ .\boot.jl:644 [inlined]
  [3] toUInt32
    @ .\boot.jl:728 [inlined]
  [4] UInt32
    @ .\boot.jl:768 [inlined]
  [5] (::VideoIO.var"#9#10"{Tuple{Int64, Int64}, DataType, Tuple{Int64, Int64}, Int64, Int64})(x::UInt8)
    @ VideoIO C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\frame_graph.jl:187
  [6] scale_gray_frames!(f::VideoIO.var"#9#10"{Tuple{Int64, Int64}, DataType, Tuple{Int64, Int64}, Int64, Int64}, dstframe::VideoIO.NestedCStruct{VideoIO.libffmpeg.AVFrame}, #unused#::Type{UInt8}, #unused#::Type{UInt8}, srcframe::VideoIO.NestedCStruct{VideoIO.libffmpeg.AVFrame})
    @ VideoIO C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\frame_graph.jl:169
  [7] exec!(s::VideoIO.GrayTransform)
    @ VideoIO C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\frame_graph.jl:200
  [8] execute_graph!
    @ C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:502 [inlined]
  [9] _retrieve!(r::VideoIO.VideoReader{true, VideoIO.GrayTransform, String}, buf::PermutedDimsArray{Gray{N0f8}, 2, (2, 1), (2, 1), Matrix{Gray{N0f8}}})
    @ VideoIO C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:513
 [10] retrieve(r::VideoIO.VideoReader{true, VideoIO.GrayTransform, String})
    @ VideoIO C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:551
 [11] read
    @ C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:653 [inlined]
 [12] iterate
    @ C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:103 [inlined]
 [13] grow_to!(dest::Vector{PermutedDimsArray{Gray{N0f8}, 2, (2, 1), (2, 1), Matrix{Gray{N0f8}}}}, itr::VideoIO.VideoReader{true, VideoIO.GrayTransform, String}, st::Int64)
    @ Base .\array.jl:882
 [14] grow_to!(dest::Vector{Any}, itr::VideoIO.VideoReader{true, VideoIO.GrayTransform, String})   
    @ Base .\array.jl:864
 [15] _collect
    @ .\array.jl:764 [inlined]
 [16] collect
    @ .\array.jl:712 [inlined]
 [17] (::VideoIO.var"#13#14")(io::VideoIO.VideoReader{true, VideoIO.GrayTransform, String})        
    @ VideoIO C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:95
 [18] openvideo(f::VideoIO.var"#13#14", args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ VideoIO C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:641
 [19] openvideo
    @ C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:638 [inlined]
 [20] #load#12
    @ C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:94 [inlined]
 [21] load(::String)
    @ VideoIO C:\Users\nicho\.julia\packages\VideoIO\V24Bg\src\avio.jl:93
 [22] top-level scope
    @ REPL[279]:1

[IanButterworth]

If you have fixes in mind for this or the other similar issue, VideoIO could do with more developers/maintainers :)

[BioTurboNick]

I'm poking at it, but video encoding/decoding is a bit of a dark art. Happy to do some things, but unsure if I'll be doing enough to take that on fully.

I think perhaps the issue is that the encoder wrote in the file that it was tv-scale 16-235 values, but what was saved was full range. So when it tries to convert the source range to the destination range, it subtracts e.g. 16 from 13 and tries to convert that to a UInt.

I can think of a couple options that would help:

• If the encoder is set to a limited color range, the output should be truncated to that range, possibly emitting a warning
• If the decoder detects a limited color range, the input should be truncated to that range, possibly emitting a warning

[galenlynch]

Looks like a problem with the logic of scale_gray_frames!. I can try to look at it this weekend if no one gets around to it by then.

I think right now VideoIO assumes that users are not familiar with color spaces and that N0f8 or UInt8 are full range, but since most decoders in the field seem to expect tv-scale 16-235 values, VideoIO will try to convert to that range. That was true of x264 videos, but I'm less familiar with what x265 players "in the wild" can handle. The user can avoid this color space conversion by providing mpeg (tv) color range inputs and setting input_colorspace_details to VideoIO.VioColorspaceDetails() . Alternatively, the conversion can be avoided by encoding in full color range which with the x264 codec was possible by setting encoder_options with color_range=2. Under the hood, VideoIO is just comparing the color space of the CodecContext constructed by ffmpeg to the color space of the transfer frame used by VideoIO.

The conversion of scale_gray_frames! done by VideoIO could definitely be improved. In fact, the whole frame_graph.jl file is pretty under developed. You can create very complicated graphs with ffmpeg, which right now can not be done through VideoIO. I couldn't get sws_scale to work that well for gray color space conversion, which is why scale_gray_frames! exists.