fps

[lizhenqi111]

Hello, author. Thank you for your work. I am currently trying to convert Camel into real-time online processing and have found that Camel is particularly sensitive to frame rate—higher frame rates significantly improve performance. Do you have any suggestions to reduce its sensitivity to frame rate, perhaps by incorporating Kalman filtering?

[VlSomers]

@lizhenqi111 thanks for you interest and by the way, feel free to open a pull request if you manage to implement something interesting for online real-time processing! Regarding your question, incorporating Kalman Filtering with some heuristic association rule would be against the philosophy of this work: we explicitly aim at removing heuristic from the MOT association module. Moreover, our temporal encoder (TE) on bbox sequences is explicitly designed to replace the Kalman Filter with a learned counterpart (Kalman Filter is a sort of fixed motion model, whereas our TE is a learned motion model). CAMELTrack do not perform well with lower framerate likely because videos with low framerate have not be seen at training time. What I would suggest is to implement a new data augmentation that would simulate various framerates to make CAMEL robust to any framerate, and then retrain the model. Let me know if this is helpful and if you need more guidance. Best

[lizhenqi111]

@VlSomers Hi, thank you for your prompt response. The data augmentation method you mentioned is a great approach, and I will give it a try.
I am a football sports enthusiast and want to use multiple cameras for full-scene football motion analysis. The dynamic fusion multi-cue capability of CamelTrack should be very helpful for occlusion scenarios. Currently, I have only done TensorRT inference conversion based on it and utilized pipeline parallel processing mechanisms to fully leverage memory and computing power, the detailed solution is in this repository link. The entire pipeline can achieve 16fps on a 4060 Ti. .
Additionally，there is another question: when the sim_threshold is set too low, such as 0.1, it may cause false matching between trajectories in the 'lost' state (due to targets going out of bounds) and newly entering targets. If set too high, it may fail to match trajectories and targets that should originally be matched. Is there significant room for optimization in metric learning? Do you have any good ideas in this regard?

Thank you again for your work.