🧑‍🌾 `tf2_ros_message_filter.multiple_frames_and_time_tolerance_deprecated` timing out flaky in Rolling/Kilted Connext

[Crola1702]

Generated by Generative AI

No response

Operating System:

Ubuntu Noble

ROS version or commit hash:

rolling

RMW implementation (if applicable):

rmw_connextdds

RMW Configuration (if applicable):

No response

Client library (if applicable):

No response

'ros2 doctor --report' output

No response

Steps to reproduce issue

1. Run a build in Rci__nightly-connext_ubuntu_noble_amd64
2. See test regression fail

Expected behavior

Test should pass

Actual behavior

Test is failing with a timeout

Additional information

Reference build:

• Rci__nightly-connext_ubuntu_noble_amd64#590

Test regressions:

• tf2_ros.tf2_ros_test_message_filter.gtest.missing_result
• projectroot.tf2_ros_test_message_filter

Timeout of 60 seconds:

8: [ RUN      ] tf2_ros_message_filter.multiple_frames_and_time_tolerance_deprecated
8: [INFO] [1762949909.211736259] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949910.211721214] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949911.211722059] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949912.211723885] [tf2_ros_message_filter]: filter callback: trigger(0)
8: RTI Connext DDS Non-commercial license is for academic, research, evaluation and personal use only. USE FOR COMMERCIAL PURPOSES IS PROHIBITED. See RTI_LICENSE.TXT for terms. Download free tools at rti.com/ncl. License issued to Non-Commercial User license@rti.com For non-production use only.
8: Expires on 00-jan-00
8: Please contact support@rti.com with any questions or comments.
8: [INFO] [1762949913.211719291] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949914.211723566] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949915.211721880] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949916.211720056] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949917.211720960] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949918.211723606] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949919.211722852] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949919.212014769] [tf2_ros_message_filter]: Message Filter dropping message: frame 'base' at time 1762949908.212 for reason 'discarding message because the queue is full'
8: [INFO] [1762949920.211721248] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949920.212044006] [tf2_ros_message_filter]: Message Filter dropping message: frame 'base' at time 1762949908.212 for reason 'discarding message because the queue is full'
8: [INFO] [1762949921.211718664] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949921.212039702] [tf2_ros_message_filter]: Message Filter dropping message: frame 'base' at time 1762949908.212 for reason 'discarding message because the queue is full'
8: [INFO] [1762949922.211719071] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949922.212017588] [tf2_ros_message_filter]: Message Filter dropping message: frame 'base' at time 1762949908.212 for reason 'discarding message because the queue is full'
8: [INFO] [1762949923.211721179] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949923.212005745] [tf2_ros_message_filter]: Message Filter dropping message: frame 'base' at time 1762949908.212 for reason 'discarding message because the queue is full'
8: [INFO] [1762949924.211722626] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949924.212027733] [tf2_ros_message_filter]: Message Filter dropping message: frame 'base' at time 1762949908.212 for reason 'discarding message because the queue is full'
8: [INFO] [1762949925.211721053] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949925.212040781] [tf2_ros_message_filter]: Message Filter dropping message: frame 'base' at time 1762949908.212 for reason 'discarding message because the queue is full'

... [A lot of these messages]

 8/14 Test  #8: tf2_ros_test_message_filter .................***Timeout  60.00 sec

If it passes it takes 10 seconds and doesn't show a lot of message filters:

8: [ RUN      ] tf2_ros_message_filter.multiple_frames_and_time_tolerance_deprecated
8: [INFO] [1762949981.499742354] [tf2_ros_message_filter]: filter callback: trigger(0)
8: [INFO] [1762949982.499760089] [tf2_ros_message_filter]: filter callback: trigger(1)
8: [INFO] [1762949983.499728804] [tf2_ros_message_filter]: filter callback: trigger(1)
8: [INFO] [1762949984.499729381] [tf2_ros_message_filter]: filter callback: trigger(2)
8: RTI Connext DDS Non-commercial license is for academic, research, evaluation and personal use only. USE FOR COMMERCIAL PURPOSES IS PROHIBITED. See RTI_LICENSE.TXT for terms. Download free tools at rti.com/ncl. License issued to Non-Commercial User license@rti.com For non-production use only.
8: Expires on 00-jan-00
8: Please contact support@rti.com with any questions or comments.
8: [INFO] [1762949985.499726436] [tf2_ros_message_filter]: filter callback: trigger(3)
8: [INFO] [1762949986.499729960] [tf2_ros_message_filter]: filter callback: trigger(4)
8: [INFO] [1762949987.499726953] [tf2_ros_message_filter]: filter callback: trigger(5)
8: [INFO] [1762949988.499752507] [tf2_ros_message_filter]: filter callback: trigger(6)
8: [       OK ] tf2_ros_message_filter.multiple_frames_and_time_tolerance_deprecated (8216 ms)

...

1/1 Test #8: tf2_ros_test_message_filter ......   Passed   10.42 sec

Is worth noting that this test fails 45% of the times, but given the fact that we run --retest-until-pass 2 it appears that it failes less time that it actually fails.

[kscottz]

We discussed this in Waffle. We're not sure this belongs in Kilted as the underlying code is deprecated.

[miguelgonrod]

Updating this issue with new flakyness

Reference builds

• https://build.ros2.org/job/Rci__nightly-connext_ubuntu_noble_amd64/718
• Kci__nightly-connext_ubuntu_noble_amd64#104

Flakyness

job_name	last_fail	first_fail	build_count	failure_count	failure_percentage
Rci__nightly-connext_ubuntu_noble_amd64	2026-03-19	2026-03-05	15	8	53.33
Kci__nightly-connext_ubuntu_noble_amd64	2026-03-19	2026-03-10	5	3	60.0

[miguelgonrod]

This issue is still happening, investigation in https://github.com/osrf/buildfarm-tools/issues/235#issuecomment-4170279256

Flakyness 15 days

job_name	last_fail	first_fail	build_count	failure_count	failure_percentage
Kci__nightly-connext_ubuntu_noble_amd64	2026-04-04	2026-03-25	4	4	100.0
Rci__nightly-connext_ubuntu_noble_amd64	2026-03-30	2026-03-27	9	2	22.22

[miguelgonrod]

Reopening this issue as this is still happening with same output. But I have to say this is happening flakier so it in fact did improved

Reference Build

• Kci__nightly-connext_ubuntu_noble_amd64#121

flakyness 15 days

job_name	last_fail	first_fail	build_count	failure_count	failure_percentage
Kci__nightly-connext_ubuntu_noble_amd64	2026-05-10	2026-05-10	5	1	20.0

Since Kci__nightly-connext_ubuntu_noble_amd64#112 it stop happening, but it is showing up again