I'm not sure that the measurements provided by the depth sensor are correct.

[arambarricalvoj]

Hello,

I am trying to send the measurements provided by the depth sensor via ROS2. I am using the default RealSense D455 in Isaac Sim. I have read about the Little-Endian encoding used by Isaac Sim. Therefore, bytes read as: “128 127 0 0” correspond to (float32) “0x00 0x00 0x7F 0x80”, i.e., NaN (Not a Number) in IEEE‑754.

However, when the object is quite close to the sensor, it detects it correctly. The object is only one unit away from the camera on the Y axis, as shown in the images below, where you can also see the different sensor values on the right terminal. Furthermore, if I make the object larger, the depth sensor detects it even when it is further away, which I do not understand why it depends on the scale (obviously if the object is very small it will not detect it if it is very far away, but this is not the case).

This sensor is supposed to detect obstacles up to 6 meters away (https://docs.isaacsim.omniverse.nvidia.com/5.1.0/assets/usd_assets_camera_depth_sensors.html).

What is going on?

[zwdoescode]

Real stereo depth cameras (and Isaac Sim's simulation of them) compute depth from disparity — the pixel offset between left and right views. The fundamental relationship is:

  depth = (focal_length × baseline) / disparity                                                           

At greater distances, the disparity shrinks. When an object is both small and far, it occupies very few pixels in the rendered image. The post-processing pipeline can't compute a reliable disparity for it — the object doesn't span enough pixels to produce a meaningful stereo match. The pipeline correctly returns invalid depth rather than an unreliable measurement. A larger object occupies more pixels → more pixel area for the disparity computation → stereo matching succeeds → you get depth values instead of NaN.

If you need ground-truth depth without stereo limitations, use the distance_to_image_plane annotator on a plain camera instead of the D455's depth pipeline.

[arambarricalvoj]

Thanks! But I'm not sure the problem I'm describing is caused by what you've described. I'm attaching a video screenshot showing what's happening, because, in my opinion, the object isn't far enough away. However, it could be that since not all the image data is printed, it appears that only NaN is detected, so I'll accept your answer as valid.
camera_performance.webm