Run Isaac ROS Foundationpose on custom dataset with ground truth segmentations

[athnzc]

Hello! I've been trying to run Foundationpose for Isaac following the tutorial for Isaac 3.1 on a custom dataset, using the Isaac Docker image with tag aarch64-ros2_humble on a Jetson with Jetpack 6.2. I was able to run the examples using the rosbag successfully after converting the score, refine models to TensorRT engines but I am having trouble running the node for my dataset. More specifically, I am running the commands below (they are all executed in the same docker container spawned with ${ISAAC_ROS_WS}/src/isaac_ros_common/scripts/run_dev.sh) :

• Run realsense camera: ros2 launch realsense2_camera rs_launch.py rgb_camera.profile:=640x480x30 depth_module.profile:=640x480x30
• Run foundationpose ros node: ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose.launch.py refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan mesh_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/IndustryShapes/textured_models/obj_000001/obj_000001.obj texture_path:=${ISAAC_ROS_WS}/isaac_ros_assets/IndustryShapes/textured_models/obj_000001/texture.png launch_rviz:=true
• Run a script to remap camera_info topic /camera/color/camera_info to /pose_estimation/camera_info.
• Run a script to stream static images to the topics that foundationpose subscribes to, according to the documentation. More specifically I publish the RGB to pose_estimation/image, the depth image to pose_estimation/depth_image, and the ground truth mask to pose_estimation/segmentation (so I am not currently using DETR or any other detector/segmentation model).

Also the origin frame of the 3D mesh of my object is at the center of the mesh, as instructed. And I can see that the rgb, depth, segmentation topics are published correctly, as their visualizations on rviz are what I expect. However, I am not getting any pose (or error for that matter), on rviz or in any of the output topics (/output, pose_estimation/pose_matrix_output). Could anyone please indicate what I could be missing?