Lower FPS Compared to OmniIsaacGymEnvs

[kojikoshi-m]

Hello,
I have transitioned from OmniIsaacGymEnvs, but I am facing an issue with computation time.
Despite using the exact same USD, task settings, and learning methods, the FPS is only about 1/3 or 1/4 of what it was with OmniIsaacGymEnvs.
Since the robot has two cameras, I expected it to be faster with TiledCamera, but the opposite happened.
It doesn’t seem to be a GPU memory issue; rather, the physical simulation processing simply appears to be slow.

I suspect it is related to the physics simulation settings, but could you advise on what I should check?

[kojikoshi-m]

I conducted a comparison of computation times between OmniIsaacGymEnvs and Isaac Lab using my custom task and the CartpoleCamera task. The CartpoleCamera task was configured with the same number of parallel environments and other settings to make the comparison as fair as possible. Below are the results highlighting the points I found noteworthy. Additionally, I have attached all the profiling data for your reference.

While the number of render calls differs significantly between the two environments, making a direct comparison difficult, it appears that rendering-related processes are slower in my custom task with Isaac Lab compared to OmniIsaacGymEnvs. This might be expected, but the camera positioning operations required for multi-camera setups also seem to be consuming considerable time.

For the Cartpole task, rendering has also become slower. However, the overall simulation speed has improved significantly in Isaac Lab, resulting in faster performance. Specifically, CartpoleCamera shows better FPS with Isaac Lab.

I would appreciate your suggestions on how to speed up my custom task compared to OmniIsaacGymEnvs.

custom task at Isaac lab

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1927   53.024    0.028   55.109    0.029 /home/sfai/conda_ws/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sim/simulation_context.py:458(render)
     2561    4.666    0.002   16.983    0.007 /home/sfai/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.isaac.core/omni/isaac/core/simulation_context/simulation_context.py:662(step)
     2560    0.010    0.000   12.346    0.005 /home/sfai/conda_ws/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sim/simulation_context.py:431(step)
     1282    0.033    0.000   32.669    0.025 /home/sfai/conda_ws/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/camera.py:281(set_world_poses)
     1293    1.313    0.001   29.585    0.023 /home/sfai/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.isaac.core/omni/isaac/core/prims/xform_prim_view.py:837(set_world_poses)
      641    0.271    0.000   94.185    0.147 /home/sfai/conda_ws/IsaacLab/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/fork_cam_tile/fork_cam_tile_env.py:596(_get_observations)

custom task at OmniIsaacGymEnvs

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        4   12.654    3.163   12.985    3.246 simulation_context.py:469(render)
     2565   18.827    0.007   39.997    0.016 simulation_context.py:440(step)
     2436   18.202    0.007   18.316    0.008 physics_context.py:569(_step)
      641    0.260    0.000   21.082    0.033 fork.py:693(get_observations)

CartpoleCamera at Isaac lab

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    1285    9.603    0.007   10.634    0.008 /home/sfai/conda_ws/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sim/simulation_context.py:458(render)
     2561    0.947    0.000    8.855    0.003 /home/sfai/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.isaac.core/omni/isaac/core/simulation_context/simulation_context.py:662(step)
     2560    0.011    0.000    7.941    0.003 /home/sfai/conda_ws/IsaacLab/source/extensions/omni.isaac.lab/omni/isaac/lab/sim/simulation_context.py:431(step)
     1281    0.086    0.000    2.689    0.002 /home/sfai/conda_ws/IsaacLab/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/cartpole/cartpole_camera_env.py:141(_get_observations)

CartpoleCamera at OmniIsaacGymEnvs

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        4    3.071    0.768    5.523    1.381 simulation_context.py:469(render)
     2563   65.814    0.026   86.769    0.034 simulation_context.py:440(step)
     1281    3.150    0.002    3.270    0.003 physics_context.py:569(_step)
     1281    0.143    0.000    2.764    0.002 cartpole_camera.py:115(get_observations)

profile_results_Cartpole_lab.txt
profile_results_Cartpole_OmniIsaac.txt
profile_results_custom_lab.txt
profile_results_custom_OmniIsaac.txt

[RandomOakForest]

Thanks for posting this. Are you using Isaac Lab 1.4.0? There shouldn't be big differences between Gym and Lab performances.

[kojikoshi-m]

Thank you for your response. I updated to Isaac Lab 1.4.0, but the significant FPS difference compared to OmniIsaacGymEnvs has not improved much.

Based on my measurements, Isaac Lab appears to have faster physical simulation performance, but it seems slower in rendering compared to OmniIsaacGymEnvs. I hope that tweaking some settings might lead to improvements.

Additionally, the unique setup of my task may be influencing these results:

• A mobile robot equipped with two cameras.
• Observations include two images from both cameras.
• In OmniIsaacGymEnvs, the robot was equipped with two cameras, and images were obtained directly from them.
• In Isaac Lab, the cameras were removed from the robot, and a TiledCamera was created.
• The camera is moved every step based on the robot's coordinates.
• Since the cameras move, it is necessary to update their positions twice per step, which requires calling 'self.sim.render()' twice.

Let me know if you have any suggestions or insights!