A rendering engine generates 2D or 3D views of a simulated robot world from its scene graph and robot and object models. It turns the virtual environment into images for a scene viewer and simulated camera sensors, much like a graphics subsystem that feeds both a developer display and a robot perception pipeline. section 5.1
Rendering is separate from physics. The physics engine updates forces, motion, positions, and orientations; the rendering engine visualizes the resulting scene. A simulator may render several images at once because its GUI and each simulated camera can require different types, parameters, and rendering settings. section 5.1
Visual realism depends on a combined rendering pipeline:
- The rendering engine determines much of the simulator’s visual quality; more capable approaches, including ray tracing, can enable higher realism. section 5.3
- Object models matter equally: mesh resolution, texture detail, and material properties can limit realism even when the renderer is more capable. section 5.3
- Choose fidelity for the task. Simpler visuals are sufficient for many simulations, while vision-based machine-learning control may need high realism for simulated training to transfer to the real world. section 5.3
Virtual camera versus scene viewer
A robot camera can be configured with a horizontal field of view, 640×480 image size, near and far clipping distances, and a 30 Hz update rate. Its rendered image becomes a camera topic for robot software, while the GUI can show the complete simulated world. These views serve different consumers rather than presenting the same information. section 5.3 section 5.1
Figure 5.6 Simulators offer varying degrees of visual realism, with Isaac Sim (left) providing high-fidelity rendering that is favorable for training vision-based machine learning models, compared to Gazebo’s (right) simpler visual realism, which is often sufficient for many tasks.
Compare a higher-fidelity Isaac Sim image with Gazebo’s simpler rendering to see why renderer and model fidelity must be selected together. section 5.3
Rendering quality is therefore part of the sim-to-real problem: differences between rendered imagery and real camera data can contribute to a gap between virtual and physical operation. Visual fidelity should be matched to whether images are merely for inspection or are inputs to robot behavior. section 5.6 section 5.3

