The geometric inconsistency problem is the difficulty text-to-image models have maintaining coherent spatial relationships among objects, depths, viewpoints, and perspectives. These models learn associations between pixels and text, but generally lack an internal representation of three-dimensional structure and physical rules. section 1.5.2 section 13.2
The inconsistency develops through several connected limitations:
- Training primarily on large, diverse collections of two-dimensional images does not provide a deep understanding of 3D geometry or spatial relationships. section 1.5.2
- Transformer positional encodings represent spatial relationships, but do not reliably enforce strict geometric constraints across complex objects. section 1.5.2
- Missing robust depth and perspective mechanisms can produce distortions, physically impossible objects, and inconsistent appearances from different angles. section 13.2
A viewpoint-dependent distortion
A generated object may appear plausible in one image but become visibly distorted when viewed from another angle, because the model does not consistently apply 3D geometry, perspective, lighting, or physics. section 1.5.2
Improving geometric consistency likely requires explicit 3D structure, physics simulations, or other mechanisms that enforce spatial coherence rather than relying only on learned two-dimensional patterns. section 1.5.2 section 13.2
