Semantic segmentation gives image generation a compositional map instead of a rigid copy. It identifies what occupies each part of a scene—such as trees, sidewalks, and streets—so a ControlNet can preserve the arrangement while inventing the visual details. section 12.5 section 12.7
A practical semantic-segmentation workflow looks like this:
- Start with a source image whose broad composition you want to retain. section 12.5
- Run the image through the “OneFormer ADE20K Segmentor” preprocessing node. section 12.5.1
- Let the preprocessor assign colored regions to recognized object categories. section 12.5.1
- Feed that segmentation map into the semantic-segmentation ControlNet and generate with your chosen prompt. section 12.5.1
The useful trick is its level of abstraction. Scribble follows an outline closely; semantic segmentation records general regions and their meanings, not texture, appearance, or fine detail. The model therefore knows “trees go here” and “sidewalk goes there” while retaining room to create. section 12.5 section 12.5.1
Cherry-blossom street
Take a spring street photograph with cherry trees and a sidewalk. The segmentor maps the trees, street, and sidewalk into colored semantic regions. Generation can then preserve their relative placement while changing the scene’s style and details instead of reproducing the photograph exactly. section 12.5
