A cross-lingual encoder maps text from multiple human languages into a shared embedding space, enabling direct comparison across languages. COMET uses XLM-RoBERTa to independently encode the source, candidate, and reference, producing contextual embeddings for each token. section 3.3.3
Within COMET’s architecture, the encoder:
- Processes source, candidate, and reference inputs separately. section 3.3.3
- Produces contextual token embeddings for each input. section 3.3.3
- Feeds multilingual representations into pooling and task-specific prediction components. section 3.3.3
Figure 3.14 Left. Estimator model architecture. The candidate, source, and reference are independently encoded using a pretrained cross-lingual encoder. The resulting embeddings are then passed through a pooling layer to create a sentence embedding for each segment. Finally, the resulting sentence embeddings are concatenated into a vector to be passed to a feed-forward regressor. Right. Ranking model architecture. It uses source, reference, a ‘better’ candidate, and a ‘worse’ candidate. They are encoded, pooled, and minimize triplet margin loss to minimize the distance between ‘better’ candidates and the ‘anchors’.
The architecture figure shows how independently encoded source, candidate, and reference representations become sentence embeddings before regression or ranking. section 3.3.3
