A weighted average multiplies each feature representation hᵢ by an attention weight αᵢ, then sums the results: x̄ = Σᵢ αᵢ · hᵢ. In PyTorch, the weights must be reshaped to align with the feature tensor for pairwise multiplication. section 10.1.3
An attention-based implementation follows this path:
- Extract one representation hᵢ for each item with a feature-extraction or backbone network. section 10.1.3
- Multiply corresponding weights and representations, then sum across the item dimension to produce one combined representation. section 10.1.3
Combining a three-item bag
For a bag containing three digit images, the backbone produces three feature vectors and the attention network assigns each a score. The Combiner forms one representation from their weighted sum, which feeds the classifier.
