FiDO is a Fusion-in-Decoder optimization that targets the memory-bandwidth bottleneck created by repeated decoder cross-attention over concatenated passage representations. section 5.6.1
FiDO changes the baseline architecture in two coordinated ways:
- Sparsifies cross-attention by reducing decoder cross-attention layers and therefore accesses to the large context key-value state. section 5.6.1
- Reallocates parameters from the encoder to the decoder, using a smaller encoder for independent passage encoding and a larger decoder for local computation. section 5.6.1
- Moves inference closer to compute-bound behavior, with reported gains in speed and task performance. section 5.6.1
FiD versus FiDO
The comparison lists FiD-Large at 770M parameters, 51.4% NQ exact match, and roughly 0.3× relative speed, versus FiDO-Large-XXL at 1.5B parameters, 52.2% exact match, and roughly 1.2× speed. section 5.6.1
Table 5.3 Comparison of FiD and FiDO models
|
Model
|
Backbone Size
|
Encoder/Decoder Parameter Ratio
|
NQ EM (%)
|
Relative Inference Speed
|
|
FiD-Base
|
220M
|
~1:1
|
48.2
|
1.0x (Baseline)
|
|
FiD-Large
|
770M
|
~1:1
|
51.4
|
~0.3x
|
|
FiDO-Large-XXL
|
1.5B
|
~1:4 (Smaller Enc, Larger Dec)
|
52.2
|
~1.2x
|
Use the comparison table to inspect parameter allocation, exact-match performance, and relative inference speed across FiD variants. section 5.6.1
