ProtGPT2 is a generative protein model for de novo design: it produces new amino-acid sequences rather than predicting properties of an existing sequence. Its 738-million-parameter GPT-2 decoder generates sequences autoregressively, in one direction. section 13.2.1
ProtGPT2 complements ESM-2, whose bidirectional architecture supports prediction tasks. Experimental validation studies found that ProtGPT2-generated sequences could fold into stable structures despite low similarity to known proteins; this does not establish that every generated sequence will do so. section 13.2.1
