Tokens per second measures how quickly a generative model emits output after generation begins. In streaming chat, it captures the pace users see once the first token arrives; it complements time to first token and time to final token, which mark the start and completion. section 4.2.3
Treat it as one part of system performance, not a stand-alone quality signal. Increasing batch size may improve throughput while worsening latency for individual users, so assess the trade-off alongside model quality and product impact. section 4.2.4 section 4.6































