time to first token
how long an AI model takes before it starts writing an answer
Before answering, a model must read the whole request, including any history and documents. Time to first token measures that wait. It grows with the length of the request, so it matters most for long conversations and AI agents. After the first word, writing speed is measured separately, in tokens a second.
Think of it as
The pause while someone reads your email before they start typing a reply.
Ideas that use this term