Prompt caching
context caching, cache, cached prefix
In one sentence
Letting the provider keep its work on the unchanged start of a prompt, such as instructions and documents, so repeated requests only process what is new, cheaper and faster.
Why it matters
A model re-reads everything from the very top on every turn, so a long, unchanging start is paid for again and again. Caching is like a bookmark: the provider keeps its work on the part that has not changed and only reads on from where the new text begins. It only works while the start is identical, so what never changes (instructions, documents) goes first and what varies goes last. Change one word at the top and the bookmark is lost. The bookmark is also temporary: minutes, not days.
Sources
Back to all termsReady to make an impact?
Turning complex ideas into clear, user-centered product experiences.