The quick read
- An answer about a policy may become outdated when the document changes, even if the question stays identical.
- Update a source and confirm that the next request receives a fresh result.
Two layers of reuse
Prompt caching can reuse processing associated with repeated input. Application caching can reuse a previous result. These mechanisms are different, and neither should be assumed to apply simply because two requests look similar.
Define validity
An answer about a policy may become outdated when the document changes, even if the question stays identical. Results may depend on permissions, language, model version and other context. Include relevant conditions in the cache design and define a suitable expiry or invalidation rule.
Test the changed case
Update a source and confirm that the next request receives a fresh result. Verify that one user’s restricted answer cannot be served to another. For a document workflow, caching unchanged extraction may be easier to reason about than caching personalised advice. Monitor incorrect reuse as well as cache-hit rates. A high hit rate is useful only when the reused work remains appropriate. The aim is to remove genuine duplication while keeping the answer current, authorised and connected to its source.
Sources & notes
AI-assisted editorial content checked against the linked sources.
developers.openai.com — official reference
Sources reviewed for the September 2026 launch edition.