Long context vs retrieval: choose how evidence reaches the model

Compare evidence delivery

Long context can hold a substantial collection directly in the request. Retrieval selects material before generation. Capacity alone does not establish that a model will notice the right passage, resolve contradictions or respect document permissions. Retrieval can also fail by omitting the decisive source.

Test the hard cases

Use questions whose answers sit in different document locations, require more than one source or are absent entirely. Measure answer support and total latency, including document preparation and search. Watch for outdated duplicates and instructions embedded inside untrusted documents.

Choose by evidence shape

A bounded document set may suit direct context. A large, changing collection may benefit from retrieval. Some applications combine both. Preserve source identifiers and make unsupported answers visible whichever delivery method you choose.

Sources & notes

An editorial decision framework, not a scored benchmark or hands-on test.

arxiv.org — research paper

developers.openai.com — official reference

Sources reviewed for the September 2026 launch edition.