Prompt caching

Learn how OpenGateway prompt caching reuses repeated prompt prefixes automatically, how long the cache is kept, and how to structure requests for more cache hits.

What is prompt caching?#

Prompt caching lets a model reuse the beginning of a prompt it has already processed, such as system instructions, tool definitions, and earlier conversation turns. It applies automatically; you do not need to add any request field. Cached input tokens use the cached-input price shown on /models.

Cache lifetime#

The prompt cache TTL (time to live) is 5 minutes.

Getting more cache hits#

  • Put content that stays the same, such as system instructions and tool definitions, at the start of the request.
  • Put content that changes, such as timestamps, user input, and retrieved documents, after that fixed part.
  • Keep the order of messages and tools, and the way you serialize JSON, identical between requests.

Checking cache usage#

The usage dashboard shows cache-read tokens separately from other input tokens, so you can see how often your requests hit the cache.

Prompt caching — OpenGateway