Kimi API Context Caching: Hits, Cost & Latency

Kimi API context caching reused the full 937-token prompt on both identical-prefix requests in our four-call control. An early five-character prefix mutation dropped the reported cache count back to zero, and restoring the original prefix restored the 937-token hit.

That is only half the result. All four responses failed the exact-answer check. Each kimi-k2.6 call used its entire 96-token completion allowance, attributed 95 tokens to reasoning, ended with finish_reason: "length" and returned no visible answer content.

Direct result: Cached prompt tokens were 0 → 937 → 0 → 937. The exact answer was 0/4. This run demonstrates a controlled token-reuse pattern, not successful task completion, a universal hit rate or a latency guarantee.

The four-call result

Frozen case	Prefix	Prompt tokens	Cached tokens	First SSE	First answer	Wall time	Cost	Exact answer
CACHE-01-FIXED-FIRST	ALPHA	937	0	871.382 ms	Not observed	3,109.826 ms	$0.00127415	Fail
CACHE-02-FIXED-IDENTICAL	ALPHA	937	937	925.083 ms	Not observed	2,718.110 ms	$0.00053392	Fail
CACHE-03-MUTATED	BRAVO	939	0	1,007.989 ms	Not observed	3,248.632 ms	$0.00127605	Fail
CACHE-04-FIXED-RESTORED	ALPHA	937	937	867.055 ms	Not observed	2,896.182 ms	$0.00053392	Fail

Every transport returned HTTP 200 text/event-stream, 98 JSON events, [DONE], complete usage and zero parse errors. “First answer” is absent because no visible content delta arrived; reasoning events are not counted as user-visible answer text.

What the official Kimi documentation says

Kimi’s current context-caching guide says caching is automatic and a reusable prefix must exceed 256 prompt tokens. It also documents an optional prompt_cache_key path.

Our run tested automatic prefix matching only. It did not send prompt_cache_key. The reported cached-token counts, timing and costs below are independent observations from the retained responses rather than vendor performance claims.

How we controlled the prefix

The synthetic system prefix was 4,461 bytes and 694 whitespace-delimited units before the constant user question. Actual API usage confirmed 937 prompt tokens for the fixed form and 939 for the mutated form, both comfortably above the documented 256-token threshold.

Only one marker near the beginning changed:

Fixed:   Marker: ALPHA
Mutated: Marker: BRAVO

Both markers contain five ASCII characters. Every other prefix byte, the user question, model, 96-token output limit and streaming configuration remained unchanged. Calls ran sequentially in this frozen order with a 1.5-second local delay:

1. fixed prefix for the first request;
2. the identical fixed prefix;
3. one early mutated prefix; and
4. the original fixed prefix restored.

The provider reported two more prompt tokens for the BRAVO form despite the equal byte length. That is an observed tokenization difference, not a fixture-length change.

What the cache pattern supports

The first fixed request reported zero cached tokens. The immediate identical request reported all 937 prompt tokens as cached. Changing the early marker produced zero cached tokens. Restoring the original exact prefix again produced all 937 prompt tokens as cached.

That controlled sequence supports these narrow statements:

• the account and endpoint exposed automatic cached-token accounting in returned usage;
• the exact fixed prefix was reusable in both later fixed calls;
• this early mutation prevented a reported hit in its one call; and
• restoring the exact original prefix recovered the full reported hit in its one call.

It does not establish how long an entry remains cached, whether another account or region would share the pattern, whether every early mutation causes a miss or whether a longer common prefix could still produce a partial hit.

Cost changed; answer quality did not improve

The frozen K2.6 rate snapshot used $0.95 per million uncached input tokens, $0.16 per million cached input tokens and $4.00 per million completion tokens. Returned usage produced:

first fixed    = (937 × $0.95 + 96 × $4.00) / 1,000,000
               = $0.00127415

identical hit  = (937 × $0.16 + 96 × $4.00) / 1,000,000
               = $0.00053392

The identical cached request cost 58.096% less than the first fixed request under that dated formula. The restored fixed request had the same token counts and calculated cost. Total usage-derived cost for all four calls was $0.00361804.

The completion charge remained in every request because each call used 96 completion tokens. A full prompt-token hit therefore did not reduce total cost to zero. More importantly, both cached and uncached requests returned empty visible content, so the lower calculated cost did not translate into a correct answer.

Use the Kimi API pricing guide and API cost calculator for general rate and arithmetic intent. This page owns the controlled cache observation, not the site’s broad pricing query.

Latency did not show one clean cache effect

Compared with the first fixed call, the identical hit had a 12.596% shorter wall time but a 6.163% slower first SSE event. The restored hit had a 6.870% shorter wall time and a 0.497% faster first SSE event.

With one observation per condition, those mixed timings cannot isolate a cache-caused latency change. Network path, provider load, queueing, model reasoning and local measurement all remained uncontrolled. We report the numbers and do not turn them into “cache makes Kimi X% faster.”

Why the exact answer failed 0/4

The frozen prompt asked for only CACHE-ANSWER-4821. Each response instead ended at the completion limit with:

• 96 completion tokens;
• 95 reasoning tokens;
• zero visible content characters;
• finish_reason: "length"; and
• no first visible answer delta.

Reasoning text was removed from public evidence, while its token and character counts were retained. The cache mechanism can reuse prompt tokens even when the model does not finish the requested answer. Applications must therefore validate output independently of cache accounting.

The protocol prohibited increasing the output cap after seeing a failure. A larger limit or different supported model configuration may produce another result, but that follow-up was not run and is not implied here.

How to design a cacheable request

Kimi performs the cache decision automatically, but request layout still determines which tokens form the reusable prefix.

1. Put stable system instructions, shared reference text and unchanged tool definitions first.
2. Put task-specific or user-specific content after the stable prefix where the application allows it.
3. Keep serialization deterministic: field order, whitespace, tool order and text normalization can change the byte sequence you send.
4. Confirm actual prompt_tokens exceeds the documented cache threshold.
5. Read cached_tokens or prompt_tokens_details.cached_tokens from returned usage instead of assuming a hit.
6. Record whether prompt_cache_key was used; this test did not use one.
7. Validate finish reason, non-empty answer and task oracle separately from cache state.
8. Treat cache behavior as an optimization. The request still needs to work correctly on a miss.

Only send content you are authorized to submit to the API. A cache optimization is not a privacy permission or a reason to include unnecessary sensitive material.

Minimal usage accounting

function summarizeKimiUsage(usage, rates) {
  const prompt = Number(usage?.prompt_tokens);
  const completion = Number(usage?.completion_tokens);
  const cached = Number(
    usage?.cached_tokens ??
    usage?.prompt_tokens_details?.cached_tokens ??
    0
  );

  if (![prompt, completion, cached].every(Number.isFinite)) {
    throw new Error("Missing usage fields");
  }
  if (cached < 0 || cached > prompt) {
    throw new Error("Invalid cached-token count");
  }

  const uncached = prompt - cached;
  const costUsd = (
    cached * rates.cachedInput +
    uncached * rates.uncachedInput +
    completion * rates.output
  ) / 1_000_000;

  return { prompt, cached, uncached, completion, costUsd };
}

Use a dated rate source and preserve the returned model ID. Do not reuse one model’s prices for another, and do not call a local calculation a provider invoice.

Common context-caching mistakes

Mistake	What goes wrong	Better evidence
Assuming the second request is a hit	Cache availability and matching are not guaranteed	Read the returned cached-token count
Changing an early instruction	Much or all of the intended prefix can stop matching	Freeze and hash the stable prefix
Comparing different tasks as latency controls	Output length and reasoning confound timing	Keep model, prompt, cap and order controlled, then repeat enough times
Calling a hit a correct answer	Token reuse and answer validity are separate	Score finish reason, content and oracle independently
Reporting input savings as total savings	Completion tokens can dominate a short request	Calculate cached input, uncached input and output separately
Hiding a miss or failed answer	The result becomes cherry-picked	Retain the frozen order and every single-pass outcome
Treating the cache as permanent storage	Cache lifetime and policy are not established by one run	Build correctness without depending on persistence

Download the controlled evidence bundle

Download the Kimi API Context Caching Evidence Bundle. It contains the frozen protocol, exact prefix template and hashes, all four sanitized requests and responses, exact-answer scores, redacted SSE transcripts, timestamped event JSONL, the track-only spend ledger, cost code, offline checks and a SHA-256 manifest.

Package SHA-256: B98F51F2B382BD1E3B6CEC15618A8ED64BAE85BB7DA7894C4722AEE403A0F05C Package size: 43,489 bytes Files: 35, including the manifest

The manifest independently rechecked 34 covered files with zero hash or byte-count mismatch. Both dependency-free offline evidence tests passed. The ZIP contains no backslash entry, no detected API-key pattern, no private account preflight and no returned reasoning text.

Methodology and limits

• Protocol version 1.0 was frozen before execution.
• Run ID: kimi-api-reliability-lab-20260807-v1.
• Official Global endpoint: https://api.moonshot.ai/v1.
• Requested and returned model: kimi-k2.6.
• Four calls ran sequentially with one attempt each and zero retries.
• No prompt_cache_key was sent.
• Prefix variants were hashed; only the one early five-character marker changed.
• Actual prompt counts exceeded 256 tokens in every response.
• Streaming was used only for consistent local timing capture.
• Inputs were synthetic and no uploaded file, production record or external action was involved.
• One call per condition cannot estimate hit probability, expiry or causal latency.
• All exact-answer checks failed at the frozen 96-token cap.
• Timing used one local client clock; network region and service load were not independently controlled.
• Cost uses returned token usage and a dated local price snapshot, not a tax invoice.

Read How We Test Kimi AI for preregistration and evidence labels. Report a correction through Sources & Corrections.

Frequently asked questions

Did Kimi cache the identical prompt?

The returned usage reported 937 cached tokens out of 937 prompt tokens on the identical request and again after the exact fixed prefix was restored. The first fixed and early-mutated calls each reported zero cached tokens.

Did the cache make the answer correct?

No. All four exact-answer checks failed because every response ended at the 96-token completion limit with empty visible content.

Did caching reduce the calculated cost?

Yes in this controlled token calculation. Each 937-token hit cost $0.00053392 versus $0.00127415 for the first fixed miss, 58.096% lower in total. The completion charge remained the same.

Did caching make the response faster?

The two hits had shorter wall times than the first fixed call, but first-SSE timing was mixed and each condition ran once. We cannot attribute a general latency improvement to caching from this sample.

Was a prompt cache key used?

No. This run tested automatic prefix matching only. If you test the optional key, disclose it as a separate configuration.

Why did BRAVO produce 939 prompt tokens when ALPHA produced 937?

The markers have equal byte length, but the API reported a two-token difference. Equal character counts do not guarantee equal tokenization.

Official sources

• Kimi API context-caching guide
• Kimi API models and exact model IDs
• Kimi API limits and billing eligibility

For setup and authentication, use the Kimi API guide. For the broader long-context K3 retrieval pilot, use the Kimi K3 Test. That page owns K3 model-task evidence; this page owns the controlled automatic-cache sequence.