Error codes

Every error, and what to do about it.

Errors come back in the chat completions error format your SDK already understands. Each one names the field, says who must act, and links here.

Request id: x-request-id, quote it when you report a problem
No retry: x-should-retry: false when a retry cannot help
Endpoint: x-served-by once an endpoint was chosen

01Client errors

Client errors.

Status	Code	Cause	Fix
400	`invalid_request`	Body is not JSON, not an object, `model` / `messages` is missing or the wrong type, or `stream` is not a boolean.	Send a chat-completions body with a string `model`, an array `messages` and, if set, a boolean `stream`.
401	`invalid_api_key`	No `Authorization: Bearer <key>` header, or the key is not in the config.	Use a key whose sha256 is in `[[keys]]` (`dwaar-gateway hash-key`).
404	`model_not_found`	The model has no routable endpoint visible to this key (unknown, not yet routable, or excluded by the key's policy). The message suggests close ids.	Use an id from `GET /v1/models`. Ids look like `<creator>/<model>`.
400	`invalid_provider_preferences`	The request's `provider` object has an unknown field, a wrong type, an unknown provider id, an empty `only`, or a list over 32 entries. `param` names the field.	Fix the `provider` object; see docs/registry.md#residency-policy.
400	`unsupported_parameter`	A request field dwaar does not forward (it moves data or enables provider-side tools), any field not listed in `registry/params.toml` (including vendor extensions sent via `extra_body`), message `content` that is neither a string nor an array of parts, a message key outside `role`, `content`, `name`, `tool_calls`, `tool_call_id`, `refusal`, `annotations`, `reasoning_content` and `reasoning`, more than 128 tools, or a capability that no endpoint passing the residency policy has (e.g. `response_format.json_schema`). `param` names the field. A JSON `null` counts as unset and is never an error.	Remove the field, or ask the operator to enable the capability on an endpoint (docs/registry.md#parameters).
404	`unknown_route`	The path does not exist.	The gateway serves `/v1/chat/completions`, `/v1/models`, `/v1/receipts/{id}`, `/v1/receipts/keys` and `/v1/registry/snapshots/{sha256}`.
405	`method_not_allowed`	The path exists but not for this method. The `Allow` header lists the methods it takes.	`POST /v1/chat/completions`; `GET` for `/v1/models`, `/v1/receipts/{id}`, `/v1/receipts/keys` and `/v1/registry/snapshots/{sha256}`.
408	`request_timeout`	The request body did not arrive within 30 s.	Check the client's network; send the body promptly.
413	`request_too_large`	The body is over 20 MiB.	Send smaller requests (large images count).

02Upstream and capacity errors

Upstream and capacity errors.

Status	Code	Cause	Fix
502	`upstream_unreachable`	Could not connect, the upstream redirected (redirects are never followed), or it closed a stream early.	Check the upstream's `base_url` and status.
502	`upstream_auth_failed`	The upstream rejected the gateway's credentials (its 401, or any 403 except a region or content-policy refusal, which reaches the client unchanged). Your client key is fine.	Operator: check the key in the upstream's `api_key_env` and its `auth_header`.
502	`upstream_response_too_large`	A non-streaming upstream response exceeded 64 MiB.	Ask for less output or stream.
502	`upstream_bad_response`	The upstream returned something unparseable, a 2xx body without `choices`, or JSON for a streaming request (or SSE for a non-streaming one). A 2xx body that is an error object is passed on as a 502 with the upstream's message.	Operator: the upstream does not speak the chat completions format for this request.
503	`upstream_not_configured`	Every endpoint for the model has its key variable unset.	Operator: set the variable named by the endpoint's `api_key_env` and restart. The startup log names it.
503	`no_india_provider`	The model exists, but the request's preferences (with the key's policy) excluded every endpoint. The message counts each rule, e.g. `2 offered; 1 excluded by zdr=true (request), 1 excluded by only (request)`; endpoints the key's own policy hides are not counted. If an endpoint passed every rule but lacked a needed capability, the answer is 400 `unsupported_parameter` instead. Not retried.	Relax the request's `provider` preferences (they cannot relax the key's policy), or pick another model.
502	`egress_denied`	The egress proxy refused to open a tunnel to the endpoint's host (its reason is in the message: `unlisted`, `port`, `non_global_address`, …). Nothing was sent. The gateway tries the next candidate once before answering this, unless the request sets `provider.allow_fallbacks: false`; `x-served-by` names the last candidate tried. Usually the gateway's and the proxy's registries differ (the host was removed and the gateway not yet restarted). Not retried.	Operator: deploy the same snapshot to proxies and gateways (proxies first when adding hosts; see docs/deploy/README.md).
502	`egress_unavailable`	The gateway could not reach the egress proxy, or the proxy dropped the connection (often a restart or rollout). Nothing was sent. Retryable.	Retry with backoff. Operator: check `egress_proxy`, the proxy's health, and the network path to it.
502	`egress_auth_failed`	The egress proxy rejected the gateway's credentials (407). Nothing was sent. Not retried.	Operator: `egress_proxy_secret_env` must hold the same secret as the proxy's `secret_env` (both sides trim whitespace).
503	`egress_overloaded`	The egress proxy is at its tunnel cap. Nothing was sent, and no other candidate is tried (it shares the proxy). Sent with `retry-after: 1`. Retryable.	Retry with backoff; operator: raise the proxy's `max_connections` (see docs/deploy/README.md#sizing).
502	`upstream_not_allowed`	The outbound URL or model failed the gateway's residency check, so nothing was sent. This is a gateway bug. Not retried.	Report `x-request-id` to the operator.
503	`overloaded`	`max_concurrent_requests` requests are already in flight, or 512 MiB of request bodies are being received at once. On `GET /v1/receipts/{id}` and `GET /v1/registry/snapshots/{sha256}`: the gateway-wide `receipt_fetch_global_per_sec` is used up, or the store's lookup slots stayed busy.	Retry with backoff, or raise the limit.
in-band	`shutting_down`	Streaming only: the gateway's shutdown grace period ended while the stream was open. With receipts on, the stream's receipt comment precedes it only if the receipt was ready within 0.5 s of the end of the grace period; otherwise `: dwaar-receipt-status unavailable` does (that receipt is stored, then voided).	Retry; another instance will serve it.
504	`upstream_timeout`	The upstream did not answer within `request_timeout_secs` (non-streaming) or `first_byte_timeout_secs` (streaming).	Raise the timeout for slow models, or retry.
upstream's	upstream's	Any other non-2xx from the upstream. The message is prefixed with the upstream id; `type`, `code` and `param` are kept; `retry-after` is kept on 429/503.	Depends on the upstream's message.

03Receipts

Receipts.

Status	Code	Cause	Fix
503	`receipt_unavailable`	The key has `receipt_policy = "required"` and its receipt could not be signed or committed (the message names the stage: `sign`, `finalize`, `store`, `commit` or `shutdown`). When the commit failed or timed out the gateway also writes a void record for the id (not when the store's error backoff refused the commit, which wrote nothing), so the receipt is never served by `GET` and verifiers report it `voided` even if its line commits later (after a restart, only if the void itself committed; a void that did not is `stage="void"`). The upstream may have processed the request, so the response carries `x-should-retry: false` and `x-request-id` (no receipt id): retrying may issue another billable request. Streams end with one in-band error event with this code and no `[DONE]`; a stream that had already failed upstream keeps its single error event, preceded by `: dwaar-receipt-status unavailable`. When the store cannot take receipts at all, the request is refused before dispatch with `receipt_store_unavailable` instead (below).	Do not retry blindly. Contact the operator with the request id. Operator: see the runbook in docs/deploy/README.md#receipts-runbook (signer down, disk full, corrupt segment).
503	`receipt_store_unavailable`	The key has `receipt_policy = "required"` and this replica's receipt store cannot take its receipt now (full or within 4 KB of `receipt_store_max_bytes`, counting the room reserved for `required` requests in flight; its commit queue full; backing off after disk errors; its commits stalled on a hung disk; or closed), so the request was refused before anything was sent upstream: nothing was billed and no receipt exists. The message says so. Sent with `retry-after: 1` and `x-request-id`; no `x-should-retry: false`, no `x-served-by`, no receipt headers. Streams get the same 503 before they start. Counted in `dwaar_receipt_refusals_total{reason}` (each retry counts), not as a receipt failure. While the state persists (not for a full queue alone) the replica also fails `/readyz`, so a retry usually reaches another one.	Retry with backoff (the SDKs do). If it persists, contact the operator with the request id. Operator: see `/readyz: receipt_store_unavailable` in the runbook.
404	`receipt_not_found`	`GET /v1/receipts/{id}` found nothing this credential may read on the replica that answered. The causes are not told apart: another replica served the request (fetch is per replica until slice 5), the receipt is not committed yet, it is past retention, it belongs to another key or to an earlier secret of this key, the id is malformed, receipts are off, or the store could not read it (logged at ERROR). Sent with `Retry-After: 1` when the id is under 5 s old.	Keep the in-band receipt (the primary record). Retry once after a second for a fresh id; otherwise ask the replica that served it, or the operator.
429	`receipt_fetch_rate_limited`	More than 10 receipt or snapshot lookups per second for this key. Sent with `Retry-After: 1`. (Past the gateway-wide `receipt_fetch_global_per_sec`, every key gets 503 `overloaded`.)	Slow down; batch audits should read the operator's store or in-band captures.
404	`snapshot_not_found`	`GET /v1/registry/snapshots/{sha256}`: this replica has not archived a signed snapshot with that `content_sha256` (dev registries are never archived).	Use the snapshot your operator published for that registry version.

04Receipts at startup

Receipts at startup.

The gateway refuses to start with one of these, naming the problem and the fix. They are for whoever runs the gateway.

Code	Cause	Fix
`receipt_key_reuses_registry_key`	A receipt key (active, published or retired) has the same public key or key id as a `registry_public_keys` entry.	Create a separate key: `dwaar-receipt keygen --id <new id> --out <file>`.
`receipt_state_dir_missing`	`receipt_signing` is set but `state_dir` is not, or `state_dir` (the receipt store, `instance_id`, the snapshot archive) cannot be created or written.	Set `state_dir` to a writable directory on a persistent per-instance volume.
`receipt_store_locked`	Another process (its pid is named) holds the lock on `state_dir/receipts/<instance_id>`.	Stop it, or give each instance its own `state_dir` (a StatefulSet with a per-pod volume); never share one between replicas.
`receipt_keys_invalid`	A key id is not `[A-Za-z0-9._-]{1,64}`, a public key does not parse, a window is invalid (`not_after` not after `not_before`, a retired key without `not_after`, the active key's `not_before` later than now, with no allowance: a receipt signed before it would fail its own window), a public key is a small-order or non-canonical Ed25519 encoding, an id or key appears twice, or there is not exactly one active key. Windows may overlap during a rotation.	Fix the entry; `dwaar-receipt keygen` (and `--retire`) print correct ones.
`receipt_policy_without_signer`	`receipt_policy = "required"` (per key or as `receipt_policy_default`) without `receipt_signing`: every such request would fail.	Set `receipt_signing`, or use `best_effort`.
`receipt_size_infeasible`	The largest receipt the field limits allow (model ids 128 bytes, ids 64, host 253, region codes 32) plus every droppable parameter in the registry's `params.toml` exceeds 2600 bytes.	Shorten the stripped and capability field lists, or the registry version.
`receipt_signer_unavailable`	The key file in `receipt_signing.path` is missing, unreadable, malformed, holds a different key id, or other users can read or write it (any world permission bit; group write only logs a warning, group read is fine for a secret mounted 0440).	Point `path` at a `dwaar-receipt keygen` file readable by the gateway, with the configured `key_id`, mode 0600 (or 0440/0400 from a secret).

05Streaming

Streaming.

A stream that fails after it started ends with one in-band event and no [DONE], which SDKs raise as an APIError while iterating:

data: {"error":{"message":"sarvam-default: upstream timed out","type":"api_error","param":null,"code":"upstream_timeout"}}

upstream_timeout means no bytes for idle_timeout_secs, or the stream ran past request_timeout_secs; upstream_unreachable means the upstream closed before finishing, sent an event over 1 MiB, or broke the connection. An upstream's own in-band error is passed through unchanged. Comment lines (: ping) every 15 s keep idle streams alive; SDKs ignore them.

06Config

Config.

dwaar-gateway check-config and startup report config problems as file:line:column: field: problem. A TOML syntax error or unknown field is reported alone (fix it and rerun); everything else is reported together. A missing file says which variable or path to set.

07Registry at startup

Registry at startup.

The gateway refuses to start, naming the file and the next command, when:

the signature fails: the snapshot is not signed by a key in registry_public_keys (add the key, or re-sign);
the format is unsupported: the snapshot's format_version is newer than this release (upgrade, or rebuild);
the payload breaks registry rules: a signed payload is re-checked like dwaar-registry lint (https, deny list, regions, ids); one that fails was not produced by build (rebuild and re-sign);
the snapshot is older: its serial is below the one recorded in state_dir, or equal to it with different content (publish a new snapshot; rolling back means re-publishing old content at a higher serial);
the rollback floor is unreadable or corrupt: fix the permissions on <state_dir>/registry-floor.json, or start once with --reset-state-floor (logged at error level; the snapshot you start with becomes the new floor);
the snapshot has expired: run dwaar-registry build and sign, or start once with --accept-expired-until <date> (at most 7 days ahead);
a key policy names an unknown provider or data centre (check-config reports it too): fix only, ignore or data_center in that key's policy (a typo in ignore would otherwise leave the provider allowed);
egress is not contained: every egress_canaries address completes a TLS handshake without the proxy (put the gateway in a network that can reach only the proxy; see docs/deploy/README.md), or registry_snapshot is set without egress_proxy or egress_canaries;
the registry is older than the proxy's: the proxy's admin /serial is newer than the gateway's snapshot (deploy the newer snapshot to the gateway);
the mode doesn't match: registry_dir without --dev-registry, --dev-registry with registry_snapshot, or --dev-registry on a non-loopback listener.

08Dev key

Dev key.

dwaar.example.toml ships the hash of the documented token dwaar-dev-key. With it configured the gateway logs a warning, and refuses to start on a non-loopback address unless --allow-dev-key is passed, because anyone who has read the README could spend your provider keys. Replace it:

printf %s 'a-long-random-token' | dwaar-gateway hash-key