> For the complete documentation index, see [llms.txt](https://docs.contextual.io/documentation-and-resources/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api.md).

# AI Gateway API

Generate text and JSON from code through your tenant's AI Routes.

The AI Gateway API sends generation requests to AI providers through your tenant's [AI Routes](https://docs.contextual.io/documentation-and-resources/components-and-data/ai-routes.md). Use it from backend services and scripts. For example, a backend service can summarize a support case or extract invoice fields as JSON. A direct generation request names an AI Route. The route chooses the provider, model, and fallbacks, and its [AI Connections](https://docs.contextual.io/documentation-and-resources/components-and-data/connections/types-of-connections/ai-connections.md) hold the provider credentials. With a saved AI Call, the application supplies a Call reference and input values, and the Gateway uses the prompt and Route saved in the Call.

## What you can do

* Generate text from a prompt or from a conversation of messages.
* Generate a JSON object or array that matches a JSON Schema.
* Generate data for new Records of an Object Type, ready to send to the Native Object API.
* Include images and files in messages.
* Stream text and reasoning as the model produces them.
* Let the model call tools that your code runs, over HTTP or a WebSocket connection.
* Call an AI Route from a client that uses the OpenAI Chat Completions format.
* Send a request in the provider's own format to OpenAI Decisions or TypeSafe AI Jev through a saved AI Connection.
* List the supported providers and models, including whether a model supports reasoning and tool calls.

The reference also lists embedding and rerank operations, which [AI Search](https://docs.contextual.io/documentation-and-resources/components-and-data/ai-search.md) uses.

## Guides

* [Run a saved AI Call](https://docs.contextual.io/documentation-and-resources/components-and-data/ai-calls/run-from-code.md) to inspect its inputs, preview the rendered prompt, and generate through its Route.
* [Client Adapters](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api/client-adapters.md): Connect AI SDK or TanStack AI to the native Gateway API with Contextual's packages.
* [Stream Responses and Run Tools](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api/streaming-and-tool-calling.md): Stream text over HTTP, run your own tools over HTTP or the WebSocket, and use web search.
* [Generate Structured Output](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api/structured-output.md): Generate JSON from your own JSON Schema, or Record data for an Object Type.

## Base URL and reference

* **API base URL:** `https://ai-gateway.<tenant-id>.my.contextual.io/api/v1`
* **Swagger reference:** `https://ai-gateway.<tenant-id>.my.contextual.io/api-docs/`
* **Raw OpenAPI:** `https://ai-gateway.<tenant-id>.my.contextual.io/api-docs/openapi.json`

Replace `<tenant-id>` with the [tenant ID from your Tenant Workspace URL](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/authentication-and-permissions.md#identify-the-tenant). You can open the reference without signing in.

On the API base URL, the hostname identifies your tenant, so the `x-org-id` header is optional. This applies to HTTP requests and WebSocket connections. If you send the header, its value must match the hostname, or the API returns `400`. The `/info` operations, which list providers and models, do not use a tenant.

The OpenAPI document uses the relative server `/api/v1`. When you import it into an API client or AI tool, set the server to the API base URL.

## Authentication and permissions

See [Authentication and Permissions](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/authentication-and-permissions.md) for credential and token guidance.

Send an access token in the `Authorization: Bearer <access-token>` header.

| Task                                                                                      | Requirements                                                                                                                                                                                                                                                |
| ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| List providers and models (`GET /info/...`)                                               | An access token                                                                                                                                                                                                                                             |
| Generate (`POST /generate`, the WebSocket, or `POST /openai-compatible/chat/completions`) | An access token from a credential that belongs to the tenant, and authorization to use the AI Route and its referenced Connections                                                                                                                          |
| Run a saved Call (`POST /generate/call`)                                                  | Read access to the Call and access to its Route and Connections. See [Run AI Calls From Code](https://docs.contextual.io/documentation-and-resources/components-and-data/ai-calls/run-from-code.md#before-you-start) for label and Object Type permissions. |
| Call OpenAI Decisions or TypeSafe AI Jev (`POST /providers/...`)                          | An access token from a credential that belongs to the tenant, and authorization to use the AI Connection                                                                                                                                                    |

The API reads AI Routes and Connections from the [Native Object API](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/native-object-api.md) with your access token. For Object Type output, it also reads the Object Type from the [Native Object Registry API](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/native-object-registry-api.md) with your access token.

## Request and response details

* Send JSON bodies with `Content-Type: application/json`.
* A direct `POST /generate` request has a `route` set to an AI Route ID, and either `prompt` or `messages`. Add `system` for instructions. For JSON output, add `schema`, and set `output` to `array` if you want a list. Supported inputs depend on the provider and model.
* For Record data, set `objectType` to `{ "id": "<object-type-id>" }` instead of `schema`. The result leaves out `_metaData` and the fields that the Native Object API generates. To generate only some fields, add `fields` inside `objectType` with a list of field paths: `{ "id": "<object-type-id>", "fields": [["name"], ["address", "city"]] }`. The result then holds only those fields. See [Generate Structured Output](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api/structured-output.md).
* `maxTokens` is optional. If you omit it, the output limit depends on the provider and model. Set `maxTokens` to request an explicit output limit within the provider and model's supported limits.
* A response has `generationType` set to `text`, with the result in `text`, or `object`, with the result in `object`. While `finishReason` is `tool-calls`, the model waits for your tool results, and an `object` response has no `object` yet. Each response also reports token usage, the model and provider used, and failed fallback attempts. `steps` contains each full provider request and response, so responses can be large.
* For all request and response fields, see `POST /generate` in the [Swagger reference](#base-url-and-reference).
* If every provider in the route fails, the API returns `400` and lists each failure in `errors`. By then, the route's fallbacks and any retries that apply have already run. Configure fallbacks and retries in the [AI Route configuration](https://docs.contextual.io/documentation-and-resources/components-and-data/ai-routes.md#configuration). If your token cannot read the AI Route or a Connection, the API returns the Native Object API's status, such as `403` or `404`.
* To stream, send `Accept: text/event-stream` with `POST /generate`. The API sends text and reasoning as server-sent events while the model produces them. On success, the last event, `generate.response`, holds the complete response. If generation fails before the first event, you receive the usual error response. If it fails after events start, the stream ends with a `generate.error` event instead. JSON and Object Type output arrive whole. After output starts, the API no longer retries or falls back to another provider. Without this header, you receive one JSON response when generation ends. See [Stream text over HTTP](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api/streaming-and-tool-calling.md#stream-text-over-http).
* For tools that your code runs, describe them in `tools`. Over HTTP, when the model calls a tool, the response has `finishReason` set to `tool-calls` and the calls in `messages`. Run the tools. Then send the request again with your earlier messages, the response's `messages` unchanged, and a `tool` message with your results. Repeat until the model answers. The API keeps no conversation state between requests. See [Run client tools over HTTP](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api/streaming-and-tool-calling.md#run-client-tools-over-http).
* The WebSocket at `wss://ai-gateway.<tenant-id>.my.contextual.io/api/v1/generate/ws` runs the same tool loop on one connection. The API sends each tool call to your code and continues with your result. To stream text over the WebSocket, add `?stream=true` to the URL. Send the `Authorization` header on the upgrade request. The browser's built-in `WebSocket` API cannot set this header. See [Run tools over the WebSocket](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api/streaming-and-tool-calling.md#run-tools-over-the-websocket).

## Use an OpenAI Chat Completions client

Clients that use the OpenAI Chat Completions format can call an AI Route through `POST /openai-compatible/chat/completions`. Configure the client with these values:

* **Base URL:** `https://ai-gateway.<tenant-id>.my.contextual.io/api/v1/openai-compatible`
* **API key:** your access token. Access tokens expire, so give the client a current token for each request.
* **Model:** the AI Route ID.

The AI Route's Connections, retries, and fallbacks still apply. Set `stream: true` to stream text. When the model calls a tool, the response has `finish_reason` set to `tool_calls`, and you send your results back as `tool` messages. The endpoint returns `400` for parameters it cannot honor, such as `n` other than 1 or a `response_format` of type `json_object`. Object Type output, provider-executed tools such as web search, and Anthropic extended thinking with your own tools need the native Generate API, over HTTP (`POST /generate`) or the WebSocket. For the supported parameters, see the [Swagger reference](#base-url-and-reference).

For a client that uses this endpoint, see the [AI SDK OpenAI Compatible provider](https://ai-sdk.dev/providers/openai-compatible-providers) or the [TanStack AI OpenAI-compatible adapter](https://tanstack.com/ai/latest/docs/adapters/openai-compatible), using the Contextual settings above. Contextual also publishes [Client Adapters](https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api/client-adapters.md) for the native Generate API.

## Call OpenAI Decisions or TypeSafe AI Jev

`POST /providers/openai/decisions` and `POST /providers/typesafe/jev` send one request in the provider's own format and return the provider's successful JSON response unchanged. These endpoints use a saved AI Connection instead of an AI Route. In the body, set `providerConnectionId` to the ID of an AI Connection of the matching type, **OpenAI Decisions** or **TypeSafe AI Jev (direct)**, and set `request` to the provider request. If the connection type does not match the endpoint, the API returns `400`. These connection types work only with these endpoints, not in AI Routes.

Each request makes one provider attempt. The API does not retry or fall back to another provider. For the request and response format, see the [Swagger reference](#base-url-and-reference).

## Example request

Send this JSON body to `POST /generate` to generate text with the AI Route `<ai-route-id>`. Include `Authorization: Bearer <access-token>` and `Content-Type: application/json` headers.

```json
{
  "route": "<ai-route-id>",
  "system": "Answer in one sentence.",
  "prompt": "What is an AI Route?"
}
```

Replace `<access-token>` with your access token and `<ai-route-id>` with the **Route ID** of an AI Route under **Connect > AI Routes** in the Admin Console. A successful response has status `200`, `generationType` set to `text`, and the answer in `text`.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.contextual.io/documentation-and-resources/tenants/tenant-apis/ai-gateway-api.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
