> For the complete documentation index, see [llms.txt](https://docs.contextual.io/documentation-and-resources/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.contextual.io/documentation-and-resources/components-and-data/ai-routes.md).

# AI Routes

### Overview

An AI Route defines how Contextual’s AI Gateway selects an AI provider and model for an AI request (for example, when using the AI Generate node). AI Routes let you:

* Define a primary provider/model and one or more fallbacks
* Retry the same provider before failing over
* Switch providers and models through route configuration

AI Routes reference **AI Connections** (Connections configured with an AI Provider).

### Creating an AI Route

1. In your tenant workspace, go to **Components → AI Routes**
2. Click **Create**
3. Configure the **General** fields: ID, Name, and Description
4. Configure the **Details** fields:
   * **Same-Provider Retries**
   * **Model Providers** (one or more provider/model entries, in priority order)
5. Save

### Configuration

#### Same-Provider Retries

The number of retry attempts to make against the currently selected provider/model before moving on to the next provider in the route. Set to `0` to disable retries.

#### Model Providers

An ordered list of provider/model pairs. Each entry contains:

* **AI Provider Connection** - Selects a Connection that has an AI Provider configured (an AI Connection). Only AI Connections appear in this picker.
* **Model** - The model identifier to use for that provider.
  * The UI can suggest known models for the selected provider and may show capability indicators (for example, Reasoning and Tool Calls).
  * You can also type any model name manually.
  * Azure OpenAI supports custom deployment/model aliases, so model identifiers may not be auto-verified.

Use an **OpenAI Compatible** Connection for a provider that AI Gateway does not offer directly and that implements the OpenAI Chat Completions API. In the AI Route's **Details** section, each **Model Providers** entry has an **AI Provider Connection** field and a **Model** field. Select your OpenAI Compatible Connection, then enter the exact model ID supplied by the provider in **Model**, including any prefix such as `organization/`. Contextual passes the ID unchanged. These connections have no shared model catalog or catalog-based capability information.

**OpenAI Compatible** Connections still use the route's retry behavior. Their calls appear in AI Gateway metrics, but estimated cost is unavailable because Contextual has no catalog or pricing data for custom model IDs.

When changing providers or configuring fallbacks, verify that each provider and model supports the inputs and features your flow uses.

Order matters: the first entry is the primary provider. If a request fails (and retries are exhausted), the next entry is attempted.

#### Example Route (UI)

* **ID**: default-text
* **Name**: Default Text Route
* **Description**: Primary OpenAI with Anthropic fallback
* **Same-Provider Retries**: 1
* **Model Providers**:
  * OpenAI connection: `openai-prod` using model `gpt-x.x`
  * Anthropic connection: `anthropic-prod` using model `claude-x-x.x`

#### Example Route for a Compatible Endpoint

After creating an [OpenAI Compatible Connection](/documentation-and-resources/components-and-data/connections/types-of-connections/ai-connections.md#openai-compatible), configure a route with:

* **ID**: custom-text
* **Name**: Custom Text Route
* **Same-Provider Retries**: 1
* **Model Providers**: Connection `custom-inference` using model `organization/custom-model`

The Connection ID and model ID are illustrative. Select your saved Connection and replace the model ID with the exact value supplied by your provider.

### Using an AI Route

* In a Flow, select the route in the AI Generate node. Advanced flows can also set it dynamically via `msg` or environment variables.
* AI Routes are versioned components and can be packaged into Services like other Components, enabling consistent promotion across environments.

## Related

* [AI Connections](/documentation-and-resources/components-and-data/connections/types-of-connections/ai-connections.md)
* [AI Generate](/documentation-and-resources/components-and-data/flows/node-reference/ai-gateway/ai-generate.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.contextual.io/documentation-and-resources/components-and-data/ai-routes.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.