AI Routes

Overview

An AI Route defines how Contextual’s AI Gateway selects an AI provider and model for an AI request (for example, when using the AI Generate node). AI Routes let you:

• Define a primary provider/model and one or more fallbacks

• Retry the same provider before failing over

• Keep flows provider-agnostic by switching routes instead of refactoring nodes

AI Routes reference AI Connections (Connections configured with an AI Provider).

Creating an AI Route

1. In your tenant workspace, go to Components → AI Routes

2. Click Create

3. Configure the General fields: ID, Name, and Description

4. Configure the Details fields:

   • Same-Provider Retries

   • Model Providers (one or more provider/model entries, in priority order)

5. Save

Configuration

Same-Provider Retries

The number of retry attempts to make against the currently selected provider/model before moving on to the next provider in the route. Set to 0 to disable retries.

Model Providers

An ordered list of provider/model pairs. Each entry contains:

• AI Provider Connection - Selects a Connection that has an AI Provider configured (an AI Connection). Only AI Connections appear in this picker.

• Model - The model identifier to use for that provider.

  • The UI can suggest known models for the selected provider and may show capability indicators (for example, Reasoning and Tool Calls).

  • You can also type any model name manually.

  • Azure OpenAI supports custom deployment/model aliases, so model identifiers may not be auto-verified.

Order matters: the first entry is the primary provider. If a request fails (and retries are exhausted), the next entry is attempted.

Example Route (UI)

• ID: default-text

• Name: Default Text Route

• Description: Primary OpenAI with Anthropic fallback

• Same-Provider Retries: 1

• Model Providers:

  • OpenAI connection: openai-prod using model gpt-x.x

  • Anthropic connection: anthropic-prod using model claude-x-x.x

Using an AI Route

• In a Flow, select the route in the AI Generate node. Advanced flows can also set it dynamically via msg or environment variables.

• AI Routes are versioned components and can be packaged into Services like other Components, enabling consistent promotion across environments.

Related

• AI Connections

• AI Generate