LLM providers, models and tasks¶

AIHub routes language-model work through semantic profiles. Business workflows always ask for a capability such as default, tiny, mini, large or reasoning; tenant and provider configuration select the concrete model. Some workflows require a particular provider API, but their model still comes from that provider's configured semantic profile.

This page covers generation and reasoning, embeddings, and Realtime and speech. These model families have independent selectors; see configuration for the complete tenant configuration.

How AIHub selects a model¶

The effective provider and model are resolved in this order:

1. A provider requirement made by the workflow, when its API is required.
2. The tenant's selected provider, if it exists in the database catalogue and supports the requested model family.
3. The database default provider, then the built-in provider if that default is absent or invalid. The built-in providers are OpenAI for LLMs and Azure for embeddings. Empty MongoDB and PostgreSQL catalogues follow the same rule.
4. A valid tenant model/profile reference within the selected provider's catalogue, then that provider's database profile/default model.
5. The centralized model fallback for the selected provider and model family.

Omitted, empty and unknown provider identifiers follow the same default chain. Model aliases, concrete names and inline model configurations must reference a model declared in the selected provider catalogue. Valid model settings and reasoning overrides are preserved. If one profile is missing, only unresolved profiles use code defaults; other valid database profiles remain active. Database connection failures propagate and do not trigger a provider switch.

When switching providers, AIHub uses the selected provider catalogue and never reuses the original provider's tenant model overrides. Model identifiers belong in configuration, centralized defaults, pricing and capability metadata; business calls cannot override them. AGENTS.md enforces this development rule.

The provider configuration is therefore authoritative during normal operation. The fallback model table below is a runtime safety net, not a fixed production contract. To change a tenant model without changing application code, update the provider catalogue or the tenant's sparse profile overrides as described in Tenant AI configuration.

Supported LLM providers¶

All four providers use the shared AIHub call boundary for task naming, token policies, structured-output handling, usage normalization and observability. Provider-side prompt caching is currently activated only for eligible OpenAI and Anthropic calls. See LLM runtime, caching and observability for the technical policy.

Anthropic API compatibility¶

The pinned Anthropic SDK uses API version 2026-10-21. Keep using ANTHROPIC_API_KEY; the SDK sends the key as a Bearer credential. AIHub's Messages, streaming, token-counting and Message Batch integrations use this version without an override to the earlier API.

AIHub does not send temperature, top_p or top_k to Anthropic; the current API rejects those parameters. Input documents belong in user messages, and non-empty text citations belong in assistant messages. Native assistant content returned for tool continuations must still be preserved. See Anthropic's API migration guide for the upstream contract changes.

Live configuration from the database¶

The live view automatically calls public GET /genai/api/v1/catalogue on this instance. MongoDB reads common_data.AIProviders; PostgreSQL reads aihub_ai_providers in the deployment schema. The existing configuration resolver applies shared provider mappings and profile inheritance, without reading tenant or application configuration overrides. Database changes appear on the next refresh; no documentation rebuild or application restart is needed.

The Source column identifies the winning layer, even when a database mapping happens to have the same model name as the code fallback. Inherited profiles also name their immediate parent. The fallback columns come directly from the current code and do not replace a successfully resolved database mapping. Profile reasoning is reported before workflow policies and explicit request overrides; Unspecified does not mean reasoning is disabled.

The model list reports configured generation models and the capabilities exposed by GenAI. GenAI profile available additionally requires the resolved model to be declared in the provider catalogue. A shared-connector fallback can therefore be shown while remaining unavailable through GenAI. Neither listing probes remote provider credentials or model access. Embedding, speech and Realtime selectors remain documented separately below and are not part of this generation catalogue.

No API key or Client-ID is requested or sent by this view. Refresh configuration reloads shared provider values. Failed loads show an unavailable state rather than presenting static fallbacks as live data. Generation and tenant-aware /models remain authenticated. The static tables remain available without JavaScript or a running API. No database configuration is embedded in the generated documentation.

The public read reuses the existing small provider-catalogue scans and safe model/profile projection. It creates no database indexes, performs no LLM calls and returns no credentials, connection settings or tenant configuration.

Provider retries¶

OpenAI, Azure OpenAI, Anthropic and OVH share a maximum of three physical attempts per provider request: the initial request plus two SDK retries. The policy lives in src/common/connectors/ai_providers/retry_policy.py and also applies to GenAI, native Decisions, document parsing, embeddings, speech, Anthropic token counting and batch API requests. Application retry decorators must not wrap these SDK retries.

The SDKs retry transient connection/timeouts and HTTP 408, 409, 429 and 5xx responses, subject to provider retry-control headers. Backoff is exponential (normally 0.5 then 1 second, with jitter); supported Retry-After hints take precedence. Authentication, permission and invalid-input errors normally fail immediately. Existing operation deadlines and cancellation still take precedence: three is an upper bound, not a guarantee that every attempt can run.

Retries repeat only the failed provider request. An embedding chunk or batch status query does not restart the entire embedding job or submit a new batch. Batch creation is a separate request; if the provider accepts it but its response is lost, retries cannot guarantee that the remote job was created only once.

Streaming retries cover request opening, not replay after a stream has been returned. Live has the same three-attempt limit for its initial WebSocket handshake within one opening deadline. The pinned SDK does not retry that initial handshake, so AIHub owns that small loop. Session reconnection, audio replay and re-execution of delegated business actions remain disabled.

The evaluation meter permits the same SDK retries, reserves budget before each physical attempt and preserves failed-attempt diagnostics and uncertain costs. See AI evaluations for accounting and observation limits.

Semantic profiles and last-resort models¶

The static table shows the code fallback, with the ordinary shared-connector reasoning policy where applicable. It is not the effective database configuration; use the live view above for this instance.

default handles general generation, tiny minimal transformations, mini short classification or normalization, large complex generation and specialist agents, and reasoning deliberate analysis.

Reasoning precedence: an explicit external GenAI request takes priority, then an explicit tenant profile setting, then the same profile in AIProviders, then the centralized semantic default. The defaults are tiny=none, mini=low, default=low, large=medium and reasoning=medium; they do not replace explicitly configured database values. Model and reasoning settings are resolved separately: changing only the tenant's model retains a configured provider effort, and a tenant can override only the effort.

Workflows select a profile instead of supplying their own reasoning level. Summaries use mini and its effort; language detection and translation use tiny. Lexi keeps its own output limits, while its reasoning comes from its selected profile. PDF and DOCX image extraction use OpenAI large, and CV parsing uses mini. No connector injects an unrelated low or high default.

On the current tiny models, none disables the dedicated thinking phase; Azure GPT-4.1 nano has no reasoning control, so AIHub omits it. OVH's current large Llama model also receives no reasoning control. This does not remove the model's ability to understand or classify an input. low allows limited reasoning, with potentially more tokens and latency; it is not a fixed token budget or a guarantee of better answers. Model capabilities still apply to configured overrides. AIHub does not lower the effort to make tools work.

Native OpenAI Decisions requests do not send reasoning_effort. These profile settings affect generation and the structured decision path, not native Decisions. GenAI Chat Completions retains its model-specific tool restrictions: when tools require none, explicitly supply that override instead of inheriting mini=low. See GenAI capability limits.

Anthropic uses output_config.effort for supported models, including Opus in normal, streaming and batch requests. Profile-based calls receive the resolved semantic effort even when the database omits it. GenAI callers selecting a concrete model without a profile or an explicit effort keep the provider's API default instead: Sonnet 5.5 defaults to high, and Haiku 5.5 to medium. For Sonnet 5.5, none maps to thinking.type=between_tools, disabling up-front thinking while allowing thinking between tools. For Haiku 5.5, it maps to thinking.type=disabled. Opus 5.5 does not support disabled thinking; its reasoning profile uses medium by default. Other supported efforts are forwarded unchanged. GenAI always gives explicit request controls priority over profile defaults. The PostgreSQL seed matches these fallback models and profile reasoning settings for all four providers. Anthropic uses Sonnet 5.5 for large, Haiku 5.5 for default/tiny/mini, and Opus 5.5 for reasoning. Seed changes apply to new provider rows; existing provider configurations and tenant overrides are not rewritten.

If a tenant or provider profile resolves successfully, its model replaces the corresponding fallback in this table. An unspecified tenant profile inherits the provider mapping. If the provider document itself has no tiny mapping, AIHub injects the provider-specific tiny fallback above rather than inheriting a larger profile. Missing model mappings for mini and large inherit default, while reasoning inherits large. Their effort still follows the requested semantic profile, not the inherited model's original profile.

Align existing provider defaults with mini¶

New provider seeds and centralized fallbacks use the same model and settings for default and mini. Existing database configurations remain authoritative until explicitly migrated. Preview the migration for the backend in use:

.venv/bin/python -m scripts.migrate_default_llm_profiles --backend mongodb
.venv/bin/python -m scripts.migrate_default_llm_profiles --backend postgres

After reviewing the preview, rerun the selected command with --apply to persist it. The tool copies each registered OpenAI, Azure, Anthropic and OVH provider's configured mini value into default, including reasoning and other settings. This deliberately replaces custom provider defaults. Tenant overrides, other profiles, model catalogues, embedding and audio configuration are preserved. No inference requests are made. Missing or invalid mini mappings abort validation before any writes; already aligned providers are unchanged.

PostgreSQL applies updates in one transaction with row locks. MongoDB guards each update against concurrent changes, but the full multi-provider operation is not atomic: a conflict can leave earlier providers updated. Resolve the conflict and rerun the idempotent migration.

GPT-6 defaults and compatibility¶

The OpenAI fallback profiles and PostgreSQL AI-provider seed use GPT-6.1 Terra for large, GPT-6.1 Sol for reasoning, and Luna for default/tiny/mini. The seed inserts missing providers only; it does not update existing provider rows or tenant overrides. Update those explicit selections separately when rolling out these defaults to an existing database.

Reasoning settings remain explicit: OpenAI calls and LangGraph agents use the shared low default when the selected profile does not configure an effort, the large and reasoning profiles use medium, and language detection, translation and conversation summaries retain none. GPT-6.1 Sol and Astra do not accept none; an explicit incompatible override is rejected instead of silently remapped. Terra supports none, but its tool calls require Responses even at that effort.

OpenAI GPT-5.4, GPT-5.5, GPT-5.6 and GPT-6 tool calls and conversations containing tool results use the Responses API, preserving reasoning effort. Requests use store=False; LangGraph retains ordered output items, including encrypted reasoning, for subsequent calls to the same model. Function schemas keep their existing strictness, tool results keep their call IDs, and streaming responses retain normalized usage and costs. Calls without tools or tool history continue to use Chat Completions. Earlier GPT-5 models also retain Chat Completions for tools, with their configured effort unchanged; AIHub never replaces it with none. No temperature is sent for these reasoning models. Continuation provenance records the requested model separately from the returned snapshot so a dated response keeps its opaque context when the configured alias is unchanged. Explicit cache options are sent only to supported GPT-5.6/GPT-6 models; GPT-5.4/GPT-5.5 retain automatic caching.

Configured o1/o3/o4 names, including dated snapshots, are also recognized as reasoning models. OpenAI and Azure preserve their requested reasoning effort and omit temperature. Recognition does not change profile mappings or enable models that are absent from the configured catalogue; provider-specific effort limits still apply.

Standard USD list prices per million tokens:

Cache-write prices replace the uncached input price for those tokens. Above 272,000 input tokens, the entire request uses twice the input/cache rates and 1.5 times the output rate. Reasoning tokens are included in output usage. Batch pricing halves all categories; AIHub's existing batch execution path remains Anthropic-only. Configured prices and provider-reported costs retain their precedence. Azure uses its own deployment profiles; direct Azure calls use the centralized Azure fallback when no configuration is supplied. The prompt engine resolves the selected provider's configured profile.

Sources: GPT-6.1 Terra, GPT-6.1 Sol, GPT-6 migration, pricing, and Responses migration.

Migrate existing OpenAI large profiles¶

The new code fallback, PostgreSQL seed and supplied evaluation configurations select gpt-6.1-terra with reasoning_effort: medium for OpenAI large. Existing database mappings remain authoritative. Preview the targeted update for the backend in use:

.venv/bin/python -m scripts.migrate_openai_large_profile --backend mongodb
# Or:
.venv/bin/python -m scripts.migrate_openai_large_profile --backend postgres

After reviewing the preview, add --apply. The script changes only a shared OpenAI large profile explicitly resolving to GPT-6.1 Sol or Terra (including snapshots), selects gpt_6_1_terra, and sets effort to medium. Terra profiles already explicitly using medium are unchanged. Other model selections and missing large mappings are left alone. It preserves other profile options, the reasoning profile, model catalogue entries, tenant overrides, embeddings and audio configuration. Source-model prices are not copied to Terra. Invalid configuration or a conflicting target alias aborts before writing.

PostgreSQL locks and updates the provider row in one transaction using its existing provider primary key. MongoDB requires exactly one shared OpenAI provider and guards the _id update against concurrent catalogue/profile edits; a conflict requires a new preview. This small, one-time catalogue scan needs no additional index. No inference calls are made. Restart application workers after applying the migration to refresh their cached provider configuration.

Migrate existing OpenAI reasoning profiles¶

The code fallback and seed do not replace provider rows already stored in a database. Preview the targeted migration for the backend in use:

.venv/bin/python -m scripts.migrate_openai_reasoning_profile --backend mongodb
# Or:
.venv/bin/python -m scripts.migrate_openai_reasoning_profile --backend postgres

After reviewing the preview, add --apply. The script changes only an OpenAI provider reasoning profile explicitly resolving to GPT-6 Astra (including snapshots). It adds gpt_6_1_sol to the catalogue and preserves other profiles, existing models, tenant overrides, embedding and audio configuration. It retains supported reasoning settings, or supplies medium when none was configured. Custom models and already migrated profiles are unchanged. Invalid settings or an alias collision abort instead of overwriting configuration.

PostgreSQL locks and updates the provider row in one transaction. MongoDB checks that exactly one OpenAI provider exists and guards the update against concurrent catalogue/profile edits. A conflict aborts and requires a new preview. No schema or index changes are needed: PostgreSQL selects its provider primary key; this bounded, one-time MongoDB catalogue lookup needs no additional index and its write selects the existing _id index. No inference requests are made. Restart application workers after applying the migration so cached provider configuration is refreshed.

Anthropic image inputs¶

The shared message adapter translates OpenAI-style image_url user blocks into native Anthropic image blocks, preserving their order alongside text. It accepts Base64 PNG, JPEG, GIF and WebP data URLs, as well as HTTPS image URLs. Malformed encodings and unsupported blocks in an image-bearing message are rejected before the provider call with stable errors that do not expose the image data. This applies to CV parsing and Chatbot image extraction; the GenAI HTTP surface retains its existing supported input types.

Claude Haiku 5.5¶

Anthropic's active model claude-haiku-5-5, released on October 7, 2026, owns AIHub's default, tiny and mini Anthropic profiles. It supports JSON Schema, function tools (including forced choices), and efforts low, medium, high, xhigh and max. AIHub additionally accepts none to disable thinking. Sampling parameters are omitted. Assistant prefill is not supported by the model.

The API enables adaptive thinking at medium by default. Thinking consumes the existing output-token budget; AIHub does not silently enlarge workflow limits. The current tokenizer counts approximately 30% more tokens for the same text than previous Haiku tokenizers. Review task budgets using the resolved model's token counts when migrating a configured workflow.

For tool continuations, preserve the returned anthropic_content blocks exactly. The shared connector rejects direct document blocks in native assistant content before creating an SDK client, including for streaming and batches. The same validation runs at the GenAI boundary and returns a sanitized HTTP 422. Documents nested inside native tool results remain unchanged and accepted. AIHub carries them through LangGraph only for the originating provider and model. Replay them through the same Anthropic account and keep earlier instructions, tools and conversation turns unchanged. Summaries of reasoning are not generated or displayed by this extension.

USD prices per million tokens:

The threshold includes cached input tokens. Batch pricing applies a 50% discount. Configured prices and provider-reported costs retain their existing precedence.

New PostgreSQL installations use the current seed. Existing MongoDB/PostgreSQL catalogues require an explicit migration. Supply the exact source model and alias from the existing catalogue as SOURCE_MODEL and SOURCE_ALIAS, then preview:

.venv/bin/python -m scripts.migrate_anthropic_haiku55 --backend mongodb \
  --source-model "$SOURCE_MODEL" --source-alias "$SOURCE_ALIAS"
# Use --backend postgres for the configured PostgreSQL schema.

Add --apply to write the reviewed changes. The command replaces only exact source identifiers with claude-haiku-5-5 / claude_haiku_5_5, updates matching provider profiles and registered tenant selectors, preserves unrelated customizations and reasoning settings, and rejects conflicting aliases. MongoDB writes use guarded _id updates; PostgreSQL uses a transaction and existing primary keys. Provider and tenant configuration collections are small; no additional index is needed. Restart workers after application to refresh cached configuration. The command never makes inference requests.

Sources: model specifications, migration requirements, and pricing.

Azure OpenAI deployments¶

Azure OpenAI runs profile-driven generation against deployments in the Azure resource configured for the AIHub process. Select it with ai_configuration.llm_provider = "azure" in the tenant configuration. Workflows that explicitly require OpenAI or Anthropic retain that provider requirement.

Set the connection variables as follows:

AZURE_OPENAI_URL=https://example-resource.openai.azure.com
AZURE_OPENAI_API_KEY=<azure-resource-api-key>

AZURE_OPENAI_URL is the resource root without a trailing slash. AIHub appends /openai/v1/ and uses the asynchronous OpenAI client for Chat Completions. The current generation connector does not read a per-model api_version field.

In the Azure provider catalogue, each model's llm_name is the deployment name sent to Azure. The catalogue alias used by llm_profiles is an AIHub configuration key; it does not have to match the deployment name. For example, the seeded configuration uses these exact deployment names:

{
  "provider": "azure",
  "llm_models": {
    "gpt_5_4_mini": {"llm_name": "gpt-5.4-mini"},
    "gpt_5_2": {"llm_name": "gpt-5.2"},
    "gpt_4_1_nano": {"llm_name": "gpt-4.1-nano"}
  },
  "llm_profiles": {
    "default": {"model": "gpt_5_4_mini", "reasoning_effort": "low"},
    "tiny": "gpt_4_1_nano",
    "mini": {"model": "gpt_5_4_mini", "reasoning_effort": "low"},
    "large": {"model": "gpt_5_4_mini", "reasoning_effort": "medium"},
    "reasoning": {"model": "gpt_5_2", "reasoning_effort": "medium"}
  }
}

All three names were present as successful deployments in the configured resource when checked on 2026-09-24. Deployment names can differ from the underlying model: the deployed model must be checked as well as the name. An entry in /openai/v1/models is a catalogue entry, not proof that a deployment exists.

These deployments must already exist in the configured resource. AIHub does not provision them. Without a resolved catalogue, the fallback names in the table above are sent as deployment names, so they must exist under those exact names. The OpenAI GPT-6 defaults do not change Azure's mappings.

The connector forwards tool definitions, tool choice, JSON mode and explicit JSON-schema response formats. Supported combinations depend on the deployed model and its API. Shared parameter handling recognizes GPT-5 and later and o1/o3/o4 from the deployment-name prefix; retain that prefix when naming those deployments so AIHub can omit temperature and forward reasoning effort. GPT-4.1 nano receives temperature without a reasoning parameter.

Migrating existing Azure configuration¶

The initialization seed inserts missing providers only. Updating source defaults does not replace existing database profiles. Preview the dedicated migration for the configured backend, then apply the reviewed update:

.venv/bin/python -m scripts.migrate_azure_provider --backend postgres
.venv/bin/python -m scripts.migrate_azure_provider --backend postgres --apply
# For MongoDB, use --backend mongodb instead.

The script adds the required catalogue entries and migrates the old standard GPT-4.1 mini / GPT-4.1 / o3-mini profile mappings to the configuration above. It fills missing profiles and preserves custom model mappings, deployment names, catalogue metadata and explicit reasoning settings. An existing target alias bound to a different deployment, or a standard profile with a missing alias, requires manual review before any write. Existing catalogue choices are retained.

The migration updates the Azure provider only. Review tenant llm_profiles overrides separately; they remain authoritative. PostgreSQL changes use one transaction and a provider row lock; MongoDB writes require the inspected fields to remain unchanged. Restart AIHub after migration to refresh cached configuration. Embedding models, dimensions, indexes and stored vectors remain unchanged.

Azure generation currently uses non-streaming Chat Completions. AIHub does not route Azure calls through its OpenAI Responses adapter or Anthropic Message Batches path, and does not send explicit prompt-cache directives to Azure. Available usage, reasoning and cached-token counts are still normalized for observability.

Configure the model's optional pricing block with the applicable Azure rates for cost estimates. Provider-reported costs take priority; without configured rates, recognized model names can use the shared OpenAI list-price table. That fallback is not an Azure billing quote, and an unrecognized deployment name may have no built-in estimate.

Sources: Azure OpenAI v1 lifecycle, Azure model availability, GPT-5.4 mini capabilities, GPT-5.2 capabilities, and GPT-4.1 nano capabilities. Model capability references do not guarantee deployment availability or Azure pricing.

OVH AI Endpoints¶

OVH uses the shared OpenAI-compatible API at https://oai.endpoints.kepler.ai.cloud.ovh.net/v1 for both Chat Completions and embeddings. Set OVH_API_KEY in the AIHub environment and select ai_configuration.llm_provider = "ovh" and/or embedding_provider = "ovh". The catalogue fields llm_name and embedding_name contain model IDs, not endpoint URLs. The connector uses a fixed OVH host and rejects legacy URL values.

Generation profiles¶

The provider seed and centralized fallbacks use the following mapping, checked against the official catalogue and public /v1/models API on 2026-09-24:

{
  "provider": "ovh",
  "llm_models": {
    "llama_3_3_70b": {"llm_name": "Meta-Llama-3_3-70B-Instruct"},
    "qwen_3_5_9b": {"llm_name": "Qwen3.5-9B"},
    "gpt_oss_120b": {"llm_name": "gpt-oss-120b"}
  },
  "llm_profiles": {
    "default": {"model": "qwen_3_5_9b", "reasoning_effort": "low"},
    "tiny": {"model": "qwen_3_5_9b", "reasoning_effort": "none"},
    "mini": {"model": "qwen_3_5_9b", "reasoning_effort": "low"},
    "large": "llama_3_3_70b",
    "reasoning": {"model": "gpt_oss_120b", "reasoning_effort": "medium"}
  }
}

Llama 3.1 8B is no longer listed and its previous model endpoint returns 404. Mistral 7B Instruct v0.3 remains listed by the models API but is absent from the web catalogue; AIHub no longer selects it by default. Catalogue absence alone is not evidence of a scheduled retirement.

Calls forward messages, temperature, the output-token budget, tool definitions, tool choice and JSON response formats. An explicit JSON schema takes precedence over JSON-object mode. The adapter forwards reasoning effort for Qwen3.5-9B (none, minimal, low, medium, high, as exposed by OVH's API schema) and gpt-oss-120b (low, medium, high); unsupported values are rejected before a request. It omits this parameter for Llama and other models without registered reasoning support. Explicit profile reasoning settings remain authoritative.

Llama supports text, function calling and structured outputs, but not image input. An image workflow explicitly selecting Llama needs a vision-capable profile; the connector does not switch models automatically. Qwen3.5-9B, selected by default, supports images. OVH streaming, asynchronous batch jobs and explicit prompt-cache directives are not implemented by this adapter.

The client has a 120-second timeout; shorter workflow deadlines still apply. Clients are closed on success, failure and cancellation. The existing LLM bounded SDK retry policy applies, without a second application retry layer. Token usage, including reported reasoning tokens, and provider-reported costs are normalized. AIHub has no built-in OVH token-price table: configure a model-level pricing block in USD when the endpoint does not report costs.

For comparison, the catalogue displayed the following EUR rates per million tokens on the verification date. These are reference prices, not an AIHub billing configuration or a guarantee of future prices:

Embeddings¶

Both existing BGE models remain in the official catalogue and models API:

The connector now calls POST /v1/embeddings with model, input and encoding_format = "float", through the OpenAI SDK. It reads vectors from data[].embedding and orders them by data[].index. Each request contains at most 25 texts, with one client reused across sequential batches. Missing, duplicate or unexpected indices and incorrect BGE vector dimensions fail the operation. HTTP errors, timeouts and cancellation propagate; the connector never returns a partial list after a failed batch.

Only the transport and configured identifiers change. Preserve the existing mongo_vector_index, mongo_vector_path, embedding_dimension, threshold_delta and embedding profile/default selections. The two BGE models and their vector storage layouts remain unchanged. OVH also lists Qwen3-Embedding-8B (32–4,096 dimensions, 32,768 input tokens), but adopting it would require a separate model migration and recomputation of stored vectors, even when choosing the same dimension.

Migrating existing configuration¶

Updating the PostgreSQL initialization seed does not overwrite existing aihub_ai_providers rows or MongoDB common_data.AIProviders documents. Prepare the configuration change together with deployment: older AIHub versions expect URLs, while the new connector requires model IDs for both generation and embeddings.

Back up the provider configuration and review tenant ai_configuration.llm_profiles overrides first. Replace tenant references to removed aliases or inline endpoint URLs with the intended new catalogue alias/model ID, preserving their explicit reasoning settings. The migration command updates the provider document only; it does not change tenant overrides, select a different embedding model or rebuild vectors/indexes.

# Preview the provider update using the configured DATABASE_URL and PG_SCHEMA.
.venv/bin/python -m scripts.migrate_ovh_provider --backend postgres

# Apply the reviewed update.
.venv/bin/python -m scripts.migrate_ovh_provider --backend postgres --apply

# MongoDB deployments use MONGO_CONNECTION_STR instead.
.venv/bin/python -m scripts.migrate_ovh_provider --backend mongodb
.venv/bin/python -m scripts.migrate_ovh_provider --backend mongodb --apply

The command replaces the standard legacy LLM entries/profile mappings, adds missing defaults, and changes the two known BGE URLs to model IDs. It preserves custom model IDs, profile options and embedding metadata. Ambiguous custom legacy URLs or conflicting catalogue aliases require manual migration and abort before writing. PostgreSQL locks the provider row in a transaction; MongoDB uses a conditional update to detect concurrent changes. Repeated application is a no-op. Restart AIHub after applying the configuration update to refresh cached provider settings.

Official sources¶

• AI Endpoints catalogue
• Public model IDs and API pricing
• Llama 3.3 70B
• Qwen3.5-9B
• Qwen3.5-9B API schema
• gpt-oss-120b
• BGE multilingual Gemma2
• BGE M3
• Qwen3 Embedding 8B

Tasks routed through each profile¶

The following table groups representative current tasks. It identifies the profile requested by the workflow; the concrete provider and model still come from runtime configuration, including provider-specific workflows below.

Conversation summaries are a specialized mini task with a 1,024-token output limit. Guardrail boundary responses use 512 output tokens. Other LangGraph mini calls use up to 2,048 output tokens and large calls use up to 4,096. Direct calls normally use the shared 4,096-token policy; known long-output workflows and Lexi-managed generation have their own limits.

Only Chatbot and MCP enable the language detector's tiny fallback; it has 15 seconds and disables prompt caching. Every other surface trusts py3langid alone. Translation gives tiny 30 seconds, disables prompt caching, and bounds the output. Both generation paths inherit the tiny profile effort (none by default); configured overrides are preserved. Native Decisions has no effort parameter. See Language detection and translation for the complete provider and persistence rules.

Workflows requiring a particular provider¶

LangGraph supervisor routing and guardrail classification select mini through OpenAI Decisions when the resolved model supports it. Azure, Anthropic, OVH and unsupported OpenAI model mappings retain structured generation on the configured provider and mini profile. Existing mappings are never overwritten and native failures do not trigger a second generative call. Guardrail doubt/block responses use a separate generative mini call on every provider. See the graph safety gate.

The same capability selection applies to memory, opportunity profile intents, objective relevance, Lexi classifications, global skill-fit checks, marketplace preselection and ambiguous-language detection. Native decisions use mini; legacy providers keep each workflow's previous profile. See the workflow and batching table.

Lexi's other phases use mini for classification and normalization, and large for job missions and skill-level descriptions. Configure the provider catalogue or tenant profiles instead of the removed LEXI_PROVIDED_MODEL environment variable. Administrative client provisioning can still write model choices into the stored profile configuration; generation uses that configuration.

Claude Sonnet 5.5¶

The Anthropic large fallback and initial provider catalogue now select claude-sonnet-5-5. Other profiles keep their mappings. The pinned Anthropic SDK (see current dependency constraints) supports native JSON, normal responses, SDK streaming and Message Batches without a dependency upgrade.

• Effort accepts low, medium, high, xhigh and max. AIHub resolves profile effort (large=medium by default); direct GenAI model selection without effort keeps the API default high. AIHub's none translates to thinking.type=between_tools. This prevents up-front thinking, but does not disable thinking between tools.
• JSON Schema goes through output_config.format. The dynamic LangChain model uses native JSON for with_structured_output() on Sonnet 5.5 and Opus 5.5, preserving Pydantic/dictionary parsing and include_raw behavior. It resolves the tenant's provider and model at invocation time. The pinned SDK transforms generated schemas into Anthropic's supported JSON Schema subset (including closed objects); Pydantic still validates the original model constraints locally.
• Tools support auto and none. Forced choices (required or a named function) are rejected before the request reaches Anthropic. GenAI discovery advertises forced_tool_choice: false; no silent conversion to auto occurs.
• No temperature, top-p, top-k or manual thinking budget is sent. Text parsing ignores thinking blocks, while tool responses retain every native block in anthropic_content, including empty signed thinking and redacted thinking. Tool continuations must replay that content unchanged, with the same preceding system prompt, tools, history and Anthropic account. The chatbot only replays native blocks to the originating provider and model.
• Internal SDK streaming still returns the assembled response. Public GenAI SSE support remains limited to the providers advertising streaming: true.
• Prompt caching requires at least 512 prompt tokens. The context window is 1,000,000 tokens; standard output supports up to 128,000 tokens. Existing application token limits still apply.

Global list prices, in USD per million tokens, are 2 input / 10 output, 0.20 cache read, 2.50 cache write (5 minutes) and 4 cache write (1 hour). Message Batches receive the standard 50% discount. The obsolete Sonnet 5 scheduled price increase is removed. Explicit configured pricing still takes precedence.

Migrating persisted Sonnet 5 settings¶

Changing the seed or fallback does not rewrite existing databases. Review the following dry-run against each configured backend before applying it:

.venv/bin/python -m scripts.migrate_anthropic_sonnet55 --backend mongodb
.venv/bin/python -m scripts.migrate_anthropic_sonnet55 --backend postgres

Add --apply only after reviewing the affected configuration count. The command uses the existing database environment settings and performs no LLM inference. It replaces exact claude-sonnet-5 identifiers and the standard claude_sonnet_5 alias with claude-sonnet-5-5 / claude_sonnet_5_5, preserving other models, custom aliases, reasoning settings, pricing overrides and prompts. Review any explicit pricing overrides separately against the rates above. Conflicting or repurposed standard catalogue aliases cause the migration to fail.

MongoDB requires exactly one Anthropic AIProviders document in common_data and visits Config collections only in databases registered through ClientConfig.clientDB. It validates all plans before writing and guards each write against concurrent changes. Writes across documents are not atomic; a partial migration can be rerun safely after resolving a conflict. Unregistered tenant databases require separate review.

PostgreSQL updates the Anthropic row in aihub_ai_providers and the aihub_config configuration of the configured schema in one transaction with row locks. Run it separately for each deployed database/schema. Dry-run also uses row locks while inspecting that transaction. Neither backend creates indexes or changes the database schema; existing provider lookups, registry scans and configuration access patterns are reused.

Sources: model specifications, migration guide, and pricing.

Claude Opus 5.5¶

The centralized Anthropic reasoning fallback is claude-opus-5-5, also used by the Message Batches helper when no resolved model is supplied. Provider documents and tenant profile overrides remain authoritative. Deploying this change does not rewrite stored settings: update existing Opus 5 profile mappings separately. Pricing overrides must also match the selected model's rates.

AIHub sends the resolved profile effort through output_config.effort for Opus 5.5: the central reasoning default is medium, and provider/tenant overrides remain authoritative. There is no connector-specific high default. External GenAI requests may override the selected profile. Structured JSON formats are preserved alongside effort. The connector sends no sampling parameters or manual thinking budget, and extracts text independently of thinking blocks. The pinned Anthropic SDK (see current dependency constraints) supports this model; no dependency upgrade is required.

The migrated workflows make independent requests without tools or assistant prefill. Opus 5.5 always uses adaptive thinking and rejects forced tool choices (any and tool) and disabled/manual thinking. AIHub rejects forced choices before opening the SDK client, including for streaming and batches. GenAI advertises forced_tool_choice: false. tool_choice="none" explicitly disables tool calls, even when tools are supplied. The chatbot uses native JSON Schema for with_structured_output() on this model, rather than forcing a tool call; Pydantic validation and include_raw retain their existing behavior. Tool responses preserve native blocks in anthropic_content, including signed and redacted thinking, for unchanged replay in append-only conversations with the same account and model. Computer-use tools are not used by the migrated workflows.

The context window is 1,000,000 tokens. Classic output is limited to 128,000 tokens; Message Batches supports 300,000 with output-300k-2026-03-24. Lexi retains these limits and the beta header. Large classic requests use SDK streaming internally, including provided-skills selection; callers still receive the assembled response. Small non-streaming requests remain supported.

Standard global API list prices in USD per million tokens:

Obsolete Opus 5 and Claude 3.5 Sonnet prices have been removed from the built-in table. Configured pricing and provider-reported costs remain authoritative. Price resolution prefers the most specific model version. Fast mode and geographic pricing premiums are not enabled by these workflows.

Sources: model specifications, migration guide, pricing, and SDK 1.8.0 changelog.

Embedding models¶

Embeddings use ai_configuration.embedding_provider and embedding_model, independently of the generation provider and its semantic LLM profiles. The tenant selects a catalogue alias; the provider's embedding_models resolves it to embedding_name, dimensions and vector storage metadata. An unspecified or unknown tenant model inherits a valid default_embedding_model, then the provider's built-in model if the database default is absent or invalid. Resolved fallbacks include dimensions, vector index/path and threshold metadata together; metadata from the rejected model is not reused.

The following table describes the repository defaults and the Lexi profile installed by the PostgreSQL migration chain. Existing database settings can differ.

The global seed marks OpenAI as the default embedding provider for tenants. Calls without resolved tenant configuration currently fall back to Azure and text-embedding-ada-002; supply the resolved configuration when the intended provider differs. The built-in OVH fallback is bge-m3 (1,024 dimensions, bge_m3_vector_index, bge_m3_vector, threshold delta 0). OpenAI and Azure use text-embedding-ada-002 (1,536 dimensions, ada_002_vector_index, ada_002_vector, threshold delta 0). These code defaults do not update stored provider documents or recreate existing vectors/indexes.

The Azure deployment inventory checked on 2026-09-24 contains both text-embedding-ada-002 and text-embedding-3-small. The OpenAI and Azure seed alias ada_003_mini points to text-embedding-3-small with 1,536 dimensions. Its alias, vector paths and index names are preserved. The Azure generation-profile migration does not alter embedding entries in existing databases.

Lexi's independent embedding profile¶

The provider-level mapping embedding_profiles.lexi selects the model used by Lexi's duplicate-detection and provided-skills matching. The seed maps it to ada_003_mini for OpenAI/Azure and bge_multilingual_gemma2 for OVH. The PostgreSQL migration retains its existing OpenAI/Azure alias ada_003_small, which resolves to the same text-embedding-3-small model, and loads the OVH mapping after creating the profile column. It preserves existing profile mappings on replay and only adds a seeded mapping when its model alias is present in the provider catalogue. General embedding defaults and stored-vector metadata are unchanged. MongoDB provider records must carry an equivalent mapping when overriding the code default.

These initialization changes do not automatically update databases where the migrations have already run; update those provider records separately.

If this profile is missing or references an unknown model, it inherits the resolved default embedding model for the selected provider. This avoids sending an OpenAI model to OVH. Calls without any resolved AI configuration retain the standalone Lexi fallback, text-embedding-3-small. These Lexi vectors are computed in memory and are not persisted.

Training's enriched-title embedding profile¶

The provider-level mapping embedding_profiles.training selects the same model for MongoDB enriched-title generation, PostgreSQL enrichment workers and chatbot training queries. It resolves catalogue aliases through the shared provider configuration, just like Lexi's embedding profile:

{
  "embedding_models": {
    "training_candidate": {
      "embedding_name": "text-embedding-3-small",
      "embedding_dimension": 1536
    }
  },
  "embedding_profiles": {"training": "training_candidate"}
}

An explicit valid mapping wins. When the mapping is absent, the shared resolver supplies a dedicated training default independent of the general embedding model: text-embedding-ada-002 for OpenAI/Azure, preserving the existing training space, and bge-m3 for OVH. These are runtime defaults, not writes to provider records. Training remains a dedicated embedding selector; generation profiles such as mini do not select vector models.

The current enriched-title storage/index contract remains 1,536 dimensions. Changing to a different dimension (including OVH's 1,024-dimensional default) requires a separately coordinated schema/index migration. Even at 1,536 dimensions, changing models requires rebuilding all stored training vectors before using the new query model. Unchanged MongoDB titles may reuse existing vectors, so ordinary reimports alone are not proof of a rebuild. This profile change performs no reembedding or index migration. Compare candidates with the embedding evaluation suite first.

Embedding calls and stored-vector compatibility¶

OpenAI and Azure use the SDK embeddings endpoint; Azure's model is a deployment name. OVH uses the shared POST /v1/embeddings endpoint with a model ID and batches of at most 25 texts; see OVH embeddings for input limits and response validation. Embedding calls do not use reasoning_effort.

Changing a model changes its vector space, even when dimensions stay the same. For persisted embeddings, regenerate the affected document vectors and query vectors together, and align the configured dimensions and MongoDB/PostgreSQL vector indexes. Changing default or large generation profiles does not affect embeddings. The Azure update retains existing embedding models and vectors.

Realtime and speech models¶

Speech and continuous voice retain dedicated model selectors. Live's delegated reasoning backend separately uses the OpenAI mini semantic profile, preserving its configured model and reasoning effort for simulations.

Each selector resolves through its corresponding catalogue and default_llm_... alias. The continuous-voice configuration field retains its name, but its transport is the Live API. There is no separate input transcription catalogue. Stored Realtime selections must be migrated before using Live; incompatible voice models are rejected rather than substituted at connection time.

Only OpenAI implements these audio capabilities. Live explicitly uses OpenAI. File STT/TTS still follows the provider selected by its existing connector. TTS defaults to alloy and MP3; Live defaults to the French voice brise and raw 24 kHz PCM16. Simulation clients can override the voice through context.voice when opening a new session.

See Live integration and configuration migration, tenant configuration, and the OpenAI Live guide.

Finding the model used by a request¶

Do not infer the effective model from the profile name alone. Inspect the resolved tenant AI configuration or the call_llm_async child run in LangSmith. The run records the task name, provider, actual model, token policy, usage and available cost information. Direct background calls execute through the same connector but appear as child runs only when an active LangGraph trace exists.

For configuration examples, see Tenant AI configuration. For caching, limits and trace fields, see LLM runtime, caching and observability.