GenAI chat completions and model discovery¶

The GenAI application provides a single generation contract across configured OpenAI, Azure OpenAI, Anthropic and OVH models. Use discovery to choose a model and its supported options before generating a response.

The /api/v1/... paths in Swagger are relative to the /genai mount. For the local service on port 8086, the SDK base URL is http://127.0.0.1:8086/genai/api/v1.

Authentication and configuration scope¶

POST /chat/completions and GET /models require an AIHub X-API-Key. The usual AIHub authentication and client-routing rules apply, including Client-ID where required by your deployment. An OpenAI provider key is not an AIHub API key. Sending only the OpenAI SDK's Authorization: Bearer ... header does not replace X-API-Key.

The authenticated configuration supplies the default provider and tenant profile mapping. The provider catalogue supplies explicit models and the profiles of other providers. PostgreSQL deployments use their configured database; MongoDB deployments use the existing client context. These endpoints do not accept a different backend URL, credentials or a different deployment environment in the JSON body.

Successful responses and GenAI error envelopes include an X-Request-ID correlation header. An optional X-GenAI-Client label, such as hrx, is used for logging when no client identifier is available. It is not an authorization or tenant-selection mechanism. Labels must contain 1–64 letters, digits, dots, underscores or hyphens.

export AIHUB_BASE_URL='http://127.0.0.1:8086/genai/api/v1'
# Set AIHUB_API_KEY to your actual AIHub key outside source code.
curl --fail-with-body "$AIHUB_BASE_URL/models" \
  -H "X-API-Key: $AIHUB_API_KEY"

Swagger request examples¶

The Chat Completions operation provides an Examples dropdown above its request body. The first example shows explicit provider/model/profile selection, reasoning, JSON Schema output, token limits and prompt-cache controls. Additional examples cover profile-based JSON output, sampling, function tools, SSE with final usage, and the minimal request using configured defaults.

Select an example before using Try it out. Model IDs and profile capabilities are illustrative: use /models or the live catalogue below to adapt them to your configuration. The explicit model overrides the profile when both are present. Sampling controls and reasoning controls are shown separately because support varies by model. Examples do not change runtime defaults.

Live catalogue¶

This view automatically reads the public GET /genai/api/v1/catalogue endpoint. It requires no API key or Client-ID and shows shared provider profiles, provenance, GenAI capabilities and independent code fallbacks. It never reads client-specific configuration. Use Refresh configuration to retrieve database changes. See live configuration details for inheritance and refresh behavior.

curl --fail-with-body "$AIHUB_BASE_URL/catalogue"

GET /catalogue takes no parameters. Its ModelList response has the same metadata structure as /models, but the default provider and profiles come only from the shared provider database. Supplied API keys or Client-ID headers do not select a client. Provider credentials, connection settings and raw provider documents are never returned. A database failure returns HTTP 502 with the stable catalogue_unavailable error code; code fallbacks are not presented as a successfully loaded database catalogue. Responses use Cache-Control: no-store.

Selecting a provider, model or profile¶

provider, model and profile are optional. Omitted and JSON null selectors have the same meaning. Empty strings are invalid. Provider and profile identifiers are case-sensitive.

The public profile values are tiny, mini, large and reasoning. The internal default profile is selected by omitting both model and profile; do not send "profile": "default".

model accepts an ID returned by /models or one of that entry's catalogue aliases. Model selection does not search other providers. For example, a Claude model without provider: "anthropic" is rejected if the configured provider is OpenAI. An invalid explicit model does not fall back to a valid profile.

When a profile is used, its configured model and reasoning_effort are preserved. An explicit reasoning_effort overrides that profile setting, including a profile that sets reasoning_effort: "none". Profile settings are defaults, not capability restrictions. For example, profile: "tiny" with reasoning_effort: "high" enables high reasoning if the resolved model supports it, even when that profile normally disables reasoning. Validation uses the effective request value and the model's capabilities; it does not reject an override because of the profile's default. When model is supplied, no reasoning setting is inherited from profile; an omitted effort uses the provider's model default. A profile named reasoning chooses a configured model/settings combination, while reasoning_effort controls effort on the model that was actually selected. They are different controls.

The shared profile resolver's configured profile mappings remain authoritative. For a different provider, its existing profile-resolution rules apply. A missing or inconsistent catalogue/profile returns unknown_model; the endpoint does not invent a model or switch providers to complete the request.

Use the configured defaults¶

{
  "messages": [
    {"role": "user", "content": "Explain skills matching in one sentence."}
  ]
}

Select a profile on Azure OpenAI¶

{
  "provider": "azure",
  "profile": "mini",
  "messages": [
    {"role": "user", "content": "Summarize the benefits of mentoring."}
  ],
  "max_completion_tokens": 1000
}

Select a model explicitly¶

{
  "provider": "openai",
  "model": "gpt-6-luna",
  "profile": "large",
  "messages": [{"role": "user", "content": "Explain mentoring in one sentence."}],
  "reasoning_effort": "low",
  "max_completion_tokens": 1000
}

In this example, gpt-6-luna wins over large. It must exist in the deployment's catalogue. The examples use models from AIHub's seeded catalogues, but /models is the source of truth for your deployment.

Discovering models and capabilities¶

GET /models returns an OpenAI-style list envelope. Use ?provider=azure to filter it. Valid filters are openai, azure, anthropic and ovh. A known provider with no configured models returns an empty data list. An invalid provider filter returns HTTP 422. There is no pagination; the response contains the configured generation catalogue, excluding embedding, audio and realtime models.

An illustrative entry is:

{
  "object": "list",
  "data": [
    {
      "id": "gpt-4o",
      "object": "model",
      "created": 0,
      "owned_by": "azure",
      "provider": "azure",
      "aliases": [
        "gpt_4o"
      ],
      "profiles": [
        "default",
        "tiny",
        "mini"
      ],
      "capabilities": {
        "reasoning": false,
        "reasoning_efforts": [],
        "json_output": true,
        "response_formats": [
          "text",
          "json_object",
          "json_schema"
        ],
        "streaming": true,
        "tools": true,
        "forced_tool_choice": true,
        "tool_reasoning_efforts": null,
        "prompt_cache_parameters": [
          "prompt_cache_key"
        ],
        "prompt_cache_retentions": [],
        "supported_parameters": [
          "frequency_penalty",
          "max_completion_tokens",
          "max_tokens",
          "parallel_tool_calls",
          "presence_penalty",
          "prompt_cache_key",
          "response_format",
          "seed",
          "stop",
          "stream",
          "stream_options",
          "temperature",
          "tool_choice",
          "tools",
          "top_p"
        ]
      }
    }
  ]
}

messages, the three selectors and stream: false are universally accepted and are not repeated in supported_parameters. Actual values and combinations remain subject to validation; presence in the list does not mean every possible value is valid. Discovery and request validation share the same capability definitions.

Availability means configured in AIHub. Discovery does not contact providers, probe credentials, check quotas, verify Azure deployments or make billable model calls. An entry can therefore still fail upstream. No credentials, connection URLs, pricing records or arbitrary private catalogue properties are returned. The same model ID can appear under more than one provider; identify it by the pair (provider, id). Conflicting catalogue configurations for one ID require selecting an unambiguous alias or fixing the catalogue.

Capabilities are deliberately conservative for unrecognized model/deployment names. In particular, an arbitrary Azure deployment name does not automatically inherit GPT reasoning/JSON capabilities. The capability definitions must recognize a model before those controls are advertised.

Request parameters¶

Unknown top-level fields are rejected. This is a documented Chat Completions subset with AIHub selection extensions, not a pass-through for every OpenAI API field. Optional generation fields omitted or set to null are not forwarded; explicit false, 0 and reasoning_effort: "none" are preserved.

The two output-token spellings are normalized to the native provider API. Omitting both leaves the provider's default unchanged, except Anthropic, whose required output limit defaults to 4,096. The GenAI endpoint does not inject the shared business-workflow sampling, reasoning or prompt-cache policy.

For non-stream requests, the HTTP service calls AIHub's existing call_llm_async() entry point. Model resolution, provider dispatch, timeouts and normalized internal usage/cost handling stay in the shared connector. A typed, optional explicit-control bundle preserves the public API parameters and native OpenAI response without changing the default policies of other AIHub workflows. The existing incremental OpenAI/Azure SSE transport remains the streaming path.

OpenAI/Azure responses use the native Chat Completions API. Anthropic uses its native Messages API, mapping JSON Schema and effort to output_config. OVH uses its OpenAI-compatible API. Unsupported explicit options return an error instead of being ignored or replaced with a different setting.

Provider differences to account for¶

Anthropic sampling controls are not exposed by this endpoint. The native API version 2026-10-21 used by the pinned SDK rejects temperature, top_p and top_k; AIHub does not send them.

GPT‑6 Astra and GPT‑6.1 Sol do not accept none or minimal effort. GPT‑6.1 Terra accepts none, low, medium, high, xhigh and max. None of these three models supports function calling through this Chat Completions endpoint, even Terra at effort none. The shared connector and LangGraph use Responses for their tool calls. The default OpenAI large profile selects Terra with medium. GPT‑5.4, GPT‑5.5, GPT‑5.6 and GPT‑6 Luna require effective effort none when tools are supplied. Set it explicitly or select a profile that configures it; AIHub will not disable reasoning automatically. /models reports these restrictions through tools and tool_reasoning_efforts. An omitted effort is rejected unless the selected profile supplies none. The built-in OpenAI and Azure mini profiles now supply low; requests using these profiles with restricted function tools must explicitly set reasoning_effort: "none". Unsupported combinations return HTTP 422 before document processing or generation.

Text, JSON mode and JSON Schema¶

The HTTP response remains a Chat Completions object in every output mode. The generated text is at choices[0].message.content. JSON output is encoded as a JSON string in content, exactly as in an OpenAI Chat Completions response. It is not returned as a replacement HTTP body or a nested object in content.

Plain text¶

Omit response_format, or send {"type":"text"}. Do not infer a JSON mode from the prompt alone.

JSON object mode¶

{
  "provider": "azure",
  "profile": "mini",
  "messages": [
    {"role": "system", "content": "Return a JSON object containing a summary string."},
    {"role": "user", "content": "Summarize a mentoring program for new engineers."}
  ],
  "response_format": {"type": "json_object"},
  "max_completion_tokens": 1000
}

Instruct the model to produce JSON in the messages. JSON mode does not enforce specific keys, types or a schema. OpenAI can reject JSON-mode requests without a JSON instruction. Refusals and token-limit truncation still need handling.

Structured JSON Schema output¶

{
  "provider": "openai",
  "model": "gpt-4o",
  "messages": [
    {"role": "user", "content": "Summarize a mentoring program for new engineers."}
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "mentoring_summary",
      "description": "A concise description of a mentoring program.",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {"summary": {"type": "string"}},
        "required": ["summary"],
        "additionalProperties": false
      }
    }
  },
  "max_completion_tokens": 1000
}

json_schema.name is required, 1–64 ASCII letters/digits/underscores/hyphens. schema is required and must be a JSON object; description and strict are optional. The selected provider enforces its supported JSON Schema dialect. AIHub validates the envelope and input budget; it does not compile arbitrary schemas, repair output, strip Markdown fences or retry generation to force validity. Anthropic receives the schema through output_config.format. OpenAI's schema name/description/strict fields are sent unchanged to OpenAI/Azure; other providers enforce their native schema semantics.

A successful response can contain:

{
  "id": "chatcmpl-example",
  "object": "chat.completion",
  "created": 1790596800,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "{\"summary\":\"Pair new engineers with experienced mentors.\"}", "refusal": null},
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {"prompt_tokens": 60, "completion_tokens": 20, "total_tokens": 80}
}

Check finish_reason, message.refusal when present, and non-null content before parsing it. A length result can contain incomplete JSON, including no visible text if reasoning consumed the budget. A content_filter result is not a successful structured answer. Tool-call responses may have content: null.

Prompt caching¶

Prompt caching is provider-side prefix reuse. It is not an AIHub result cache, document store or conversation-history store. Sending the same key does not return a previous completion, and cache hits are not guaranteed.

• prompt_cache_key is forwarded unchanged to supported OpenAI/Azure models.
• prompt_cache_retention controls cache lifetime on supported models before GPT‑5.6. GPT‑5.5 accepts 24h; older supported models expose their permitted values in /models. GPT‑5.6 and GPT‑6 reject this legacy field with HTTP 422, including when it is combined with prompt_cache_options.
• prompt_cache_options.mode chooses implicit or explicit caching on supported OpenAI models. For GPT‑5.6 and GPT‑6, ttl: "30m" is the currently exposed minimum lifetime. Their discovery metadata lists no legacy retention values.
• Explicit breakpoints are preserved on text content parts. At most four can be supplied. With explicit-only mode and no breakpoints, no cache write is requested.
• AIHub does not manufacture breakpoints, generate a key or retry without caching after the provider rejects an explicit cache setting.
• An empty prompt_cache_parameters list means these request controls are not exposed; it does not mean the provider never performs automatic caching.

{
  "provider": "openai",
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "developer",
      "content": [
        {
          "type": "text",
          "text": "You summarize mentoring programs using the supplied reusable policy.",
          "prompt_cache_breakpoint": {"mode": "explicit"}
        }
      ]
    },
    {"role": "user", "content": "Summarize this program: weekly mentoring for new engineers."}
  ],
  "prompt_cache_key": "mentoring-policy-v1",
  "prompt_cache_options": {"mode": "explicit", "ttl": "30m"},
  "reasoning_effort": "low",
  "max_completion_tokens": 1000
}

This short example demonstrates the request shape; it is too short to guarantee cache eligibility. Provider minimum prefix lengths, prefix matching, cache-write pricing and retention rules still apply. Put stable reusable content first and varying user content last. Treat cache keys as opaque identifiers, not a place for personal data or credentials.

When reported, cache usage appears in usage.prompt_tokens_details.cached_tokens; reasoning usage appears in usage.completion_tokens_details.reasoning_tokens. These are subsets of input/output usage, not extra tokens to add to total_tokens. OpenAI-compatible native usage fields are retained. Anthropic usage is normalized into the same names when available; missing usage remains null.

Function tools¶

Supply OpenAI function definitions in tools. tool_choice can be auto, none, required, or {"type":"function","function":{"name":"submit_summary"}}. A required/named choice needs tool definitions; a named choice must reference a defined function. Other tool types, including hosted web-search/computer tools, are not exposed by this contract.

Check capabilities.forced_tool_choice in model discovery before requiring a function. Sonnet 5.5 and Opus 5.5 support tools with auto and none, but reject required and named choices. Use response_format.type=json_schema for structured JSON. For all Anthropic models, none explicitly disables tool calls even when tool definitions are supplied; it is never omitted or converted to auto. Sonnet's effort values are none, low, medium, high, xhigh and max; none disables up-front thinking using Anthropic's between_tools mode, which can still think between tools. A profile selects its configured or central effort; a concrete model selected without a profile or request effort keeps the API default high. Request effort overrides the profile.

A function call is returned in choices[0].message.tool_calls; its function.arguments is a JSON string. AIHub does not execute the function. The caller executes its own authorized operation, then sends the assistant message and corresponding role: "tool" messages with matching tool_call_id values.

For Haiku 5.5, reasoning_effort: "none" disables thinking. Profile-based calls use the resolved profile effort (mini/default=low, tiny=none). Selecting a concrete model without a profile or request effort keeps the API default medium. Other supported levels are low, medium, high, xhigh and max. Unlike Sonnet 5.5 and Opus 5.5, Haiku 5.5 accepts forced function choices. For every Anthropic model, including Haiku, Sonnet and Opus 5.5, when continuing a tool call, retain the returned assistant message's anthropic_content extension unmodified alongside its tool_calls. It contains native content and opaque thinking signatures, not reasoning text for display. Keep prior system/tools/messages unchanged and use the same Anthropic account.

Native replay must also respect API version 2026-10-21: a document block directly inside an assistant's anthropic_content is rejected locally with HTTP 422 and error.code="invalid_request", before configuration lookup or a provider call. Supply new documents as user file parts through the document contract above. Documents nested inside native tool-result blocks remain accepted and unchanged, as do assistant text citations and signed thinking. AIHub never moves or strips native blocks to repair an invalid history.

Streaming¶

Set stream: true for a provider/model advertising streaming: true. The endpoint returns Content-Type: text/event-stream and Cache-Control: no-cache. It relays OpenAI Chat Completions chunks and finishes with:

data: {"id":"chatcmpl-example","object":"chat.completion.chunk","created":1790596800,"model":"gpt-4o","choices":[{"index":0,"delta":{"role":"assistant","content":"Hello"},"finish_reason":null}]}

data: [DONE]

SSE preserves the OpenAI Chat Completions HTTP contract and works with OpenAI SDK streaming clients. Generation needs a request followed by a server-to-client stream, so a separate WebSocket endpoint is unnecessary. A future bidirectional real-time interaction, such as continuous audio, would require a separate contract. Browser clients should use streaming fetch for this POST request with its JSON body and authentication headers; native EventSource uses GET.

Request "stream_options":{"include_usage":true} for the upstream usage chunk. It commonly has choices: []; do not assume every event has a first choice. Usage is not requested automatically, and interrupted streams may never deliver it.

JSON output streams as text fragments in choices[].delta.content. Concatenate them and wait for successful completion before parsing JSON. Tool argument deltas also need assembly. Do not parse individual chunks as complete output objects.

curl --fail-with-body --no-buffer "$AIHUB_BASE_URL/chat/completions" \
  -H "X-API-Key: $AIHUB_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"provider":"openai","model":"gpt-4o","messages":[{"role":"user","content":"Explain mentoring briefly."}],"stream":true,"stream_options":{"include_usage":true}}'

Failures before streaming starts use a regular HTTP error. Once HTTP 200 and the stream have started, a later failure is emitted as an SSE data: event containing the error envelope, followed by [DONE]; it cannot change the HTTP status already sent. Client disconnect/cancellation closes the upstream stream. Anthropic and OVH streaming are not exposed by this endpoint and return HTTP 422.

Discovery provenance fields¶

GET /models retains the OpenAI object: "list" and data envelope and adds:

On authenticated /models, the optional provider query filter applies to both data and providers. Public /catalogue always lists the shared catalogue. An unused fallback does not add a model to data. Responses use Cache-Control: no-store; values are read on each request, and database failures retain the sanitized error contract. No configuration is embedded into the static Swagger schema or shared documentation site. Swagger examples are illustrative: use Try it out on /models for authenticated client-specific values, or the live public catalogue above for shared provider values.

For example, a valid explicit database mapping still has source: "provider" when its model name equals fallback.model. An invalid tenant override that falls back to a valid provider mapping also reports source: "provider". Profile settings are not the final request options: explicit generation controls and existing per-workflow policies still apply as documented above.

Documents¶

Messages can contain OpenAI-style inline file parts:

{
  "provider": "azure",
  "profile": "mini",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Summarize this document."},
        {"type": "file", "file": {"filename": "program.pdf", "file_data": "data:application/pdf;base64,REPLACE_WITH_ACTUAL_BASE64_BYTES"}}
      ]
    }
  ]
}

Replace the Base64 placeholder with actual file bytes. Supported MIME types are application/pdf and application/vnd.openxmlformats-officedocument.wordprocessingml.document. This existing extraction workflow uses a configured OpenAI document model even when the final completion selects another provider. PDF pages, including scans, are sent in small page groups with the reasoning effort resolved from that profile. DOCX paragraphs and tables are read in order, with embedded images sent for transcription. Extracted text remains user content.

Only send documents approved for that processing path. Selecting another completion provider does not change document extraction routing. Incomplete extraction fails with document_extraction_failed. AIHub does not persist uploaded documents and does not log their content or filenames. URLs and provider file IDs are not accepted as replacements for inline file bytes. General image/audio/video message parts are not exposed by this endpoint.

Errors, limits and operational behavior¶

Generation/discovery errors use this envelope:

{
  "error": {
    "message": "The selected model does not support the requested option. See GET /models.",
    "type": "invalid_request_error",
    "param": "reasoning_effort",
    "code": "unsupported_capability"
  },
  "request_id": "example-request-id"
}

Request-validation errors do not echo the input body. param is populated for recognized unsupported generation fields and can otherwise be null. Provider exception messages, credentials and payloads are not copied into public errors. Malformed JSON and unknown fields produce the same sanitized validation envelope. AIHub's authentication middleware and infrastructure can additionally enforce their existing error/rate-limit policies.

Non-stream completion execution has a 120-second timeout. OpenAI/Azure streaming uses the 120-second SDK timeout, including upstream reads; this is not a promise of unlimited idle stream duration. Document extraction happens before completion and has its own bounded processing. Generation does not retry with a different model/provider or remove rejected options. All supported providers use at most three attempts per request through the shared SDK retry policy. Transient connection errors, timeouts and retryable HTTP errors use exponential backoff; permanent errors fail without retries. The 120-second non-stream deadline includes retries and backoff. Streaming may retry opening the request but never replays a stream already returned to the caller. See provider retries.

Application logs identify the request, client when available, selected model, provider, status, elapsed time and reported input/output usage. They exclude prompt text, generated content, files and cache keys. A response is not stored as chatbot conversation history by these endpoints.

Using the OpenAI Python client¶

import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8086/genai/api/v1",
    api_key=os.environ["AIHUB_API_KEY"],
    default_headers={"X-API-Key": os.environ["AIHUB_API_KEY"]},
)

completion = client.chat.completions.create(
    model="gpt-4o",  # Choose an ID from GET /models.
    messages=[
        {"role": "system", "content": "Return a JSON object with a summary field."},
        {"role": "user", "content": "Describe mentoring for new engineers."},
    ],
    response_format={"type": "json_object"},
    max_completion_tokens=1000,
    extra_body={"provider": "openai"},
)
choice = completion.choices[0]
if choice.finish_reason == "stop" and not choice.message.refusal and choice.message.content:
    summary = json.loads(choice.message.content)
    print(summary["summary"])
else:
    print("Handle refusal, truncation, filtering or tool calls before using the result.")

provider and profile are AIHub extensions, so use the SDK's extra_body for those fields. The SDK's typed chat.completions.create requires a model; for profile-only or fully default requests, use a direct HTTP request with the bodies shown above. Supplying both SDK model and extra_body.profile still selects the model, not the profile. The official OpenAI /models response has fewer fields; AIHub adds capability metadata without changing the list envelope.

Migrating from the initial GenAI contract¶

The old profile:... and combined provider/model selector syntax is replaced; it is not a second supported routing path. Update callers before deploying this contract change. Authentication and inline document extraction use the existing AIHub mechanisms.

Contract references¶

• OpenAI structured outputs
• OpenAI prompt caching
• OpenAI reasoning controls
• GPT-6 Luna capabilities
• GPT-6.1 Terra capabilities
• GPT-6.1 Sol capabilities
• GPT-6 Astra capabilities
• AIHub provider configuration

The deployed Swagger schema and /models response describe the supported AIHub contract. Native provider documentation describes upstream behavior and may include features that this endpoint does not expose.

The generated genai HTTP reference is the authoritative source for parameters, schemas, responses and examples.