Shared input security¶
AIHub combines shared input budgets with protections where data is used. SQL, JavaScript and prompt-injection examples remain valid text: keyword blacklists would reject legitimate content without preventing injection.
This guide describes runtime behavior and reusable interfaces. The
input inventory records protection decisions,
exemptions and remaining risks. AGENTS.md delegates the review procedure for
new endpoints and changed inputs to .agents/skills/aihub-input-safety-review/.
Shared profiles and budgets¶
The text profiles in src/common/security/text_validation.py preserve Unicode,
punctuation, whitespace and code examples without rewriting or truncation.
| Type | Constraint | Use |
|---|---|---|
UntrustedText |
Reject C0/C1 controls except TAB, CR and LF | Base for domain profiles |
SkillSearchText |
Same controls, at most 500 characters | search_query on both skills_single routes |
ChatUserText |
Same controls, at most 20,000 characters | Chat input message, including replacement transcriptions |
SafetyChecked[T] in src/common/security/endpoint_inputs.py checks nested
text and structures while preserving the original object. Its default budgets
apply per checked argument:
| Budget | Limit |
|---|---|
| Individual string | 200,000 characters |
| Cumulative text | 2,000,000 characters |
| Visited nodes | 50,000 |
| Depth | 20 |
Existing tighter domain limits still apply. These checks preserve OpenAPI
schemas, aliases, defaults, coercion, nullable fields, extras and response
models. Their runtime limits need not appear as maxLength in OpenAPI.
Credentials and binary contents use dedicated boundaries instead.
The three *_in_consumer_format imports use a dedicated limit of 250,000
nodes per record, counting object keys and values. Their shared flows,
standardization and import prompt validators use the same limit, including
when invoked by queue consumers. CONSUMER_IMPORT_NODE_LIMIT in
src/common/security/endpoint_inputs.py defines this budget; individual and
cumulative text limits and maximum depth remain unchanged. The default node
budget for other inputs remains 50,000.
Consumer-format user imports at
POST /data_exchange/api/v1/users/import_user_in_consumer_format accept DEL
(U+007F) and C1 characters (U+0080–U+009F) in these text fields:
achievers[].reason;education[].educationalInstitute.instituteName(including legacy U+0092);previousExperiences[].companyName;previousExperiences[].jobTitle;previousExperiences[].experienceDescription;clientExperiences[].experienceDescription;- standardized
experiences[].experienceTitle,experiences[].experienceDescriptionandexperiences[].description.
The company-name exception applies only to previousExperiences[].companyName.
clientExperiences[].clientPrimaryPartyName and standardized
experiences[].companyName retain strict validation.
The institute-name exception applies only to the nested value in each education
list item. Other education fields and JSON keys remain strict. Education is still
accepted for compatibility and ignored by user standardization.
The field-specific profile in src/entity/user/input_validation.py is reused by
the user import flow and standardization. User inference also accepts DEL/C1 in the
corresponding job titles and descriptions after transformation, while retaining
them in user-role messages. Its internal job mapping uses the imported titles
as root keys; only those title keys and description values have this tolerance.
Nested metadata keys and date fields remain strict. Values are preserved
verbatim, without attempting to repair their encoding. Identifiers, keys in the
received JSON and other fields remain strict; C0 controls except TAB, CR and LF
and the dedicated import budgets still apply.
Shared profiles on other endpoints, including chat and skill search, remain
strict.
The other two consumer-format imports use the same LEGACY_NARRATIVE_CONTROLS range
from src/common/security/text_validation.py, with their own field allowlists:
Endpoint under /data_exchange/api/v1 |
Narrative values accepting DEL/C1 |
|---|---|
/opportunities/import_opportunity_in_consumer_format |
jobOpportunityDescription, standardized description, skills[].professionalSkillDefinition |
/feedback/import_feedback_in_consumer_format |
feedbackText, declineReason |
Opportunity validation is reused in its flow and standardization, then for
descriptions in skill and title inference prompts. Job and training skill
inference retains strict validation. Feedback validation is reused in its
score-update flow; the PwC transformation continues to use identifiers and
flags and ignores the optional narrative fields. Both imports preserve the
optional empty body, open object schema and 201 response. Titles, identifiers,
flags, mapping keys and other fields retain their existing control restrictions.
C0 controls other than TAB, CR and LF remain rejected in all text. The validators
do not normalize or repair text encoding.
Invalid typed HTTP inputs return 422; manually parsed chat payloads return
400, and simulation keeps its existing error envelope. Empty skill queries
still reach service validation and return 400. These are character/structure
limits after parsing, not request-body byte ceilings. The
attachment limits and
IP/API-key quotas complement them.
Rejected payload values are not logged.
Validation diagnostics¶
FastAPI validation failures emit WARNING logs with event
http_validation_error, the HTTP route template, and the existing request/client
correlation. validation_errors lists field paths and rule codes, for example
body.clientExperiences[0].experienceDescription with control_characters and
codepoint=U+0000, or text_limit with limit=200000 and observed=200001.
Ordinary schema failures use their Pydantic rule code, such as int_parsing.
Shared input budgets collect all rejected attributes within the inspected
structure. Diagnostics are split into numbered batches of ten per log line;
the list is not truncated after the first failure or batch. The cumulative
total_text_limit refers to the whole checked input. For depth_limit and
node_limit, the reported path identifies where inspection stops; descendants
of an oversized container are not scanned. An already oversized string is
reported by its length check without further content scanning.
Logs contain no field values, request bodies, headers or raw validation
messages. Paths are limited to 256 characters; non-identifier mapping keys are
masked as <field>, and <key> marks validation of a mapping key itself.
Safety checks remain enabled, including for consumer-format user imports.
OpenAPI and the public 422 envelope/message retain their existing behavior.
Reusable interfaces¶
For an existing request model, SafetyChecked[Payload] adds shared budgets at
the endpoint boundary without adding them to persistence or response models.
With an explicit FastAPI Body, Form, Path or Query default, use
payload: SafetyChecked[Payload] = checked_parameter(Body(...)).
Both helpers come from src.common.security.endpoint_inputs; the second retains
the validator when FastAPI copies the parameter default's metadata.
For a field requiring a domain limit, the base type can be reused as follows (the 10,000-character limit is illustrative):
from typing import Annotated
from pydantic import BaseModel, Field
from src.common.security.text_validation import UntrustedText
type DescriptionText = Annotated[UntrustedText, Field(max_length=10_000)]
class DescriptionInput(BaseModel):
description: DescriptionText
Manual parsers use the same model or validator at their existing error boundary.
Replacement values such as transcriptions are validated again: assignment or
model_copy(update=...) does not necessarily run Pydantic validation.
LLM trust boundaries¶
src/common/security/prompt_context.py owns the common system policy and JSON
encoding of labeled external text. Stored summaries, preferences, experience
skills, descriptions, languages and user-authored tasks remain user-role data,
separate from application instructions. Persistence does not make text trusted.
Tenant-owned system prompts remain privileged configuration.
The Oria graph requires the classifier's allow
decision before knowledge retrieval, long-term preference extraction or agent
execution. block and doubt stop the turn. Errors, malformed responses or a
15-second timeout return a stable temporary-failure message in French or
English, preserving the JSON response shape or final SSE run.failed event.
Cancellation propagates. This gate adds latency and makes chat availability
depend on the classifier.
Authentication, HTTP language resolution, conversation metadata and attachment or transcription processing can precede that gate. Direct memory writes use deterministic validation. Simulation, MCP and other standalone LLM workflows do not automatically receive the Oria classifier's protection.
Simulation profiles and scenarios are user-role data. Live browser frames are limited to 100,000 characters and validated against a closed event schema. Only Base64 PCM16 audio (up to one second per frame), microphone controls, monotonic playback counters and close requests are accepted. Instructions, tool results, model selection and provider session updates remain server-owned. Simulation voice has its own scenario instructions and does not inherit the Oria graph classifier. See the frontend contract.
Neither JSON quoting nor a classifier guarantees detection of prompt injection. Model outputs, retrieved documents and tool results remain untrusted. Tool argument validation, tenant/user authorization, parameterized queries and output escaping are independent protections.
Files, imports and outputs¶
| Boundary | Current behavior |
|---|---|
| Chat attachments and frontend configuration images | Base64 length is bounded before decoding and actual bytes are checked afterward; the default upload limit is 10 MiB. Chat fileWeight must match decoded bytes when supplied. Invalid images keep the configuration 400 envelope; see the chat reference for attachment errors and metadata rules. |
| Office archives through the common upload reader | Before document parsing, ZIP expansion is checked in a worker thread: at most 1,000 entries and 50 MiB expanded, in addition to original byte limits. Excess returns 413. |
| Imported entities and queue messages | Raw records are checked before persistence or LLM work, including standardization and skill embedding preparation. Budgets apply per record; existing consumer/batch retry and continuation behavior remains in charge. |
| Identifier/status spreadsheets and prompt-engine files | Consumed IDs are validated before processing (422 on invalid input). Prompt rows are checked before background scheduling, and grouped context is checked again before generation. |
| Downloaded Lexi catalogues | 20 MiB transport limit, 10,000 skills, depth 20 and 250,000 nodes. Individual/cumulative text limits are each 20 × 1,024 × 1,024 characters, with control-character validation. Destination and scope checks are recorded in the inventory. |
| XLSX and Lexi CSV exports | XLSX formula-looking values and headers are literal text; numeric cells retain their types. Lexi CSV prefixes strings starting with =, +, - or @ (including after whitespace) with an apostrophe. JSON results are unchanged. |
| HTTP attachment names | Special characters use RFC 5987 encoding; ordinary names retain their existing header representation. |
| Email attachment names | Brevo serializes names as JSON fields; JSON files gain a .txt suffix without changing their contents. The SMTP fallback preserves original filenames and uses the shared attachment_disposition encoder to prevent MIME header injection. |
SQL values remain bound, matching sort keys use allowlists and PwC deep filters use fixed-key/numeric parsers. URLs, paths, collection names and privileged configuration have consumer-specific boundaries, detailed in the inventory.
Limits and remaining risks¶
Text budgets do not prevent SQL injection, SSRF, unsafe rendering or unauthorized actions by themselves. Request-byte ceilings before parsing and frontend HTML/Markdown escaping belong to their respective server/client boundaries. Mocked tests establish validation, message roles and blocking order, not a real classifier detection rate; no live-provider adversarial evaluation was performed. The inventory tracks these limits and the unresolved frontend configuration identity check. This documentation is not a penetration-test report.