Skills Library¶

AI Hub Main API on this instance: Swagger · ReDoc · Live OpenAPI JSON

Administration API on this instance: Swagger · ReDoc · Live OpenAPI JSON

Main HTTP reference · Admin HTTP reference

src/skills_library resolves free-text skill labels against the tenant skill catalog. It owns request schemas and backend dispatch, while the reusable PostgreSQL query implementation is shared from src/entity/skill.

Search algorithm¶

For a single label, the service runs semantic and lexical retrieval and fuses their ranks with weighted reciprocal-rank fusion (RRF). Results can be filtered by repository metadata, limited and semantically deduplicated. The caller's optional threshold remains accepted for compatibility but is ignored: the base threshold is 0.93 in MongoDB and 0.91 in PostgreSQL. MongoDB adds the embedding model's configured threshold_delta.

In MongoDB mode, the query language is detected before vector generation. Non-English text is translated through the cache and DeepL when possible; the Atlas autocomplete branch always searches the original text against title. Detection or translation failure keeps the case-normalized query for vector search. The translation-cache lookup resolves every distinct translation stored under deterministic capitalization variants. Each translation contributes vector candidates alongside the normalized original query; the service keeps every distinct skill with its best similarity score. When no cached capitalization variant is available, DeepL is called once. A missing DeepL key or its first network, HTTP or response-validation failure immediately triggers one translation call through the tenant provider's tiny LLM profile. A successful DeepL or LLM translation is added to the same cache. If both providers fail, the normalized original query remains the vector-search input and the request continues. Failures are logged by safe type or status without exposing response payloads. Query capitalization therefore does not select one translation at the expense of the others. The input language does not filter MongoDB candidates.

Skill autocomplete uses lucene.standard with foldDiacritics=true for source titles and legacy skillTitle.en. The analyzer splits words and lowercases them without language-specific stemming. For example, animation remains a complete searchable word in Animation de formation, rather than becoming the French stem anim. The same definition applies to titles in every language.

The text query preserves Unicode letters and marks, normalizes canonical Unicode composition and whitespace, and lets the index handle case and accent folding. No French accent variants or shortened-word workarounds are generated. Existing sequential token matching, one-edit fuzzy matching, repository filters and the 100-candidate text limit remain in place. Language-neutral tokenization does not provide language-specific morphology or dictionary-based word segmentation.

Existing MongoDB indexes must be reconciled after deploying this definition; changing Python code alone does not update an index. See the skill index rollout procedure.

MongoDB skill import also resolves missing English titles in groups: one cache lookup covers all unique labels, only cache misses enter grouped DeepL requests, and a failed DeepL group triggers one structured tiny LLM call for that same group. Successful translations are persisted together before embeddings are generated.

For each repository-priority pass, the vector branches run concurrently while Atlas autocomplete runs once on the original query. Vector candidates are merged by skill identifier before text/vector RRF, retaining the highest cosine score when several embeddings find the same skill. The fixed semantic threshold, repository priority, cross-repository semantic deduplication and requested results_number limit then apply to the merged candidates as usual.

In PostgreSQL mode, full-text/trigram and HNSW cosine searches run concurrently against skill_search_v1. Each candidate is a specific translation row, so language filters that row directly. RRF combines the two rankings and retains only the best translation of each skill when no language is requested. The embedding input is case-normalized just as it is for MongoDB. This branch never reads or writes the MongoDB Translations cache; relational translations are the only translation source for PostgreSQL search.

See Language detection and translation for the shared provider chain and backend boundaries.

Endpoints¶

Both single-skill routes validate search_query with the shared SkillSearchText profile: at most 500 characters, without C0/C1 controls other than TAB, CR and LF. Invalid values return 422. Optional/nullable input remains accepted by the schema; an empty search still returns the existing service error 400. Unicode, punctuation and legitimate code examples are preserved. This validation is not a SQL or prompt-injection detector. See the shared input-security guide.

POST /api/v1/skills-library/skills_single¶

Both single-skill routes log their parsed input, including default values, as compact JSON at INFO with the prefix skills input parameters:. The shared logging filter still applies credential redaction and output-size limits.

The current authenticated endpoint resolves the tenant from the API key first and Client-ID second. It selects PostgreSQL automatically when relational resources exist; otherwise it uses the normalized MongoDB skill collection, then returns ranked skill documents with similarity metadata.

POST /skills-library/skills_single¶

The historical unauthenticated route retains the request body's tenant and collection fields. It invokes the same search service and fixed backend thresholds, including automatic PostgreSQL selection when available.

POST /admin/api/v1/skills-library/embeddings/regenerate¶

This administrator and MongoDB-only operation regenerates embeddings from title_en for the selected skill collection and provider/model. Collection names pass through the same environment normalization used by search and skill imports when a collection name is supplied.

Omit the optional collection_name query parameter to process every collection listed in reference_data, sequentially in alphabetical order. This includes all collection names, without normalizing discovered names or restricting them to the current environment. A supplied client_id does not limit this global mode. Even a collection named Skills is processed inside reference_data; explicitly selecting collection_name=Skills retains the tenant database routing. Logs identify the current name and progress, for example Skill embedding regeneration: collection 3 of 12: skills_integration_smartmobility started, followed by a completion log. An empty database completes without model calls.

For example, POST /admin/api/v1/skills-library/embeddings/regenerate?update_embeddings=true completes missing English titles and vectors across all reference collections. Authentication and provider/model query parameters remain the same.

Its modes are:

Incremental selection happens in MongoDB. A cursor projects only identifiers and titles, and processing reads at most 400 documents per application batch. Existing vectors are not downloaded. Each batch is saved before processing the next. The endpoint returns HTTP 202 with a task ID and executes the workflow in the background. Completion, including translation failures, is reported by email; previously saved batches remain saved. See background operations.

Existing nonempty title_en values are reused. For missing English titles, the shared language detector processes unique source titles in groups. Titles identified as English are copied unchanged. Other or undetermined languages use the translation cache, then batched DeepL translation with the existing tiny-LLM fallback. DeepL batches respect its 50-text and request-size limits. Legacy documents without a usable title can use skillTitle.en as their source. Documents without any usable source or English title are skipped and counted in a warning.

Every selected skill gets a vector calculated from its English title. This also replaces an existing vector when title_en had to be populated. Both fields are saved together, without changing the original title. Both modes use existing indexes; index creation belongs to client provisioning and the administrative index reconciliation endpoint.

If neither translator succeeds, the current batch is not saved and the API reports a failed operation by email with failure_code=operation_failed and a task ID for the causal logs. Earlier batches or collections may already have been saved; retry with update_embeddings=true to complete the remaining documents. A collection failure stops the background operation before subsequent collections are started. A translation-cache read or write failure is best effort and does not discard successful translations. No MariaDB skill import or PostgreSQL regeneration is performed by this endpoint.

Skill ingestion and standardization are documented under Entities and Administration.