Development guide¶

Setup¶

Prerequisites¶

Python 3.14 and Git;
access to the databases and configuration used by the development environment: MongoDB by default, or the PostgreSQL backend selected by DATABASE_URL;
credentials for the AI providers used by that configuration.

See Configuration for database provisioning, tenant settings and optional services such as RabbitMQ, Datalake and LangSmith.

Install¶

git clone https://gitlab.neobrain.tech/data-ia/aihub.git
cd aihub
python3.14 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip check
pre-commit install

Configure the environment¶

Create a local .env with the infrastructure and provider credentials for your development environment. For a MongoDB environment using OpenAI, a starting point is:

MONGO_CONNECTION_STR='mongodb://localhost:27017'
OPENAI_API_KEY='<openai-api-key>'
ENV='integration'

The databases must contain the required tenant and provider configuration; these variables alone do not provision them. Adapt the settings to the selected backend using the configuration guide. APP_IDENTIFIER_SHORT selects the runtime environment when configured; ENV is used when it is absent, empty or mutu.

Designing AI features¶

Select a semantic profile for every generation or decision workflow, and resolve both its model and its reasoning through the shared configuration. Do not add local effort constants, task-specific thinking budgets or minimum-effort logic. Provider/tenant llm_profiles can override central defaults; preserve those settings and translate them only for the selected provider's capabilities. External GenAI controls remain caller-configurable, and native Decisions has no effort parameter. Speech and embeddings retain their dedicated selectors.

Include the profile and nominal/boundary evaluation scenarios in the feature plan. The aihub-llm-evaluation skill reviews profile propagation through normal, streaming, batch and provider-specific paths. Technical tests cover configuration precedence and SDK payloads; live evaluations retain their semantic criteria. See reasoning configuration and AI evaluations. Paid runs require their own authorization.

Run locally¶

With the repository virtual environment active, start the application:

uvicorn src.main:app --host 0.0.0.0 --port 8000 --reload

Useful local URLs:

URL	Content
`http://localhost:8000/health-check`	Health check
`http://localhost:8000/main/docs`	Main FastAPI OpenAPI UI
`http://localhost:8000/admin/docs`	Administration OpenAPI UI
`http://localhost:8000/chatbot/docs`	Chatbot OpenAPI UI
`http://localhost:8000/data_exchange/docs`	Data Exchange OpenAPI UI
`http://localhost:8000/lexi/docs`	Lexi OpenAPI UI
`http://localhost:8000/docs/`	Generated English Zensical site

The generated documentation, Swagger, ReDoc and OpenAPI schemas are public. For protected API operations, supply X-API-Key through Swagger's Authorize action. See Authentication for standard and administrator keys, tenant routing and endpoint-specific requirements.

Check that the main application is serving requests:

curl 'http://localhost:8000/health-check'
# {"message":"API is up and running!"}

Code conventions¶

Ruff is the source of truth for formatting and linting:

ruff check .
ruff format --check .
mypy
pyright

Use Google-style docstrings for public modules and functions. Keep API contract examples in docs/ and use absolute Python imports rooted at src, scripts or tests. Relative imports are rejected by Ruff. The pre-commit Ruff hook applies safe fixes only; preview and review an unsafe fix before applying it to a specific violation.

Ruff's D rules enforce the production-source docstring contract. Public modules describe their ownership and architectural boundary; public classes describe their role, state and invariants; public callables explain their intent, non-obvious behavior and side effects. Add Args, Returns, Raises or Yields sections only when they convey information beyond the type annotations. Do not add a docstring that merely repeats the symbol name, and do not suppress a D finding without a narrowly documented reason. Application code and operational scripts share this contract; tests, Alembic revisions and repository-agent tooling remain outside the gate. Ruff also enables ERA so commented-out code is rejected automatically.

Mypy checks every Python module under src and scripts with the repository's strict module policy. This includes all application surfaces, business domains, connectors, repositories, workers, batch flows, LangGraph orchestration and operational entry points; a new module in either tree is therefore checked automatically. Official Pandas stubs are pinned in requirements.txt. AsyncSSH supplies its own type annotations. Dynamic typing remains limited to justified SDK, database-row and JSON-document boundaries rather than being used to bypass checks.

Pyright uses the same analysis engine as Pylance. It checks src, tests and scripts in basic mode and treats deprecated APIs and typing forms as errors through reportDeprecated. Its project configuration is kept in pyrightconfig.json, so local editors, pre-commit and GitLab CI apply the same rules. Do not use a weaker local configuration when preparing a commit.

Tests¶

bash scripts/run_tests.sh
AIHUB_TEST_WORKERS=1 bash scripts/run_tests.sh
pytest tests/chatbot/
pytest --cov=src --cov-report=html

The local pre-commit suite measures both statement and branch coverage for all modules under src and requires 100% coverage. GitLab CI does not run this intentionally local suite. The complete runner uses four isolated pytest workers by default; set AIHUB_TEST_WORKERS to adjust concurrency or use 1 for sequential debugging. A configured Alembic test database forces sequential execution to protect its shared schema. Focused plain pytest commands remain sequential. Tests must use deterministic fakes for models, databases, queues and external SDKs. The test runner disables LangChain and LangSmith tracing before pytest. Pytest treats warnings as errors, including DeprecationWarning and PendingDeprecationWarning; narrowly documented third-party exceptions are the only allowed ignores.

Changes to provider normalization, prompt caching, ranking or bounded batch execution should include unit tests that do not require live external services.

See the testing guide for focused suite examples and the complete local coverage command.

PostgreSQL migrations¶

Alembic is the source of truth for PostgreSQL schema changes. Generate a new migration when you add, alter or remove a PostgreSQL table, column or type, and apply it locally with the same connection settings used by the application:

export DATABASE_URL='postgresql://postgres:postgres@localhost:5432/aihub_test'
export PG_SCHEMA='mysql'
alembic revision -m 'describe the change'
alembic upgrade head

Alembic logs migration internals at DEBUG and SQL statements at INFO while masking bound parameter values. Use the explicit debug flag when investigating a migration that appears to wait or stop without an exception:

alembic -x debug=true upgrade head

The command emits an Alembic debug logging is enabled marker before opening the PostgreSQL connection.

GitLab CI does not run migrations against shared environments. It only runs the PostgreSQL smoke migration test against an ephemeral pgvector-enabled service. Infrastructure-owned deployment workflows remain responsible for applying alembic upgrade head to staging and production.

Use a dedicated disposable database when running the smoke test locally:

export ALEMBIC_TEST_DATABASE_URL='postgresql://postgres:postgres@localhost:5432/aihub_test'
pytest --confcutdir=tests/common/connectors/postgres \
  tests/common/connectors/postgres/test_alembic_migration_smoke.py

The isolated smoke command does not load the global FastAPI test fixtures and therefore requires neither APP_IDENTIFIER_SHORT nor ENV.

Project version tags¶

AI Hub keeps a global Git release tag alongside the versions of its individual FastAPI applications. Code changes retain the existing workflow: increment the affected API versions in source, then let scripts/project_version.py calculate one global increment from the last applicable vX.Y.Z tag.

Changes since the applicable tag	Global increment
One or more API patch increases	Patch (`+0.0.1`)
One or more API minor increases, including accumulated minor and patch changes	Minor (`+0.1.0`), patch reset to zero
`requirements.txt` only	Patch (`+0.0.1`)
Documentation only	Patch (`+0.0.1`)
Any tracked change under `tests/`, including fixtures	Patch (`+0.0.1`)
API, requirements, documentation and test changes together	One increment; minor takes precedence
Already versioned content promoted to another branch	Reuse the existing version
No API version, requirements, documentation or test change	No new tag

Documentation inputs are docs/, README.md, README_CHATBOT.md and zensical.toml. Any requirements content change counts, including additions, removals, comments and dependency upgrades or downgrades. Changes are compared with the last applicable tag, so skipped pipelines do not lose pending updates. All files under tests/ count, including unit/integration tests, shared helpers and fixtures. Additions, edits, deletions and renames trigger the same single patch increment; the number of test files or commits does not multiply it. Returning to the tagged contents before a release does not require a new tag. API major changes and version decreases remain errors. Adding an application produces one global minor increment.

The single create-project-version-tag job runs on pushes to develop and every release/* branch, after successful quality and applicable build checks, and before deployment. It fetches tags and the current branch head, serializes tag creation with a shared resource group, skips superseded pipelines, and calls the version script once. Other branches and merge-request pipelines do not publish versions. Tag pipelines remain disabled.

The job requires GITLAB_RELEASE_TOKEN when a new tag must be created. Configure its availability for develop and every release/* branch, including any branch protection required by the GitLab variable settings. Keep the token masked and protected, protect develop and the release/* pattern, and grant its owner permission to create protected v* tags. A missing token or failed tag publication fails the job and prevents deployment. Rundeck and GitOps jobs run only after the upstream checks succeed.

An applicable tag belongs to the branch history or to an identical committed Git tree represented in that history. This preserves the version across parallel release merges whose commit IDs differ but complete contents match. Equal API version numbers alone do not establish equivalence. Existing tags are immutable: a collision with different content fails, and rerunning a published release does not create another version. The initial version remains v1.0.0.

Documentation displays API versions read directly from source and a dependency list read from requirements.txt. Generation happens only in the isolated build cache; the site title stays unversioned. Git tags are a CI release concern and are not required to build the site. Docker and source copies deployed through Rundeck/Ansible use the same build path without a version argument or additional metadata file. Documentation-only, requirements-only and test-only releases receive their global patch tag without an artificial API version change.

Documentation workflow¶

The Markdown sources are docs/; site/ is ignored generated output. When a public endpoint, environment setting, event, index or operational behavior changes:

update the relevant English functional pages under docs/ and the navigation in zensical.toml;
after API version changes, refresh HTTP sources with python -m scripts.update_http_reference when contracts, routes, schemas, mounts, metadata or dependencies change;
commit the documentation sources, including docs/api/http/ and docs/includes/http/, with the behavior they describe;
let the conditional hooks check HTTP freshness and create or reuse the strict English build in the content-addressed cache outside the worktree.

git status --short docs zensical.toml

For standalone validation, run python -m scripts.build_documentation. It generates API version banners and the dependency list from source files in the isolated project, then runs the strict build. Its cache depends only on build inputs. Git and release metadata are unnecessary for documentation. Invalid API version literals and build errors still fail validation. CI does not run a documentation build. Changes created by Renovate or directly in GitLab do not pass through the local documentation hooks; refresh and check the HTTP references locally before promoting those changes. Application startup invokes the same builder in a background subprocess after API initialization, including outside Docker. /docs/ returns HTTP 503 until the completed site is ready, while health checks and Swagger remain available. See the runtime lifecycle. Never stage a local site/ directory.

The site enables breadcrumbs, a table of contents that follows the current section, and URL anchor tracking. These navigation features use the active deployment URL and do not require a canonical site address.

For a static documentation preview, build and serve the printed cache directory:

python -m scripts.build_documentation
python -m http.server 8000 --directory /path/printed/by/the/build

Git workflow¶

The mandatory commit hook owns repository checks. Development commits run the complete test suite; promotion merges into release/* reuse those results and skip that suite while retaining other hooks and GitLab quality checks. Run pre-commit run --all-files only for explicit repository-wide verification or quality-tool configuration changes.

git switch -c feature/my-feature develop
git add <files>
git commit -m 'feat(scope): describe the change'
git push origin feature/my-feature

Do not commit .env, tokens, tenant data or generated debugging output.

For release promotion, ask the agent to use $aihub-release-merge with a target such as staging or release/prod. It merges the same pinned develop commit into release/integration and release/int_pg concurrently, using separate worktrees and Python environments. Once integration is validated and pushed, the integration → staging → preprod → prod chain continues up to the requested target without waiting for int_pg. Stages within that chain remain sequential. Each stage pulls its destination with git pull --ff-only, validates the merge with the repository commit gates, and pushes before continuing. Pure promotions retain existing API versions. A failed pull, unresolved conflict, validation failure or rejected push stops only the dependent path; the independent path can finish and completed pushes remain published. The agent waits for both paths before reporting the overall result, including partial completion if either path failed.

For a combined request such as "commit on develop, push, then merge through prod", the agent first completes $aihub-prepare-commit and the requested push. It verifies that develop is synchronized with origin/develop before starting $aihub-release-merge.