> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brainstormer.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent Provisioning

> In-platform architecture of the URL-to-agent orchestrator: the single AgentProvisioningService, the phase FSM, Redis-to-SSE fan-out, the agent_builds state machine, service-account JWT minting, and plan/credit pre-flight.

## Overview

Agent Provisioning turns a creator's source URL (a YouTube or Instagram channel) into a fully-provisioned, chat-ready agent. A single server-side orchestrator — `AgentProvisioningService` (`services/bot/src/services/agent-provisioning.service.ts`) — owns the entire chain: detect the source, scrape recent content, analyze the creator's voice, create the knowledge base and agent, link and index sources, and (optionally) publish a public slug.

<Note>
  This page documents the **internal platform architecture**. For the consumer-facing async API, see the [Provisioning quickstart](/developer/provisioning-quickstart) and the [`agents:from-url` reference](/api-reference/provisioning/from-url).
</Note>

## Callers

Two entry points run the same `provision()` method. Behavioural differences are encoded as input flags.

| Caller           | Entry route                          | Backend                                                   | `waitForIndexing` | Default `mode` |
| ---------------- | ------------------------------------ | --------------------------------------------------------- | ----------------- | -------------- |
| In-app wizard    | `POST /bots/provision`               | enqueue build, poll `GET /bots/provision/builds/:buildId` | `false`           | `full`         |
| Public async API | `POST /v1/agents:from-url` (gateway) | `POST /agent-builds` -> BullMQ worker                     | `true`            | `preview`      |

Both paths enqueue a BullMQ `agent-provisioning` job and return `202` with a `buildId` plus poll/stream URLs; neither blocks the HTTP request. The wizard returns as soon as the agent exists and indexes in the background. The public API waits for the bounded index to reach a terminal state before emitting `ready`.

## Phase State Machine

`provision()` advances a monotonic phase FSM, emitting one `ProvisioningEvent` per transition with an interpolated `progress` weight.

```
queued -> detecting -> scraping -> analyzing -> provisioning -> indexing -> ready
                                                                          \-> failed
```

| Phase          | Progress | What happens                                                                                                                     |
| -------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `queued`       | 0        | Job accepted, build row created                                                                                                  |
| `detecting`    | 5        | Platform + handle parsed from URL; plan/credit pre-flight                                                                        |
| `scraping`     | 15       | `POST /knowledge/light-scrape` reads recent public content + brand                                                               |
| `analyzing`    | 35       | `CreatorAnalysisService` derives the system prompt, welcome, description                                                         |
| `provisioning` | 65       | KB created, sources batched, `bots` + RBAC + tools + prompt config + widget config                                               |
| `indexing`     | 82       | (async only) poll KB sources until terminal or timeout                                                                           |
| `ready`        | 100      | Agent chat-ready; result returned/persisted                                                                                      |
| `failed`       | 100      | Carries a `ProvisioningErrorCode` (`scrape_failed`, `analysis_failed`, `kb_required`, `plan_limit`, `insufficient_credits`, ...) |

Document-only inputs skip `scraping`/`analyzing` and produce a blank agent with a warning. Failures throw `ProvisioningServiceError`, which carries the error code, an HTTP status, and a `retryable` flag.

## Redis-to-SSE Fan-out

Progress is decoupled from the SSE connection through Redis, so the SSE client can land on a **different bot replica** than the one running the job.

```mermaid theme={null}
graph LR
    A[Provisioning worker] -->|onEvent| B[handleEvent]
    B -->|persist| C[(agent_builds)]
    B -->|publish| D[Redis pub/sub<br/>provision:progress:buildId]
    B -->|terminal only| H[HMAC webhook]
    D --> E[provision-bridge<br/>psubscribe]
    E -->|emit progress:buildId| F[SSE route]
    F -->|text/event-stream| G[Client]
```

Each event is fanned out three ways by `handleEvent` in `provisioning.queue.ts`: persisted to `agent_builds`, published to Redis `provision:progress:{buildId}`, and — on terminal events only — delivered to the consumer's optional HMAC-signed webhook (`provision-bridge.service.ts`: `services/bot/src/services/provision-bridge.service.ts`).

The SSE route maps phases to SSE event names (`ready` -> `done`, `failed` -> `error`, else `progress`), uses the event's `emittedAt` as the SSE `id:` for **`Last-Event-ID` replay**, replays the current snapshot on connect, and closes immediately if the build is already terminal.

## The `agent_builds` State Machine

Migration `080_agent_provisioning.sql` adds `agent_builds` — the durable state machine behind every build (both callers; `application_id` is `NULL` for the wizard path). It powers:

* **Snapshot / poll** — `GET /agent-builds/:buildId` returns the latest `ProvisioningEvent` (stored in `last_event`).
* **SSE replay** — `last_event` + `last_event_id` let a reconnecting client resume.
* **Idempotency** — a client `Idempotency-Key` short-circuits duplicate submits via `findByIdempotencyKey`.
* **Handle dedup** — `findReusableByHandle` returns a fresh, published agent for the same `(org, platform, normalized_handle)`, making repeat submits instant and free.
* **Preview TTL** — `preview` builds get `expires_at = now + 7 days`.
* **Full result** — migration `081` adds a `result JSONB` column so the wizard's step-2 review hydrates from a polled build.

A node-cron worker (`workers/provisioning-cleanup.worker.ts`, every 6h, env-gated `PROVISIONING_CLEANUP_ENABLED=true`) soft-deletes expired preview resources (KB links + knowledge base) and stamps `cleaned_at`.

## Service-Account JWT & Cost Attribution

Bot-side writes run **in-process** via the existing repositories. Knowledge-side work (light-scrape, KB create, `sources/batch`) and the three AI suggestion endpoints run over **internal HTTP**, authenticated with a short-lived (600s) service-account JWT minted via `signJwt(..., config.JWT_SECRET)`:

```ts theme={null}
signJwt(
  {
    userId: input.createdBy, // service-account UUID (API) or real user UUID (wizard)
    email: "service-account@brainstormer.internal",
    platformRole: "user",
    organizations: [{ id: input.organizationId, role: input.orgRole || "member" }],
  },
  config.JWT_SECRET,
  600,
);
```

For the public API, every API key resolves to a real `service_account_user_id` (migration `080`, `api_applications.service_account_user_id`). That UUID becomes `created_by`/`owned_by`, the `resource_permissions` grant target, and the **billing `user_id`** — so all provisioning costs attribute to the service account, not a synthetic string. Each provider-cost building block (creator analysis, light-scrape, KB embedding, AI suggestions) sets its own narration at its own boundary; the orchestrator records no un-narrated direct cost.

## Knowledge Base Readiness for Auto-Publish

When `input.publish` is `true`, the orchestrator verifies that the agent has at least one linked knowledge base with at least one indexed document **before** publishing the public slug. If the KB is missing or still indexing, the build fails with:

| Error code    | Meaning                                                                          |
| ------------- | -------------------------------------------------------------------------------- |
| `kb_required` | No linked KB, or no indexed documents yet. The build cannot publish a live slug. |

This prevents public agents from going live without grounded context. The in-app wizard path (`publish: false`) creates the agent as a draft and defers the readiness check to the user-driven **Go Live** step.

## Plan-Limit & Credit Pre-flight

`assertWithinLimits()` runs server-side **before** any work, so both callers are covered. It reads `billing.getPlanLimitsAndUsage()` and throws `ProvisioningServiceError(..., httpStatus 402)`:

* **Credit balance** is always enforced — `creditBalance <= 0` throws `insufficient_credits` even for superadmins.
* **Resource limits** (`maxAgents`, `maxKbs`) throw `plan_limit`, but are skipped when `bypassPlanLimits` is set (superadmins, mirroring the gateway billing-guard).

The sync wizard route maps the 402 straight back to the client; the async worker converts it into a `failed` event. Orgs with no billing account are treated as unmetered.

## Agent Creation Contract Compliance

The orchestrator honors the full [Agent Creation Contract](/developer/architecture/prompt-config) — `POST /bots` alone is insufficient. In one run it:

1. Creates the `bots` row + RBAC grant (`createBot`, owner = `owned_by`).
2. Links the KB via `kb_agent_links` (`upsertKBLinks`).
3. Writes `bot_tools` including `builtin:kb_context_injection` (and `hitl:escalate_to_human` for support intent).
4. Writes `bot_prompt_configs/system` (`mode: "fixed"`) so the editor + version history populate.
5. Writes `bots.widget_config.welcome` (+ brand + starters) when configured.
6. Pins the latest prompt versions, then publishes a unique public slug when `publish` is set.

## Bounded Preview Ingestion

`preview` is a `mode` flag on the same `provision()` path. It defaults to `maxItems: 8`, the only ingestion hint plumbed to connectors via `sources/batch` (`recentFirst` / `transcriptsOnly` are accepted but best-effort). The async submit route caps `ingestion.maxItems` at `50`. The `indexing` wait uses a `120s` deadline for preview (vs `180s` for full) and proceeds to `ready` on timeout, letting background indexing continue.

## Key Files

| File                                                                   | Role                                                         |
| ---------------------------------------------------------------------- | ------------------------------------------------------------ |
| `services/bot/src/services/agent-provisioning.service.ts`              | The orchestrator (`provision()`)                             |
| `services/bot/src/queues/provisioning.queue.ts`                        | BullMQ worker, 3-way event fan-out, SSRF-guarded webhook     |
| `services/bot/src/services/provision-bridge.service.ts`                | Redis pub/sub -> in-process SSE emitter                      |
| `services/bot/src/routes/provisioning.routes.ts`                       | `agent-builds` create / poll / SSE routes                    |
| `services/gateway/src/routes/public-api.ts`                            | `/v1/agents:from-url` surface (scope, rate-limit, Turnstile) |
| `services/bot/src/workers/provisioning-cleanup.worker.ts`              | Preview TTL cleanup cron                                     |
| `infrastructure/postgres/migrations/080_agent_provisioning.sql`, `081` | `agent_builds` table, service-account user, `result` column  |

Full internal contract: `docs/architecture/AGENT_PROVISIONING_API.md`.