> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brainstormer.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Documents

> Upload, manage, and process documents in knowledge bases. Add URL sources and connectors.

# Documents

Upload documents to knowledge bases for indexing, manage document lifecycle, and configure URL/feed sources for automatic content ingestion.

<Note>
  All API requests require a valid JWT token in the `Authorization: Bearer <token>` header. The API Gateway decodes the JWT and forwards auth context (`user-id`, `organization-id`, `user-email`, `x-platform-role`, `x-org-role`) as headers to downstream services.
</Note>

<Warning>
  This endpoint incurs provider cost and records a billing event via `record_external_cost_event()`. Credits are deducted from the organization's wallet based on the configured margin multiplier.
</Warning>

## Upload Document

<ParamField method="POST" path="/api/knowledge/kb/:id/documents" />

Upload a file to a knowledge base. The document is stored and queued for asynchronous processing (text extraction, chunking, embedding, and optional graph extraction).

### Path Parameters

<ParamField path="id" type="string" required>KB UUID.</ParamField>

### Request Body

Multipart file upload. Send the file as a `multipart/form-data` request.

### Supported File Types

| Type   | Extensions                               |
| ------ | ---------------------------------------- |
| PDF    | `.pdf`                                   |
| Word   | `.docx`, `.doc`                          |
| Text   | `.txt`, `.md`, `.csv`                    |
| Web    | `.html`, `.htm`                          |
| Images | `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp` |
| Audio  | `.mp3`, `.wav`, `.m4a`, `.ogg`           |
| Video  | `.mp4`, `.webm`, `.mov`                  |

### Response (201)

<ResponseField name="document" type="object">
  The created document record.

  <Expandable title="Document object">
    <ResponseField name="id" type="string">Document UUID.</ResponseField>
    <ResponseField name="knowledgeBaseId" type="string">Parent KB UUID.</ResponseField>
    <ResponseField name="title" type="string">Document title (filename).</ResponseField>
    <ResponseField name="mimeType" type="string">File MIME type.</ResponseField>
    <ResponseField name="fileSizeBytes" type="number">File size in bytes.</ResponseField>
    <ResponseField name="processingStatus" type="string">Current status (see below).</ResponseField>
    <ResponseField name="createdAt" type="string">ISO 8601 timestamp.</ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="jobId" type="string">
  BullMQ job ID for tracking processing progress.
</ResponseField>

<ResponseField name="message" type="string">
  Confirmation message.
</ResponseField>

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST https://your-domain.com/api/knowledge/kb/KB_ID/documents \
    -H "Authorization: Bearer $TOKEN" \
    -F "file=@/path/to/document.pdf"
  ```

  ```javascript JavaScript theme={null}
  const formData = new FormData();
  formData.append("file", fileBlob, "document.pdf");

  const response = await fetch(
    `https://your-domain.com/api/knowledge/kb/${kbId}/documents`,
    {
      method: "POST",
      headers: { Authorization: `Bearer ${token}` },
      body: formData,
    }
  );

  const { document, jobId } = await response.json();
  ```
</CodeGroup>

### Processing Pipeline

After upload, documents go through this async pipeline:

1. **Load**: LlamaParse (for PDF/DOCX) or LangChain fallback loaders
2. **Chunk**: Split into chunks (1000 characters, 200 overlap)
3. **Embed**: Generate multimodal embeddings via Gemini Embedding 2 (3072 dimensions)
4. **Store**: Upsert vectors to Pinecone (production) or ChromaDB (local)
5. **Graph** (optional): Extract entities and relationships
6. **Summarize** (optional): Generate document summary for KB map

### Processing Statuses

| Status       | Description                        |
| ------------ | ---------------------------------- |
| `pending`    | Queued for processing              |
| `processing` | Currently being processed          |
| `completed`  | Successfully indexed               |
| `failed`     | Processing failed (can be retried) |

***

## List Documents

<ParamField method="GET" path="/api/knowledge/kb/:id/documents" />

List all documents in a knowledge base.

### Path Parameters

<ParamField path="id" type="string" required>KB UUID.</ParamField>

### Query Parameters

<ParamField query="limit" type="number" default="50">Maximum results.</ParamField>
<ParamField query="offset" type="number" default="0">Pagination offset.</ParamField>

### Response (200)

<ResponseField name="documents" type="object[]">Array of document objects.</ResponseField>
<ResponseField name="total" type="number">Total document count.</ResponseField>
<ResponseField name="limit" type="number">Applied limit.</ResponseField>
<ResponseField name="offset" type="number">Applied offset.</ResponseField>

```bash curl theme={null}
curl https://your-domain.com/api/knowledge/kb/KB_ID/documents?limit=20 \
  -H "Authorization: Bearer $TOKEN"
```

***

## Delete Document

<ParamField method="DELETE" path="/api/knowledge/kb/:id/documents/:docId" />

Delete a document and its associated chunks and embeddings from a knowledge base.

### Path Parameters

<ParamField path="id" type="string" required>KB UUID.</ParamField>
<ParamField path="docId" type="string" required>Document UUID.</ParamField>

### Response (200)

```json theme={null}
{
  "message": "Document deleted successfully",
  "documentId": "doc-uuid",
  "chunksDeleted": 12
}
```

<ResponseField name="message" type="string">Confirmation.</ResponseField>
<ResponseField name="documentId" type="string">Deleted document UUID.</ResponseField>
<ResponseField name="chunksDeleted" type="number">Number of chunk/vector rows removed.</ResponseField>

***

## Retry Failed Document

<ParamField method="POST" path="/api/knowledge/kb/:id/documents/:docId/retry" />

Re-queue a failed document for processing. Resets the status to `pending` and creates a new processing job.

### Path Parameters

<ParamField path="id" type="string" required>KB UUID.</ParamField>
<ParamField path="docId" type="string" required>Document UUID.</ParamField>

### Response (200)

<ResponseField name="message" type="string">Confirmation.</ResponseField>
<ResponseField name="documentId" type="string">Document UUID.</ResponseField>
<ResponseField name="jobId" type="string">New processing job ID.</ResponseField>

***

## URL Sources

### Add URL Source

<ParamField method="POST" path="/api/knowledge/kb/:id/sources/url" />

Add a URL or feed source to a knowledge base. Content is fetched, processed, and indexed. Supports single URLs, RSS feeds, and web crawling.

#### Path Parameters

<ParamField path="id" type="string" required>KB UUID.</ParamField>

#### Request Body

<ParamField body="url" type="string" required>
  Source URL (webpage, RSS feed, or social media profile).
</ParamField>

<ParamField body="sourceType" type="string" default="url">
  Source type: `url`, `rss`, `linkedin`, `twitter`, `medium`.
</ParamField>

<ParamField body="syncEnabled" type="boolean" default="false">
  Enable automatic periodic re-sync.
</ParamField>

<ParamField body="syncFrequencyMinutes" type="number" default="60">
  Sync interval in minutes (when `syncEnabled` is true).
</ParamField>

<ParamField body="maxItems" type="number">
  Maximum items to fetch from feeds.
</ParamField>

<ParamField body="maxPages" type="number">
  Maximum pages to crawl.
</ParamField>

<ParamField body="crawlDepth" type="number">
  Link crawl depth (0 = single page only).
</ParamField>

<ParamField body="mediaEmbeddingStrategy" type="string">
  Strategy for media content: `native` (embed directly) or `transcription` (convert to text first).
</ParamField>

<ParamField body="visualEntityExtraction" type="boolean">
  Enable visual entity extraction from images.
</ParamField>

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST https://your-domain.com/api/knowledge/kb/KB_ID/sources/url \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://blog.example.com/feed",
      "sourceType": "rss",
      "syncEnabled": true,
      "syncFrequencyMinutes": 120
    }'
  ```

  ```javascript JavaScript theme={null}
  await fetch(
    `https://your-domain.com/api/knowledge/kb/${kbId}/sources/url`,
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${token}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        url: "https://blog.example.com/feed",
        sourceType: "rss",
        syncEnabled: true,
        syncFrequencyMinutes: 120,
      }),
    }
  );
  ```
</CodeGroup>

***

## Light Scrape

<ParamField method="POST" path="/api/knowledge/light-scrape" />

Quick URL scrape for content preview. Fetches and extracts text from up to 10 URLs without full KB indexing. Used by the Creator Wizard for AI profile generation.

### Request Body

<ParamField body="urls" type="object[]" required>
  Array of URLs to scrape (max 10).

  <Expandable title="URL object">
    <ParamField body="url" type="string" required>URL to scrape.</ParamField>
    <ParamField body="sourceType" type="string" required>Source type label.</ParamField>
  </Expandable>
</ParamField>

### Response (200)

<ResponseField name="results" type="object[]">
  Scrape results for each URL.

  <Expandable title="Result object">
    <ResponseField name="url" type="string">The scraped URL.</ResponseField>
    <ResponseField name="sourceType" type="string">Source type.</ResponseField>
    <ResponseField name="title" type="string">Page title.</ResponseField>
    <ResponseField name="textContent" type="string">Extracted text (max 2000 chars per URL).</ResponseField>
    <ResponseField name="author" type="string">Author if detected.</ResponseField>
    <ResponseField name="excerpt" type="string">Page excerpt if available.</ResponseField>
    <ResponseField name="error" type="string">Error message if scrape failed for this URL.</ResponseField>
  </Expandable>
</ResponseField>

```bash curl theme={null}
curl -X POST https://your-domain.com/api/knowledge/light-scrape \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": [
      {"url": "https://example.com/about", "sourceType": "website"},
      {"url": "https://blog.example.com/post-1", "sourceType": "blog"}
    ]
  }'
```

<Note>
  Light scrape has a 10-second timeout per URL and returns a maximum of 2000 characters of text per URL. For full content indexing, use the document upload or URL source endpoints.
</Note>

***

## Text Extraction

<ParamField method="POST" path="/api/knowledge/extract-text" />

Extract text and images from an uploaded file without indexing it into a knowledge base. Useful for previewing content or processing attachments.

### Request Body

Multipart file upload.

### Response (200)

Returns extracted text content and any images found in the document.