> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brainstormer.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Service

> Knowledge service: RAG pipeline, document processing, content connectors, vector search, embedding service, and knowledge graph extraction.

## Overview

**Port:** 4005

The Knowledge Service handles the RAG pipeline including document processing, content connectors, multimodal vector embeddings, knowledge graph extraction, and unified search across multiple knowledge bases.

## Endpoints

### Knowledge Base CRUD

| Route               | Method | Purpose              |
| ------------------- | ------ | -------------------- |
| `/knowledge/kb`     | POST   | Create KB            |
| `/knowledge/kb`     | GET    | List KBs (paginated) |
| `/knowledge/kb/:id` | GET    | Get KB details       |
| `/knowledge/kb/:id` | PATCH  | Update KB            |
| `/knowledge/kb/:id` | DELETE | Delete KB            |

### Documents

| Route                                | Method | Purpose                                  |
| ------------------------------------ | ------ | ---------------------------------------- |
| `/knowledge/kb/:id/documents`        | POST   | Upload documents (multipart)             |
| `/knowledge/kb/:id/documents`        | GET    | List documents                           |
| `/knowledge/kb/:id/documents/:docId` | DELETE | Delete document                          |
| `/knowledge/extract-text`            | POST   | Extract text + images from uploaded file |

### Search

| Route                          | Method | Purpose                                                               |
| ------------------------------ | ------ | --------------------------------------------------------------------- |
| `/knowledge/search`            | POST   | Unified search: semantic + image + graph-informed across multiple KBs |
| `/knowledge/kb/:id/search/log` | GET    | Search log history (paginated, filterable)                            |
| `/knowledge/kb/:id/search/log` | DELETE | Clear search log history                                              |
| `/knowledge/kb/:id/map`        | GET    | KB document registry + summaries                                      |

### Knowledge Graph

| Route                                              | Method  | Purpose                                           |
| -------------------------------------------------- | ------- | ------------------------------------------------- |
| `/knowledge/kb/:id/graph/search`                   | POST    | Knowledge graph entity search                     |
| `/knowledge/kb/:id/enhance-graph`                  | POST    | Entity resolution + cross-document inference      |
| `/knowledge/kb/:id/graph/entities/:entityId`       | PATCH   | Rename entity (old name becomes alias)            |
| `/knowledge/kb/:id/graph/entities/:entityId/merge` | POST    | Merge entities into canonical                     |
| `/knowledge/kb/:id/graph/entities/:entityId`       | DELETE  | Delete entity and its relationships               |
| `/knowledge/kb/:id/graph/pinned-entities`          | GET/PUT | Manage user-defined known entities for extraction |
| `/knowledge/kb/:id/entities`                       | GET     | List extracted entities                           |

### Graph Optimization

| Route                                              | Method | Purpose                                    |
| -------------------------------------------------- | ------ | ------------------------------------------ |
| `/knowledge/kb/:id/graph/optimize/stats`           | GET    | Suggestion counts by type/status           |
| `/knowledge/kb/:id/graph/optimize/suggestions`     | GET    | List optimization suggestions (filterable) |
| `/knowledge/kb/:id/graph/optimize/generate`        | POST   | Trigger suggestion generation (queue job)  |
| `/knowledge/kb/:id/graph/optimize/ai-resolve`      | POST   | AI auto-resolve pending suggestions        |
| `/knowledge/kb/:id/graph/optimize/suggestions/:id` | PATCH  | Approve/reject single suggestion           |
| `/knowledge/kb/:id/graph/optimize/apply`           | POST   | Apply approved suggestions                 |

### Agent Links

| Route                               | Method | Purpose                |
| ----------------------------------- | ------ | ---------------------- |
| `/knowledge/kb/:id/agents/:agentId` | POST   | Link KB to agent       |
| `/knowledge/kb/:id/agents/:agentId` | DELETE | Unlink KB from agent   |
| `/knowledge/agents/:agentId/kbs`    | GET    | Get agent's linked KBs |

### Utilities

| Route                     | Method | Purpose                        |
| ------------------------- | ------ | ------------------------------ |
| `/knowledge/light-scrape` | POST   | Quick URL scrape (max 10 URLs) |

## RAG Pipeline

### Indexing Flow

```mermaid theme={null}
graph LR
    A[Upload / URL] --> B[ConnectorRegistry]
    B --> C["ContentConnector<br/>(YouTube, Instagram, etc.)"]
    C --> D["NormalizedContent[]"]
    D --> E["Content Pipeline<br/>OCR, Transcription, Chunking"]
    E --> F["Embed<br/>(Gemini 3072d)"]
    F --> G["Store<br/>(Pinecone/Chroma)"]
    E --> H["Graph Extract<br/>(Entity Seeding)"]
    H --> I["Summarize"]
```

**Pipeline steps:**

1. **Upload** -- Document, URL, or social media source
2. **Load** -- LlamaParse (PDF/docx) or LangChain fallback loaders
3. **Chunk** -- 1000 characters, 200 overlap
4. **Extract Media** -- Vision text extraction for images/PDFs, transcription for audio/video
5. **Embed** -- Gemini Embedding 2 (3072 dimensions, multimodal)
6. **Store** -- Pinecone (production) or ChromaDB (local)
7. **Graph Extract** -- Entity seeding from existing KB entities + LLM extraction
8. **Summarize** -- Per-document summaries for KB map

### Graph Enhancement

After document-level extraction, an optional enhance-graph pass runs:

<Steps>
  <Step title="Entity Resolution">
    LLM-confirmed duplicate detection via name similarity (Levenshtein, substring, abbreviation matching). Merged entities consolidate relationships, aliases, and mention counts.
  </Step>

  <Step title="Cross-Document Inference">
    Discovers relationships between entities appearing in different documents but never explicitly connected in any single document.
  </Step>
</Steps>

### Unified Search Flow

```
POST /knowledge/search
  -> query + images + files + kbIds
  -> Parallel: semantic embedding search + image embedding search + graph-informed document discovery
  -> Merge + deduplicate + rank
  -> Tagged results with discoveryMethods (semantic/image/graph)
  -> relationshipPath + suggestedFollowUps
```

## Content Connectors

All KB source ingestion flows through a unified pipeline built on the **ContentConnector** pattern.

| Connector       | Source Types                                               |
| --------------- | ---------------------------------------------------------- |
| `youtube`       | YouTube video/channel/playlist URLs                        |
| `instagram`     | Instagram profile/post URLs                                |
| `twitter`       | Twitter/X profile/post URLs                                |
| `rss`           | RSS/Atom feed URLs                                         |
| `blog-platform` | Substack, Medium, Ghost, and other blog platforms          |
| `url`           | Generic web URLs (fallback)                                |
| `document`      | Uploaded files (PDF, CSV, TXT, DOCX, images, audio, video) |

### NormalizedContent

All connectors produce `NormalizedContent[]` with:

* `externalId` (dedup key), `contentHash` (change detection via SHA-256)
* `title`, `textContent`, `media[]` (image/video/audio attachments)
* `source` metadata (connector name, URL, platform, author, publishedAt)
* Optional: `thumbnailUrl`, `engagementMetrics`, `mediaType`

## Core Services

| Service                               | Purpose                                                              |
| ------------------------------------- | -------------------------------------------------------------------- |
| `vector-store.service.ts`             | Pinecone (prod) / ChromaDB (local) abstraction                       |
| `embedding.service.ts`                | Gemini Embedding 2 multimodal vectors (3072 dims)                    |
| `document-loader.service.ts`          | LlamaParse (PDF/docx) or LangChain fallback loaders                  |
| `content-pipeline.service.ts`         | Chunk text, OCR images, transcribe audio/video, upsert embeddings    |
| `queue.service.ts`                    | BullMQ async jobs: document, URL, graph indexing                     |
| `graph-extraction.service.ts`         | Extract entities and relationships (multimodal, with entity seeding) |
| `unified-search.service.ts`           | Orchestrates semantic, image, and graph-informed search              |
| `graph-query.service.ts`              | Graph traversal, entity lookup                                       |
| `entity-resolution.service.ts`        | LLM-powered duplicate entity detection and merging                   |
| `cross-document-inference.service.ts` | Infer relationships across documents                                 |
| `graph-optimizer.service.ts`          | Generate, AI-resolve, and apply graph optimization suggestions       |
| `document-summary.service.ts`         | Generate document summaries + KB map                                 |
| `sync-scheduler.service.ts`           | BullMQ repeatable jobs for RSS/URL auto-sync                         |
| `transcription.service.ts`            | OpenRouter Whisper audio/video to text                               |
| `platform-adapters.service.ts`        | LinkedIn, Twitter, RSS, Medium connectors                            |

## Database Tables

| Table                  | Purpose                                                     |
| ---------------------- | ----------------------------------------------------------- |
| `knowledge_bases`      | KB metadata (name, org, settings)                           |
| `kb_documents`         | Documents (title, file\_path, processing\_status, platform) |
| `kb_document_chunks`   | Text chunks with embedding IDs                              |
| `kb_sources`           | RSS/URL sources with sync config                            |
| `kb_search_log`        | Unified search logging (manual + agent RAG)                 |
| `kb_source_items`      | Social media posts / feed items                             |
| `kb_agent_links`       | Junction table linking KBs to bots                          |
| `kg_document_registry` | Per-document summaries and topics                           |
| `kg_entities`          | Extracted entities (name, type, aliases, mention\_count)    |
| `kg_relationships`     | Entity relationships (source, target, type, weight)         |

## External Integrations

| Integration        | Config                                                            |
| ------------------ | ----------------------------------------------------------------- |
| Gemini Embedding 2 | `GEMINI_API_KEY`, `EMBEDDING_MODEL`, `EMBEDDING_DIMENSIONS`       |
| Pinecone           | `PINECONE_API_KEY`, `PINECONE_INDEX_NAME`, `PINECONE_ENVIRONMENT` |
| ChromaDB (local)   | `CHROMA_HOST`, `CHROMA_PORT`                                      |
| LlamaParse         | `LLAMA_CLOUD_API_KEY` (optional)                                  |
| OpenRouter Whisper | `OPENROUTER_API_KEY` (transcription)                              |
| Redis/BullMQ       | `REDIS_HOST`, `REDIS_PORT`                                        |

## Critical Patterns

<AccordionGroup>
  <Accordion title="Billing">
    All external calls MUST log costs via `record_external_cost_event()`.
  </Accordion>

  <Accordion title="Vector Store Abstraction">
    Use `vectorStoreService` abstraction -- never call Pinecone or Chroma directly.
  </Accordion>

  <Accordion title="Async Processing">
    BullMQ for async processing: max 5 concurrent jobs, 3 retries.
  </Accordion>

  <Accordion title="Multimodal Chunks">
    Chunks are tagged with `embeddingType: "text" | "image" | "video" | "audio" | "transcript"`.
  </Accordion>

  <Accordion title="Authentication">
    `request.user.organizationId` from JWT middleware, never raw headers.
  </Accordion>

  <Accordion title="RBAC">
    `checkResourceAccess()` before all KB operations.
  </Accordion>
</AccordionGroup>