Knowledge Service

Overview

Port: 4005 The Knowledge Service handles the RAG pipeline including document processing, content connectors, multimodal vector embeddings, knowledge graph extraction, and unified search across multiple knowledge bases.

Endpoints

Knowledge Base CRUD

Route	Method	Purpose
`/knowledge/kb`	POST	Create KB
`/knowledge/kb`	GET	List KBs (paginated)
`/knowledge/kb/:id`	GET	Get KB details
`/knowledge/kb/:id`	PATCH	Update KB
`/knowledge/kb/:id`	DELETE	Delete KB

Documents

Route	Method	Purpose
`/knowledge/kb/:id/documents`	POST	Upload documents (multipart)
`/knowledge/kb/:id/documents`	GET	List documents
`/knowledge/kb/:id/documents/:docId`	DELETE	Delete document
`/knowledge/extract-text`	POST	Extract text + images from uploaded file

Search

Route	Method	Purpose
`/knowledge/search`	POST	Unified search: semantic + image + graph-informed across multiple KBs
`/knowledge/kb/:id/search/log`	GET	Search log history (paginated, filterable)
`/knowledge/kb/:id/search/log`	DELETE	Clear search log history
`/knowledge/kb/:id/map`	GET	KB document registry + summaries

Knowledge Graph

Route	Method	Purpose
`/knowledge/kb/:id/graph/search`	POST	Knowledge graph entity search
`/knowledge/kb/:id/enhance-graph`	POST	Entity resolution + cross-document inference
`/knowledge/kb/:id/graph/entities/:entityId`	PATCH	Rename entity (old name becomes alias)
`/knowledge/kb/:id/graph/entities/:entityId/merge`	POST	Merge entities into canonical
`/knowledge/kb/:id/graph/entities/:entityId`	DELETE	Delete entity and its relationships
`/knowledge/kb/:id/graph/pinned-entities`	GET/PUT	Manage user-defined known entities for extraction
`/knowledge/kb/:id/entities`	GET	List extracted entities

Graph Optimization

Route	Method	Purpose
`/knowledge/kb/:id/graph/optimize/stats`	GET	Suggestion counts by type/status
`/knowledge/kb/:id/graph/optimize/suggestions`	GET	List optimization suggestions (filterable)
`/knowledge/kb/:id/graph/optimize/generate`	POST	Trigger suggestion generation (queue job)
`/knowledge/kb/:id/graph/optimize/ai-resolve`	POST	AI auto-resolve pending suggestions
`/knowledge/kb/:id/graph/optimize/suggestions/:id`	PATCH	Approve/reject single suggestion
`/knowledge/kb/:id/graph/optimize/apply`	POST	Apply approved suggestions

Agent Links

Route	Method	Purpose
`/knowledge/kb/:id/agents/:agentId`	POST	Link KB to agent
`/knowledge/kb/:id/agents/:agentId`	DELETE	Unlink KB from agent
`/knowledge/agents/:agentId/kbs`	GET	Get agent’s linked KBs

Utilities

Route	Method	Purpose
`/knowledge/light-scrape`	POST	Quick URL scrape (max 10 URLs)

RAG Pipeline

Indexing Flow

Pipeline steps:

Upload — Document, URL, or social media source
Load — LlamaParse (PDF/docx) or LangChain fallback loaders
Chunk — 1000 characters, 200 overlap
Extract Media — Vision text extraction for images/PDFs, transcription for audio/video
Embed — Gemini Embedding 2 (3072 dimensions, multimodal)
Store — Pinecone (production) or ChromaDB (local)
Graph Extract — Entity seeding from existing KB entities + LLM extraction
Summarize — Per-document summaries for KB map

Graph Enhancement

After document-level extraction, an optional enhance-graph pass runs:

Entity Resolution

LLM-confirmed duplicate detection via name similarity (Levenshtein, substring, abbreviation matching). Merged entities consolidate relationships, aliases, and mention counts.

Cross-Document Inference

Discovers relationships between entities appearing in different documents but never explicitly connected in any single document.

Unified Search Flow

POST /knowledge/search
  -> query + images + files + kbIds
  -> Parallel: semantic embedding search + image embedding search + graph-informed document discovery
  -> Merge + deduplicate + rank
  -> Tagged results with discoveryMethods (semantic/image/graph)
  -> relationshipPath + suggestedFollowUps

Content Connectors

All KB source ingestion flows through a unified pipeline built on the ContentConnector pattern.

Connector	Source Types
`youtube`	YouTube video/channel/playlist URLs
`instagram`	Instagram profile/post URLs
`twitter`	Twitter/X profile/post URLs
`rss`	RSS/Atom feed URLs
`blog-platform`	Substack, Medium, Ghost, and other blog platforms
`url`	Generic web URLs (fallback)
`document`	Uploaded files (PDF, CSV, TXT, DOCX, images, audio, video)

NormalizedContent

All connectors produce NormalizedContent[] with:

externalId (dedup key), contentHash (change detection via SHA-256)
title, textContent, media[] (image/video/audio attachments)
source metadata (connector name, URL, platform, author, publishedAt)
Optional: thumbnailUrl, engagementMetrics, mediaType

Core Services

Service	Purpose
`vector-store.service.ts`	Pinecone (prod) / ChromaDB (local) abstraction
`embedding.service.ts`	Gemini Embedding 2 multimodal vectors (3072 dims)
`document-loader.service.ts`	LlamaParse (PDF/docx) or LangChain fallback loaders
`content-pipeline.service.ts`	Chunk text, OCR images, transcribe audio/video, upsert embeddings
`queue.service.ts`	BullMQ async jobs: document, URL, graph indexing
`graph-extraction.service.ts`	Extract entities and relationships (multimodal, with entity seeding)
`unified-search.service.ts`	Orchestrates semantic, image, and graph-informed search
`graph-query.service.ts`	Graph traversal, entity lookup
`entity-resolution.service.ts`	LLM-powered duplicate entity detection and merging
`cross-document-inference.service.ts`	Infer relationships across documents
`graph-optimizer.service.ts`	Generate, AI-resolve, and apply graph optimization suggestions
`document-summary.service.ts`	Generate document summaries + KB map
`sync-scheduler.service.ts`	BullMQ repeatable jobs for RSS/URL auto-sync
`transcription.service.ts`	OpenRouter Whisper audio/video to text
`platform-adapters.service.ts`	LinkedIn, Twitter, RSS, Medium connectors

Database Tables

Table	Purpose
`knowledge_bases`	KB metadata (name, org, settings)
`kb_documents`	Documents (title, file_path, processing_status, platform)
`kb_document_chunks`	Text chunks with embedding IDs
`kb_sources`	RSS/URL sources with sync config
`kb_search_log`	Unified search logging (manual + agent RAG)
`kb_source_items`	Social media posts / feed items
`kb_agent_links`	Junction table linking KBs to bots
`kg_document_registry`	Per-document summaries and topics
`kg_entities`	Extracted entities (name, type, aliases, mention_count)
`kg_relationships`	Entity relationships (source, target, type, weight)

External Integrations

Integration	Config
Gemini Embedding 2	`GEMINI_API_KEY`, `EMBEDDING_MODEL`, `EMBEDDING_DIMENSIONS`
Pinecone	`PINECONE_API_KEY`, `PINECONE_INDEX_NAME`, `PINECONE_ENVIRONMENT`
ChromaDB (local)	`CHROMA_HOST`, `CHROMA_PORT`
LlamaParse	`LLAMA_CLOUD_API_KEY` (optional)
OpenRouter Whisper	`OPENROUTER_API_KEY` (transcription)
Redis/BullMQ	`REDIS_HOST`, `REDIS_PORT`

Critical Patterns

Billing

All external calls MUST log costs via record_external_cost_event().

Vector Store Abstraction

Use vectorStoreService abstraction — never call Pinecone or Chroma directly.

Async Processing

BullMQ for async processing: max 5 concurrent jobs, 3 retries.

Multimodal Chunks

Chunks are tagged with embeddingType: "text" | "image" | "video" | "audio" | "transcript".

Authentication

request.user.organizationId from JWT middleware, never raw headers.

RBAC

checkResourceAccess() before all KB operations.

Overview

Services

Architecture

Agent Provisioning API

Deployment

Design System (Neo)

Overview

Endpoints

Knowledge Base CRUD

Documents

Search

Knowledge Graph

Graph Optimization

Agent Links

Utilities

RAG Pipeline

Indexing Flow

Graph Enhancement

Unified Search Flow

Content Connectors

NormalizedContent

Core Services

Database Tables

External Integrations

Critical Patterns

​Overview

​Endpoints

​Knowledge Base CRUD

​Documents

​Search

​Knowledge Graph

​Graph Optimization

​Agent Links

​Utilities

​RAG Pipeline

​Indexing Flow

​Graph Enhancement

​Unified Search Flow

​Content Connectors

​NormalizedContent

​Core Services

​Database Tables

​External Integrations

​Critical Patterns

Overview

Endpoints

Knowledge Base CRUD

Documents

Search

Knowledge Graph

Graph Optimization

Agent Links

Utilities

RAG Pipeline

Indexing Flow

Graph Enhancement

Unified Search Flow

Content Connectors

NormalizedContent

Core Services

Database Tables

External Integrations

Critical Patterns