Skip to main content

Changelog

All notable changes to Brainstormer V2 are documented here. This project follows Semantic Versioning.

Unreleased

ChromaDB upgraded from 0.6.3 to 1.5.9

The ChromaDB server has been upgraded from chromadb/chroma:0.6.3 (Python/SQLite) to chromadb/chroma:1.5.9 (Rust server) to fix the SQLite dangling-transaction deadlock that caused all POST/write requests to hang after abrupt connection drops.
  • Docker image pinned to chromadb/chroma:1.5.9 in docker-compose.yml
  • Volume mount changed from /chroma/chroma to /data (1.x default persist path)
  • Healthcheck added (bash TCP check on port 8000) for depends_on: condition: service_healthy
  • _initPromise bypass removed from vector-store.service.ts — the 1.x Rust server handles concurrent access natively
  • withChromaLock single-flight queue removed — no longer needed with the concurrent-safe 1.x server
  • All existing KB collections migrated in place with zero data loss

Jest tests now resolve @brainstormer/shared/ssrf correctly

services/knowledge/src/services/__tests__/brand-extraction.service.test.ts failed at suite load — the jest moduleNameMapper catch-all expected

Configurable knowledge-graph and document-summary models

Admins can now change the models used for knowledge-graph extraction and document summarization directly from Admin → System Config → Knowledge Base, without rebuilding or redeploying the knowledge service.
  • New managed platform keys: GRAPH_EXTRACTION_MODEL and SUMMARY_MODEL.
  • Live reads: both keys are resolved at extraction time via platformConfig.get() and invalidated through the existing Redis bus.
  • Save-time validation: model names are checked against the OpenRouter /models list before they are persisted, preventing typos or sunset models from silently breaking ingestion.
  • Failure visibility: graph extraction now records per-call failures and maps them to failed, partial, or completed on kb_documents.graph_status. Repeated failures are logged as errors and flow through the existing GlitchTip → Slack pipeline.

Signup plan cards now match the plan catalog

The Choose your plan signup page now derives numeric plan bullets (monthly credits, agents, and knowledge bases) directly from the live plan configuration in the database. Plan descriptions are now editable by superadmins under Admin → Plans, so the signup page stays in sync with the catalog.
  • Non-enterprise plans show bullets computed from monthlyCredits, maxAgents, and maxKbs.
  • Enterprise bullets continue to come from the plan features field.

Ghost blog platform support for knowledge bases

Ghost is now available as a selectable blog platform in the Add a New Source dialog. Users can add Ghost blogs (.ghost.io or custom domains with /ghost/ path) as knowledge base sources — posts are discovered via the Ghost RSS feed and indexed automatically.
  • Frontend — Ghost appears in the source type grid alongside Substack, Medium, and WordPress with teal brand styling. URL auto-detection works for .ghost.io and /ghost/ paths in both the KB source manager and Creator Wizard.
  • Backend — No changes needed; the existing blog-platform connector and GhostAdapter already handle Ghost feed discovery and ingestion.

Group-based access control for agent distribution

Published agents can now restrict public access to organization members and specific org groups.
  • Backendagent_distribution_settings gained require_org_membership and allowed_group_ids UUID[] (migration 098). NULL/{} means all groups allowed. A unified checkPublicAccess() guard enforces identity, membership, and group checks on GET /public/agents/:slug and related public conversation paths.
  • Repository helpersisUserInGroups() and isUserOrgMember() query org_group_members and organization_members directly in the bot service.
  • Gateway — the /public/* lane now strips inbound identity headers, then verifies any optional Authorization: Bearer token and forwards verified user-id/organization-id headers to the bot service.
  • Marketplace — agents with group or org-membership restrictions are excluded from GET /marketplace/agents because anonymous browsers cannot satisfy them.
  • Frontend — the Distribute / Web channel page has a new Require organization membership toggle and a group multi-select. MultiSelectDropdown was renamed to MultiSelect and a reusable Toggle component was extracted.

Visitor question preview in the Creator Wizard

Creators can now see the pre-chat questionnaire exactly as visitors will experience it before publishing an agent.
  • A Preview visitor flow button in Step 2 of the Creator Wizard opens the live visitor questionnaire overlay, including brand colors and logo.
  • A warning banner appears when more than three visitor questions are configured, reminding creators that each extra question increases drop-off.
  • The warning is stronger when every question is required.

Creator Wizard now explains when Continue is disabled

On the Your Content step, the Continue button stays disabled until at least one source is added. A helper hint now appears next to the button so users know what to do next:
  • When no source is added: “Add at least one source to continue”
  • When a URL, file, or shop domain is entered but not yet staged: “Add the source above to continue”
The hint disappears once a source is successfully added and the button becomes active.

Friendly provider-error messages in chat

Chat no longer surfaces raw OpenRouter billing or rate-limit errors to end users. Provider-level failures are mapped to user-friendly messages before reaching the UI. When the provider key hits its credit or token limit, the response uses code: "provider_limit_reached" and shows:
“This assistant is temporarily unavailable. Please try again later.”
The raw provider error, including any API key URLs, is sanitized before logging. Applies to authenticated chat, public widget chat, and chat-core headless consumers.

Indexing-status polling — eliminated double round-trip and error-banner flicker

The GET /bots/:id/indexing-status endpoint was making two identical requests to the knowledge service on every call. The redundant checkKbReadiness() pre-check was removed, cutting knowledge-service load in half across the three frontend components that poll every 15 seconds.
  • Backend — removed the duplicate fetch; the fallback message now surfaces the knowledge service’s error detail when available, falling back to a generic message on network failures.
  • FrontendsetIndexingError(null) now runs only after a successful poll, so the error banner no longer unmounts and remounts on every 15-second tick during sustained failures.

Mandatory knowledge-base readiness before agents go live

Agents can no longer be published without a linked, indexed knowledge base.
  • Backend gatesPOST /bots, PUT /bots/:id (when setting isDraft: false), POST /bots/:id/publish, POST /bots/:id/approve, POST /bots/:botId/distribution/publish, and the async provisioning worker all verify readiness before publishing.
  • Fail-closed — if the knowledge service is unreachable, publishing is blocked.
  • Frontend guards — the classic setup flow requires a KB before continuing, the tabbed builder disables the publish toggle without a linked KB, and the Distribute / Go Live pages disable Publish until the readiness check passes.
  • New proxy endpointGET /bots/:id/indexing-status surfaces indexing status through the bot service.

KB ingestion failure banner

Knowledge-base ingestion failures are now surfaced prominently on the agent editor and Go Live screens instead of sitting silently in the database.
  • Failure banner — when a linked KB has failed sources, a red Alert banner appears on the agent edit Knowledge Base tab and the Go Live screen, showing the error reason from lastSyncError.
  • One-click retry — each failed source has a [Retry] button that triggers a re-sync via the existing POST /kb/:id/sources/:sourceId/sync endpoint.
  • Deep link to details — a [Details] link opens the KB detail page for full sync history.
  • BackendGET /knowledge/agents/:agentId/kbs and GET /knowledge/kb now return syncHealth, failedSourceCount, and failedSources[] with {sourceId, sourceUrl, lastSyncError} per failed source.

Smarter retrieval — hybrid search, reranking, and diverse context

The knowledge-base retrieval pipeline was overhauled end to end.
  • Hybrid retrieval — every search now runs a semantic (vector) channel and a keyword (full-text) channel in parallel and fuses them with Reciprocal Rank Fusion. Exact identifiers — SKUs, error codes, names — reliably surface even when the wording differs from your documents.
  • Cross-encoder reranking — fused candidates are re-scored by a reranker that reads the query and each chunk together, for sharper relevance than similarity alone.
  • Diverse, budgeted context — before an agent answers, near-duplicate chunks are dropped (MMR) and total knowledge-base context is capped by a token budget, so retrieval never crowds out the answer.
  • Verified end to end by a new live Playwright suite that uploads a document, waits for indexing, searches, and asserts a grounded, cited answer.

Knowledge-base analytics & the learning loop

A new Analytics dashboard on every knowledge base shows what your content is actually doing — and retrieval now improves on its own as customers use it.
  • Outcome attribution — per-document resource performance (impressions, thumbs, helpfulness tier), a feedback-driven knowledge-gap list (“questions your customers asked that you couldn’t answer well”), and an impression → click → conversion funnel.
  • Learning Progress — a weighted score of the customer signals you’ve collected, the three learning stages it unlocks (smart ranking → learned weights → custom reranker), and how much more signal is needed.
  • Self-improving ranking — resources your customers engage with are automatically boosted in retrieval, floor-gated so early clicks never skew results. Toggle it per knowledge base from the analytics dashboard.

Refer & Earn — referral loop

Share your referral link, and both you and your invitees earn credits when they publish their first agent.
  • Earn your code on first publish — the moment you publish your first agent, a unique referral link is emailed to you and appears on the new Refer & Earn page in the sidebar.
  • Easy sharing — new users can follow your ?ref=CODE link (the code is auto-applied at signup) or enter the code manually in the “Referral code” field on the sign-up form.
  • Both sides earn on activation — when an invitee publishes their first agent, both you and the invitee receive add-on credits (amounts set by platform admins, defaults: 100 for referrers, 50 for referees). The invitee also unlocks their own referral code, keeping the loop going.
  • Per-code cap — each code has a maximum number of successful referrals (default 10) after which it is marked exhausted and no further rewards are issued.
  • Platform admin controls — superadmins can adjust credit amounts, the per-code cap, and toggle the program on/off from Admin → System Config without a redeploy.
  • All reward credits land in the non-expiring add-on credit pool and are emailed when earned.

Faster path to publishing a claimed agent

Publishing an agent after signup is more obvious and less error-prone, and the distribution section is easier to navigate.
  • Land on the publish page after claim signup — users who sign up by claiming an agent now arrive directly on that agent’s Web / Public Page settings (where the URL and Publish button live) instead of the dashboard, so they can publish without hunting for it.
  • URL first — the Web / Public Page now shows the URL slug and public URL at the top, above the landing-page and chat-widget settings.
  • “Channels” is now “Distribute” — the agent-editor tab was renamed, and the per-channel pages show a breadcrumb (Distribute › Web / Public Page) so the active channel is always clear.
  • Conflict-free default slug — the auto-filled URL slug is now checked for uniqueness and suffixed when needed, so a freshly claimed agent no longer fails to publish with a “slug already exists” error on a value the system chose.
  • Plan picker — selecting a plan during signup now scrolls the Continue button into view.
Vision text extraction on indexed images is more reliable and tunable, so editorial overlay text (headlines, venue names, award banners) shows up more consistently in knowledge-base search.
  • Tunable noise floor — the minimum extracted-text length is now a platform setting (Vision OCR Min Text Length, default 20) editable from Admin → System Config, no redeploy required.
  • Transient-failure retry — a failed extraction retries once after a short backoff before giving up, recovering from provider blips that previously lost the text silently.
  • Re-sync backfill — when a synced source re-runs with unchanged media, existing vision text is preserved (no wasted re-extraction) and any image still missing it is filled in, so overlay text becomes searchable without re-embedding unchanged images.

Landing → app agent claim handoff

Agents provisioned from the marketing landing page now transfer into a user’s account when they sign up. A first-party Funnel organization owns the agent and its knowledge base while the visitor previews it, absorbing all pre-signup cost. On signup email verification, the agent and KB re-parent into the new user’s organization — counting against their plan, with no re-indexing and no duplicated LLM cost. The handoff is keyed on the verified email and scoped strictly to the funnel application, so no other tenant’s agent is ever eligible for transfer. First-party funnel only; not a tenant-facing capability.

Searchable knowledge-base linking in the agent editor

The agent Knowledge tab scales to large workspaces and makes attached knowledge bases clear at a glance.
  • Linked knowledge bases appear in their own section at the top, always visible regardless of the search term, each with per-KB retrieval settings and a link to its detail page.
  • A search box finds knowledge bases by name or description; the picker shows a capped set of matches with a “Showing X of Y” hint instead of the whole list.
  • The picker scopes to the agent’s organization, so the agent’s own linked knowledge bases always load.

Read-only mode for the embeddable widget

The embeddable chat can now be locked to a read-only view — messages stay visible but the composer is hidden — so host pages can overlay their own login/payment wall (e.g. after a free-message limit) or show a shared, read-only conversation.
  • Activate on load with ?readonly=1 (or ?mode=readonly) on the iframe URL, or data-readonly="true" on the embed script.
  • Toggle at runtime from host gating logic via BrainstormerWidget.setReadOnly(true|false) — the one inbound command in an otherwise one-way bridge.
  • A new widget:readonly event (with the current flag + counts) fires whenever the mode flips, so the host can sync its overlay.
See Embeddable Chat Engine → Read-only mode.
  • Message copy control: the copy button now sits below each assistant message and is always visible (ChatGPT/Claude style) instead of appearing in a hover-only toolbar — so it works on touch devices. The non-functional “share” control was removed.
  • Direct sign-up landing: marketing traffic can now land straight on the sign-up form by appending ?mode=signup (also ?mode=register or ?signup=1) to the app URL, instead of always opening on the sign-in form.

Widget event counts continue across resumed conversations

When a returning visitor resumed an existing conversation in the embeddable widget, the message:sent / message:received event counts restarted at zero, ignoring the messages the conversation was loaded with. The counts now seed from the loaded message history, so login/payment-wall gating thresholds carry across page reloads instead of resetting.A new conversation:resumed event fires once an existing conversation finishes loading, carrying userMessageCount / assistantMessageCount / messageCount — so the host can re-apply a wall on reload without waiting for the visitor’s next message (widget:ready fires before the conversation loads and cannot carry counts). See Embeddable Chat Engine → Host page events.

Embed widget emits events to the host page

The embeddable chat now posts one-way events to the page that frames it, so host sites can build their own gating — login walls, paywalls, usage analytics — without any two-way coupling.
  • Events: widget:ready, conversation:started, message:sent, message:received, widget:error, delivered via postMessage in a namespaced envelope (source: "brainstormer-widget"). Payloads carry metadata and running per-session message counts — never message text.
  • Loader API: embed.js exposes BrainstormerWidget.on(type, cb) (and "*" for all events), with origin verification handled for you. Raw <iframe> embedders can listen for message directly.
  • Counts are a client-side UX signal — enforce real limits server-side too. See Embeddable Chat Engine → Host page events.

Choose API-key scopes in the Developer Portal

The Generate Key form lets you pick which scopes a brs_live_ key carries, so keys can reach the Agent Provisioning API.
  • Scope picker: chat, kb:read, and agents:write are selectable when generating a key (/developer). New keys default to chat + kb:read; tick agents:write to allow POST /v1/agents:from-url.
  • The key table now shows each key’s granted scopes.
  • A key missing a required scope is rejected by the gateway with 403 insufficient_scope.

Drop-in embed widget + anonymous operator presence

The embeddable chat widget is now live, and public visitors get real-time operator presence.
  • Embed Widget: a dependency-free /embed.js loader injects a floating chat launcher → iframe of a chrome-minimal /embed/:slug page. Configure + copy the snippet in the agent’s Channels → Embed Widget tab (data-slug, data-color, data-position). The agent must be published and not require login.
  • Anonymous operator-presence SSE: GET /api/public/agents/:slug/conversations/:id/events?anonymousId= streams operator messages / takeover / resolution to public visitors (no JWT; ownership scoped by anonymousId). Wired into chat-core’s PublicTransport.subscribeEvents. This also removes the spurious auth errors the public chat history panel used to log.

Anonymous file uploads for embedded chat

Visitors on a published agent page (or the embeddable widget) can now attach files to their messages without logging in — when the agent has file uploads enabled.
  • POST /api/public/agents/:slug/attachments: anonymous, slug + anonymousId scoped multipart upload. Returns an attachment ref ({ id, accessToken }) to pass in the chat message’s new attachments array.
  • Hardened: per-IP / per-anonymousId rate limits, size + MIME allowlist, and magic-byte sniffing (a file whose bytes don’t match its declared type is rejected). See the Anonymous Chat reference.
  • Billing narrations on the provisioning path are now cross-linked (source → agent → KB) instead of blank on the ledger.

Agent Provisioning & Chat API

A public, API-key-authenticated capability that lets any client app build a chat agent from a creator’s YouTube or Instagram URL — observe the build asynchronously, then chat anonymously with streaming responses grounded in the creator’s content. No logged-in Brainstormer user required.
  • POST /v1/agents:from-url: kick off an async build from a channel URL, returning 202 with a buildId — non-blocking. Supports mode (bounded preview vs. full), intent, model, Idempotency-Key, and deliver.{stream,webhookUrl}.
  • Live progress (SSE): GET /v1/agents/builds/:id/events streams one ProvisioningEvent per phase transition (queued → detecting → scraping → analyzing → provisioning → indexing → ready/failed) with phase-weighted, display-ready progress copy. Replays state on connect; closes on terminal.
  • Polling snapshot: GET /v1/agents/builds/:id returns the latest ProvisioningEvent (same shape as SSE) for stateless / email-me-a-link backends.
  • HMAC-signed webhook: optional out-of-band delivery of the terminal event, signed X-Brainstormer-Signature: sha256=<hmac>, with retry/backoff.
  • Anonymous chat: once ready, browsers chat directly via the slug-based /api/public/agents/:slug/* endpoints — no API key, with optional SSE token streaming and starterQuestions to seed the UI.
  • Auth & abuse controls: agents:write scope (403 insufficient_scope), no-secrets-in-browser Pattern A backend proxy, Cloudflare Turnstile, per-key + per-IP rate limits (429 + Retry-After), idempotency, and handle dedup.
  • Structured error model: a flat ProvisioningErrorCode enum, each carrying retryable semantics, plus partial-success warning for thin content.
  • Developer docs: new API reference (from-url, build snapshot, SSE, webhook, anonymous chat), a 5-minute quickstart with a Next.js Pattern A proxy, auth & rate-limits and concepts guides, plus a machine-readable llms.txt.

Real-Time SSE Streaming for Chat

Chat responses now stream token-by-token in real-time instead of waiting for the full response. First token appears in under 1 second (previously 40-86 seconds of blank waiting).
  • Real SSE streaming: Tokens arrive via Server-Sent Events through all 4 layers (LangChain → Bot Service → Gateway → Frontend)
  • Non-streaming model fallback: Models that don’t support streaming automatically fall back to invoke-then-send
  • Parallelized setup: Independent pre-LLM calls (prompt resolution, conversation history, bot config) run concurrently via Promise.all
  • Latency instrumentation: Every chat request logs a structured timing breakdown (KB retrieval, TTFT, tokens/sec, post-processing) for ongoing optimization
  • Citation instructions: KB context now instructs the LLM to include [1], [2] citation markers in responses
  • Auto-linked @mentions: Instagram @handles in chat responses are now clickable links. URLs auto-linked via GitHub Flavored Markdown.
  • Chat button on editor: Published agents now have a “Chat” button in the editor header for quick access

Enhanced Creator Voice Analysis

The creator wizard now analyzes actual social media content (Instagram posts, videos, images) to generate voice-accurate system prompts instead of scraping empty homepage HTML.
  • Platform adapter integration: Instagram, YouTube, TikTok, Twitter URLs use Apify adapters to fetch real posts with captions, engagement metrics, and media
  • Media processing: Up to 5 media items get transcribed (video/audio) or vision-extracted (images) during light-scrape for richer voice analysis
  • Improved LLM prompt: Creator analysis now extracts verbal patterns, catchphrases, few-shot examples, and top-performing topics
  • Wizard UX: Phase-based progress messages (“Fetching posts…”, “Transcribing videos…”), skip link, enrichment summary badge
  • Generate from KB button: Re-generate a voice-accurate prompt anytime from the prompt editor using existing KB content
  • Graceful degradation: Every processing step has fallbacks — Apify fails → URLFetcher, transcription fails → caption only, vision fails → skip

Human-in-the-Loop (HITL) System

Complete operator oversight system for AI agent conversations — escalation, approval workflows, queue management, and real-time operator dashboard.
  • Escalation workflow: AI-triggered or manual operator takeover with configurable escalation mode (both, AI-only, manual-only)
  • Approval workflow: Hold AI responses for operator review before delivery (off, all responses, or AI-selected)
  • Queue management: Auto-assign queued escalations when operators connect, configurable timeout (60s–24h), outside-hours behavior (queue, message-only, fallback email)
  • Operator dashboard: Real-time escalation queue, stats bar, quick actions, WebSocket-powered live updates
  • Browser notifications: Notification API alerts for new escalations, assignments, and messages when tab is unfocused
  • In-app notification bell: Sidebar badge with pending count and recent alerts dropdown, visible from any page
  • Customizable messages: 8 user-facing messages configurable per-agent (escalation started, operator joined, queued, expired, approval pending, etc.)
  • Concurrency safety: Atomic claim/approval operations prevent double-claim race conditions in multi-operator environments
  • Reusable email verification: Generic OTP-based email verification system with <VerifiedEmailInput> component, used for fallback email config and dashboard email verification banner

Chat Stabilization — Routing, Citations, Prompt Editor

Several fixes consolidating the chat experience after the widget config rollout.
  • Path-based conversation URLs via [[...conversationId]] catch-all for both dashboard and public chat pages
  • Chat history continues conversations via path, not query param — prevents the variable form from re-prompting on existing conversations
  • Citation persistence: non-streaming path embeds sources in message metadata; frontend reads persisted metadata first, falls back to transient lastResponse
  • KB context prompt strengthened with explicit citation format rules and examples
  • CitationTooltip: richer hover tooltips with thumbnails and purple badges
  • EnhancedChatInterface: SSE + HITL updates aligned with new routing and metadata-sourced citations

Incremental Crawl Ingestion + Centralized Scraping

Knowledge-base ingestion now streams results progressively, and an internal meta-task model is configurable.
  • Crawl workers deliver pages incrementally as they’re fetched instead of waiting for the full batch; default page limit lowered to reduce over-crawl on large sites
  • Centralized scraping service consolidates URL fetching behind one adapter used by the crawler, light-scrape, and manual-sync paths
  • Internal “meta-task” model (used for creator analysis, parameter suggestion, summarization) is now configurable via platform config + org-level override — organizations can choose a cheaper/faster model for internal LLM work without affecting chat

Widget Config, Appearance Tab, Editor Reorganization

Agent appearance and end-user experience are now configured from a single Appearance tab on the editor, persisted in a new widget_config JSONB on bots (with org-level defaults in widget_defaults).
  • Editor reorganized from 8 tabs to 6 with header actions for Save / Preview / Publish
  • New Appearance tab — branding, chat styling, variable form sections
  • ChatThemeWrapper injects widget config as CSS variables into the chat surface
  • VariableFormOverlay — pre-chat form component rendered when the agent declares required template variables
  • Widget theming + variable form integrated into both authenticated and public chat interfaces
  • Org-level widget defaults configurable from Organization Settings → Brand Defaults
  • Draft preview banner on chat page when previewing an unpublished agent
  • Welcome message moved out of the tool system and into widget_config; builtin:variable_collection tool (non-functional) removed
  • Template variable selector added to the welcome message prompt editor
  • Public agent endpoint returns the merged org-default + agent widget config
  • New endpoints: GET /bots/:id/widget-config, PUT /bots/:id/widget-config, GET/PUT /auth/organizations/:id/widget-defaults
  • Shared WidgetConfig types + merge utility in packages/shared
  • Distribution simplified: visibility removed in favour of URL-based access; old /distribute page redirects to the editor Distribute tab

Unified Onboarding + Intent-Driven Agent Creation

Dual onboarding wizards (OrgOnboardingWizard, OnboardingWizard) replaced with a single UnifiedOnboardingWizard that handles plan selection and team setup in one flow. Agent creation now asks for an intent up front.
  • Five intents: Creator/Brand, Customer Support, Knowledge Assistant, Sales & Outreach, Personal Knowledge Base
  • Intent drives the generated system prompt, default tool selection, KB source ordering, and creator-analysis prompts
  • Billing plans moved from hardcoded ONBOARDING_PLANS to a DB-driven API (GET /auth/billing-plans) with Stripe-ready columns
  • New DB: bots.intent column, billing_plans table with Stripe pricing columns, intent-specific system prompt defaults
  • KBSourceAdder reorders sources based on the chosen intent
  • Fixed onboarding redirect loop + KB source icon mapping
  • Fixed race where navigating to CreatorWizard would briefly bounce through the onboarding redirect guard

Variable Form + Prompt Editor Reliability

Cluster of fixes around the variable form flow and prompt editor interactions.
  • Delay conversation creation until the variable form is submitted (previously created an orphan conversation)
  • Load stored conversation variables during chat + wait for widget config before deciding whether to show the variable form
  • Fix stale closure in the variable form → conversation creation flow
  • Prevent “Update Agent” from overwriting prompt editor changes made since load
  • Serialize template variables as {{name}} in the rich-text markdown output so they round-trip through saves
  • Normalize public agent response to include templateVariables on the agent object (matches authenticated shape)

Agent Prompt Configuration System

Unified, versioned prompt management replacing all hardcoded LLM prompts platform-wide.
  • New tables: prompt_type_registry, system_prompt_defaults, system_prompt_default_versions, bot_prompt_configs, bot_prompt_config_versions, conversation_variables; streaming_enabled added to bots
  • Three-tier prompt resolution: bot override, system default, code constant fallback
  • fixed (static) and generated (LLM-driven) modes per prompt type, per agent
  • Welcome message returned in POST /bots/:id/conversations response; supports streaming in generated mode
  • Dynamic variables ({{variable_name}}) with five-source precedence chain (per-message > sensitive > client > agent > system)
  • Sensitive variables stored encrypted via POST endpoint or signed JWT context tokens
  • Semantic versioning (major.minor) with auto-generated changelogs and rollback on all prompt configs
  • streaming_enabled field on bots; effective streaming = bot setting AND per-request stream param
  • Shared prompt rendering engine: packages/shared/src/prompt-renderer.ts
  • Bot service CRUD endpoints: GET/PUT/DELETE /bots/:id/prompt-configs/:type, version history, rollback
  • Auth service superadmin endpoints: prompt type registry CRUD, system defaults management with versioning
  • All endpoints proxied through gateway
  • PromptConfigEditor React component for agent edit page and admin UI
  • Creator wizard extended: AI analysis now returns suggestedWelcomeMessage + suggestedWelcomePrompt
  • Existing bots.system_prompt values migrated to bot_prompt_configs (backward compat preserved)
  • All previously hardcoded prompts seeded as system defaults
  • Billing integrated: generated mode prompts call record_external_cost_event()

Knowledge Graph + Document Registry + Summary Index

PostgreSQL-based knowledge graph for enhanced RAG retrieval.
  • 4 new database tables: kg_document_registry, kg_entities, kg_relationships, kg_communities; new columns on kb_documents for graph metadata
  • Multimodal entity extraction from text, images (vision model), video frames, and audio transcripts
  • Document summary index generates per-document summaries for KB map overview
  • Graph query service with entity search, relationship traversal, and community detection
  • User-controlled visual entity extraction configurable at source and document levels
  • New API endpoints: /map, /search/enhanced, /graph/search, /graph/stats, /reindex-graph, /graph/entities, /graph/visualization
  • Enhanced bot retrieval with KB map + graph context injected into agent chat
  • Frontend Graph Explorer page with force-directed visualization, Document Knowledge Panel, and Graph Stats section
  • Async graph processing via BullMQ graph-indexing queue
  • PostgreSQL-based graph storage with recursive CTE traversal (no Neo4j dependency)

Unified Content Processing Pipeline

Introduced ContentConnector pattern with ConnectorRegistry for all KB source ingestion.
  • All connectors (YouTube, Instagram, Twitter, RSS, blog-platform, URL, document) produce standardized NormalizedContent[] with dedup keys and content hashes
  • Vision text extraction via OpenRouter for images and PDF pages
  • Transcription service for audio and video content
  • ContentProcessingPipeline orchestrates extraction, chunking, embedding, and vector storage in a single flow
  • Extensible: new source types require only a new connector implementing the ContentConnector interface

Gemini Multimodal Embeddings Integration

Replaced OpenAI text-only embeddings with Gemini Embedding 2 (gemini-embedding-exp-03-07) for native multimodal RAG across the Knowledge Base system.
  • Text, image, video, and audio content embedded natively via @google/genai at 3072 dimensions (up from 1536)
  • User-configurable media embedding strategy per KB: “native” (embed media directly) or “transcription” (Whisper to text to embed)
  • Per-post document creation for social media (YouTube, Instagram) instead of concatenated text blobs
  • Media storage service with ffmpeg-based video/audio splitting for content exceeding Gemini limits
  • Per-modality billing integration (text per 1k chars, image flat rate, video/audio per second) via record_external_cost_event
  • Enhanced bot citations with platform badges, thumbnails, published dates, and “View Original” links
  • KB analytics dashboard with 5 new components: QueryTrendsChart, AgentUsageChart, KnowledgeGapsTable, ContentUtilizationTable, CostBreakdownCard
  • PATCH endpoint for KB settings updates (media embedding strategy toggle)
  • Section-based analytics API for lazy-loaded dashboard sections

Gemini Embedding 2 Multimodal RAG POC

Completed proof-of-concept validating native text-to-video, text-to-image, cross-language (Spanish/Hindi to English), and image-to-image retrieval using gemini-embedding-2-preview (8/9 tests passed, 89%). PDF native embedding unreliable (workaround: convert to images); multimodal queries underperform.

Light Scrape Creator Wizard

Replaced the 4-step creator wizard (Sources, Build Knowledge, AI Profile, Launch) with a 3-step flow (Sources, AI Profile, Launch) that eliminates the blocking KB sync wait.
  • New POST /knowledge/light-scrape endpoint extracts URL content instantly without full indexing
  • New POST /knowledge/kb/:id/sources/batch endpoint for bulk source creation after agent is created
  • analyzeScrapedContent() accepts raw scraped content directly instead of requiring a knowledgeBaseId
  • POST /bots/creator-analysis now accepts either { knowledgeBaseId } or { scrapedContent[] }, keeping backward compatibility
  • CreatorWizard rewritten: step 1 collects draft sources, step 2 runs light scrape + AI analysis in parallel, step 3 creates agent + KB + enqueues async sync
  • Net result: wizard completes in seconds instead of 5+ minutes; full KB sync runs asynchronously in the background

AI-Powered Creator Onboarding Wizard

Redesigned creator wizard from 5-step (profile, sources, ingest, prompt, launch) to a 4-step AI-driven flow (sources, ingest, analyze, launch).
  • New creator-analysis.service.ts analyzes ingested KB content via OpenRouter (claude-3-haiku) to extract creator profile and system prompt automatically
  • New POST /bots/creator-analysis endpoint accepts a KB ID and returns creatorProfile + systemPrompt
  • Profile step eliminated: creator identity is inferred from their own content rather than entered manually

Additional Features

  • Added onboarding path architecture for /agents/create to support multiple low-friction onboarding tracks (creator, classic)
  • Enhanced chat message rendering with repositioned actions within bubbles
  • Improved error handling with input validation in LangChain service
  • Complete multi-modal architecture design (WebRTC + SSE + WebSocket)
  • API Gateway service with health checks and service proxy
  • Auth Service with JWT and organization context
  • PostgreSQL database schema with multi-tenant support
  • Comprehensive development tooling and Git workflow

Bug Fixes

  • Fixed message actions positioning for better UX
  • Added null checks and filtering for invalid chat messages
  • Improved input validation with detailed error messages

Documentation Updates

  • Detailed architecture documentation for multi-modal platform
  • Implementation plan with parallel development strategy
  • Contributing guide with sophisticated code management protocol
  • Complete development setup and testing procedures

Build Infrastructure

  • Monorepo setup with Turborepo
  • Comprehensive TypeScript configuration
  • ESLint and Prettier code formatting
  • Husky Git hooks with conventional commits
  • Automated versioning and changelog generation

v0.1.0

Foundation Release

Initial project setup establishing the complete foundation for Brainstormer V2 — a next-generation AI chatbot platform built for multi-modal interactions (text, voice, and video) with enterprise-grade features.
  • Initial project setup with microservices architecture
  • Foundation for multi-modal AI chatbot platform
  • Enterprise-grade multi-tenant organization support
  • Real-time communication protocol design
Key Achievements:
  • Production-ready JWT authentication with organization context
  • Multi-tenant architecture: B2B-ready with custom billing rates per organization
  • API Gateway with service discovery, health checks, and proxy routing
  • Optimized PostgreSQL schema with proper relationships
  • Excellent development tooling with hot-reload and comprehensive logging

Initial Build Setup

  • Project structure with monorepo approach
  • Complete development environment setup
  • Docker Compose for local development
  • PostgreSQL database with proper migrations