Changelog
All notable changes to Brainstormer V2 are documented here. This project follows Semantic Versioning.Unreleased
ChromaDB upgraded from 0.6.3 to 1.5.9
The ChromaDB server has been upgraded fromchromadb/chroma:0.6.3 (Python/SQLite) to chromadb/chroma:1.5.9 (Rust server) to fix the SQLite dangling-transaction deadlock that caused all POST/write requests to hang after abrupt connection drops.- Docker image pinned to
chromadb/chroma:1.5.9indocker-compose.yml - Volume mount changed from
/chroma/chromato/data(1.x default persist path) - Healthcheck added (
bash TCP check on port 8000) fordepends_on: condition: service_healthy _initPromisebypass removed fromvector-store.service.ts— the 1.x Rust server handles concurrent access nativelywithChromaLocksingle-flight queue removed — no longer needed with the concurrent-safe 1.x server- All existing KB collections migrated in place with zero data loss
Jest tests now resolve @brainstormer/shared/ssrf correctly
services/knowledge/src/services/__tests__/brand-extraction.service.test.ts failed at suite load — the jest moduleNameMapper catch-all expectedConfigurable knowledge-graph and document-summary models
Admins can now change the models used for knowledge-graph extraction and document summarization directly from Admin → System Config → Knowledge Base, without rebuilding or redeploying the knowledge service.- New managed platform keys:
GRAPH_EXTRACTION_MODELandSUMMARY_MODEL. - Live reads: both keys are resolved at extraction time via
platformConfig.get()and invalidated through the existing Redis bus. - Save-time validation: model names are checked against the OpenRouter
/modelslist before they are persisted, preventing typos or sunset models from silently breaking ingestion. - Failure visibility: graph extraction now records per-call failures and maps them to
failed,partial, orcompletedonkb_documents.graph_status. Repeated failures are logged as errors and flow through the existing GlitchTip → Slack pipeline.
Signup plan cards now match the plan catalog
The Choose your plan signup page now derives numeric plan bullets (monthly credits, agents, and knowledge bases) directly from the live plan configuration in the database. Plan descriptions are now editable by superadmins under Admin → Plans, so the signup page stays in sync with the catalog.- Non-enterprise plans show bullets computed from
monthlyCredits,maxAgents, andmaxKbs. - Enterprise bullets continue to come from the plan
featuresfield.
Ghost blog platform support for knowledge bases
Ghost is now available as a selectable blog platform in the Add a New Source dialog. Users can add Ghost blogs (.ghost.io or custom domains with /ghost/ path) as knowledge base sources — posts are discovered via the Ghost RSS feed and indexed automatically.- Frontend — Ghost appears in the source type grid alongside Substack, Medium, and WordPress with teal brand styling. URL auto-detection works for
.ghost.ioand/ghost/paths in both the KB source manager and Creator Wizard. - Backend — No changes needed; the existing
blog-platformconnector andGhostAdapteralready handle Ghost feed discovery and ingestion.
Group-based access control for agent distribution
Published agents can now restrict public access to organization members and specific org groups.- Backend —
agent_distribution_settingsgainedrequire_org_membershipandallowed_group_ids UUID[](migration098).NULL/{}means all groups allowed. A unifiedcheckPublicAccess()guard enforces identity, membership, and group checks onGET /public/agents/:slugand related public conversation paths. - Repository helpers —
isUserInGroups()andisUserOrgMember()queryorg_group_membersandorganization_membersdirectly in the bot service. - Gateway — the
/public/*lane now strips inbound identity headers, then verifies any optionalAuthorization: Bearertoken and forwards verifieduser-id/organization-idheaders to the bot service. - Marketplace — agents with group or org-membership restrictions are excluded from
GET /marketplace/agentsbecause anonymous browsers cannot satisfy them. - Frontend — the Distribute / Web channel page has a new Require organization membership toggle and a group multi-select.
MultiSelectDropdownwas renamed toMultiSelectand a reusableTogglecomponent was extracted.
Visitor question preview in the Creator Wizard
Creators can now see the pre-chat questionnaire exactly as visitors will experience it before publishing an agent.- A Preview visitor flow button in Step 2 of the Creator Wizard opens the live visitor questionnaire overlay, including brand colors and logo.
- A warning banner appears when more than three visitor questions are configured, reminding creators that each extra question increases drop-off.
- The warning is stronger when every question is required.
Creator Wizard now explains when Continue is disabled
On the Your Content step, the Continue button stays disabled until at least one source is added. A helper hint now appears next to the button so users know what to do next:- When no source is added: “Add at least one source to continue”
- When a URL, file, or shop domain is entered but not yet staged: “Add the source above to continue”
Friendly provider-error messages in chat
Chat no longer surfaces raw OpenRouter billing or rate-limit errors to end users. Provider-level failures are mapped to user-friendly messages before reaching the UI. When the provider key hits its credit or token limit, the response usescode: "provider_limit_reached" and shows:“This assistant is temporarily unavailable. Please try again later.”The raw provider error, including any API key URLs, is sanitized before logging. Applies to authenticated chat, public widget chat, and
chat-core headless consumers.Indexing-status polling — eliminated double round-trip and error-banner flicker
TheGET /bots/:id/indexing-status endpoint was making two identical requests to the knowledge service on every call. The redundant checkKbReadiness() pre-check was removed, cutting knowledge-service load in half across the three frontend components that poll every 15 seconds.- Backend — removed the duplicate fetch; the fallback
messagenow surfaces the knowledge service’s error detail when available, falling back to a generic message on network failures. - Frontend —
setIndexingError(null)now runs only after a successful poll, so the error banner no longer unmounts and remounts on every 15-second tick during sustained failures.
Mandatory knowledge-base readiness before agents go live
Agents can no longer be published without a linked, indexed knowledge base.- Backend gates —
POST /bots,PUT /bots/:id(when settingisDraft: false),POST /bots/:id/publish,POST /bots/:id/approve,POST /bots/:botId/distribution/publish, and the async provisioning worker all verify readiness before publishing. - Fail-closed — if the knowledge service is unreachable, publishing is blocked.
- Frontend guards — the classic setup flow requires a KB before continuing, the tabbed builder disables the publish toggle without a linked KB, and the Distribute / Go Live pages disable Publish until the readiness check passes.
- New proxy endpoint —
GET /bots/:id/indexing-statussurfaces indexing status through the bot service.
KB ingestion failure banner
Knowledge-base ingestion failures are now surfaced prominently on the agent editor and Go Live screens instead of sitting silently in the database.- Failure banner — when a linked KB has failed sources, a red
Alertbanner appears on the agent edit Knowledge Base tab and the Go Live screen, showing the error reason fromlastSyncError. - One-click retry — each failed source has a [Retry] button that triggers a re-sync via the existing
POST /kb/:id/sources/:sourceId/syncendpoint. - Deep link to details — a [Details] link opens the KB detail page for full sync history.
- Backend —
GET /knowledge/agents/:agentId/kbsandGET /knowledge/kbnow returnsyncHealth,failedSourceCount, andfailedSources[]with{sourceId, sourceUrl, lastSyncError}per failed source.
Smarter retrieval — hybrid search, reranking, and diverse context
The knowledge-base retrieval pipeline was overhauled end to end.- Hybrid retrieval — every search now runs a semantic (vector) channel and a keyword (full-text) channel in parallel and fuses them with Reciprocal Rank Fusion. Exact identifiers — SKUs, error codes, names — reliably surface even when the wording differs from your documents.
- Cross-encoder reranking — fused candidates are re-scored by a reranker that reads the query and each chunk together, for sharper relevance than similarity alone.
- Diverse, budgeted context — before an agent answers, near-duplicate chunks are dropped (MMR) and total knowledge-base context is capped by a token budget, so retrieval never crowds out the answer.
- Verified end to end by a new live Playwright suite that uploads a document, waits for indexing, searches, and asserts a grounded, cited answer.
Knowledge-base analytics & the learning loop
A new Analytics dashboard on every knowledge base shows what your content is actually doing — and retrieval now improves on its own as customers use it.- Outcome attribution — per-document resource performance (impressions, thumbs, helpfulness tier), a feedback-driven knowledge-gap list (“questions your customers asked that you couldn’t answer well”), and an impression → click → conversion funnel.
- Learning Progress — a weighted score of the customer signals you’ve collected, the three learning stages it unlocks (smart ranking → learned weights → custom reranker), and how much more signal is needed.
- Self-improving ranking — resources your customers engage with are automatically boosted in retrieval, floor-gated so early clicks never skew results. Toggle it per knowledge base from the analytics dashboard.
Refer & Earn — referral loop
Share your referral link, and both you and your invitees earn credits when they publish their first agent.- Earn your code on first publish — the moment you publish your first agent, a unique referral link is emailed to you and appears on the new Refer & Earn page in the sidebar.
- Easy sharing — new users can follow your
?ref=CODElink (the code is auto-applied at signup) or enter the code manually in the “Referral code” field on the sign-up form. - Both sides earn on activation — when an invitee publishes their first agent, both you and the invitee receive add-on credits (amounts set by platform admins, defaults: 100 for referrers, 50 for referees). The invitee also unlocks their own referral code, keeping the loop going.
- Per-code cap — each code has a maximum number of successful referrals (default 10) after which it is marked exhausted and no further rewards are issued.
- Platform admin controls — superadmins can adjust credit amounts, the per-code cap, and toggle the program on/off from Admin → System Config without a redeploy.
- All reward credits land in the non-expiring add-on credit pool and are emailed when earned.
Faster path to publishing a claimed agent
Publishing an agent after signup is more obvious and less error-prone, and the distribution section is easier to navigate.- Land on the publish page after claim signup — users who sign up by claiming an agent now arrive directly on that agent’s Web / Public Page settings (where the URL and Publish button live) instead of the dashboard, so they can publish without hunting for it.
- URL first — the Web / Public Page now shows the URL slug and public URL at the top, above the landing-page and chat-widget settings.
- “Channels” is now “Distribute” — the agent-editor tab was renamed, and the
per-channel pages show a breadcrumb (
Distribute › Web / Public Page) so the active channel is always clear. - Conflict-free default slug — the auto-filled URL slug is now checked for uniqueness and suffixed when needed, so a freshly claimed agent no longer fails to publish with a “slug already exists” error on a value the system chose.
- Plan picker — selecting a plan during signup now scrolls the Continue button into view.
Sharper image OCR for knowledge-base search
Vision text extraction on indexed images is more reliable and tunable, so editorial overlay text (headlines, venue names, award banners) shows up more consistently in knowledge-base search.- Tunable noise floor — the minimum extracted-text length is now a platform
setting (
Vision OCR Min Text Length, default 20) editable from Admin → System Config, no redeploy required. - Transient-failure retry — a failed extraction retries once after a short backoff before giving up, recovering from provider blips that previously lost the text silently.
- Re-sync backfill — when a synced source re-runs with unchanged media, existing vision text is preserved (no wasted re-extraction) and any image still missing it is filled in, so overlay text becomes searchable without re-embedding unchanged images.
Landing → app agent claim handoff
Agents provisioned from the marketing landing page now transfer into a user’s account when they sign up. A first-party Funnel organization owns the agent and its knowledge base while the visitor previews it, absorbing all pre-signup cost. On signup email verification, the agent and KB re-parent into the new user’s organization — counting against their plan, with no re-indexing and no duplicated LLM cost. The handoff is keyed on the verified email and scoped strictly to the funnel application, so no other tenant’s agent is ever eligible for transfer. First-party funnel only; not a tenant-facing capability.Searchable knowledge-base linking in the agent editor
The agent Knowledge tab scales to large workspaces and makes attached knowledge bases clear at a glance.- Linked knowledge bases appear in their own section at the top, always visible regardless of the search term, each with per-KB retrieval settings and a link to its detail page.
- A search box finds knowledge bases by name or description; the picker shows a capped set of matches with a “Showing X of Y” hint instead of the whole list.
- The picker scopes to the agent’s organization, so the agent’s own linked knowledge bases always load.
Read-only mode for the embeddable widget
The embeddable chat can now be locked to a read-only view — messages stay visible but the composer is hidden — so host pages can overlay their own login/payment wall (e.g. after a free-message limit) or show a shared, read-only conversation.- Activate on load with
?readonly=1(or?mode=readonly) on the iframe URL, ordata-readonly="true"on the embed script. - Toggle at runtime from host gating logic via
BrainstormerWidget.setReadOnly(true|false)— the one inbound command in an otherwise one-way bridge. - A new
widget:readonlyevent (with the current flag + counts) fires whenever the mode flips, so the host can sync its overlay.
Cleaner chat message actions + direct sign-up links
- Message copy control: the copy button now sits below each assistant message and is always visible (ChatGPT/Claude style) instead of appearing in a hover-only toolbar — so it works on touch devices. The non-functional “share” control was removed.
- Direct sign-up landing: marketing traffic can now land straight on the
sign-up form by appending
?mode=signup(also?mode=registeror?signup=1) to the app URL, instead of always opening on the sign-in form.
Widget event counts continue across resumed conversations
When a returning visitor resumed an existing conversation in the embeddable widget, themessage:sent / message:received event counts restarted at zero,
ignoring the messages the conversation was loaded with. The counts now seed from
the loaded message history, so login/payment-wall gating thresholds carry across
page reloads instead of resetting.A new conversation:resumed event fires once an existing conversation
finishes loading, carrying userMessageCount / assistantMessageCount /
messageCount — so the host can re-apply a wall on reload without waiting for
the visitor’s next message (widget:ready fires before the conversation loads
and cannot carry counts). See
Embeddable Chat Engine → Host page events.Embed widget emits events to the host page
The embeddable chat now posts one-way events to the page that frames it, so host sites can build their own gating — login walls, paywalls, usage analytics — without any two-way coupling.- Events:
widget:ready,conversation:started,message:sent,message:received,widget:error, delivered viapostMessagein a namespaced envelope (source: "brainstormer-widget"). Payloads carry metadata and running per-session message counts — never message text. - Loader API:
embed.jsexposesBrainstormerWidget.on(type, cb)(and"*"for all events), with origin verification handled for you. Raw<iframe>embedders can listen formessagedirectly. - Counts are a client-side UX signal — enforce real limits server-side too. See Embeddable Chat Engine → Host page events.
Choose API-key scopes in the Developer Portal
The Generate Key form lets you pick which scopes abrs_live_ key carries, so
keys can reach the Agent Provisioning API.- Scope picker:
chat,kb:read, andagents:writeare selectable when generating a key (/developer). New keys default tochat+kb:read; tickagents:writeto allowPOST /v1/agents:from-url. - The key table now shows each key’s granted scopes.
- A key missing a required scope is rejected by the gateway with
403 insufficient_scope.
Drop-in embed widget + anonymous operator presence
The embeddable chat widget is now live, and public visitors get real-time operator presence.- Embed Widget: a dependency-free
/embed.jsloader injects a floating chat launcher → iframe of a chrome-minimal/embed/:slugpage. Configure + copy the snippet in the agent’s Channels → Embed Widget tab (data-slug,data-color,data-position). The agent must be published and not require login. - Anonymous operator-presence SSE:
GET /api/public/agents/:slug/conversations/:id/events?anonymousId=streams operator messages / takeover / resolution to public visitors (no JWT; ownership scoped byanonymousId). Wired intochat-core’sPublicTransport.subscribeEvents. This also removes the spurious auth errors the public chat history panel used to log.
Anonymous file uploads for embedded chat
Visitors on a published agent page (or the embeddable widget) can now attach files to their messages without logging in — when the agent has file uploads enabled.POST /api/public/agents/:slug/attachments: anonymous, slug +anonymousIdscoped multipart upload. Returns an attachment ref ({ id, accessToken }) to pass in the chat message’s newattachmentsarray.- Hardened: per-IP / per-
anonymousIdrate limits, size + MIME allowlist, and magic-byte sniffing (a file whose bytes don’t match its declared type is rejected). See the Anonymous Chat reference. - Billing narrations on the provisioning path are now cross-linked (source → agent → KB) instead of blank on the ledger.
Agent Provisioning & Chat API
A public, API-key-authenticated capability that lets any client app build a chat agent from a creator’s YouTube or Instagram URL — observe the build asynchronously, then chat anonymously with streaming responses grounded in the creator’s content. No logged-in Brainstormer user required.POST /v1/agents:from-url: kick off an async build from a channel URL, returning202with abuildId— non-blocking. Supportsmode(boundedpreviewvs.full),intent,model,Idempotency-Key, anddeliver.{stream,webhookUrl}.- Live progress (SSE):
GET /v1/agents/builds/:id/eventsstreams oneProvisioningEventper phase transition (queued → detecting → scraping → analyzing → provisioning → indexing → ready/failed) with phase-weighted, display-ready progress copy. Replays state on connect; closes on terminal. - Polling snapshot:
GET /v1/agents/builds/:idreturns the latestProvisioningEvent(same shape as SSE) for stateless / email-me-a-link backends. - HMAC-signed webhook: optional out-of-band delivery of the terminal event,
signed
X-Brainstormer-Signature: sha256=<hmac>, with retry/backoff. - Anonymous chat: once
ready, browsers chat directly via the slug-based/api/public/agents/:slug/*endpoints — no API key, with optional SSE token streaming andstarterQuestionsto seed the UI. - Auth & abuse controls:
agents:writescope (403 insufficient_scope), no-secrets-in-browser Pattern A backend proxy, Cloudflare Turnstile, per-key + per-IP rate limits (429+Retry-After), idempotency, and handle dedup. - Structured error model: a flat
ProvisioningErrorCodeenum, each carryingretryablesemantics, plus partial-successwarningfor thin content. - Developer docs: new API reference (
from-url, build snapshot, SSE, webhook, anonymous chat), a 5-minute quickstart with a Next.js Pattern A proxy, auth & rate-limits and concepts guides, plus a machine-readablellms.txt.
Real-Time SSE Streaming for Chat
Chat responses now stream token-by-token in real-time instead of waiting for the full response. First token appears in under 1 second (previously 40-86 seconds of blank waiting).- Real SSE streaming: Tokens arrive via Server-Sent Events through all 4 layers (LangChain → Bot Service → Gateway → Frontend)
- Non-streaming model fallback: Models that don’t support streaming automatically fall back to invoke-then-send
- Parallelized setup: Independent pre-LLM calls (prompt resolution, conversation history, bot config) run concurrently via Promise.all
- Latency instrumentation: Every chat request logs a structured timing breakdown (KB retrieval, TTFT, tokens/sec, post-processing) for ongoing optimization
- Citation instructions: KB context now instructs the LLM to include
[1],[2]citation markers in responses - Auto-linked @mentions: Instagram @handles in chat responses are now clickable links. URLs auto-linked via GitHub Flavored Markdown.
- Chat button on editor: Published agents now have a “Chat” button in the editor header for quick access
Enhanced Creator Voice Analysis
The creator wizard now analyzes actual social media content (Instagram posts, videos, images) to generate voice-accurate system prompts instead of scraping empty homepage HTML.- Platform adapter integration: Instagram, YouTube, TikTok, Twitter URLs use Apify adapters to fetch real posts with captions, engagement metrics, and media
- Media processing: Up to 5 media items get transcribed (video/audio) or vision-extracted (images) during light-scrape for richer voice analysis
- Improved LLM prompt: Creator analysis now extracts verbal patterns, catchphrases, few-shot examples, and top-performing topics
- Wizard UX: Phase-based progress messages (“Fetching posts…”, “Transcribing videos…”), skip link, enrichment summary badge
- Generate from KB button: Re-generate a voice-accurate prompt anytime from the prompt editor using existing KB content
- Graceful degradation: Every processing step has fallbacks — Apify fails → URLFetcher, transcription fails → caption only, vision fails → skip
Human-in-the-Loop (HITL) System
Complete operator oversight system for AI agent conversations — escalation, approval workflows, queue management, and real-time operator dashboard.- Escalation workflow: AI-triggered or manual operator takeover with configurable escalation mode (both, AI-only, manual-only)
- Approval workflow: Hold AI responses for operator review before delivery (off, all responses, or AI-selected)
- Queue management: Auto-assign queued escalations when operators connect, configurable timeout (60s–24h), outside-hours behavior (queue, message-only, fallback email)
- Operator dashboard: Real-time escalation queue, stats bar, quick actions, WebSocket-powered live updates
- Browser notifications: Notification API alerts for new escalations, assignments, and messages when tab is unfocused
- In-app notification bell: Sidebar badge with pending count and recent alerts dropdown, visible from any page
- Customizable messages: 8 user-facing messages configurable per-agent (escalation started, operator joined, queued, expired, approval pending, etc.)
- Concurrency safety: Atomic claim/approval operations prevent double-claim race conditions in multi-operator environments
- Reusable email verification: Generic OTP-based email verification system with
<VerifiedEmailInput>component, used for fallback email config and dashboard email verification banner
Chat Stabilization — Routing, Citations, Prompt Editor
Several fixes consolidating the chat experience after the widget config rollout.- Path-based conversation URLs via
[[...conversationId]]catch-all for both dashboard and public chat pages - Chat history continues conversations via path, not query param — prevents the variable form from re-prompting on existing conversations
- Citation persistence: non-streaming path embeds sources in message metadata; frontend reads persisted metadata first, falls back to transient
lastResponse - KB context prompt strengthened with explicit citation format rules and examples
- CitationTooltip: richer hover tooltips with thumbnails and purple badges
- EnhancedChatInterface: SSE + HITL updates aligned with new routing and metadata-sourced citations
Incremental Crawl Ingestion + Centralized Scraping
Knowledge-base ingestion now streams results progressively, and an internal meta-task model is configurable.- Crawl workers deliver pages incrementally as they’re fetched instead of waiting for the full batch; default page limit lowered to reduce over-crawl on large sites
- Centralized scraping service consolidates URL fetching behind one adapter used by the crawler, light-scrape, and manual-sync paths
- Internal “meta-task” model (used for creator analysis, parameter suggestion, summarization) is now configurable via platform config + org-level override — organizations can choose a cheaper/faster model for internal LLM work without affecting chat
Widget Config, Appearance Tab, Editor Reorganization
Agent appearance and end-user experience are now configured from a single Appearance tab on the editor, persisted in a newwidget_config JSONB on bots (with org-level defaults in widget_defaults).- Editor reorganized from 8 tabs to 6 with header actions for Save / Preview / Publish
- New Appearance tab — branding, chat styling, variable form sections
ChatThemeWrapperinjects widget config as CSS variables into the chat surfaceVariableFormOverlay— pre-chat form component rendered when the agent declares required template variables- Widget theming + variable form integrated into both authenticated and public chat interfaces
- Org-level widget defaults configurable from Organization Settings → Brand Defaults
- Draft preview banner on chat page when previewing an unpublished agent
- Welcome message moved out of the tool system and into
widget_config;builtin:variable_collectiontool (non-functional) removed - Template variable selector added to the welcome message prompt editor
- Public agent endpoint returns the merged org-default + agent widget config
- New endpoints:
GET /bots/:id/widget-config,PUT /bots/:id/widget-config,GET/PUT /auth/organizations/:id/widget-defaults - Shared
WidgetConfigtypes + merge utility inpackages/shared - Distribution simplified:
visibilityremoved in favour of URL-based access; old/distributepage redirects to the editor Distribute tab
Unified Onboarding + Intent-Driven Agent Creation
Dual onboarding wizards (OrgOnboardingWizard, OnboardingWizard) replaced with a single UnifiedOnboardingWizard that handles plan selection and team setup in one flow. Agent creation now asks for an intent up front.- Five intents: Creator/Brand, Customer Support, Knowledge Assistant, Sales & Outreach, Personal Knowledge Base
- Intent drives the generated system prompt, default tool selection, KB source ordering, and creator-analysis prompts
- Billing plans moved from hardcoded
ONBOARDING_PLANSto a DB-driven API (GET /auth/billing-plans) with Stripe-ready columns - New DB:
bots.intentcolumn,billing_planstable with Stripe pricing columns, intent-specific system prompt defaults KBSourceAdderreorders sources based on the chosen intent- Fixed onboarding redirect loop + KB source icon mapping
- Fixed race where navigating to
CreatorWizardwould briefly bounce through the onboarding redirect guard
Variable Form + Prompt Editor Reliability
Cluster of fixes around the variable form flow and prompt editor interactions.- Delay conversation creation until the variable form is submitted (previously created an orphan conversation)
- Load stored conversation variables during chat + wait for widget config before deciding whether to show the variable form
- Fix stale closure in the variable form → conversation creation flow
- Prevent “Update Agent” from overwriting prompt editor changes made since load
- Serialize template variables as
{{name}}in the rich-text markdown output so they round-trip through saves - Normalize public agent response to include
templateVariableson the agent object (matches authenticated shape)
Agent Prompt Configuration System
Unified, versioned prompt management replacing all hardcoded LLM prompts platform-wide.- New tables:
prompt_type_registry,system_prompt_defaults,system_prompt_default_versions,bot_prompt_configs,bot_prompt_config_versions,conversation_variables;streaming_enabledadded tobots - Three-tier prompt resolution: bot override, system default, code constant fallback
fixed(static) andgenerated(LLM-driven) modes per prompt type, per agent- Welcome message returned in
POST /bots/:id/conversationsresponse; supports streaming in generated mode - Dynamic variables (
{{variable_name}}) with five-source precedence chain (per-message > sensitive > client > agent > system) - Sensitive variables stored encrypted via POST endpoint or signed JWT context tokens
- Semantic versioning (major.minor) with auto-generated changelogs and rollback on all prompt configs
streaming_enabledfield on bots; effective streaming = bot setting AND per-requeststreamparam- Shared prompt rendering engine:
packages/shared/src/prompt-renderer.ts - Bot service CRUD endpoints:
GET/PUT/DELETE /bots/:id/prompt-configs/:type, version history, rollback - Auth service superadmin endpoints: prompt type registry CRUD, system defaults management with versioning
- All endpoints proxied through gateway
PromptConfigEditorReact component for agent edit page and admin UI- Creator wizard extended: AI analysis now returns
suggestedWelcomeMessage+suggestedWelcomePrompt - Existing
bots.system_promptvalues migrated tobot_prompt_configs(backward compat preserved) - All previously hardcoded prompts seeded as system defaults
- Billing integrated: generated mode prompts call
record_external_cost_event()
Knowledge Graph + Document Registry + Summary Index
PostgreSQL-based knowledge graph for enhanced RAG retrieval.- 4 new database tables:
kg_document_registry,kg_entities,kg_relationships,kg_communities; new columns onkb_documentsfor graph metadata - Multimodal entity extraction from text, images (vision model), video frames, and audio transcripts
- Document summary index generates per-document summaries for KB map overview
- Graph query service with entity search, relationship traversal, and community detection
- User-controlled visual entity extraction configurable at source and document levels
- New API endpoints:
/map,/search/enhanced,/graph/search,/graph/stats,/reindex-graph,/graph/entities,/graph/visualization - Enhanced bot retrieval with KB map + graph context injected into agent chat
- Frontend Graph Explorer page with force-directed visualization, Document Knowledge Panel, and Graph Stats section
- Async graph processing via BullMQ
graph-indexingqueue - PostgreSQL-based graph storage with recursive CTE traversal (no Neo4j dependency)
Unified Content Processing Pipeline
Introduced ContentConnector pattern with ConnectorRegistry for all KB source ingestion.- All connectors (YouTube, Instagram, Twitter, RSS, blog-platform, URL, document) produce standardized
NormalizedContent[]with dedup keys and content hashes - Vision text extraction via OpenRouter for images and PDF pages
- Transcription service for audio and video content
- ContentProcessingPipeline orchestrates extraction, chunking, embedding, and vector storage in a single flow
- Extensible: new source types require only a new connector implementing the
ContentConnectorinterface
Gemini Multimodal Embeddings Integration
Replaced OpenAI text-only embeddings with Gemini Embedding 2 (gemini-embedding-exp-03-07) for native multimodal RAG across the Knowledge Base system.- Text, image, video, and audio content embedded natively via
@google/genaiat 3072 dimensions (up from 1536) - User-configurable media embedding strategy per KB: “native” (embed media directly) or “transcription” (Whisper to text to embed)
- Per-post document creation for social media (YouTube, Instagram) instead of concatenated text blobs
- Media storage service with ffmpeg-based video/audio splitting for content exceeding Gemini limits
- Per-modality billing integration (text per 1k chars, image flat rate, video/audio per second) via
record_external_cost_event - Enhanced bot citations with platform badges, thumbnails, published dates, and “View Original” links
- KB analytics dashboard with 5 new components: QueryTrendsChart, AgentUsageChart, KnowledgeGapsTable, ContentUtilizationTable, CostBreakdownCard
- PATCH endpoint for KB settings updates (media embedding strategy toggle)
- Section-based analytics API for lazy-loaded dashboard sections
Gemini Embedding 2 Multimodal RAG POC
Completed proof-of-concept validating native text-to-video, text-to-image, cross-language (Spanish/Hindi to English), and image-to-image retrieval usinggemini-embedding-2-preview (8/9 tests passed, 89%). PDF native embedding unreliable (workaround: convert to images); multimodal queries underperform.Light Scrape Creator Wizard
Replaced the 4-step creator wizard (Sources, Build Knowledge, AI Profile, Launch) with a 3-step flow (Sources, AI Profile, Launch) that eliminates the blocking KB sync wait.- New
POST /knowledge/light-scrapeendpoint extracts URL content instantly without full indexing - New
POST /knowledge/kb/:id/sources/batchendpoint for bulk source creation after agent is created analyzeScrapedContent()accepts raw scraped content directly instead of requiring aknowledgeBaseIdPOST /bots/creator-analysisnow accepts either{ knowledgeBaseId }or{ scrapedContent[] }, keeping backward compatibility- CreatorWizard rewritten: step 1 collects draft sources, step 2 runs light scrape + AI analysis in parallel, step 3 creates agent + KB + enqueues async sync
- Net result: wizard completes in seconds instead of 5+ minutes; full KB sync runs asynchronously in the background
AI-Powered Creator Onboarding Wizard
Redesigned creator wizard from 5-step (profile, sources, ingest, prompt, launch) to a 4-step AI-driven flow (sources, ingest, analyze, launch).- New
creator-analysis.service.tsanalyzes ingested KB content via OpenRouter (claude-3-haiku) to extract creator profile and system prompt automatically - New
POST /bots/creator-analysisendpoint accepts a KB ID and returnscreatorProfile+systemPrompt - Profile step eliminated: creator identity is inferred from their own content rather than entered manually
Additional Features
- Added onboarding path architecture for
/agents/createto support multiple low-friction onboarding tracks (creator,classic) - Enhanced chat message rendering with repositioned actions within bubbles
- Improved error handling with input validation in LangChain service
- Complete multi-modal architecture design (WebRTC + SSE + WebSocket)
- API Gateway service with health checks and service proxy
- Auth Service with JWT and organization context
- PostgreSQL database schema with multi-tenant support
- Comprehensive development tooling and Git workflow
Bug Fixes
- Fixed message actions positioning for better UX
- Added null checks and filtering for invalid chat messages
- Improved input validation with detailed error messages
Documentation Updates
- Detailed architecture documentation for multi-modal platform
- Implementation plan with parallel development strategy
- Contributing guide with sophisticated code management protocol
- Complete development setup and testing procedures
Build Infrastructure
- Monorepo setup with Turborepo
- Comprehensive TypeScript configuration
- ESLint and Prettier code formatting
- Husky Git hooks with conventional commits
- Automated versioning and changelog generation
v0.1.0
Foundation Release
Initial project setup establishing the complete foundation for Brainstormer V2 — a next-generation AI chatbot platform built for multi-modal interactions (text, voice, and video) with enterprise-grade features.- Initial project setup with microservices architecture
- Foundation for multi-modal AI chatbot platform
- Enterprise-grade multi-tenant organization support
- Real-time communication protocol design
- Production-ready JWT authentication with organization context
- Multi-tenant architecture: B2B-ready with custom billing rates per organization
- API Gateway with service discovery, health checks, and proxy routing
- Optimized PostgreSQL schema with proper relationships
- Excellent development tooling with hot-reload and comprehensive logging

