Documentation Index
Fetch the complete documentation index at: https://docs.brainstormer.io/llms.txt
Use this file to discover all available pages before exploring further.
A knowledge base gets its content from sources — files you upload or URLs you connect. Once a source is added, Brainstormer processes it in the background, splits it into searchable chunks, and makes it available to any agent linked to the KB.
Create a knowledge base and add sources
Open Knowledge Bases
In the sidebar, click Knowledge. You’ll see a list of all knowledge bases in your organization.
Create a new KB
Click Create KB. Give it a name and, optionally, a description. You can also set the visibility:
- Private — only visible to your organization
- Shared — visible within your organization
- Public — visible to anyone with access
Add a source
Inside the KB, click Add Source. A panel opens where you can choose the source type (see the sections below). Select a type, fill in the required details, and click Add Source.
Wait for processing
Your source will appear in the source list with a status badge. Processing runs in the background — the status updates automatically as it progresses.
Link the KB to an agent
Go to your agent’s settings and open the Knowledge Base tab. Toggle on the KB you want the agent to use. The agent will search it during every conversation.
Processing time depends on the size and type of the source. A short web page may finish in seconds; a large PDF or a social media profile with many posts may take a few minutes.
Source types
Documents
Web pages & sites
Feeds & blogs
Social media
Upload a file directly from your computer. Brainstormer splits the document into chunks and embeds them for semantic search.Supported formats| Category | Formats |
|---|
| Documents | PDF, DOC, DOCX, TXT, CSV, MD, JSON, PPTX, XLSX |
| Images | PNG, JPG, JPEG |
| Audio | MP3, WAV, M4A |
| Video | MP4, MOV, WebM |
- Click Add Source and select File Upload.
- Drag and drop your file onto the upload area, or click browse to pick a file.
- Choose a media embedding strategy:
- Native — embeds video and audio directly so the agent can search visual and audio content.
- Transcription — converts audio and video to text before embedding. Choose this for longer recordings or when you only need the spoken content to be searchable.
- Optionally enable Visual Entity Extraction to identify people, brands, and products in images and video using AI vision.
- The file is uploaded immediately. Processing begins in the background.
Once processing is complete, the document appears in the document list with a completed status badge and a chunk count. Add content from the web by entering a URL. Three options are available depending on how much of a site you want to index.Web pageAdd a single public URL. Brainstormer fetches the page and indexes its text content.
- Good for: product pages, help articles, individual blog posts, any public URL.
- Enter the full URL, for example
https://example.com/docs/getting-started.
Website crawlCrawl an entire website by following links from a starting URL.
- Configure the maximum number of pages (10–500) and crawl depth (1–5 link levels).
- Good for: documentation sites, marketing sites, any multi-page property you want fully indexed.
SitemapIndex all pages listed in an XML sitemap.
- Enter the URL of the sitemap file, for example
https://example.com/sitemap.xml.
- Good for: large sites where you already have a sitemap.
For documentation sites, a sitemap is usually the most reliable approach — it gives you full coverage without needing to tune crawl depth.
Connect a feed or blog to keep your KB updated as new content is published.RSS / Atom feedEnter the URL of any RSS or Atom feed. Feed items are fetched and indexed automatically.
- Good for: news feeds, blog feeds, podcast feeds, any content published via RSS.
- Example:
https://example.com/feed.xml
SubstackEnter your Substack publication URL. Newsletter posts are discovered and indexed via the Substack feed.MediumEnter an author profile URL. Published articles are discovered and indexed.WordPressEnter the blog URL. Posts are discovered via the WordPress REST API or RSS feed.
Auto-syncFor feeds and blogs, you can enable auto-sync to re-fetch content on a schedule:| Frequency | Use case |
|---|
| Every 15–30 min | Breaking news, high-frequency feeds |
| Every hour | Daily blogs, active newsletters |
| Every 6–24 hours | Low-frequency content |
Connect social media profiles to index posts, captions, and media content.| Platform | What is indexed |
|---|
| YouTube | Video descriptions, transcripts (via transcription or native embedding) |
| Instagram | Post captions, images |
| Twitter / X | Tweets, replies, media |
| LinkedIn | Company page posts |
| TikTok | Post descriptions |
- Click Add Source and select the platform.
- Enter the profile or channel URL (e.g.
https://youtube.com/@channelname or https://instagram.com/username).
- Choose how many posts to sync (10–100 most recent posts).
- Choose a media embedding strategy (Native or Transcription).
- Optionally enable Visual Entity Extraction to identify people and brands in images and video.
- Click Add Source.
Social media sources index the most recent posts at the time of sync. Enable auto-sync to pick up new content automatically.
Document processing statuses
After adding a source, each document in the list shows a status badge:
| Status | Meaning |
|---|
| Pending | Queued and waiting to start processing |
| Processing | Currently being parsed, chunked, and embedded |
| Completed | Ready to search — the chunk count is shown |
| Failed | Processing failed. Click Retry Processing to try again |
Managing sources
From the source list you can:
- Sync now — re-fetch a URL source immediately to pick up new content.
- Delete — remove a source and all documents indexed from it.
Deleting a source removes its documents from the KB, which may affect your agent’s ability to answer questions that relied on that content.