> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brainstormer.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Cloud Storage

> URL-based file serving architecture: token authentication, storage paths, security model, and LLM provider integration.

## Overview

The Cloud Storage System is a URL-based file serving system for multimodal LLM processing. Files are stored to disk or S3 and accessed via secure, token-authenticated URLs that multimodal AI providers fetch directly.

## Architecture

```mermaid theme={null}
graph TB
    subgraph "Frontend"
        UI[Chat Interface]
        FU[File Upload Component]
    end

    subgraph "API Gateway"
        GW[Gateway :4000]
    end

    subgraph "Bot Service"
        FS[File Upload Service]
        AR[Attachment Repository]
        CS[Chat Service]
    end

    subgraph "Storage"
        LS[Local Storage]
        S3[AWS S3]
        DB[(PostgreSQL)]
    end

    subgraph "External"
        LLM[LLM Providers]
    end

    UI --> GW --> FS
    FU --> GW
    FS --> AR --> DB
    FS --> LS
    FS --> S3
    CS --> LLM
    LLM -.->|Direct URL Access| LS
    LLM -.->|Direct URL Access| S3
```

## File Upload Flow

<Steps>
  <Step title="Client Upload">
    Frontend sends multipart form data to the API Gateway.
  </Step>

  <Step title="Authentication">
    Gateway verifies user permissions and organization context.
  </Step>

  <Step title="Validation">
    File type, size, and organization limits are checked.
  </Step>

  <Step title="Storage">
    File saved to disk/cloud with secure naming convention.
  </Step>

  <Step title="Database Record">
    Attachment record created with metadata and 64-character access token.
  </Step>

  <Step title="Response">
    Returns attachment metadata and secure access URL.
  </Step>
</Steps>

## Database Schema

### message\_attachments

```sql theme={null}
CREATE TABLE message_attachments (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  message_id UUID REFERENCES messages(id) ON DELETE CASCADE,
  organization_id UUID NOT NULL REFERENCES organizations(id),

  -- File Information
  original_filename VARCHAR(255) NOT NULL,
  stored_filename VARCHAR(255) NOT NULL,
  file_path VARCHAR(500) NOT NULL,
  file_size_bytes BIGINT NOT NULL,
  content_type VARCHAR(100) NOT NULL,
  file_hash VARCHAR(128),

  -- Classification
  file_type VARCHAR(20) NOT NULL
    CHECK (file_type IN ('image', 'document', 'audio', 'video', 'text', 'other')),
  processing_status VARCHAR(20) DEFAULT 'pending',
  extracted_text TEXT,
  extracted_metadata JSONB DEFAULT '{}',

  -- Access Control
  is_public BOOLEAN DEFAULT FALSE,
  expires_at TIMESTAMP,
  access_token VARCHAR(128),
  download_count INTEGER DEFAULT 0
);
```

### file\_storage\_config

```sql theme={null}
CREATE TABLE file_storage_config (
  id SERIAL PRIMARY KEY,
  organization_id UUID NOT NULL REFERENCES organizations(id),
  max_file_size_mb INTEGER DEFAULT 50,
  max_total_storage_gb DECIMAL(10,2) DEFAULT 10.0,
  allowed_file_types TEXT[],
  default_expiry_days INTEGER DEFAULT 90,
  enable_ocr BOOLEAN DEFAULT TRUE,
  enable_image_analysis BOOLEAN DEFAULT TRUE,
  enable_audio_transcription BOOLEAN DEFAULT TRUE,
  UNIQUE(organization_id)
);
```

## Security Architecture

### Token-Based Authentication

* Each file gets a unique access token (64-character hex string)
* Tokens required for file downloads
* Configurable expiration times per organization

### Organization Isolation

* All files partitioned by organization ID
* Cross-organization access prevented at database level
* Storage paths include organization-specific directories

### Storage Path Structure

```
/storage/attachments/{org_id}/{year}/{month}/{timestamp}_{random}_{filename}
```

### File Validation Pipeline

```
Upload -> MIME Type Validation -> File Size Check -> Extension Validation -> Storage
```

## API Endpoints

<Tabs>
  <Tab title="Upload">
    ```
    POST /api/agents/attachments/upload
    Content-Type: multipart/form-data

    Response:
    {
      "attachment": {
        "id": "uuid",
        "originalFilename": "document.pdf",
        "fileType": "document",
        "fileSizeBytes": 1048576,
        "accessUrl": "/api/agents/attachments/abc123.../file",
        "processingStatus": "pending"
      },
      "processingQueued": true
    }
    ```
  </Tab>

  <Tab title="Download">
    ```
    GET /api/agents/attachments/:accessToken/file
    Headers: Organization-ID: uuid

    Response: File stream with Content-Type and Content-Disposition headers
    ```
  </Tab>

  <Tab title="Chat with Files">
    ```json theme={null}
    POST /api/agents/:agentId/chat
    {
      "message": "Analyze this document",
      "attachments": [
        {
          "id": "uuid",
          "url": "https://api.brainstormer.com/.../file",
          "type": "document",
          "filename": "report.pdf"
        }
      ]
    }
    ```
  </Tab>
</Tabs>

## LLM Provider Integration

LLM services access files directly via URL -- no base64 encoding needed:

<CodeGroup>
  ```typescript OpenAI GPT-4 Vision theme={null}
  const response = await openai.chat.completions.create({
    model: "gpt-4-vision-preview",
    messages: [{
      role: "user",
      content: [
        { type: "text", text: "Analyze this document" },
        { type: "image_url", image_url: { url: fileAccessUrl } }
      ]
    }]
  });
  ```

  ```typescript Anthropic Claude theme={null}
  const response = await anthropic.messages.create({
    model: "claude-3-sonnet-20240229",
    messages: [{
      role: "user",
      content: [
        { type: "text", text: "Analyze this document" },
        { type: "image", source: { type: "url", url: fileAccessUrl } }
      ]
    }]
  });
  ```
</CodeGroup>

## Performance Improvements

| Metric           | Base64 Approach       | URL-based Approach | Improvement    |
| ---------------- | --------------------- | ------------------ | -------------- |
| Max file size    | 25MB                  | 2GB+               | 8000% increase |
| API payload size | File size x 1.33      | \~200 bytes        | 99% reduction  |
| Memory usage     | Full file in memory   | Streaming          | 75% reduction  |
| Processing time  | High for large files  | Consistent         | 60% faster     |
| LLM API costs    | High (large payloads) | Optimized          | 40% reduction  |

## Error Handling

```typescript theme={null}
// Fallback mechanism for file access
async function getFileAccessUrl(attachmentId: string): Promise<string> {
  try {
    return await generateSecureUrl(attachmentId);
  } catch (error) {
    // Fallback: Return base64 if URL access fails
    logger.warn("URL access failed, falling back to base64", { attachmentId });
    return await generateBase64Fallback(attachmentId);
  }
}
```