> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brainstormer.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Migration Strategy

> Base64 to URL-based cloud storage migration: phases, implementation status, performance benchmarks, and rollback strategy.

## Overview

This document outlines the migration strategy from base64-encoded file handling to the URL-based cloud storage system. The migration eliminates critical limitations of base64 encoding while ensuring zero downtime and backward compatibility.

## Migration Overview

<Columns>
  <Card title="Before (Base64)" icon="file-code">
    ```
    Frontend Upload
      -> Base64 Encoding
      -> API Payload
      -> LLM Processing
    ```

    * 33% file size overhead
    * 25MB max file size
    * Full file in memory
    * Model-specific compatibility issues
  </Card>

  <Card title="After (URL-based)" icon="link">
    ```
    Frontend Upload
      -> Secure Storage
      -> URL Generation
      -> LLM URL Access
    ```

    * No encoding overhead
    * 2GB+ file support
    * Streaming access
    * Universal LLM compatibility
  </Card>
</Columns>

## Migration Phases

### Phase 1: Infrastructure Preparation

**Status:** Complete

* Database schema: `message_attachments` and `file_storage_config` tables
* Local filesystem storage with secure paths
* Organization-based storage partitioning
* Access token generation

### Phase 2: Core Service Implementation

**Status:** Complete

* `FileUploadService`: validation, storage, token-based access, processing queue
* `AttachmentRepository`: CRUD operations, processing status, cleanup

### Phase 3: API Integration

**Status:** Complete

* Upload endpoint: `POST /api/agents/attachments/upload`
* File access endpoint: `GET /api/agents/attachments/:accessToken/file`
* Chat integration: attachments passed as URL references

### Phase 4: Frontend Integration

**Status:** Complete

* Drag-and-drop file upload
* Progress tracking and error handling
* File type validation on client
* Preview generation
* Message display with download links

### Phase 5: LLM Provider Integration

**Status:** Complete

All major LLM providers supported with URL-based file access:

<CodeGroup>
  ```typescript OpenAI theme={null}
  { type: "image_url", image_url: { url: attachmentUrl } }
  ```

  ```typescript Anthropic theme={null}
  { type: "image", source: { type: "url", url: attachmentUrl } }
  ```

  ```typescript Google Gemini theme={null}
  { inlineData: { mimeType: contentType, data: attachmentUrl } }
  ```
</CodeGroup>

**Fallback mechanism:** If URL access fails, the system automatically falls back to base64 encoding.

## Performance Benchmarks

| Metric            | Base64                | URL-based   | Improvement    |
| ----------------- | --------------------- | ----------- | -------------- |
| Maximum file size | 25MB                  | 2GB+        | 8000% increase |
| API payload size  | File size x 1.33      | \~200 bytes | 99% reduction  |
| Memory usage      | Full file in memory   | Streaming   | 75% reduction  |
| Processing time   | High for large files  | Consistent  | 60% faster     |
| LLM API costs     | High (large payloads) | Optimized   | 40% reduction  |

## Risk Mitigation

<AccordionGroup>
  <Accordion title="Data Integrity">
    SHA-256 hash verification for all files. Automatic integrity checks before LLM processing.
  </Accordion>

  <Accordion title="Security">
    64-character cryptographically secure access tokens. Organization-based access control. Regular token rotation and expiration.
  </Accordion>

  <Accordion title="Performance">
    Optimized database queries with strategic indexing. Connection pooling. CDN integration planned.
  </Accordion>

  <Accordion title="Compatibility">
    Comprehensive testing with all LLM providers. Automatic fallback to base64 when URLs are not supported.
  </Accordion>
</AccordionGroup>

## Rollback Strategy

If critical issues are encountered:

<Steps>
  <Step title="Immediate Rollback">
    Disable new file upload endpoint. Route uploads through legacy base64 processing.
  </Step>

  <Step title="Gradual Migration Back">
    Convert recent URL-based attachments to base64 storage. Maintain database records for audit.
  </Step>

  <Step title="Re-enablement">
    Implement feature flags for gradual re-enablement after fixes.
  </Step>
</Steps>

## Success Metrics

| Category  | Metric                        | Target               |
| --------- | ----------------------------- | -------------------- |
| Technical | File processing success rate  | `>99.5%`             |
| Technical | API response time             | `<500ms`             |
| Technical | Storage efficiency            | 75% memory reduction |
| Technical | Error rate                    | `<0.1%`              |
| Business  | LLM processing cost reduction | 40%                  |

## Future Enhancements

* **Amazon S3**: Full integration with signed URLs
* **Google Cloud Storage**: GCS for Google Cloud deployments
* **Azure Blob Storage**: Azure ecosystem integration
* **CDN**: Global content delivery network
* **Image Processing**: Real-time resizing and optimization