ArchiTools/docs/architecture/STORAGE-LAYER.md

Storage Layer Architecture

Internal reference for the ArchiTools storage abstraction layer.

---

Overview

ArchiTools uses an adapter-based storage abstraction that decouples all module data operations from the underlying persistence mechanism. The default adapter uses localStorage for zero-infrastructure demo/development mode. The architecture supports swapping to REST API, MinIO, or database backends without changing module code.

Key principle: No module may import localStorage, fetch to a storage endpoint, or any persistence API directly. All data access flows through the StorageService interface via the useStorage hook.

---

StorageService Interface

// src/lib/storage/types.ts

interface StorageService {
  /** Retrieve a single item by namespace and key. Returns null if not found. */
  get<T>(namespace: string, key: string): Promise<T | null>;

  /** Persist a value under namespace/key. Overwrites existing value. */
  set<T>(namespace: string, key: string, value: T): Promise<void>;

  /** Remove a single item. No-op if key does not exist. */
  delete(namespace: string, key: string): Promise<void>;

  /** List all keys within a namespace. Returns empty array if namespace is empty. */
  list(namespace: string): Promise<string[]>;

  /** Query all items in a namespace, filtering by predicate. */
  query<T>(namespace: string, predicate: (item: T) => boolean): Promise<T[]>;

  /** Remove all data within a namespace. */
  clear(namespace: string): Promise<void>;

  /** Export entire namespace as a key-value record. Used for migration and backup. */
  export(namespace: string): Promise<Record<string, unknown>>;

  /** Import a key-value record into a namespace. Merges with existing data by default. */
  import(namespace: string, data: Record<string, unknown>): Promise<void>;
}

All methods are async. Even LocalStorageAdapter returns promises to maintain interface compatibility and allow drop-in replacement with async backends.

---

Adapter Pattern

Architecture

Module code
    ↓
useStorage(namespace) hook
    ↓
StorageProvider context
    ↓
StorageService interface
    ↓
┌─────────────────────────────────┐
│  LocalStorageAdapter (default)  │
│  ApiStorageAdapter  (future)    │
│  MinioAdapter       (future)    │
└─────────────────────────────────┘

Adapter Implementations

LocalStorageAdapter

File: src/lib/storage/adapters/local-storage.ts

• Default adapter, used in demo mode and local development.
• Client-only. Must not be invoked during SSR; all calls should be guarded or deferred to useEffect.
• Storage key format: architools::{namespace}::{key}
• Values are JSON-serialized via JSON.stringify / JSON.parse.
• Enforces per-namespace size quota (default: 2 MB per namespace, configurable).
• Total localStorage budget: 5 MB (browser limit). The adapter tracks usage and throws StorageQuotaExceededError when limits are hit.

// Key format
function storageKey(namespace: string, key: string): string {
  return `architools::${namespace}::${key}`;
}

Size tracking:

// Approximate size calculation for quota enforcement
function byteSize(value: string): number {
  return new Blob([value]).size;
}

ApiStorageAdapter (Future)

File: src/lib/storage/adapters/api-storage.ts

• Communicates with a REST API backend (STORAGE_API_URL).
• Endpoint mapping:
  • GET /storage/{namespace}/{key} — get
  • PUT /storage/{namespace}/{key} — set
  • DELETE /storage/{namespace}/{key} — delete
  • GET /storage/{namespace} — list
  • POST /storage/{namespace}/query — query (sends predicate as filter object, not function)
  • DELETE /storage/{namespace} — clear
  • GET /storage/{namespace}/export — export
  • POST /storage/{namespace}/import — import
• Authentication: Bearer token from Authentik session (when auth is enabled).
• Query method: The API adapter cannot serialize JavaScript predicates. Instead, query fetches all items and filters client-side, or accepts a structured filter object for server-side filtering.

MinioAdapter (Future)

File: src/lib/storage/adapters/minio-storage.ts

• For file/object storage (signatures, stamps, templates, uploaded documents).
• Not a general key-value store; used alongside ApiStorageAdapter for binary assets.
• Object path format: {bucket}/{namespace}/{key}
• Presigned URLs for browser uploads/downloads.
• Server-side only (access keys must not be exposed to client).

---

Namespace Isolation

Every module operates within its own namespace. Namespaces are string identifiers that partition data to prevent collisions.

Assigned Namespaces

Module	Namespace
Dashboard	dashboard
Registratura	registratura
Email Signature	email-signature
Word XML Generators	word-xml
Digital Signatures	digital-signatures
Password Vault	password-vault
IT Inventory	it-inventory
Address Book	address-book
Prompt Generator	prompt-generator
Word Templates	word-templates
Tag Manager	tag-manager
Mini Utilities	mini-utilities
AI Chat	ai-chat
System/Settings	system

Namespace Rules

1. Namespaces are lowercase kebab-case strings.
2. A module must never read or write to another module's namespace directly.
3. Cross-module data access happens through shared services (e.g., the tag service reads from tag-manager namespace on behalf of any module).
4. The system namespace is reserved for platform-level configuration (theme preference, sidebar state, feature flag overrides).

---

StorageProvider and useStorage Hook

StorageProvider

// src/lib/storage/provider.tsx

import { createContext, useContext, useMemo } from 'react';

interface StorageContextValue {
  service: StorageService;
}

const StorageContext = createContext<StorageContextValue | null>(null);

interface StorageProviderProps {
  adapter?: StorageService;
  children: React.ReactNode;
}

export function StorageProvider({ adapter, children }: StorageProviderProps) {
  const service = useMemo(() => {
    return adapter ?? createDefaultAdapter();
  }, [adapter]);

  return (
    <StorageContext.Provider value={{ service }}>
      {children}
    </StorageContext.Provider>
  );
}

The StorageProvider wraps the application in src/app/layout.tsx. Adapter selection is determined at startup based on the NEXT_PUBLIC_STORAGE_ADAPTER environment variable.

Adapter Factory

// src/lib/storage/factory.ts

export function createDefaultAdapter(): StorageService {
  const adapterType = process.env.NEXT_PUBLIC_STORAGE_ADAPTER ?? 'localStorage';

  switch (adapterType) {
    case 'localStorage':
      return new LocalStorageAdapter();
    case 'api':
      return new ApiStorageAdapter(process.env.STORAGE_API_URL!);
    default:
      throw new Error(`Unknown storage adapter: ${adapterType}`);
  }
}

useStorage Hook

// src/lib/storage/hooks.ts

export function useStorage<T = unknown>(namespace: string) {
  const context = useContext(StorageContext);
  if (!context) {
    throw new Error('useStorage must be used within StorageProvider');
  }
  const { service } = context;

  return useMemo(() => ({
    get: (key: string) => service.get<T>(namespace, key),
    set: (key: string, value: T) => service.set<T>(namespace, key, value),
    delete: (key: string) => service.delete(namespace, key),
    list: () => service.list(namespace),
    query: (predicate: (item: T) => boolean) => service.query<T>(namespace, predicate),
    clear: () => service.clear(namespace),
    export: () => service.export(namespace),
    import: (data: Record<string, unknown>) => service.import(namespace, data),
  }), [service, namespace]);
}

Usage in a module:

function RegistraturaPage() {
  const storage = useStorage<RegistryEntry>('registratura');

  async function loadEntries() {
    const keys = await storage.list();
    // ...
  }

  async function saveEntry(entry: RegistryEntry) {
    await storage.set(entry.id, entry);
  }
}

---

Data Serialization

Rules

1. All structured data is serialized as JSON via JSON.stringify.
2. Dates are stored as ISO 8601 strings ("2025-03-15T10:30:00.000Z"), never as Date objects.
3. Binary data (images, files) is not stored in the key-value layer. Use MinIO or base64-encoded strings (for small assets under 100 KB only, e.g., signature images in digital-signatures).
4. undefined values are stripped during serialization. Use null for intentional absence.
5. Keys must be valid URL path segments: lowercase alphanumeric, hyphens, and underscores only.

Validation

const KEY_PATTERN = /^[a-z0-9][a-z0-9_-]*$/;

function validateKey(key: string): void {
  if (!KEY_PATTERN.test(key)) {
    throw new StorageKeyError(`Invalid storage key: "${key}"`);
  }
}

---

Migration Strategy

localStorage to API Backend

The export/import methods on StorageService enable namespace-level data migration.

Migration flow:

1. User triggers export from admin panel
2. For each namespace:
   a. Call oldAdapter.export(namespace) → JSON record
   b. Download as backup file (optional)
   c. Call newAdapter.import(namespace, data)
   d. Verify item count matches
3. Switch NEXT_PUBLIC_STORAGE_ADAPTER to 'api'
4. Restart application

Migration script (admin utility):

async function migrateStorage(
  source: StorageService,
  target: StorageService,
  namespaces: string[]
): Promise<MigrationReport> {
  const report: MigrationReport = { namespaces: [], errors: [] };

  for (const ns of namespaces) {
    try {
      const data = await source.export(ns);
      const keyCount = Object.keys(data).length;
      await target.import(ns, data);
      const verifyKeys = await target.list(ns);

      report.namespaces.push({
        namespace: ns,
        exported: keyCount,
        imported: verifyKeys.length,
        success: keyCount === verifyKeys.length,
      });
    } catch (error) {
      report.errors.push({ namespace: ns, error: String(error) });
    }
  }

  return report;
}

Schema Versioning

Each namespace should store a _meta key containing schema version:

interface NamespaceMeta {
  schemaVersion: number;
  lastMigration: string; // ISO 8601
  itemCount: number;
}

On application startup, the storage layer checks _meta.schemaVersion and runs registered migration functions if the version is behind.

---

Configuration

Environment Variables

Variable	Default	Description
NEXT_PUBLIC_STORAGE_ADAPTER	localStorage	Active adapter: localStorage, api
STORAGE_API_URL	—	REST API endpoint (for api adapter)
MINIO_ENDPOINT	—	MinIO server address
MINIO_ACCESS_KEY	—	MinIO access key (server-side only)
MINIO_SECRET_KEY	—	MinIO secret key (server-side only)
MINIO_BUCKET	architools	MinIO bucket name

Variables prefixed with NEXT_PUBLIC_ are available on the client. MinIO credentials are server-side only and must never be exposed to the browser.

---

Size Limits and Quotas

localStorage Adapter

Limit	Value	Notes
Browser total	~5 MB	Varies by browser. Chrome/Firefox: 5 MB per origin.
Per-namespace soft limit	2 MB	Configurable in adapter constructor.
Per-value max size	1 MB	Prevents accidental storage of large blobs.
Key length max	128 characters	Including namespace prefix.

When a quota is exceeded, the adapter throws StorageQuotaExceededError. Module code must handle this gracefully — typically by showing a user-facing error and suggesting data export/cleanup.

Quota Enforcement

class LocalStorageAdapter implements StorageService {
  private namespaceQuota: number; // bytes, default 2 * 1024 * 1024

  async set<T>(namespace: string, key: string, value: T): Promise<void> {
    const serialized = JSON.stringify(value);
    const size = byteSize(serialized);

    if (size > this.maxValueSize) {
      throw new StorageQuotaExceededError(
        `Value exceeds max size: ${size} bytes (limit: ${this.maxValueSize})`
      );
    }

    const currentUsage = await this.getNamespaceSize(namespace);
    if (currentUsage + size > this.namespaceQuota) {
      throw new StorageQuotaExceededError(
        `Namespace "${namespace}" quota exceeded: ${currentUsage + size} bytes (limit: ${this.namespaceQuota})`
      );
    }

    localStorage.setItem(storageKey(namespace, key), serialized);
  }
}

---

Error Handling

Error Types

// src/lib/storage/errors.ts

export class StorageError extends Error {
  constructor(message: string, public readonly code: string) {
    super(message);
    this.name = 'StorageError';
  }
}

export class StorageKeyError extends StorageError {
  constructor(message: string) {
    super(message, 'INVALID_KEY');
  }
}

export class StorageQuotaExceededError extends StorageError {
  constructor(message: string) {
    super(message, 'QUOTA_EXCEEDED');
  }
}

export class StorageNotFoundError extends StorageError {
  constructor(namespace: string, key: string) {
    super(`Key "${key}" not found in namespace "${namespace}"`, 'NOT_FOUND');
  }
}

export class StorageNetworkError extends StorageError {
  constructor(message: string) {
    super(message, 'NETWORK_ERROR');
  }
}

export class StorageSerializationError extends StorageError {
  constructor(message: string) {
    super(message, 'SERIALIZATION_ERROR');
  }
}

Error Handling Patterns

In adapters: Adapters catch raw errors (JSON parse failures, network timeouts, QuotaExceededError from the browser) and re-throw as typed StorageError subclasses.

In hooks/modules: Use try/catch with specific error type checks:

try {
  await storage.set(entry.id, entry);
} catch (error) {
  if (error instanceof StorageQuotaExceededError) {
    toast.error('Spațiul de stocare este plin. Exportați datele vechi.');
  } else if (error instanceof StorageNetworkError) {
    toast.error('Eroare de conexiune. Încercați din nou.');
  } else {
    toast.error('Eroare la salvare.');
    console.error('[Storage]', error);
  }
}

---

Update Patterns

Optimistic Updates

Used for fast UI response when data loss risk is low (e.g., updating a tag, toggling a flag).

function useOptimisticUpdate<T extends BaseEntity>(namespace: string) {
  const storage = useStorage<T>(namespace);
  const [items, setItems] = useState<T[]>([]);

  async function update(id: string, patch: Partial<T>) {
    // 1. Apply optimistically to local state
    const previous = [...items];
    setItems(items.map(item =>
      item.id === id ? { ...item, ...patch, updatedAt: new Date().toISOString() } : item
    ));

    try {
      // 2. Persist to storage
      const current = await storage.get(id);
      if (current) {
        await storage.set(id, { ...current, ...patch, updatedAt: new Date().toISOString() });
      }
    } catch (error) {
      // 3. Rollback on failure
      setItems(previous);
      throw error;
    }
  }

  return { items, setItems, update };
}

Pessimistic Updates

Used when data integrity is critical (e.g., registry entries, password vault). The UI waits for confirmation before updating local state.

async function saveEntry(entry: RegistryEntry) {
  setLoading(true);
  try {
    await storage.set(entry.id, entry);
    // Only update UI after successful persistence
    setEntries(prev => [...prev, entry]);
  } catch (error) {
    // UI remains unchanged; show error
    handleStorageError(error);
  } finally {
    setLoading(false);
  }
}

Guidelines

Scenario	Pattern	Reason
Tag edits, dashboard widget reorder	Optimistic	Low risk, frequent interaction
Registry entry creation	Pessimistic	Data integrity matters
Password vault writes	Pessimistic	Security-sensitive
Bulk import	Pessimistic with progress	Large operation, needs feedback
Settings/preferences	Optimistic	Trivial to re-apply

---

Caching Strategy

localStorage Adapter

No caching layer needed. localStorage is synchronous and in-memory in the browser. The async wrapper adds negligible overhead.

API Adapter (Future)

The API adapter should implement a write-through cache:

class ApiStorageAdapter implements StorageService {
  private cache: Map<string, { value: unknown; timestamp: number }> = new Map();
  private cacheTTL = 30_000; // 30 seconds

  async get<T>(namespace: string, key: string): Promise<T | null> {
    const cacheKey = `${namespace}::${key}`;
    const cached = this.cache.get(cacheKey);

    if (cached && Date.now() - cached.timestamp < this.cacheTTL) {
      return cached.value as T;
    }

    const result = await this.fetchFromApi<T>(namespace, key);
    if (result !== null) {
      this.cache.set(cacheKey, { value: result, timestamp: Date.now() });
    }
    return result;
  }

  async set<T>(namespace: string, key: string, value: T): Promise<void> {
    await this.putToApi(namespace, key, value);
    // Write-through: update cache immediately
    this.cache.set(`${namespace}::${key}`, { value, timestamp: Date.now() });
  }
}

Cache invalidation happens on set, delete, and clear. The query method always bypasses cache and fetches fresh data.

---

Cross-Tab Synchronization

Problem

When using localStorage, multiple browser tabs may read stale data if one tab writes and another has already loaded its state.

Solution

The LocalStorageAdapter listens to the storage event, which fires when another tab modifies localStorage:

// src/lib/storage/adapters/local-storage.ts

class LocalStorageAdapter implements StorageService {
  private listeners: Map<string, Set<(key: string) => void>> = new Map();

  constructor() {
    if (typeof window !== 'undefined') {
      window.addEventListener('storage', this.handleStorageEvent);
    }
  }

  private handleStorageEvent = (event: StorageEvent) => {
    if (!event.key?.startsWith('architools::')) return;

    const [, namespace, key] = event.key.split('::');
    const nsListeners = this.listeners.get(namespace);
    if (nsListeners) {
      nsListeners.forEach(listener => listener(key));
    }
  };

  /** Subscribe to changes in a namespace from other tabs. */
  onExternalChange(namespace: string, callback: (key: string) => void): () => void {
    if (!this.listeners.has(namespace)) {
      this.listeners.set(namespace, new Set());
    }
    this.listeners.get(namespace)!.add(callback);

    // Return unsubscribe function
    return () => {
      this.listeners.get(namespace)?.delete(callback);
    };
  }

  destroy() {
    if (typeof window !== 'undefined') {
      window.removeEventListener('storage', this.handleStorageEvent);
    }
  }
}

Hook Integration

export function useStorageSync(namespace: string, onSync: () => void) {
  const { service } = useContext(StorageContext)!;

  useEffect(() => {
    if ('onExternalChange' in service) {
      const unsubscribe = (service as LocalStorageAdapter).onExternalChange(
        namespace,
        () => onSync()
      );
      return unsubscribe;
    }
  }, [service, namespace, onSync]);
}

Modules that display lists (registratura, address book) should call useStorageSync to re-fetch data when another tab makes changes.

Limitations

• The storage event only fires in other tabs, not the tab that made the change.
• The API adapter does not support cross-tab sync natively. If needed, it would require WebSocket or polling.
• Cross-tab sync is best-effort, not transactional. Two tabs writing the same key simultaneously can cause last-write-wins conflicts.

---

File Structure

src/lib/storage/
├── types.ts              # StorageService interface, NamespaceMeta
├── errors.ts             # Error classes
├── factory.ts            # createDefaultAdapter()
├── provider.tsx          # StorageProvider context
├── hooks.ts              # useStorage, useStorageSync
├── constants.ts          # Namespace list, quota defaults
└── adapters/
    ├── local-storage.ts  # LocalStorageAdapter
    ├── api-storage.ts    # ApiStorageAdapter (stub)
    └── minio-storage.ts  # MinioAdapter (stub)

---

Testing

• Adapters are tested with a shared test suite that runs against the StorageService interface. Swap the adapter instance to test each implementation identically.
• LocalStorageAdapter tests use a mock localStorage (e.g., jest-localstorage-mock or a simple in-memory Map).
• Integration tests verify namespace isolation: writing to namespace A must never affect namespace B.
• Quota tests verify that StorageQuotaExceededError is thrown at the correct thresholds.