docs: document N+1 performance bug findings and prevention rules

This commit is contained in:
AI Assistant
2026-02-27 22:27:07 +02:00
parent c45a30ec14
commit db9bcd7192
2 changed files with 153 additions and 78 deletions
+27 -7
View File
@@ -18,6 +18,7 @@ git push origin main # auto-deploys via Portainer webhook
## Project Overview ## Project Overview
**ArchiTools** is a modular internal web dashboard for an architecture/engineering office group of 3 companies: **ArchiTools** is a modular internal web dashboard for an architecture/engineering office group of 3 companies:
- **Beletage** (architecture) - **Beletage** (architecture)
- **Urban Switch** (urbanism) - **Urban Switch** (urbanism)
- **Studii de Teren** (geotechnics) - **Studii de Teren** (geotechnics)
@@ -25,8 +26,9 @@ git push origin main # auto-deploys via Portainer webhook
It runs on an on-premise Ubuntu server at `10.10.10.166`, containerized with Docker, managed via Portainer, served by Nginx Proxy Manager. It runs on an on-premise Ubuntu server at `10.10.10.166`, containerized with Docker, managed via Portainer, served by Nginx Proxy Manager.
### Stack ### Stack
| Layer | Technology | | Layer | Technology |
|---|---| | ------------ | ---------------------------------------------------------------------------- |
| Framework | Next.js 16.x, App Router, TypeScript (strict) | | Framework | Next.js 16.x, App Router, TypeScript (strict) |
| Styling | Tailwind CSS v4, shadcn/ui | | Styling | Tailwind CSS v4, shadcn/ui |
| Database | PostgreSQL (10.10.10.166:5432) via Prisma v6 ORM | | Database | PostgreSQL (10.10.10.166:5432) via Prisma v6 ORM |
@@ -38,6 +40,7 @@ It runs on an on-premise Ubuntu server at `10.10.10.166`, containerized with Doc
| Language | Code in **English**, UI in **Romanian** | | Language | Code in **English**, UI in **Romanian** |
### Architecture Principles ### Architecture Principles
- **Module platform, not monolith** — each module isolated with own types/services/hooks/components - **Module platform, not monolith** — each module isolated with own types/services/hooks/components
- **Feature flags** gate module loading (disabled = zero bundle cost) - **Feature flags** gate module loading (disabled = zero bundle cost)
- **Storage abstraction**: `StorageService` interface with adapters (database default via Prisma, localStorage fallback) - **Storage abstraction**: `StorageService` interface with adapters (database default via Prisma, localStorage fallback)
@@ -94,7 +97,7 @@ legacy/ # Original HTML tools for reference
## Implemented Modules (14/14 — zero placeholders) ## Implemented Modules (14/14 — zero placeholders)
| # | Module | Route | Key Features | | # | Module | Route | Key Features |
|---|---|---|---| | --- | ---------------------- | --------------------- | --------------------------------------------------------------------------------------------------- |
| 1 | **Dashboard** | `/` | KPI cards (6), activity feed (last 20), module grid, external tools | | 1 | **Dashboard** | `/` | KPI cards (6), activity feed (last 20), module grid, external tools |
| 2 | **Email Signature** | `/email-signature` | Multi-company branding, address toggle, live preview, zoom/copy/download | | 2 | **Email Signature** | `/email-signature` | Multi-company branding, address toggle, live preview, zoom/copy/download |
| 3 | **Word XML Generator** | `/word-xml` | Category-based XML gen, simple/advanced mode, ZIP export | | 3 | **Word XML Generator** | `/word-xml` | Category-based XML gen, simple/advanced mode, ZIP export |
@@ -113,6 +116,7 @@ legacy/ # Original HTML tools for reference
### Registratura — Legal Deadline Tracking (Termene Legale) ### Registratura — Legal Deadline Tracking (Termene Legale)
The Registratura module includes a full legal deadline tracking engine for Romanian construction permitting: The Registratura module includes a full legal deadline tracking engine for Romanian construction permitting:
- **16 deadline types** across 5 categories (Avize, Completări, Analiză, Autorizare, Publicitate) - **16 deadline types** across 5 categories (Avize, Completări, Analiză, Autorizare, Publicitate)
- **Working days vs calendar days** with Romanian public holiday support (including Orthodox Easter via Meeus algorithm) - **Working days vs calendar days** with Romanian public holiday support (including Orthodox Easter via Meeus algorithm)
- **Backward deadlines** (e.g., AC extension: 45 working days BEFORE expiry) - **Backward deadlines** (e.g., AC extension: 45 working days BEFORE expiry)
@@ -121,6 +125,7 @@ The Registratura module includes a full legal deadline tracking engine for Roman
- **Tabbed UI**: "Registru" tab (existing registry) + "Termene legale" tab (deadline dashboard) - **Tabbed UI**: "Registru" tab (existing registry) + "Termene legale" tab (deadline dashboard)
Key files: Key files:
- `services/working-days.ts` — Romanian holidays, `addWorkingDays()`, `isWorkingDay()` - `services/working-days.ts` — Romanian holidays, `addWorkingDays()`, `isWorkingDay()`
- `services/deadline-catalog.ts` — 16 `DeadlineTypeDef` entries - `services/deadline-catalog.ts` — 16 `DeadlineTypeDef` entries
- `services/deadline-service.ts``createTrackedDeadline()`, `resolveDeadline()`, `aggregateDeadlines()` - `services/deadline-service.ts``createTrackedDeadline()`, `resolveDeadline()`, `aggregateDeadlines()`
@@ -134,7 +139,7 @@ Key files:
### Server: `10.10.10.166` (Ubuntu) ### Server: `10.10.10.166` (Ubuntu)
| Service | Port | Purpose | | Service | Port | Purpose |
|---|---|---| | ----------------------- | ---------------------- | ----------------------------------- |
| **ArchiTools** | 3000 | This app (tools.beletage.ro) | | **ArchiTools** | 3000 | This app (tools.beletage.ro) |
| **Gitea** | 3002 | Git hosting (git.beletage.ro) | | **Gitea** | 3002 | Git hosting (git.beletage.ro) |
| **PostgreSQL** | 5432 | App database (Prisma ORM) | | **PostgreSQL** | 5432 | App database (Prisma ORM) |
@@ -164,6 +169,7 @@ git push origin main
``` ```
### Docker ### Docker
- `Dockerfile`: 3-stage build (deps → builder → runner), `node:20-alpine`, non-root user - `Dockerfile`: 3-stage build (deps → builder → runner), `node:20-alpine`, non-root user
- `Dockerfile` includes `npx prisma generate` before build step - `Dockerfile` includes `npx prisma generate` before build step
- `docker-compose.yml`: single service, port 3000, **all env vars hardcoded** (Portainer CE can't inject env vars) - `docker-compose.yml`: single service, port 3000, **all env vars hardcoded** (Portainer CE can't inject env vars)
@@ -175,12 +181,14 @@ git push origin main
## Development Rules ## Development Rules
### TypeScript Strict Mode Gotchas ### TypeScript Strict Mode Gotchas
- `array.split()[0]` returns `string | undefined` — use `.slice(0, 10)` instead - `array.split()[0]` returns `string | undefined` — use `.slice(0, 10)` instead
- `Record<string, T>[key]` returns `T | undefined` — always guard with null check - `Record<string, T>[key]` returns `T | undefined` — always guard with null check
- Spread of possibly-undefined objects: `{ ...obj[key], field }` — check existence first - Spread of possibly-undefined objects: `{ ...obj[key], field }` — check existence first
- lucide-react Icons: cast through `unknown``React.ComponentType<{ className?: string }>` - lucide-react Icons: cast through `unknown``React.ComponentType<{ className?: string }>`
### Conventions ### Conventions
- **Code**: English - **Code**: English
- **UI text**: Romanian - **UI text**: Romanian
- **Components**: functional, `'use client'` directive where needed - **Components**: functional, `'use client'` directive where needed
@@ -189,8 +197,18 @@ git push origin main
- **Dates**: ISO strings (`YYYY-MM-DD` for display, full ISO for timestamps) - **Dates**: ISO strings (`YYYY-MM-DD` for display, full ISO for timestamps)
- **No emojis** in code or UI unless explicitly requested - **No emojis** in code or UI unless explicitly requested
### Storage Performance Rules
- **NEVER** use `storage.list()` followed by `storage.get()` in a loop — this is an N+1 query bug
- `list()` fetches ALL items (keys+values) from DB but discards values, then each `get()` re-fetches individually
- **ALWAYS** use `storage.exportAll()` (namespaced) or `storage.export(namespace)` (service-level) to batch-load
- Filter items client-side after a single fetch: `for (const [key, value] of Object.entries(all)) { ... }`
- After mutations (add/update), either do optimistic local state update or a single `refresh()` — never both
### Module Development Pattern ### Module Development Pattern
Every module follows: Every module follows:
``` ```
src/modules/<name>/ src/modules/<name>/
├── components/ # React components ├── components/ # React components
@@ -202,6 +220,7 @@ src/modules/<name>/
``` ```
### Before Pushing ### Before Pushing
1. `npx next build` — must pass with zero errors 1. `npx next build` — must pass with zero errors
2. Test the feature manually on `localhost:3000` 2. Test the feature manually on `localhost:3000`
3. Commit with descriptive message 3. Commit with descriptive message
@@ -212,7 +231,7 @@ src/modules/<name>/
## Company IDs ## Company IDs
| ID | Name | Prefix | | ID | Name | Prefix |
|---|---|---| | ----------------- | --------------- | ------ |
| `beletage` | Beletage | B | | `beletage` | Beletage | B |
| `urban-switch` | Urban Switch | US | | `urban-switch` | Urban Switch | US |
| `studii-de-teren` | Studii de Teren | SDT | | `studii-de-teren` | Studii de Teren | SDT |
@@ -223,7 +242,7 @@ src/modules/<name>/
## Current Integrations ## Current Integrations
| Feature | Status | Notes | | Feature | Status | Notes |
|---|---|---| | ------------------- | ---------------------- | ------------------------------------------------------- |
| **Authentik SSO** | ✅ Active | NextAuth v4 + OIDC, group→role/company mapping | | **Authentik SSO** | ✅ Active | NextAuth v4 + OIDC, group→role/company mapping |
| **PostgreSQL** | ✅ Active | Prisma ORM, `KeyValueStore` model, `/api/storage` route | | **PostgreSQL** | ✅ Active | Prisma ORM, `KeyValueStore` model, `/api/storage` route |
| **MinIO** | Client configured | 10.10.10.166:9002, bucket `tools`, adapter pending | | **MinIO** | Client configured | 10.10.10.166:9002, bucket `tools`, adapter pending |
@@ -235,7 +254,7 @@ src/modules/<name>/
## Model Recommendations ## Model Recommendations
| Task Type | Claude | OpenAI | Google | Notes | | Task Type | Claude | OpenAI | Google | Notes |
|---|---|---|---|---| | ----------------------------- | -------------- | ------------- | ---------------- | ----------------------------------------------- |
| **Bug fixes, config** | Haiku 4.5 | GPT-4o-mini | Gemini 2.5 Flash | Fast, cheap | | **Bug fixes, config** | Haiku 4.5 | GPT-4o-mini | Gemini 2.5 Flash | Fast, cheap |
| **Features, tests, UI** | **Sonnet 4.6** | GPT-5.2 | Gemini 3 Flash | Best value — Opus-class quality at Sonnet price | | **Features, tests, UI** | **Sonnet 4.6** | GPT-5.2 | Gemini 3 Flash | Best value — Opus-class quality at Sonnet price |
| **New modules, architecture** | Opus 4.6 | GPT-5.3-Codex | Gemini 3 Pro | Complex multi-file, business logic | | **New modules, architecture** | Opus 4.6 | GPT-5.3-Codex | Gemini 3 Pro | Complex multi-file, business logic |
@@ -243,6 +262,7 @@ src/modules/<name>/
**Default: Sonnet 4.6** for most work. See `ROADMAP.md` for per-task recommendations. **Default: Sonnet 4.6** for most work. See `ROADMAP.md` for per-task recommendations.
### Session Handoff Tips ### Session Handoff Tips
- Read this `CLAUDE.md` first — it has all context - Read this `CLAUDE.md` first — it has all context
- Read `ROADMAP.md` for the complete task list with dependencies - Read `ROADMAP.md` for the complete task list with dependencies
- Check `docs/` for deep dives on specific systems - Check `docs/` for deep dives on specific systems
@@ -256,7 +276,7 @@ src/modules/<name>/
## Documentation Index ## Documentation Index
| Doc | Path | Content | | Doc | Path | Content |
|---|---|---| | ------------------- | ------------------------------------------ | -------------------------------------------- |
| System Architecture | `docs/architecture/SYSTEM-ARCHITECTURE.md` | Overall architecture, module platform design | | System Architecture | `docs/architecture/SYSTEM-ARCHITECTURE.md` | Overall architecture, module platform design |
| Module System | `docs/architecture/MODULE-SYSTEM.md` | Module registry, lifecycle, config format | | Module System | `docs/architecture/MODULE-SYSTEM.md` | Module registry, lifecycle, config format |
| Feature Flags | `docs/architecture/FEATURE-FLAGS.md` | Flag system, env overrides | | Feature Flags | `docs/architecture/FEATURE-FLAGS.md` | Flag system, env overrides |
+55
View File
@@ -4,6 +4,61 @@
--- ---
## Session — 2026-02-27 late night #2 (GitHub Copilot - Claude Opus 4.6)
### Context
Performance investigation: Registratura loading extremely slowly with only 6 entries. Rack numbering inverted.
### Root Cause Analysis — Findings
**CRITICAL BUG: N+1 Query Pattern in ALL Storage Hooks**
The `DatabaseStorageAdapter.list()` method fetches ALL items (keys + values) from PostgreSQL in one HTTP request, but **discards the values** and returns only the key names. Then every hook calls `storage.get(key)` for EACH key individually — making a separate HTTP request + DB query per item.
With 6 registry entries + ~10 contacts + ~20 tags, the Registratura page fired **~40 sequential HTTP requests** on load (**1 list + N gets per hook × 3 hooks**). Each request goes through: browser → Next.js API route → Prisma → PostgreSQL → back. This is a textbook N+1 query problem.
**Additional issues found:**
- `addEntry()` called `getAllEntries()` for number generation, then `refresh()` called `getAllEntries()` again → double-fetch
- `closeEntry()` called `updateEntry()` (which refreshes), then manually called `refresh()` again → double-refresh
- Every single module hook had the same pattern (11 hooks total)
- Rack visualization rendered U1 at top instead of bottom
### Fix Applied
**Strategy:** Replace `list()` + N × `get()` with a single `exportAll()` call that fetches all items in one HTTP request and filters client-side.
**Files fixed (13 total):**
- `src/modules/registratura/services/registry-service.ts` — added `exportAll` to `RegistryStorage` interface, rewrote `getAllEntries`
- `src/modules/registratura/hooks/use-registry.ts``addEntry` uses optimistic local state update instead of double-refresh; `closeEntry` batches saves with `Promise.all` + single refresh
- `src/modules/address-book/hooks/use-contacts.ts``exportAll` batch load
- `src/modules/it-inventory/hooks/use-inventory.ts``exportAll` batch load
- `src/modules/password-vault/hooks/use-vault.ts``exportAll` batch load
- `src/modules/word-templates/hooks/use-templates.ts``exportAll` batch load
- `src/modules/prompt-generator/hooks/use-prompt-generator.ts``exportAll` batch load
- `src/modules/hot-desk/hooks/use-reservations.ts``exportAll` batch load
- `src/modules/email-signature/hooks/use-saved-signatures.ts``exportAll` batch load
- `src/modules/digital-signatures/hooks/use-signatures.ts``exportAll` batch load
- `src/modules/ai-chat/hooks/use-chat.ts``exportAll` batch load
- `src/core/tagging/tag-service.ts` — uses `storage.export()` instead of N+1
- `src/modules/it-inventory/components/server-rack.tsx` — reversed slot rendering (U1 at bottom)
**Performance impact:** ~90% reduction in HTTP requests on page load. Registratura: from ~40 requests to 3 (one per namespace: registratura, address-book, tags).
### Prevention Rules (for future development)
> **NEVER** use the `storage.list()` + loop `storage.get()` pattern.
> Always use `storage.exportAll()` to load all items in a namespace, then filter client-side.
> This is the #1 performance pitfall in the storage layer.
### Commits
- `c45a30e` perf: fix N+1 query pattern across all modules + rack numbering
---
## Session — 2026-02-27 late night (GitHub Copilot - Claude Opus 4.6) ## Session — 2026-02-27 late night (GitHub Copilot - Claude Opus 4.6)
### Context ### Context