gitadmin/ArchiTools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: document N+1 performance bug findings and prevention rules

Browse Source
This commit is contained in:
AI Assistant
2026-02-27 22:27:07 +02:00
parent c45a30ec14
commit db9bcd7192
2 changed files with 153 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+98 -78
View File
@@ -18,6 +18,7 @@ git push origin main # auto-deploys via Portainer webhook
## Project Overview
**ArchiTools** is a modular internal web dashboard for an architecture/engineering office group of 3 companies:
- **Beletage** (architecture)
- **Urban Switch** (urbanism)
- **Studii de Teren** (geotechnics)
@@ -25,19 +26,21 @@ git push origin main # auto-deploys via Portainer webhook
It runs on an on-premise Ubuntu server at `10.10.10.166`, containerized with Docker, managed via Portainer, served by Nginx Proxy Manager.
### Stack
| Layer | Technology |
|---|---|
| Framework | Next.js 16.x, App Router, TypeScript (strict) |
| Styling | Tailwind CSS v4, shadcn/ui |
| Database | PostgreSQL (10.10.10.166:5432) via Prisma v6 ORM |
| Storage | `DatabaseStorageAdapter` (PostgreSQL) — localStorage fallback available |
| Layer | Technology |
| ------------ | ---------------------------------------------------------------------------- |
| Framework | Next.js 16.x, App Router, TypeScript (strict) |
| Styling | Tailwind CSS v4, shadcn/ui |
| Database | PostgreSQL (10.10.10.166:5432) via Prisma v6 ORM |
| Storage | `DatabaseStorageAdapter` (PostgreSQL) — localStorage fallback available |
| File Storage | MinIO (10.10.10.166:9002 API / 9003 UI) — client configured, adapter pending |
| Auth | NextAuth v4 + Authentik OIDC (auth.beletage.ro) |
| Deploy | Docker multi-stage, Portainer CE, Nginx Proxy Manager |
| Repo | Gitea at `https://git.beletage.ro/gitadmin/ArchiTools` |
| Language | Code in **English**, UI in **Romanian** |
| Auth | NextAuth v4 + Authentik OIDC (auth.beletage.ro) |
| Deploy | Docker multi-stage, Portainer CE, Nginx Proxy Manager |
| Repo | Gitea at `https://git.beletage.ro/gitadmin/ArchiTools` |
| Language | Code in **English**, UI in **Romanian** |
### Architecture Principles
- **Module platform, not monolith** — each module isolated with own types/services/hooks/components
- **Feature flags** gate module loading (disabled = zero bundle cost)
- **Storage abstraction**: `StorageService` interface with adapters (database default via Prisma, localStorage fallback)
@@ -93,26 +96,27 @@ legacy/ # Original HTML tools for reference
## Implemented Modules (14/14 — zero placeholders)
| # | Module | Route | Key Features |
|---|---|---|---|
| 1 | **Dashboard** | `/` | KPI cards (6), activity feed (last 20), module grid, external tools |
| 2 | **Email Signature** | `/email-signature` | Multi-company branding, address toggle, live preview, zoom/copy/download |
| 3 | **Word XML Generator** | `/word-xml` | Category-based XML gen, simple/advanced mode, ZIP export |
| 4 | **Registratura** | `/registratura` | CRUD registry, stats, filters, **legal deadline tracking** |
| 5 | **Tag Manager** | `/tag-manager` | CRUD tags, category/scope/color, US/SDT seeds, mandatory categories |
| 6 | **IT Inventory** | `/it-inventory` | Equipment tracking, Address Book contact link, type/status/company filters |
| 7 | **Address Book** | `/address-book` | CRUD contacts, card grid, vCard export, Registratura reverse lookup, **dynamic types** |
| 8 | **Password Vault** | `/password-vault` | CRUD credentials, email field, clickable URLs, strength meter, company scope |
| 9 | **Mini Utilities** | `/mini-utilities` | Text case, char counter, percentage, area converter, U→R, artifact cleaner, MDLPA, PDF reducer, OCR |
| 10 | **Prompt Generator** | `/prompt-generator` | Template-driven prompt builder, 14 builtin templates |
| 11 | **Digital Signatures** | `/digital-signatures` | CRUD assets, drag-and-drop file upload, tag chips |
| 12 | **Word Templates** | `/word-templates` | Template library, 8 categories, version tracking, .docx placeholder auto-detection |
| 13 | **AI Chat** | `/ai-chat` | Session-based chat UI, demo mode (no API keys yet) |
| 14 | **Hot Desk** | `/hot-desk` | 4 desks, week-ahead calendar, room layout (window+door), reserve/cancel |
| # | Module | Route | Key Features |
| --- | ---------------------- | --------------------- | --------------------------------------------------------------------------------------------------- |
| 1 | **Dashboard** | `/` | KPI cards (6), activity feed (last 20), module grid, external tools |
| 2 | **Email Signature** | `/email-signature` | Multi-company branding, address toggle, live preview, zoom/copy/download |
| 3 | **Word XML Generator** | `/word-xml` | Category-based XML gen, simple/advanced mode, ZIP export |
| 4 | **Registratura** | `/registratura` | CRUD registry, stats, filters, **legal deadline tracking** |
| 5 | **Tag Manager** | `/tag-manager` | CRUD tags, category/scope/color, US/SDT seeds, mandatory categories |
| 6 | **IT Inventory** | `/it-inventory` | Equipment tracking, Address Book contact link, type/status/company filters |
| 7 | **Address Book** | `/address-book` | CRUD contacts, card grid, vCard export, Registratura reverse lookup, **dynamic types** |
| 8 | **Password Vault** | `/password-vault` | CRUD credentials, email field, clickable URLs, strength meter, company scope |
| 9 | **Mini Utilities** | `/mini-utilities` | Text case, char counter, percentage, area converter, U→R, artifact cleaner, MDLPA, PDF reducer, OCR |
| 10 | **Prompt Generator** | `/prompt-generator` | Template-driven prompt builder, 14 builtin templates |
| 11 | **Digital Signatures** | `/digital-signatures` | CRUD assets, drag-and-drop file upload, tag chips |
| 12 | **Word Templates** | `/word-templates` | Template library, 8 categories, version tracking, .docx placeholder auto-detection |
| 13 | **AI Chat** | `/ai-chat` | Session-based chat UI, demo mode (no API keys yet) |
| 14 | **Hot Desk** | `/hot-desk` | 4 desks, week-ahead calendar, room layout (window+door), reserve/cancel |
### Registratura — Legal Deadline Tracking (Termene Legale)
The Registratura module includes a full legal deadline tracking engine for Romanian construction permitting:
- **16 deadline types** across 5 categories (Avize, Completări, Analiză, Autorizare, Publicitate)
- **Working days vs calendar days** with Romanian public holiday support (including Orthodox Easter via Meeus algorithm)
- **Backward deadlines** (e.g., AC extension: 45 working days BEFORE expiry)
@@ -121,6 +125,7 @@ The Registratura module includes a full legal deadline tracking engine for Roman
- **Tabbed UI**: "Registru" tab (existing registry) + "Termene legale" tab (deadline dashboard)
Key files:
- `services/working-days.ts` — Romanian holidays, `addWorkingDays()`, `isWorkingDay()`
- `services/deadline-catalog.ts` — 16 `DeadlineTypeDef` entries
- `services/deadline-service.ts` — `createTrackedDeadline()`, `resolveDeadline()`, `aggregateDeadlines()`
@@ -133,23 +138,23 @@ Key files:
### Server: `10.10.10.166` (Ubuntu)
| Service | Port | Purpose |
|---|---|---|
| **ArchiTools** | 3000 | This app (tools.beletage.ro) |
| **Gitea** | 3002 | Git hosting (git.beletage.ro) |
| **PostgreSQL** | 5432 | App database (Prisma ORM) |
| **Portainer** | 9000 | Docker management |
| **Nginx Proxy Manager** | 81 (admin) | Reverse proxy + SSL termination |
| **Uptime Kuma** | 3001 | Service monitoring |
| **MinIO** | 9002 (API) / 9003 (UI) | Object storage |
| **Authentik** | 9100 | SSO (auth.beletage.ro) — **active** |
| **N8N** | 5678 | Workflow automation (future) |
| **Stirling PDF** | 8087 | PDF tools |
| **IT-Tools** | 8085 | Developer utilities |
| **FileBrowser** | 8086 | File management |
| **Netdata** | 19999 | System monitoring |
| **Dozzle** | 9999 | Docker log viewer |
| **CrowdSec** | 8088 | Security |
| Service | Port | Purpose |
| ----------------------- | ---------------------- | ----------------------------------- |
| **ArchiTools** | 3000 | This app (tools.beletage.ro) |
| **Gitea** | 3002 | Git hosting (git.beletage.ro) |
| **PostgreSQL** | 5432 | App database (Prisma ORM) |
| **Portainer** | 9000 | Docker management |
| **Nginx Proxy Manager** | 81 (admin) | Reverse proxy + SSL termination |
| **Uptime Kuma** | 3001 | Service monitoring |
| **MinIO** | 9002 (API) / 9003 (UI) | Object storage |
| **Authentik** | 9100 | SSO (auth.beletage.ro) — **active** |
| **N8N** | 5678 | Workflow automation (future) |
| **Stirling PDF** | 8087 | PDF tools |
| **IT-Tools** | 8085 | Developer utilities |
| **FileBrowser** | 8086 | File management |
| **Netdata** | 19999 | System monitoring |
| **Dozzle** | 9999 | Docker log viewer |
| **CrowdSec** | 8088 | Security |
### Deployment Pipeline
@@ -164,6 +169,7 @@ git push origin main
```
### Docker
- `Dockerfile`: 3-stage build (deps → builder → runner), `node:20-alpine`, non-root user
- `Dockerfile` includes `npx prisma generate` before build step
- `docker-compose.yml`: single service, port 3000, **all env vars hardcoded** (Portainer CE can't inject env vars)
@@ -175,12 +181,14 @@ git push origin main
## Development Rules
### TypeScript Strict Mode Gotchas
- `array.split()[0]` returns `string | undefined` — use `.slice(0, 10)` instead
- `Record<string, T>[key]` returns `T | undefined` — always guard with null check
- Spread of possibly-undefined objects: `{ ...obj[key], field }` — check existence first
- lucide-react Icons: cast through `unknown` → `React.ComponentType<{ className?: string }>`
### Conventions
- **Code**: English
- **UI text**: Romanian
- **Components**: functional, `'use client'` directive where needed
@@ -189,8 +197,18 @@ git push origin main
- **Dates**: ISO strings (`YYYY-MM-DD` for display, full ISO for timestamps)
- **No emojis** in code or UI unless explicitly requested
### Storage Performance Rules
- **NEVER** use `storage.list()` followed by `storage.get()` in a loop — this is an N+1 query bug
- `list()` fetches ALL items (keys+values) from DB but discards values, then each `get()` re-fetches individually
- **ALWAYS** use `storage.exportAll()` (namespaced) or `storage.export(namespace)` (service-level) to batch-load
- Filter items client-side after a single fetch: `for (const [key, value] of Object.entries(all)) { ... }`
- After mutations (add/update), either do optimistic local state update or a single `refresh()` — never both
### Module Development Pattern
Every module follows:
```
src/modules/<name>/
├── components/ # React components
@@ -202,6 +220,7 @@ src/modules/<name>/
```
### Before Pushing
1. `npx next build` — must pass with zero errors
2. Test the feature manually on `localhost:3000`
3. Commit with descriptive message
@@ -211,38 +230,39 @@ src/modules/<name>/
## Company IDs
| ID | Name | Prefix |
|---|---|---|
| `beletage` | Beletage | B |
| `urban-switch` | Urban Switch | US |
| `studii-de-teren` | Studii de Teren | SDT |
| `group` | Grup | G |
| ID | Name | Prefix |
| ----------------- | --------------- | ------ |
| `beletage` | Beletage | B |
| `urban-switch` | Urban Switch | US |
| `studii-de-teren` | Studii de Teren | SDT |
| `group` | Grup | G |
---
## Current Integrations
| Feature | Status | Notes |
|---|---|---|
| **Authentik SSO** | ✅ Active | NextAuth v4 + OIDC, group→role/company mapping |
| **PostgreSQL** | ✅ Active | Prisma ORM, `KeyValueStore` model, `/api/storage` route |
| **MinIO** | Client configured | 10.10.10.166:9002, bucket `tools`, adapter pending |
| **AI Chat API** | UI complete, demo mode | No API keys yet; supports Claude/GPT/Ollama |
| **N8N automations** | Webhook URL configured | For notifications, backups, workflows |
| Feature | Status | Notes |
| ------------------- | ---------------------- | ------------------------------------------------------- |
| **Authentik SSO** | ✅ Active | NextAuth v4 + OIDC, group→role/company mapping |
| **PostgreSQL** | ✅ Active | Prisma ORM, `KeyValueStore` model, `/api/storage` route |
| **MinIO** | Client configured | 10.10.10.166:9002, bucket `tools`, adapter pending |
| **AI Chat API** | UI complete, demo mode | No API keys yet; supports Claude/GPT/Ollama |
| **N8N automations** | Webhook URL configured | For notifications, backups, workflows |
---
## Model Recommendations
| Task Type | Claude | OpenAI | Google | Notes |
|---|---|---|---|---|
| **Bug fixes, config** | Haiku 4.5 | GPT-4o-mini | Gemini 2.5 Flash | Fast, cheap |
| **Features, tests, UI** | **Sonnet 4.6** | GPT-5.2 | Gemini 3 Flash | Best value — Opus-class quality at Sonnet price |
| **New modules, architecture** | Opus 4.6 | GPT-5.3-Codex | Gemini 3 Pro | Complex multi-file, business logic |
| Task Type | Claude | OpenAI | Google | Notes |
| ----------------------------- | -------------- | ------------- | ---------------- | ----------------------------------------------- |
| **Bug fixes, config** | Haiku 4.5 | GPT-4o-mini | Gemini 2.5 Flash | Fast, cheap |
| **Features, tests, UI** | **Sonnet 4.6** | GPT-5.2 | Gemini 3 Flash | Best value — Opus-class quality at Sonnet price |
| **New modules, architecture** | Opus 4.6 | GPT-5.3-Codex | Gemini 3 Pro | Complex multi-file, business logic |
**Default: Sonnet 4.6** for most work. See `ROADMAP.md` for per-task recommendations.
### Session Handoff Tips
- Read this `CLAUDE.md` first — it has all context
- Read `ROADMAP.md` for the complete task list with dependencies
- Check `docs/` for deep dives on specific systems
@@ -255,21 +275,21 @@ src/modules/<name>/
## Documentation Index
| Doc | Path | Content |
|---|---|---|
| Doc | Path | Content |
| ------------------- | ------------------------------------------ | -------------------------------------------- |
| System Architecture | `docs/architecture/SYSTEM-ARCHITECTURE.md` | Overall architecture, module platform design |
| Module System | `docs/architecture/MODULE-SYSTEM.md` | Module registry, lifecycle, config format |
| Feature Flags | `docs/architecture/FEATURE-FLAGS.md` | Flag system, env overrides |
| Storage Layer | `docs/architecture/STORAGE-LAYER.md` | StorageService interface, adapters |
| Tagging System | `docs/architecture/TAGGING-SYSTEM.md` | Cross-module tags |
| Security & Roles | `docs/architecture/SECURITY-AND-ROLES.md` | Visibility, auth, roles |
| Module Dev Guide | `docs/guides/MODULE-DEVELOPMENT.md` | How to create a new module |
| HTML Integration | `docs/guides/HTML-TOOL-INTEGRATION.md` | Legacy tool migration |
| UI Design System | `docs/guides/UI-DESIGN-SYSTEM.md` | Design tokens, component patterns |
| Docker Deployment | `docs/guides/DOCKER-DEPLOYMENT.md` | Full Docker/Portainer/Nginx guide |
| Coding Standards | `docs/guides/CODING-STANDARDS.md` | TS strict, naming, patterns |
| Testing Strategy | `docs/guides/TESTING-STRATEGY.md` | Testing approach |
| Configuration | `docs/guides/CONFIGURATION.md` | Env vars, flags, companies |
| Data Model | `docs/DATA-MODEL.md` | All entity schemas |
| Repo Structure | `docs/REPO-STRUCTURE.md` | Directory layout |
| Prompt Generator | `docs/modules/PROMPT-GENERATOR.md` | Prompt module deep dive |
| Module System | `docs/architecture/MODULE-SYSTEM.md` | Module registry, lifecycle, config format |
| Feature Flags | `docs/architecture/FEATURE-FLAGS.md` | Flag system, env overrides |
| Storage Layer | `docs/architecture/STORAGE-LAYER.md` | StorageService interface, adapters |
| Tagging System | `docs/architecture/TAGGING-SYSTEM.md` | Cross-module tags |
| Security & Roles | `docs/architecture/SECURITY-AND-ROLES.md` | Visibility, auth, roles |
| Module Dev Guide | `docs/guides/MODULE-DEVELOPMENT.md` | How to create a new module |
| HTML Integration | `docs/guides/HTML-TOOL-INTEGRATION.md` | Legacy tool migration |
| UI Design System | `docs/guides/UI-DESIGN-SYSTEM.md` | Design tokens, component patterns |
| Docker Deployment | `docs/guides/DOCKER-DEPLOYMENT.md` | Full Docker/Portainer/Nginx guide |
| Coding Standards | `docs/guides/CODING-STANDARDS.md` | TS strict, naming, patterns |
| Testing Strategy | `docs/guides/TESTING-STRATEGY.md` | Testing approach |
| Configuration | `docs/guides/CONFIGURATION.md` | Env vars, flags, companies |
| Data Model | `docs/DATA-MODEL.md` | All entity schemas |
| Repo Structure | `docs/REPO-STRUCTURE.md` | Directory layout |
| Prompt Generator | `docs/modules/PROMPT-GENERATOR.md` | Prompt module deep dive |
SESSION-LOG.md
+55
View File
@@ -4,6 +4,61 @@
---
## Session — 2026-02-27 late night #2 (GitHub Copilot - Claude Opus 4.6)
### Context
Performance investigation: Registratura loading extremely slowly with only 6 entries. Rack numbering inverted.
### Root Cause Analysis — Findings
**CRITICAL BUG: N+1 Query Pattern in ALL Storage Hooks**
The `DatabaseStorageAdapter.list()` method fetches ALL items (keys + values) from PostgreSQL in one HTTP request, but **discards the values** and returns only the key names. Then every hook calls `storage.get(key)` for EACH key individually — making a separate HTTP request + DB query per item.
With 6 registry entries + ~10 contacts + ~20 tags, the Registratura page fired **~40 sequential HTTP requests** on load (**1 list + N gets per hook × 3 hooks**). Each request goes through: browser → Next.js API route → Prisma → PostgreSQL → back. This is a textbook N+1 query problem.
**Additional issues found:**
- `addEntry()` called `getAllEntries()` for number generation, then `refresh()` called `getAllEntries()` again → double-fetch
- `closeEntry()` called `updateEntry()` (which refreshes), then manually called `refresh()` again → double-refresh
- Every single module hook had the same pattern (11 hooks total)
- Rack visualization rendered U1 at top instead of bottom
### Fix Applied
**Strategy:** Replace `list()` + N × `get()` with a single `exportAll()` call that fetches all items in one HTTP request and filters client-side.
**Files fixed (13 total):**
- `src/modules/registratura/services/registry-service.ts` — added `exportAll` to `RegistryStorage` interface, rewrote `getAllEntries`
- `src/modules/registratura/hooks/use-registry.ts` — `addEntry` uses optimistic local state update instead of double-refresh; `closeEntry` batches saves with `Promise.all` + single refresh
- `src/modules/address-book/hooks/use-contacts.ts` — `exportAll` batch load
- `src/modules/it-inventory/hooks/use-inventory.ts` — `exportAll` batch load
- `src/modules/password-vault/hooks/use-vault.ts` — `exportAll` batch load
- `src/modules/word-templates/hooks/use-templates.ts` — `exportAll` batch load
- `src/modules/prompt-generator/hooks/use-prompt-generator.ts` — `exportAll` batch load
- `src/modules/hot-desk/hooks/use-reservations.ts` — `exportAll` batch load
- `src/modules/email-signature/hooks/use-saved-signatures.ts` — `exportAll` batch load
- `src/modules/digital-signatures/hooks/use-signatures.ts` — `exportAll` batch load
- `src/modules/ai-chat/hooks/use-chat.ts` — `exportAll` batch load
- `src/core/tagging/tag-service.ts` — uses `storage.export()` instead of N+1
- `src/modules/it-inventory/components/server-rack.tsx` — reversed slot rendering (U1 at bottom)
**Performance impact:** ~90% reduction in HTTP requests on page load. Registratura: from ~40 requests to 3 (one per namespace: registratura, address-book, tags).
### Prevention Rules (for future development)
> **NEVER** use the `storage.list()` + loop `storage.get()` pattern.
> Always use `storage.exportAll()` to load all items in a namespace, then filter client-side.
> This is the #1 performance pitfall in the storage layer.
### Commits
- `c45a30e` perf: fix N+1 query pattern across all modules + rack numbering
---
## Session — 2026-02-27 late night (GitHub Copilot - Claude Opus 4.6)
### Context