Compare commits
2 Commits
f49fdb1da0
...
b62132ab9e
| Author | SHA1 | Date | |
|---|---|---|---|
| b62132ab9e | |||
| 28c870fb12 |
@@ -0,0 +1,285 @@
|
|||||||
|
# Plan 001 — DB GIS Central Multi-App
|
||||||
|
|
||||||
|
**Status:** DRAFT v1 — neaprobat, de discutat
|
||||||
|
**Autor:** Claude + Marius Tarau
|
||||||
|
**Data:** 2026-04-18
|
||||||
|
**Scope:** Arhitectură unificată GIS pentru ArchiTools, eTerra.live, SmartCity360, Planhub ERP + servicii viitoare
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Context & motivație
|
||||||
|
|
||||||
|
Date GIS (cadastru ANCPI, enrichment, PUG/PUZ, UTR) sunt folosite de multiple platforme:
|
||||||
|
- **ArchiTools** — portal intern Beletage (există, producție)
|
||||||
|
- **eTerra.live** — SaaS topografi (proxy eTerra, repo nou gitadmin/eterra-live)
|
||||||
|
- **SmartCity360** — portal GIS primării (în arhitectură)
|
||||||
|
- **Planhub ERP** — ERP arhitecți cu geoportal (în arhitectură)
|
||||||
|
- Viitoare servicii
|
||||||
|
|
||||||
|
Nevoie: **single source of truth GIS** cu acces controlat, sync centralizat, cache/CDN partajat, izolare multi-tenant.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Topologie
|
||||||
|
|
||||||
|
**All on-prem.** Nimic în cloud public.
|
||||||
|
- Servere: satra (10.10.10.166), shop (10.10.10.84), proxy (10.10.10.199), orchi (10.10.40.60 — CJ2 via VPN)
|
||||||
|
- Gateway public: Traefik pe proxy, Sophos DNAT 80/443
|
||||||
|
- Cloudflare doar pentru domenii SaaS public-facing (eTerra.live, SmartCity360)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. DB Central — Shop server (10.10.10.84)
|
||||||
|
|
||||||
|
**Resurse disponibile (verificat 2026-04-18):**
|
||||||
|
- Xeon Gold 6430, 128 vCPU, 251GB RAM, 3.5TB NVMe (1% ocupat)
|
||||||
|
- Rulează deja: Supabase stack, WordPress, Tegola
|
||||||
|
- Load avg ~0.5, headroom masiv
|
||||||
|
|
||||||
|
**Cluster Postgres nou, dedicat:**
|
||||||
|
- Container `postgres-gis` (separat de Supabase existent — izolare critică)
|
||||||
|
- Postgres 16 + PostGIS 3.4 + pg_cron
|
||||||
|
- SRID nativ 3844 (Stereo70)
|
||||||
|
- Port intern, expus doar pe 10.10.10.0/24 + VPN CJ2 (10.10.40.0/24)
|
||||||
|
- NICIODATĂ expus direct pe internet
|
||||||
|
|
||||||
|
### Schema split
|
||||||
|
|
||||||
|
| Schema | Conținut | Access |
|
||||||
|
|---|---|---|
|
||||||
|
| `gis_core` | Cadastru național ANCPI: UAT, terenuri, clădiri, administrativ | Public read, write doar sync daemon |
|
||||||
|
| `gis_urban` | PUG, PUZ, PUD, UTR, zone protecție, zone reglementate | Read public (layer public), write per tenant primărie (RLS) |
|
||||||
|
| `gis_enrichment` | CF, proprietari, adrese, categorii folosință — **GDPR sensibil** | Auth obligatoriu, RLS pe rol |
|
||||||
|
| `gis_meta` | Sync runs, rules, status, audit | Internal only |
|
||||||
|
| `tenants` | Primării, firme arh, clienți SaaS | Control plane |
|
||||||
|
|
||||||
|
### Roluri Postgres
|
||||||
|
|
||||||
|
- `gis_sync_rw` — write pe `gis_core` + `gis_meta` (sync daemons)
|
||||||
|
- `gis_public_ro` — read pe `gis_core` + layer public din `gis_urban` (Martin public tiles)
|
||||||
|
- `gis_app_ro` — read authenticated cu RLS pe `gis_urban` privat + `gis_enrichment`
|
||||||
|
- `gis_urban_rw` — write pe `gis_urban` doar pentru tenant-ul propriu
|
||||||
|
|
||||||
|
### RLS pe multi-tenant
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- exemplu pentru gis_urban tabele private
|
||||||
|
CREATE POLICY tenant_isolation ON gis_urban.puz_private
|
||||||
|
USING (siruta = ANY(current_setting('app.allowed_sirutas', true)::text[]));
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Layer acces — public/auth endpoints
|
||||||
|
|
||||||
|
| Subdomain | Serviciu | Auth | Scop |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `tiles.beletage.ro` | Martin (instanță dedicată) | Niciun auth pe layer public | Tile server public cadastru + PUG public |
|
||||||
|
| `pmtiles.beletage.ro` | nginx static | Niciun auth | Overview Romania, PMTiles pre-generated |
|
||||||
|
| `gis-api.beletage.ro` | Next.js API sau PostgREST | JWT Authentik | Enrichment, query spațial, mutations |
|
||||||
|
| `sync.beletage.ro` | Orchestrator UI | Authentik admin | Control panel sync central |
|
||||||
|
| `eterra.live` | SaaS topografi | Authentik | Portal topografi |
|
||||||
|
| `planhub.beletage.ro` | ERP arhitecți | Authentik | Geoportal + PM |
|
||||||
|
| `<primarie>.smartcity360.ro` | Portal primărie | Cetățeni anonim + admin Authentik | Subdomain per tenant |
|
||||||
|
|
||||||
|
**Traefik** — reverse proxy unic, SSL auto. **Cloudflare** în față pe domenii SaaS public (eTerra.live, SmartCity360) cu rate limit + CDN cache tile-uri (TTL 24h pentru UAT).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. ACL Enrichment (per rol)
|
||||||
|
|
||||||
|
| Rol | Vede enrichment |
|
||||||
|
|---|---|
|
||||||
|
| `topograf` (eTerra.live) | Full — CF, proprietari, adrese |
|
||||||
|
| `architect` (Planhub ERP) | Full + PUG complet |
|
||||||
|
| `admin_primaria` (SmartCity360) | Full pentru UAT-ul primăriei + date PUG private |
|
||||||
|
| `cetatean` (SmartCity360 public) | Doar nr. cadastral + suprafață + UTR |
|
||||||
|
| `admin` (ArchiTools) | Tot |
|
||||||
|
|
||||||
|
Implementat prin RLS + `current_setting('app.role')`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Shared enrichment pool — insight cheie
|
||||||
|
|
||||||
|
**De-duplicare globală.** Dacă topograf A cere enrichment pe parcela X:
|
||||||
|
1. Salvat în `gis_enrichment` central
|
||||||
|
2. Topograf B cere aceeași parcelă → **hit instant**, zero request la eTerra
|
||||||
|
3. TTL default 90 zile → după expire auto-refresh
|
||||||
|
|
||||||
|
Beneficii:
|
||||||
|
- Economii masive de sesiuni eTerra
|
||||||
|
- Viteză maximă pentru request-uri repetate
|
||||||
|
- Baza de date crește cu datele "cele mai cerute" de topografi (natural prioritization)
|
||||||
|
|
||||||
|
**TBD:** TTL fix 90 zile vs decizie manuală a topografului pentru "force refresh"?
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Sync workers — centralizat
|
||||||
|
|
||||||
|
**Extrage din ArchiTools într-un serviciu standalone** `gis-sync-orchestrator`:
|
||||||
|
- Un singur scheduler (nu duplicat per app)
|
||||||
|
- Workers:
|
||||||
|
- `eterra-sync` — delta sync cadastru (nocturn automat + on-demand per UAT)
|
||||||
|
- `enrichment-worker` — pulls CF/owners pe cerere
|
||||||
|
- `pug-ingest` (viitor) — import PUG din shapefiles/PDFs
|
||||||
|
- **Queue:** `pg-boss` (Postgres-backed, zero infra extra) sau BullMQ+Redis
|
||||||
|
- **Delta sync** folosește `gis_uats.last_updated_dtm` deja existent
|
||||||
|
- **Notificare apps** la sync complete: `Postgres LISTEN/NOTIFY` + webhook → fiecare app se abonează
|
||||||
|
- **PMTiles rebuild** trigger-uit de orchestrator după sync major (webhook existent satra:9876)
|
||||||
|
|
||||||
|
### Session pool eTerra per-user
|
||||||
|
|
||||||
|
- Tabel `eterra_sessions` keyed by `user_id` (fiecare topograf = sesiune proprie)
|
||||||
|
- Sync on-demand eTerra.live folosește credențialele topografului logat
|
||||||
|
- Sync nocturn automat folosește cont service (actualul hardcoded)
|
||||||
|
- Avantaj: toți topografii + sistemul automat contribuie la menținerea DB-ului actual
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Consumers
|
||||||
|
|
||||||
|
| App | Schemas | Write | DB user | Deploy |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| ArchiTools | core + urban + enrichment | enrichment | `archi_rw` | satra |
|
||||||
|
| eTerra.live | core + enrichment (ACL topograf) | enrichment via sync | `eterra_ro` + `eterra_sync` | satra/shop TBD |
|
||||||
|
| SmartCity360 | core + urban (filter siruta) | urban (PUG/PUZ) | `sc_rw` | TBD |
|
||||||
|
| Planhub ERP | core + urban | proiecte proprii (schemă `erp`) | `erp_rw` | TBD |
|
||||||
|
|
||||||
|
Fiecare app:
|
||||||
|
- Own Authentik OIDC app
|
||||||
|
- Own DB user + permisiuni minimale
|
||||||
|
- Own MinIO bucket
|
||||||
|
- Own Infisical project pentru secrete
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Securitate
|
||||||
|
|
||||||
|
- **DB niciodată expus direct pe internet** → tot prin Martin / API cu auth
|
||||||
|
- **Enrichment** (CF, proprietari) → NU în tile-uri publice, doar via API auth'd
|
||||||
|
- **Rate limit Cloudflare** pe `tiles.*` (ex: 100 req/s per IP)
|
||||||
|
- **Audit log** pe `gis_urban` în `gis_meta.audit` — cine modifică PUG primăriei
|
||||||
|
- **Secrete** în Infisical, proiecte separate per app (zero reuse)
|
||||||
|
- **Backup:** `pg_basebackup` + WAL archiving zilnic → MinIO satra + replică async CJ2 (PITR 14 zile)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Performanță
|
||||||
|
|
||||||
|
- Tabele simplificate pe zoom (deja există pentru UAT: z0/5/8/12) — extinde pe `gis_terenuri` dacă nevoie
|
||||||
|
- **Materialized views** pentru query-uri grele cross-tenant (refresh nocturn)
|
||||||
|
- **Indexe GIST** pe toate `geom`, **BRIN** pe `created_at` pentru audit
|
||||||
|
- Martin pool: 16 conexiuni per instanță, 2 instanțe behind Traefik LB
|
||||||
|
- **Read replica Postgres** când QPS > 500/s (streaming replication)
|
||||||
|
- **PMTiles** pentru tot ce nu se schimbă zilnic (UAT, administrativ) — mult mai rapid decât Martin live
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Scale target an 1
|
||||||
|
|
||||||
|
- ~50 arhitecți (Planhub ERP)
|
||||||
|
- ~100 topografi (eTerra.live)
|
||||||
|
- ~2 primării × 5 admin = 10 admin_primaria (SmartCity360)
|
||||||
|
- Citizen public access (SmartCity360) — nedeterminat
|
||||||
|
|
||||||
|
**Concluzie:** Shop server (128 vCPU, 251GB RAM) = zero probleme la scale-ul ăsta. Rezerve pentru 10x creștere.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Migrare graduală — fără downtime ArchiTools
|
||||||
|
|
||||||
|
1. Setup `postgres-gis` container pe shop, config de bază
|
||||||
|
2. Replicare initial snapshot `architools_db` → `postgres-gis` (pg_dump + restore)
|
||||||
|
3. Creează schemas noi (`gis_urban`, `tenants`), mută tabele existente (`ALTER TABLE SET SCHEMA`)
|
||||||
|
4. Creează roluri + RLS policies, testează cu user nou
|
||||||
|
5. Extrage sync în serviciu standalone — paritate funcțională cu ArchiTools
|
||||||
|
6. Lansează Martin dedicat pe `tiles.beletage.ro`
|
||||||
|
7. ArchiTools trece pe endpoint nou (feature flag)
|
||||||
|
8. Verify paritate + performanță → cutover final
|
||||||
|
9. eTerra.live + SmartCity360 + Planhub ERP conectate la DB central
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 13. Date GIS existente (autoritative)
|
||||||
|
|
||||||
|
| Tabel | Conținut | Features aprox |
|
||||||
|
|---|---|---|
|
||||||
|
| `gis_uats` | Limite UAT Romania (+ views z0/5/8/12 simplified) | 3181 UAT-uri |
|
||||||
|
| `gis_administrativ` | Intravilan + arii speciale ANCPI | — |
|
||||||
|
| `gis_terenuri` | Parcele cadastrale eTerra | Cluj 774K (extrapolare ~25M total Romania) |
|
||||||
|
| `gis_cladiri` | Clădiri cadastrale eTerra | Cluj — (similar magnitude) |
|
||||||
|
| `gis_terenuri_status` / `gis_cladiri_status` | Parcele/clădiri + flag enrichment/legal | — |
|
||||||
|
| `gis_features` | Raw eTerra (attributes JSON + enrichment JSON) | — |
|
||||||
|
| `gis_sync_rules`, `gis_sync_runs` | Orchestrare sync | — |
|
||||||
|
|
||||||
|
Status actual: Cluj complet sincronizat (81 UAT, ~8.3h), restul României de populat (estimare 15-25h inițial, 1-2h delta).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 14. Date GIS de adăugat pentru SmartCity360 + ERP
|
||||||
|
|
||||||
|
**Urbanism PUG:**
|
||||||
|
- `gis_urban.utr` — Unități Teritoriale Referință (code, zone_type, regulation JSON cu POT/CUT/H_max, geom)
|
||||||
|
- `gis_urban.zone_reglementate` — zone funcționale PUG
|
||||||
|
- `gis_urban.zone_protectie` — monumente, arheologie, peisaj, sanitară, hidrologică
|
||||||
|
|
||||||
|
**PUZ/PUD:**
|
||||||
|
- `gis_urban.puz` / `gis_urban.pud` — doc_ref, HCL_number, approval_date, status, geom perimetru
|
||||||
|
|
||||||
|
**Administrativ suplimentar:**
|
||||||
|
- `gis_urban.strazi`
|
||||||
|
- `gis_urban.numerotare`
|
||||||
|
- `gis_urban.dotari_publice` (școli, spitale, parcuri)
|
||||||
|
|
||||||
|
**Fază 2 (opțional):**
|
||||||
|
- Rețele utilități (apă, canal, gaz, electric, telecom)
|
||||||
|
- Cadastru verde (arbori, scuaruri)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 15. Întrebări deschise — TBD
|
||||||
|
|
||||||
|
### Tehnice
|
||||||
|
|
||||||
|
1. **Supabase vs Postgres raw pe shop** — schemă nouă în Supabase existent (câștig: RLS auto, PostgREST gratis, auth) sau container Postgres separat (izolare totală, simplu)? **Recomandare curentă:** separat.
|
||||||
|
|
||||||
|
2. **Domain SaaS** — `eterra.live` deja deținut? `smartcity360.ro` de cumpărat? Subdomain per primărie (`cluj-napoca.smartcity360.ro`) vs path-based (`smartcity360.ro/cluj-napoca`)?
|
||||||
|
|
||||||
|
3. **Portal cetățeni SmartCity** — acces complet anonim (zero login, doar rate limit) vs email/captcha pentru abuse prevention?
|
||||||
|
|
||||||
|
4. **TTL enrichment** — 90 zile auto-refresh implicit? Topograf poate forța refresh manual?
|
||||||
|
|
||||||
|
5. **Backup offsite** — replică Postgres async la CJ2 (10.10.40.x) via VPN? Sau doar MinIO snapshot zilnic?
|
||||||
|
|
||||||
|
6. **Migrare ArchiTools** — DB-ul actual rămâne paralel cu noul central, apoi cutover? Sau migrare directă cu downtime controlat?
|
||||||
|
|
||||||
|
7. **Martin shared vs per-app** — o singură instanță Martin publică pentru toate app-urile, sau instanțe separate per app (branding URL + izolare)?
|
||||||
|
|
||||||
|
### Business / organizaționale
|
||||||
|
|
||||||
|
8. **Billing / metering** — eTerra.live SaaS plătit: how measured? Per sync on-demand? Per enrichment request? Per user/lună?
|
||||||
|
|
||||||
|
9. **Data ownership** — dacă topograf X face enrichment pe parcela Y, datele sunt ale lui privat sau contribuie automat la pool-ul central public? (răspuns parțial: shared pool, dar GDPR?)
|
||||||
|
|
||||||
|
10. **PUG private pentru primării** — ce date specific sunt "private" vs "public"? (ex: observații interne, documentație anexă)
|
||||||
|
|
||||||
|
11. **SLA public tiles** — uptime target? Recovery time după outage?
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 16. Concluzii curente
|
||||||
|
|
||||||
|
- Arhitectura e fezabilă pe infrastructura existentă fără hardware nou
|
||||||
|
- Shop server = candidat ideal DB central
|
||||||
|
- Shared enrichment pool = insight major cu ROI imediat
|
||||||
|
- Migrarea se poate face gradual fără downtime
|
||||||
|
- Securitate + multi-tenancy rezolvate cu RLS + Authentik per-app
|
||||||
|
|
||||||
|
**Next steps când e aprobat:**
|
||||||
|
1. Răspuns la întrebările deschise (secțiunea 15)
|
||||||
|
2. Design detaliat schema `gis_urban` (DDL concret)
|
||||||
|
3. Design API `gis-api.beletage.ro` (endpoints + auth flow)
|
||||||
|
4. Plan migrare ArchiTools pas-cu-pas
|
||||||
|
5. POC postgres-gis pe shop + test Martin + 1 app consumer
|
||||||
@@ -0,0 +1,822 @@
|
|||||||
|
# Plan 001 v2 — DB GIS Central Multi-App
|
||||||
|
|
||||||
|
**Status:** FINAL APPROVED — 2026-04-20 (replaces v1 DRAFT)
|
||||||
|
**Autor:** Marius Tarau + Claude Code
|
||||||
|
**Scope:** Arhitectură unificată GIS pentru ArchiTools, eterra.live, pug.digital, planhub.ro + viitoare servicii
|
||||||
|
**Execution trigger:** user sends "go" in new session → orchestrator begins Sprint 1
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. Cum recunoaște Claude "GO"
|
||||||
|
|
||||||
|
În sesiunea următoare, la cuvântul **"go"** singular:
|
||||||
|
1. Re-read `/home/orchestrator/Code/ArchiTools/docs/plans/001-v2-central-gis-db-multi-app.md` complet
|
||||||
|
2. Read `project_plan001_GO_sequence.md` din memorii
|
||||||
|
3. Verifică secrete pre-loaded în Infisical (listă la §20)
|
||||||
|
4. Pornește **Sprint 1 — Day 1** (§16)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Executive summary
|
||||||
|
|
||||||
|
**Ce construim:** platformă GIS centralizată pe server shop (10.10.10.84) care servește 4 produse (ArchiTools intern, eterra.live SaaS topografi, pug.digital SaaS primării, planhub.ro SaaS arhitecți) + viitoare servicii.
|
||||||
|
|
||||||
|
**Durată:** 5 săptămâni dev + 1 weekend cutover + 1 săptămână cleanup = ~7 săptămâni total.
|
||||||
|
|
||||||
|
**Echipă:** 1 dev (Marius) + Claude Code pair.
|
||||||
|
|
||||||
|
**Rezultat:**
|
||||||
|
- DB unică (PostgreSQL 18.3 + PostGIS 3.6.3) schemă split, RLS multi-tenant
|
||||||
|
- Tile serving unificat (Martin + TiTiler + PMTiles + Cloudflare CDN)
|
||||||
|
- Sync orchestrator cu multi-account ANCPI shuffle
|
||||||
|
- Backup multi-tier (NAS + B2 + viitor Azure + viitor CJ2)
|
||||||
|
- Dashboard unificat admin eterra.live
|
||||||
|
- MDLPA export compliance pentru PUG/PUZ/PUD/PMUD
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. State curent (baseline)
|
||||||
|
|
||||||
|
### Infrastructură
|
||||||
|
- **Shop** (10.10.10.84): Xeon Gold 6430, 128 vCPU, 251GB RAM, 3.3TB NVMe free
|
||||||
|
- **Satra** (10.10.10.166): Ubuntu, 194GB disk (61% plin), architools_postgres curent
|
||||||
|
- **Proxy** (10.10.10.199): Traefik v3.6.8
|
||||||
|
- **NAS NewAmun** (10.10.10.10): NETGEAR ReadyNAS, 40.88TB free, btrfs
|
||||||
|
|
||||||
|
### App-uri curente
|
||||||
|
- ArchiTools (tools.beletage.ro) — Next.js, Prisma, conectat architools_postgres satra
|
||||||
|
- eterra.live — Next.js v5 NextAuth, per-user AES-256-GCM, același DB
|
||||||
|
- Martin tile server v1.4.0 pe satra port 3010
|
||||||
|
|
||||||
|
### POC postgres-gis (2026-04-19)
|
||||||
|
- Deployed pe shop la `/home/dnz/postgres-gis/`
|
||||||
|
- PG 18.3 + PostGIS 3.6.3 + pgBouncer v1.24.1-p1
|
||||||
|
- `architools_db` restored from satra: 16GB, 24 tabele, 100% paritate row counts
|
||||||
|
- Performance test: **9.4× faster** ST_Intersects vs satra
|
||||||
|
- Status: **RUNNING sandbox**, zero impact production
|
||||||
|
|
||||||
|
### Securitate rotate 2026-04-19
|
||||||
|
- AUTHENTIK_CLIENT_SECRET ✅
|
||||||
|
- NEXTAUTH_SECRET ✅
|
||||||
|
- DB_PASS Postgres ✅
|
||||||
|
- MINIO_ACCESS/SECRET_KEY ✅ (user nou gen-hex)
|
||||||
|
- NOTIFICATION_CRON_SECRET ✅
|
||||||
|
- ETERRA_USERNAME/PASSWORD ✅ (mutate Infisical)
|
||||||
|
|
||||||
|
### Backup infrastructure testată 2026-04-20
|
||||||
|
- NAS rsync via SSH key RSYNC-ONLY flag, `gisbkp/` → `/HDD/gisbkp` (symlink), 50 MB/s LAN
|
||||||
|
- Backblaze B2 bucket `beletage-gis-backups` (EU, Object Lock Compliance, SSE-B2), key scoped
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Decizii arhitecturale (finalizate)
|
||||||
|
|
||||||
|
### Golden rule (aplicat întreg planul)
|
||||||
|
Preferă investment upfront pentru câștiguri pe termen lung în: **safety, speed, resources, future-proof**.
|
||||||
|
|
||||||
|
### Tehnologii
|
||||||
|
- **PostgreSQL 18.3** (async I/O, skip scans, temporal constraints, UUID v7)
|
||||||
|
- **PostGIS 3.6.3** (ST_RemoveIrrelevantPointsForView, SFCGAL modern)
|
||||||
|
- **pgBouncer 1.24.1-p1** (transaction mode, 1000 max clients, 50 default pool)
|
||||||
|
- **Martin 1.5.0** tile server (vector MVT)
|
||||||
|
- **TiTiler** (raster COG)
|
||||||
|
- **Next.js 16 + Hono + Prisma + Zod** pentru gis-api
|
||||||
|
- **pg-boss** queue (NU Redis)
|
||||||
|
- **pgBackRest** backup engine
|
||||||
|
- **Docker Compose** (NU Kubernetes)
|
||||||
|
- **MinIO** object storage
|
||||||
|
- **Authentik** OIDC (existent)
|
||||||
|
|
||||||
|
### Versioning policy
|
||||||
|
- Latest stable ≥3 luni în producție + ≥1 patch
|
||||||
|
- Pin exact în compose (NU `:latest`)
|
||||||
|
- Manual upgrades după test pe snapshot
|
||||||
|
- Periodic review trimestrial
|
||||||
|
|
||||||
|
### Domains (brand strategy)
|
||||||
|
| Domain | Rol |
|
||||||
|
|---|---|
|
||||||
|
| `gis.ac` | Infra neutru — tiles, api, pmtiles, s3 |
|
||||||
|
| `eterra.live` | SaaS topografi |
|
||||||
|
| `pug.digital` | SaaS primării (fost SmartCity360) |
|
||||||
|
| `planhub.ro` | SaaS multi-tenant arhitecți |
|
||||||
|
| `app.beletage.ro` | Beletage tenant CNAME rebrand |
|
||||||
|
| `vreau.digital` | Rezervă — portal cetățeni |
|
||||||
|
| `buildini.ai` | Rezervă — AI design |
|
||||||
|
| `puz.digital` | SEO redirect → pug.digital |
|
||||||
|
| `beletage.ro` | Intern + admin |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Target stack shop (13 containere)
|
||||||
|
|
||||||
|
```
|
||||||
|
shop (10.10.10.84) — /home/dnz/postgres-gis/ + /opt/gis-stack/:
|
||||||
|
|
||||||
|
├── postgres-gis PG 18.3 + PostGIS 3.6.3 [✅ POC]
|
||||||
|
├── postgres-gis-pgbouncer transaction mode, 1000 clients [✅ POC]
|
||||||
|
├── martin-public vector tiles publice, zero auth [Sprint 1]
|
||||||
|
├── martin-private vector tiles auth via JWT ForwardAuth [Sprint 1]
|
||||||
|
├── titiler raster COG + DEM proxy [Sprint 1]
|
||||||
|
├── gis-api Next.js 16 + Hono + Prisma + Zod [Sprint 2]
|
||||||
|
├── gis-sync-orchestrator pg-boss workers + session pool [Sprint 3]
|
||||||
|
├── nginx-pmtiles static serve MinIO pmtiles [Sprint 1]
|
||||||
|
├── minio-gis buckets: pmtiles, cog, cf-pdfs, dxf, backups [Sprint 1]
|
||||||
|
├── pgadmin intern doar (pgadmin.beletage.ro) [Sprint 1]
|
||||||
|
├── prometheus scrape metrics [Sprint 4]
|
||||||
|
├── grafana dashboards [Sprint 4]
|
||||||
|
└── alertmanager alerts → n8n webhook [Sprint 4]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Resource utilization estimat:** ~8% CPU shop, ~15% RAM shop. Loaded la plural, idle majority.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Schema DB split
|
||||||
|
|
||||||
|
```
|
||||||
|
postgres-gis / gis (database):
|
||||||
|
|
||||||
|
├── schema: gis_core Cadastru ANCPI (public readable)
|
||||||
|
│ ├── terenuri Parcele (25M estimate)
|
||||||
|
│ ├── cladiri Clădiri cadastrale
|
||||||
|
│ ├── uats 3,186 UAT România
|
||||||
|
│ ├── administrativ Intravilan + arii protejate
|
||||||
|
│ └── uats_z0/5/8/12 Simplified views per zoom
|
||||||
|
│
|
||||||
|
├── schema: gis_urban PUG/PUZ/PUD (multi-tenant RLS per siruta)
|
||||||
|
│ ├── plan_spatial MDLPA-compliant root (SIRUTA, Judet, HCL, stadiu)
|
||||||
|
│ ├── zf_existenta, zf_propusa Zone funcționale + HILUCS codes
|
||||||
|
│ ├── zona_reglementare Zone protecție (monumente, sanitară, etc.)
|
||||||
|
│ ├── utr Unități Teritoriale Referință + regulament JSONB
|
||||||
|
│ ├── puz, pud Sub-plans
|
||||||
|
│ ├── cai_comunicatie Drumuri
|
||||||
|
│ ├── retele_edilitare Apă, canal, gaz, electric, telecom
|
||||||
|
│ ├── echipare_edilitara Hidranți, stații (puncte)
|
||||||
|
│ ├── regulament_local Text regulament
|
||||||
|
│ ├── avize_acorduri PDF metadata
|
||||||
|
│ ├── *_drafts Versiuni work-in-progress arhitecți
|
||||||
|
│ └── puz_in_avizare Public preview în timpul avizării
|
||||||
|
│
|
||||||
|
├── schema: gis_enrichment GDPR sensitive (CF, proprietari, adrese)
|
||||||
|
│ ├── cf_extracts Criptat, RLS strict
|
||||||
|
│ ├── proprietari Nume + CNP
|
||||||
|
│ ├── adrese Normalizare nomenclator
|
||||||
|
│ └── shared_pool GDPR-safe shareable (nr_cad, suprafață, UTR)
|
||||||
|
│
|
||||||
|
├── schema: gis_meta Orchestrare + audit
|
||||||
|
│ ├── sync_runs Log sync eTerra
|
||||||
|
│ ├── sync_rules Planning sync
|
||||||
|
│ ├── audit ENCRYPTED cu pgcrypto (audit metadata)
|
||||||
|
│ ├── eterra_sessions Session pool persistent
|
||||||
|
│ ├── eterra_accounts Multi-account ANCPI shuffle
|
||||||
|
│ ├── eterra_account_usage_log Per-action audit
|
||||||
|
│ └── raster_sources COG registry (upload metadata)
|
||||||
|
│
|
||||||
|
├── schema: tenants Control plane multi-tenancy
|
||||||
|
│ ├── tenants id, name, tenant_type, is_beletage_group, siruta_scope[]
|
||||||
|
│ ├── members user → tenant mapping
|
||||||
|
│ └── enrichment_scopes per-user flag (none/basic/full)
|
||||||
|
│
|
||||||
|
├── schema: eterra App-specific eterra.live
|
||||||
|
├── schema: pug App-specific pug.digital
|
||||||
|
├── schema: archi App-specific ArchiTools
|
||||||
|
├── schema: planhub App-specific planhub.ro (multi-tenant)
|
||||||
|
├── schema: queue pg-boss queue
|
||||||
|
└── schema: public Extensii (postgis, pg_stat_statements, pgcrypto, pg_trgm)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Super-tenant Beletage group
|
||||||
|
Tabela `tenants.is_beletage_group BOOLEAN`. Tenants marked (Beletage SRL, Studii de Teren, Urban Switch, Cubitron, + viitoare firme asociate) au acces NERESTRICȚIONAT la toate datele. Controlat prin admin Beletage.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Roluri Postgres + RLS
|
||||||
|
|
||||||
|
### DB users
|
||||||
|
- `gis_app_rw` — gis-api main
|
||||||
|
- `gis_sync_rw` — orchestrator worker (UNICUL scriitor în gis_core)
|
||||||
|
- `gis_public_ro` — Martin public (read-only)
|
||||||
|
- `gis_private_ro` — Martin-private (read-only cu RLS)
|
||||||
|
- `gis_titiler_ro` — TiTiler raster
|
||||||
|
- `gis_admin_dba` — pgAdmin + Marius (NO BYPASSRLS — Golden)
|
||||||
|
|
||||||
|
### Session variables setate la fiecare tranzacție
|
||||||
|
```sql
|
||||||
|
SET LOCAL app.user_id = ...;
|
||||||
|
SET LOCAL app.tenant_id = ...;
|
||||||
|
SET LOCAL app.is_beletage_group = 'true'|'false';
|
||||||
|
SET LOCAL app.enrichment_scope = 'none'|'basic'|'full';
|
||||||
|
SET LOCAL app.allowed_sirutas = 'csv,list';
|
||||||
|
SET LOCAL app.roles = 'csv,list';
|
||||||
|
```
|
||||||
|
|
||||||
|
### RLS policy patterns
|
||||||
|
- `gis_core`: zero RLS, public read
|
||||||
|
- `gis_urban.*_drafts`: tenant-isolated (own tenant_id OR Beletage group)
|
||||||
|
- `gis_urban.*_aprobate`: public read, write admin_primaria sau Beletage
|
||||||
|
- `gis_enrichment.cf_extracts`:
|
||||||
|
```sql
|
||||||
|
USING (
|
||||||
|
current_setting('app.is_beletage_group', true)::boolean = true
|
||||||
|
OR fetched_by_user_id = current_setting('app.user_id', true)::uuid
|
||||||
|
OR fetched_by_tenant_id = current_setting('app.tenant_id', true)::uuid
|
||||||
|
OR (is_shareable = true AND current_setting('app.enrichment_scope', true) IN ('basic', 'full'))
|
||||||
|
)
|
||||||
|
```
|
||||||
|
- `FORCE ROW LEVEL SECURITY` pe toate tabelele RLS (fallback deny)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Auth flow (Authentik OIDC)
|
||||||
|
|
||||||
|
### JWT custom claims
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"sub": "user-uuid",
|
||||||
|
"email": "...",
|
||||||
|
"tenant_id": "tenant-uuid",
|
||||||
|
"is_beletage_group": true,
|
||||||
|
"roles": ["topograf", "firm_admin"],
|
||||||
|
"enrichment_scope": "full",
|
||||||
|
"allowed_sirutas": ["54975", "155243"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### User attributes sync
|
||||||
|
- **Webhook HMAC-signed** din planhub/eterra.live la user create/update
|
||||||
|
- Idempotency key dedupe 24h
|
||||||
|
- Exponential backoff retry (6 tentative: 1,2,4,8,16,32s)
|
||||||
|
- Dead Letter Queue în pg-boss `auth-sync-dlq`
|
||||||
|
|
||||||
|
### Super-tenant check
|
||||||
|
**Dynamic DB query** la fiecare JWT emit (Authentik custom expression), cache 60s.
|
||||||
|
|
||||||
|
### JWT expiry
|
||||||
|
- Access: 1h
|
||||||
|
- Refresh: 30d
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Sync orchestrator + multi-account ANCPI
|
||||||
|
|
||||||
|
### Componente
|
||||||
|
- **Queue:** pg-boss (schema `queue`), max 8 workers, retention 30d
|
||||||
|
- **Workers:**
|
||||||
|
- `eterra-sync-worker` (delta/full UAT sync)
|
||||||
|
- `enrichment-worker` (CF fetch + GDPR pool split)
|
||||||
|
- `pug-ingest-worker` (GPKG upload → gis_urban drafts)
|
||||||
|
- `pmtiles-rebuild-worker` (tippecanoe → MinIO)
|
||||||
|
- `cache-invalidate-worker` (Cloudflare purge API)
|
||||||
|
|
||||||
|
### Multi-account ANCPI shuffle
|
||||||
|
```sql
|
||||||
|
gis_meta.eterra_accounts (
|
||||||
|
id, username, password_encrypted (AES-256-GCM, key GIS_SYNC_AES_KEY),
|
||||||
|
account_type, quota_per_hour DEFAULT 500,
|
||||||
|
usage_current_hour, last_reset_at,
|
||||||
|
status, notes
|
||||||
|
)
|
||||||
|
```
|
||||||
|
Round-robin pe conturi active cu quota disponibilă. Failover auto dacă blocked. Distribution load ANCPI = zero detection singular-account patterns.
|
||||||
|
|
||||||
|
### Session pool persistent (TTL 20 min)
|
||||||
|
Înlocuiește in-memory din eterra-live + hardcoded din ArchiTools.
|
||||||
|
|
||||||
|
### LISTEN/NOTIFY events cross-app
|
||||||
|
- `sync:uat:done` → apps refresh UI
|
||||||
|
- `pmtiles:rebuild:done` → cache invalidate
|
||||||
|
- `enrichment:new` → user UI notify
|
||||||
|
|
||||||
|
### Scheduler (pg-boss cron)
|
||||||
|
- Delta sync: Lu-Vi 2:00
|
||||||
|
- Deep sync: Weekend 23:00
|
||||||
|
- PMTiles rebuild: Lu-Vi 4:00
|
||||||
|
- Cleanup sessions: 6h
|
||||||
|
|
||||||
|
### ArchiTools/eterra-live post-migration
|
||||||
|
- **ArchiTools:** DELETE eterra-client.ts + setInterval. Read-only consumer + trigger sync via API.
|
||||||
|
- **eterra-live:** Păstrează UI eterra login (per-user), trigger sync prin orchestrator API.
|
||||||
|
- **Orchestrator = UNICUL scriitor gis_core.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Tile serving (4 subdomenii)
|
||||||
|
|
||||||
|
### `tiles.gis.ac` (PUBLIC)
|
||||||
|
Martin 1 instanță, zero auth, CF rate limit 100 req/s/IP.
|
||||||
|
|
||||||
|
Layer groups:
|
||||||
|
- `/cadastru`, `/uat`, `/pug-public`, `/zone-protectie`, `/drumuri`, `/retele-publice`
|
||||||
|
- `/puz-aprobate`, `/pud-aprobate`, `/analize-publice`, `/puz-in-avizare`
|
||||||
|
|
||||||
|
### `tiles-private.gis.ac` (AUTH)
|
||||||
|
Martin-private container + Traefik ForwardAuth JWT validation. Zero cache.
|
||||||
|
|
||||||
|
Layer groups:
|
||||||
|
- `/pug-drafts/{tenant}`, `/puz-drafts/{tenant}`, `/pud-drafts/{tenant}`
|
||||||
|
- `/observatii-primarie/{siruta}`, `/analize-interne/{tenant}`
|
||||||
|
|
||||||
|
### `raster.gis.ac` + `dem.gis.ac`
|
||||||
|
TiTiler (NU Martin — Martin only vector).
|
||||||
|
- `raster.gis.ac` serves COG din MinIO bucket
|
||||||
|
- `dem.gis.ac` = **proxy cu cache CF** la ANCPI MNT (geoportal.ancpi.ro/maps/rest/services/ANCPI/MNT)
|
||||||
|
- Hillshade + contour generate local din DEM
|
||||||
|
|
||||||
|
### `pmtiles.gis.ac`
|
||||||
|
nginx static serve din MinIO bucket `pmtiles`. CF cache 7 zile.
|
||||||
|
|
||||||
|
### CF TTL config
|
||||||
|
| Path | TTL |
|
||||||
|
|---|---|
|
||||||
|
| tiles.gis.ac/uat | 30 zile |
|
||||||
|
| tiles.gis.ac/cadastru | 6h + auto-purge la sync |
|
||||||
|
| tiles.gis.ac/pug-public | 24h |
|
||||||
|
| tiles.gis.ac/puz-in-avizare | 1h |
|
||||||
|
| tiles.gis.ac/puz-aprobate, pud | 24h |
|
||||||
|
| pmtiles.gis.ac | 7 zile |
|
||||||
|
| tiles-private.gis.ac | zero |
|
||||||
|
| raster.gis.ac, dem.gis.ac | 30 zile |
|
||||||
|
|
||||||
|
Bot Fight Mode: Medium default.
|
||||||
|
|
||||||
|
### PUZ lifecycle visibility
|
||||||
|
| Stadiu | Vizibil | Cache |
|
||||||
|
|---|---|---|
|
||||||
|
| Draft intern | tiles-private (tenant only) | zero |
|
||||||
|
| Avizare publică | tiles.gis.ac/puz-in-avizare | 1h |
|
||||||
|
| Aprobat | tiles.gis.ac/puz-aprobate (overlay pug-public) | 24h |
|
||||||
|
|
||||||
|
### Raster Library module (admin eterra.live)
|
||||||
|
- Upload TIFF/GeoTIFF → background gdal_translate → COG + overviews
|
||||||
|
- Preview thumbnail 512px
|
||||||
|
- Public/private toggle
|
||||||
|
- Delete/replace
|
||||||
|
- DB: `gis_meta.raster_sources`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. API layer (api.gis.ac)
|
||||||
|
|
||||||
|
### Stack
|
||||||
|
Next.js 16 + Hono router intern + Prisma + Zod + Authentik JWT middleware.
|
||||||
|
|
||||||
|
### Endpoint exemple
|
||||||
|
```
|
||||||
|
POST /enrichment/parcela auth: tier≥basic
|
||||||
|
GET /parcela/{id} auth: optional
|
||||||
|
POST /pug/zone-functionala auth: admin_primaria
|
||||||
|
POST /sync/uat auth: admin Beletage
|
||||||
|
POST /raster/upload auth: architect cu scope
|
||||||
|
GET /search?q=... auth: optional
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rate limiting (Redis per JWT sub)
|
||||||
|
| Tier | Req/oră | Enrichments/zi |
|
||||||
|
|---|---|---|
|
||||||
|
| Free | 100 | 10 |
|
||||||
|
| Basic | 1000 | 100 |
|
||||||
|
| Pro | 5000 | unlimited |
|
||||||
|
| Admin | nelimitat | nelimitat |
|
||||||
|
|
||||||
|
Managed prin admin eterra.live.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. MDLPA compliance (Ordin 904/2023)
|
||||||
|
|
||||||
|
**Abordare Golden:** schema internă flexibilă + export layer separat conform MDLPA.
|
||||||
|
|
||||||
|
### Internal schema gis_urban.plan
|
||||||
|
Include ALL MDLPA required fields + extras (history, drafts, workflows, comments).
|
||||||
|
|
||||||
|
### PG18 temporal constraint
|
||||||
|
```sql
|
||||||
|
ALTER TABLE gis_urban.plan_spatial ADD CONSTRAINT no_overlap_plan
|
||||||
|
EXCLUDE USING gist (siruta WITH =, doc_type WITH =,
|
||||||
|
daterange(data_aprob, data_exp, '[)') WITH &&);
|
||||||
|
```
|
||||||
|
Garantează zero PUG-uri suprapuse pe aceeași UAT.
|
||||||
|
|
||||||
|
### Export worker
|
||||||
|
- `pug-export-worker` (pg-boss job)
|
||||||
|
- Input: `{plan_id}`
|
||||||
|
- Output: ZIP cu 5 subdirectoare + GPKG MDLPA-compliant + PDF-uri
|
||||||
|
- Tooling: ogr2ogr (GDAL) pentru GPKG, custom orchestrator Node
|
||||||
|
- Drop Z/M dimensions la export
|
||||||
|
- Default HILUCS_N1 dacă missing intern
|
||||||
|
- ZIP naming: `^[A-Z][A-Z]_.*?_\d{1,10}_(PUG|PUZ|PATJ|PMUD)_[0-9]{8}`
|
||||||
|
|
||||||
|
### Validator MDLPA
|
||||||
|
**Faza 1:** manual pre-submit by user (download Validator_2.0.5, run local).
|
||||||
|
**Faza 2:** Marius primește repo validator de la friends MDLPA → containerizăm.
|
||||||
|
**Faza 3:** custom pre-validation 80% reguli (blocker-i obvious) înaintea ZIP download.
|
||||||
|
|
||||||
|
### Doc types în scope
|
||||||
|
PUG (prio 1), PUZ (prio 1), PUD (prio 2), PMUD (da), PATJ (opțional, max 1 județ).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Backup strategy (multi-tier)
|
||||||
|
|
||||||
|
### Tier 1 — NAS local (LAN fast)
|
||||||
|
- Path: `gis-backup@10.10.10.10:gisbkp/` (symlink → `/HDD/gisbkp`)
|
||||||
|
- btrfs + COW + compression
|
||||||
|
- Retention: 30 daily + 12 monthly + 5 yearly snapshots
|
||||||
|
- Protocol: rsync over SSH, RSYNC-ONLY key flag
|
||||||
|
|
||||||
|
### Tier 2 — Backblaze B2 (offsite hot)
|
||||||
|
- Bucket: `beletage-gis-backups` (region EU)
|
||||||
|
- Object Lock Compliance mode 90d + SSE-B2
|
||||||
|
- Weekly full + monthly archive encrypted AES-256 client-side
|
||||||
|
|
||||||
|
### Tier 3 — Azure (deferred, credit €2400/year)
|
||||||
|
- Azure PostgreSQL Flexible Server (DR cloud fallback, ~50-80€/lună)
|
||||||
|
- Azure CDN + Blob Hot (PMTiles secondary, ~20€/lună)
|
||||||
|
- When: scale 10k+ users sau geo-redundanță
|
||||||
|
|
||||||
|
### Tier 4 — CJ2 standby (deferred ~oct 2026)
|
||||||
|
- Streaming replication async, RPO~0, RTO<5min
|
||||||
|
- Aștept workstation AI nou + disk dedicat
|
||||||
|
|
||||||
|
### pgBackRest setup
|
||||||
|
- 2 repos: NAS + B2
|
||||||
|
- PITR window: **30 zile**
|
||||||
|
- Continuous WAL archive
|
||||||
|
- Compress zstd + encrypt AES-256
|
||||||
|
- Parallel restore workers
|
||||||
|
|
||||||
|
### DR drill
|
||||||
|
- **Trimestrial** manual full recovery simulation
|
||||||
|
- **Săptămânal** smoke test (read-only integrity check)
|
||||||
|
- **Lunar** test restore automation (ephemeral container, 30 min, 20GB temp)
|
||||||
|
|
||||||
|
### Backup Dashboard
|
||||||
|
Modul în admin.eterra.live, tab "Backup & Recovery":
|
||||||
|
- Cards pentru fiecare tier (NAS, B2, Azure placeholder, CJ2 placeholder)
|
||||||
|
- Plugin architecture `BackupTarget` interface — extensibil
|
||||||
|
- Slot-uri viitoare: email archival, Gitea repos, workstations, configs
|
||||||
|
- Alerting Golden low-noise (email doar la red)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 13. Monitoring (Dashboard admin eterra.live)
|
||||||
|
|
||||||
|
### 6 cards simple (RO tooltip clar)
|
||||||
|
1. Hărți (req/s)
|
||||||
|
2. Baza de date (query latency)
|
||||||
|
3. Spațiu disc (%)
|
||||||
|
4. Topografi activi
|
||||||
|
5. Sincronizare (ultim)
|
||||||
|
6. Backup (ultim)
|
||||||
|
|
||||||
|
### Culori: verde/galben/roșu
|
||||||
|
### Alert doar la roșu (zero spam)
|
||||||
|
### Grafana full = collapsed "Detalii avansate"
|
||||||
|
|
||||||
|
### Prometheus metrics scraped
|
||||||
|
- Martin (/metrics), Postgres (pg_stat_statements + exporter), pgBouncer, MinIO, gis-api, orchestrator
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 14. Securitate — posture final
|
||||||
|
|
||||||
|
### Authentication
|
||||||
|
- Authentik OIDC pentru toate apps
|
||||||
|
- JWT 1h access + 30d refresh
|
||||||
|
- JWKS cache 1h
|
||||||
|
|
||||||
|
### Authorization
|
||||||
|
- RLS multi-tenant (siruta + tenant_id + is_beletage_group)
|
||||||
|
- Session variables via SET LOCAL (zero leak)
|
||||||
|
- Zero Postgres role cu BYPASSRLS permanent
|
||||||
|
|
||||||
|
### Secrets management
|
||||||
|
- Infisical single source of truth
|
||||||
|
- Rotate schedule: annual pentru keys critice, automatic pentru JWT
|
||||||
|
- Encryption keys: Infisical + paper print sealed Beletage safe (YubiKey viitor)
|
||||||
|
|
||||||
|
### Network
|
||||||
|
- PG 5433 bind 127.0.0.1 (intern shop)
|
||||||
|
- pgBouncer 6432 bind 127.0.0.1
|
||||||
|
- Martin/TiTiler/gis-api prin Traefik only
|
||||||
|
- Sophos DNAT doar 80/443 către proxy
|
||||||
|
|
||||||
|
### Audit
|
||||||
|
- `gis_meta.audit` ENCRYPTED cu pgcrypto (key AUDIT_ENCRYPTION_KEY)
|
||||||
|
- Retention: 1 an hot PG + 5 ani archive MinIO
|
||||||
|
- Decryption doar la query admin explicit
|
||||||
|
|
||||||
|
### Data at rest
|
||||||
|
- Backups encrypted (pgBackRest + client-side AES pentru B2)
|
||||||
|
- ENCRYPTION_SECRET rotate pending (re-encrypt 15 vault entries)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 15. Ce DISPARE post-cutover
|
||||||
|
|
||||||
|
- `setInterval` cron-in-cod din ArchiTools
|
||||||
|
- `eterra-client.ts` din ArchiTools (mut în orchestrator)
|
||||||
|
- In-memory session pool eterra-live (persistent DB)
|
||||||
|
- PG 5432 exposed 0.0.0.0 satra (shop bind LAN)
|
||||||
|
- PMTiles webhook broken (înlocuit de pmtiles-rebuild-worker)
|
||||||
|
- systemd pmtiles-webhook parole în args (container nou)
|
||||||
|
- Hardcoded ETERRA_USERNAME/PASSWORD ArchiTools (shared pool)
|
||||||
|
- AES IV=16 eterra-live (upgrade 12)
|
||||||
|
- Rate limit in-memory eterra-live (Redis)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 16. Migration timeline (6 săptămâni)
|
||||||
|
|
||||||
|
### Sprint 1 — Foundation stack (săpt 1)
|
||||||
|
**Day 1** (GO trigger):
|
||||||
|
- Deploy `martin-public` container pe shop
|
||||||
|
- Deploy `martin-private` container + Traefik ForwardAuth middleware
|
||||||
|
- Deploy `titiler` container
|
||||||
|
- Deploy `nginx-pmtiles` container
|
||||||
|
- Deploy `minio-gis` container + bucket setup
|
||||||
|
|
||||||
|
**Day 2:**
|
||||||
|
- Deploy `pgadmin` (intern pgadmin.beletage.ro)
|
||||||
|
- Traefik routes pentru toate subdomeniile GIS
|
||||||
|
|
||||||
|
**Day 3-4:**
|
||||||
|
- Cloudflare zones: add `gis.ac`, `eterra.live`, `pug.digital`, `planhub.ro`
|
||||||
|
- DNS records + Page Rules TTL + Rate Limits + Bot Fight Medium
|
||||||
|
- Test tile serving basic
|
||||||
|
|
||||||
|
**Day 5:**
|
||||||
|
- Schema split pe shop:
|
||||||
|
- `CREATE SCHEMA gis_core, gis_urban, gis_enrichment, gis_meta, tenants, eterra, pug, archi, planhub, queue`
|
||||||
|
- `ALTER TABLE public.X SET SCHEMA gis_core` pentru tabele existente
|
||||||
|
- Create views + roles
|
||||||
|
|
||||||
|
### Sprint 2 — Auth + data layer (săpt 2)
|
||||||
|
**Day 1-2:**
|
||||||
|
- Authentik custom property mappings (tenant_id, is_beletage_group, enrichment_scope, allowed_sirutas)
|
||||||
|
- Webhook signed HMAC din eterra.live → Authentik sync
|
||||||
|
- PG roluri aplicație + grants
|
||||||
|
|
||||||
|
**Day 3-4:**
|
||||||
|
- RLS policies per tabel (gis_urban, gis_enrichment)
|
||||||
|
- FORCE ROW LEVEL SECURITY pe toate
|
||||||
|
- gis-api boilerplate (Next.js 16 + Hono + Prisma + Zod)
|
||||||
|
- Auth middleware JWT verify + SET LOCAL
|
||||||
|
|
||||||
|
**Day 5:**
|
||||||
|
- Cypress test suite RLS (6 scenarii obligatorii)
|
||||||
|
|
||||||
|
### Sprint 3 — Sync orchestrator (săpt 3)
|
||||||
|
**Day 1-2:**
|
||||||
|
- `gis-sync-orchestrator` container
|
||||||
|
- pg-boss setup schema queue
|
||||||
|
- eTerra client extract în shared lib
|
||||||
|
|
||||||
|
**Day 3:**
|
||||||
|
- Session pool persistent `gis_meta.eterra_sessions` + multi-account `gis_meta.eterra_accounts`
|
||||||
|
- Workers: eterra-sync, enrichment, pug-ingest
|
||||||
|
|
||||||
|
**Day 4:**
|
||||||
|
- Workers: pmtiles-rebuild, cache-invalidate (CF API)
|
||||||
|
- LISTEN/NOTIFY events implementation
|
||||||
|
|
||||||
|
**Day 5:**
|
||||||
|
- Admin UI module eterra.live — sync jobs control panel
|
||||||
|
|
||||||
|
### Sprint 4 — Backup + Dashboard (săpt 4)
|
||||||
|
**Day 1-2:**
|
||||||
|
- pgBackRest deploy shop
|
||||||
|
- 2 repos: NAS (rsync push via cron) + B2 (direct S3)
|
||||||
|
- Scheduled jobs: daily incremental, weekly full, monthly archive
|
||||||
|
- WAL archive continuous
|
||||||
|
|
||||||
|
**Day 3:**
|
||||||
|
- Backup Dashboard UI (admin eterra.live tab "Backup & Recovery")
|
||||||
|
- Plugin architecture `BackupTarget` interface
|
||||||
|
- Cards: NAS, B2, Azure placeholder, CJ2 placeholder
|
||||||
|
|
||||||
|
**Day 4:**
|
||||||
|
- DR runbook scripts
|
||||||
|
- Monthly restore test automation (ephemeral container cron)
|
||||||
|
- Prometheus + Grafana + Alertmanager deploy
|
||||||
|
|
||||||
|
**Day 5:**
|
||||||
|
- Grafana dashboards (GIS Overview, Martin Perf, PG Health, API Rate Limits, Raster Library)
|
||||||
|
- 6 simple cards admin eterra.live
|
||||||
|
- Alertmanager → n8n webhook (email + Telegram on red)
|
||||||
|
|
||||||
|
### Sprint 5 — Test & parity (săpt 5)
|
||||||
|
**Day 1-2:**
|
||||||
|
- Shadow sync: orchestrator rulează în paralel cu satra setInterval (both read from ANCPI)
|
||||||
|
- Compare row counts ambele, checksum-uri, latency
|
||||||
|
|
||||||
|
**Day 3:**
|
||||||
|
- Load test: 1000 concurrent API requests, 500 QPS tile server
|
||||||
|
- Performance comparisons satra vs shop end-to-end
|
||||||
|
|
||||||
|
**Day 4:**
|
||||||
|
- Security audit final: TLS scan, dependency audit, pentest basic
|
||||||
|
- CORS, CSP, rate limits verify
|
||||||
|
|
||||||
|
**Day 5:**
|
||||||
|
- GO/NO-GO decision meeting
|
||||||
|
- Final checklist review
|
||||||
|
|
||||||
|
### Weekend cutover (if GO)
|
||||||
|
**Friday 22:00:**
|
||||||
|
- Notify users (eterra.live + ArchiTools) maintenance 4h
|
||||||
|
- Traefik maintenance page
|
||||||
|
- Stop ArchiTools + eterra-live containers
|
||||||
|
- pg_dump satra final snapshot → shop (delta apply)
|
||||||
|
- Diff check paritate
|
||||||
|
- Switch DATABASE_URL containere → shop
|
||||||
|
- Deploy ArchiTools nou (cu eterra-client DELETED)
|
||||||
|
- Cloudflare DNS update: tiles.gis.ac → shop Martin
|
||||||
|
- Start orchestrator cron jobs
|
||||||
|
- Smoke test full flow
|
||||||
|
|
||||||
|
**Saturday 06:00:**
|
||||||
|
- Monitor dashboard
|
||||||
|
- On-call standby
|
||||||
|
|
||||||
|
**Saturday 20:00:**
|
||||||
|
- All green → GO production
|
||||||
|
- satra architools_postgres READ-ONLY (fallback 30 zile safety)
|
||||||
|
|
||||||
|
**Sunday:**
|
||||||
|
- Observation + bug bash
|
||||||
|
|
||||||
|
### Cleanup (săpt 6)
|
||||||
|
- git filter-repo ArchiTools (rescrie istoria, notify re-clone)
|
||||||
|
- Structural fix compose (env_file + stack.env in .gitignore)
|
||||||
|
- Remove satra containers (după 30 zile)
|
||||||
|
- ENCRYPTION_SECRET rotate + re-encrypt 15 vault entries
|
||||||
|
- Documentation final în busc-infra
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 17. Runbooks (DR scenarios)
|
||||||
|
|
||||||
|
### R1 — Corruption parcela X la 14:30
|
||||||
|
```bash
|
||||||
|
pgbackrest --stanza=gis --type=time --target="2026-04-20 14:29:00" restore
|
||||||
|
# Verify integrity + promote
|
||||||
|
```
|
||||||
|
|
||||||
|
### R2 — Crash shop total → failover CJ2 (când activ)
|
||||||
|
1. Promote CJ2 standby to primary
|
||||||
|
2. Update DNS: tiles.gis.ac, api.gis.ac → CJ2 IP
|
||||||
|
3. Restart apps pointing CJ2
|
||||||
|
4. RTO target: <5 min
|
||||||
|
|
||||||
|
### R3 — Ransomware
|
||||||
|
- B2 Object Lock Compliance = fișiere immutable
|
||||||
|
- Restore latest clean snapshot pre-ransomware
|
||||||
|
- Wipe compromised + rebuild from B2
|
||||||
|
|
||||||
|
### R4 — Accidental DROP TABLE
|
||||||
|
```bash
|
||||||
|
pgbackrest restore --type=time --target="<moment înainte de DROP>"
|
||||||
|
# Replay WAL până exact înainte de comanda distructivă
|
||||||
|
```
|
||||||
|
|
||||||
|
### R5 — Disaster regional (shop + CJ2 down)
|
||||||
|
- Restore din B2 arhive lunar
|
||||||
|
- Bootstrap new cluster pe alt host
|
||||||
|
- RTO: 4-8 ore
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 18. Next features post-production
|
||||||
|
|
||||||
|
### Imediat după stable (month 1-2)
|
||||||
|
- MDLPA validator containerizat (după repo de la friends MDLPA)
|
||||||
|
- Geopackage export pipeline complet
|
||||||
|
- Raster Library module + upload UI
|
||||||
|
|
||||||
|
### Medium term (month 3-6)
|
||||||
|
- PMUD workflow (rețele transport, piste, parcare)
|
||||||
|
- SmartCity360 / pug.digital launch (primul tenant primărie)
|
||||||
|
- planhub.ro SaaS pivot multi-tenant (primii arhitecți externi)
|
||||||
|
- CJ2 standby activation (după workstation AI nou)
|
||||||
|
|
||||||
|
### Long term (month 6-12)
|
||||||
|
- PATJ suport (1 județ pilot)
|
||||||
|
- Azure activation (Azure PostgreSQL DR + Blob Hot CDN)
|
||||||
|
- Cross-region geo-redundanță dacă scale justifies
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 19. Risk register
|
||||||
|
|
||||||
|
| Risc | Probabilitate | Impact | Mitigare |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Cutover fail | Low | High | Rollback DNS → satra, 30d safety window |
|
||||||
|
| Data loss cutover window | Very Low | High | Shadow sync săpt 5, delta final |
|
||||||
|
| Orchestrator bugs prod | Medium | Medium | Săpt 5 shadow prinde majoritatea |
|
||||||
|
| ANCPI block multi-account | Low | High | Shuffle + quota + alerting |
|
||||||
|
| B2 cloud outage | Very Low | Medium | NAS + CJ2 (când activ) alternative |
|
||||||
|
| Marius unavailable | Medium | High | Claude Code has full plan + memory |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 20. Secrete pre-loaded în Infisical (verificate 2026-04-20)
|
||||||
|
|
||||||
|
### Active (ArchiTools production)
|
||||||
|
- AUTHENTIK_CLIENT_ID, AUTHENTIK_CLIENT_SECRET
|
||||||
|
- NEXTAUTH_SECRET
|
||||||
|
- DB_PASS (satra Postgres user, will rotate at cutover)
|
||||||
|
- ETERRA_USERNAME, ETERRA_PASSWORD
|
||||||
|
- NOTIFICATION_CRON_SECRET
|
||||||
|
- BREVO_BELETAGE_API_KEY
|
||||||
|
|
||||||
|
### Shop stack (pre-loaded pentru sprint 1)
|
||||||
|
- POSTGRES_GIS_SUPERUSER_PASSWORD
|
||||||
|
- POSTGRES_GIS_ARCHITOOLS_PASS
|
||||||
|
- POSTGRES_GIS_ETERRA_PASS
|
||||||
|
- POSTGRES_GIS_MARTIN_PASS
|
||||||
|
- POSTGRES_GIS_SYNC_PASS
|
||||||
|
- POSTGRES_GIS_PUG_PASS
|
||||||
|
- POSTGRES_GIS_PLANHUB_PASS
|
||||||
|
- MINIO_GIS_ROOT_USER
|
||||||
|
- MINIO_GIS_ROOT_PASSWORD
|
||||||
|
- PGADMIN_DEFAULT_PASSWORD
|
||||||
|
|
||||||
|
### Security/Crypto
|
||||||
|
- GIS_SYNC_AES_KEY
|
||||||
|
- AUTHENTIK_WEBHOOK_HMAC_SECRET
|
||||||
|
- AUDIT_ENCRYPTION_KEY
|
||||||
|
- BACKUP_DASHBOARD_ADMIN_TOKEN
|
||||||
|
|
||||||
|
### Backup
|
||||||
|
- BACKUP_ARCHIVE_KEY (AES-256 B2 client-side)
|
||||||
|
- PGBACKREST_REPO1_CIPHER_PASS
|
||||||
|
- STANDBY_REPLICATION_PASS (pentru CJ2 viitor)
|
||||||
|
- B2_APPLICATION_KEY_ID
|
||||||
|
- B2_APPLICATION_KEY
|
||||||
|
- B2_BUCKET_NAME (beletage-gis-backups)
|
||||||
|
- B2_ENDPOINT (s3.eu-central-003.backblazeb2.com)
|
||||||
|
|
||||||
|
### De generat la moment (viitor)
|
||||||
|
- POSTGRES_GIS_TITILER_PASS (day 1 sprint 1)
|
||||||
|
- POSTGRES_GIS_APP_RW_PASS (day 1 sprint 2)
|
||||||
|
- GRAFANA_ADMIN_PASSWORD (sprint 4)
|
||||||
|
- PROMETHEUS_BASIC_AUTH (sprint 4)
|
||||||
|
- AZURE_STORAGE_ACCOUNT, AZURE_STORAGE_SAS (când activăm)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 21. Hardware/infrastructure check-list pre-GO
|
||||||
|
|
||||||
|
- ✅ Shop 10.10.10.84 reachable SSH (user dnz, groups docker+wheel)
|
||||||
|
- ✅ Shop docker 29.2.1 instalat
|
||||||
|
- ✅ Shop disk 3.3TB free
|
||||||
|
- ✅ Shop RAM 225GB avail
|
||||||
|
- ✅ Shop ports 5433, 6432 libere (POC ocupate)
|
||||||
|
- ✅ Satra SSH (user bulibasa) reachable
|
||||||
|
- ✅ Proxy Traefik v3.6.8 (10.10.10.199)
|
||||||
|
- ✅ Sophos 80/443 only WAN (LAN internal safe)
|
||||||
|
- ✅ NAS 10.10.10.10 rsync key deployed, symlink OK
|
||||||
|
- ✅ Backblaze B2 bucket + scoped key tested
|
||||||
|
- ✅ Authentik auth.beletage.ro v2025.2.4 operational
|
||||||
|
- ✅ Infisical toate secretele pre-loaded
|
||||||
|
- ✅ Cloudflare API token disponibil
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 22. Acceptance criteria (Definition of Done)
|
||||||
|
|
||||||
|
Production considerat stable când:
|
||||||
|
- [ ] Toate 13 containere shop running + healthy
|
||||||
|
- [ ] Cypress RLS suite 100% pass
|
||||||
|
- [ ] Load test 500 QPS tiles + 100 QPS api sub 200ms p95
|
||||||
|
- [ ] Backup test restore successful (monthly)
|
||||||
|
- [ ] ArchiTools + eterra.live live pe shop DATABASE_URL
|
||||||
|
- [ ] Orchestrator UNICUL scriitor gis_core
|
||||||
|
- [ ] Zero error logs 72h continuous
|
||||||
|
- [ ] Dashboard admin eterra.live arată toate verde
|
||||||
|
- [ ] DR runbook validat (scenario R1 + R4 testat)
|
||||||
|
- [ ] Grafana dashboards populated
|
||||||
|
- [ ] Users notificați + feedback pozitiv
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 23. Contacts & ownership
|
||||||
|
|
||||||
|
- **Owner:** Marius Tarau (m.tarau@beletage.ro)
|
||||||
|
- **Pair:** Claude Code (via Anthropic Opus 4.7 1M context)
|
||||||
|
- **Escalation:** m.tarau@beletage.ro + (telefon Beletage)
|
||||||
|
- **On-call during cutover weekend:** Marius primary, Claude assist
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 24. Changelog
|
||||||
|
|
||||||
|
- **v1 DRAFT** (2026-04-18) — initial propunere
|
||||||
|
- **v2 FINAL** (2026-04-20) — approved, includes all decisions from Pas 1-7, POC results, security rotation status, multi-account ANCPI shuffle, super-tenant Beletage group, Azure credit alloc, NAS + B2 tested, Backup Dashboard plugin architecture
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**END OF PLAN.**
|
||||||
|
Next action: when Marius sends "go" → Sprint 1 Day 1 begins.
|
||||||
@@ -0,0 +1,178 @@
|
|||||||
|
# Plan 002 — ArchiTools thin-client migration: review feedback from eterra.live side
|
||||||
|
|
||||||
|
**Date:** 2026-05-17
|
||||||
|
**Reviewer:** Claude session in `/home/orchestrator/Code/eterra-live` (Marius)
|
||||||
|
**Context:** Reviews the "Faza 0..7" plan proposed by the architools claude that routes architools enrichment through eterra.live.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## TL;DR — change of direction
|
||||||
|
|
||||||
|
The original plan made **eterra.live** the gateway for architools/planhub enrichment via `POST /api/internal/enrich`. **This is the wrong boundary.** The correct gateway is **`api.gis.ac`** (the `gis-api` service on shop), which is already designed for cross-app consumption (JWT + RLS + scope filtering, Hono on `10.10.10.84:3100`, fronted by Traefik).
|
||||||
|
|
||||||
|
eterra.live is a single-instance Next.js end-user product. Putting architools' geoportal behind it makes eterra.live a SPOF — any deploy/restart breaks architools cadastre. The right model treats both architools AND eterra.live as **peer consumers** of api.gis.ac.
|
||||||
|
|
||||||
|
eterra.live already migrated to api.gis.ac for `CfExtract` (Sprint 2 Day 3-4, 2026-04-20). Same migration pattern applies to architools, just bigger surface area.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What's already in place (don't reinvent)
|
||||||
|
|
||||||
|
`api.gis.ac` exists in production:
|
||||||
|
|
||||||
|
- Repo: `gitadmin/gis-api`, local clone `~/Code/gis-api`
|
||||||
|
- Stack: Next.js 16.1 + Hono 4 + Prisma 6.19 + Zod 4 + jose 6
|
||||||
|
- Auth: HS256 JWT today (`GIS_API_DEV_JWT_SECRET`) — **migrating to Authentik OIDC before architools cutover** (decision locked 2026-05-17)
|
||||||
|
- Claims: `{ sub, org_ids[], is_beletage_group, enrichment_scope: "none"|"basic"|"full", email? }` — `tenant` claim being added
|
||||||
|
- RLS: `withUserContext()` sets `app.user_id / org_ids / is_beletage_group / enrichment_scope` per Postgres tx
|
||||||
|
- Live endpoints (under `/api/v1/`):
|
||||||
|
- `GET /me` — claims echo
|
||||||
|
- `GET /parcela/:id` — full `GisFeature` read
|
||||||
|
- `GET /search?q=&limit=` — UAT + cadastralRef text search
|
||||||
|
- `GET/POST/PATCH /enrichment/cf*` — CfExtract CRUD with RLS
|
||||||
|
- `GET /enrichment/catalog/:nrCadastral` — catalog metadata
|
||||||
|
- `GET/POST /enrichment/cf/:id/pdf` — MinIO PDF stream
|
||||||
|
- RLS test suite: 6 scenarios, asserts BOTH HTTP and direct-SQL paths (CI: `.gitea/workflows/rls-test.yml`)
|
||||||
|
|
||||||
|
What it does NOT have yet (gaps to close before architools cutover):
|
||||||
|
|
||||||
|
1. **Authentik OIDC** (HS256 still in use)
|
||||||
|
2. **Live-fetch proxy endpoints** to orchestrator (parcel/tech, parcel/units, building/tech, building/condo-owners, imm-apps)
|
||||||
|
3. **Scope-based field filtering** on `/parcela/:id` (currently leaks `enrichment.PROPRIETARI` regardless of scope)
|
||||||
|
4. **`tenant` claim** in JWT
|
||||||
|
5. **CORS allowlist** for cross-origin app calls
|
||||||
|
6. **Per-tenant rate limiting**
|
||||||
|
7. **Audit log table** (`gis_meta.api_audit`)
|
||||||
|
|
||||||
|
These will be added in a dedicated gis-api session before architools starts consuming. Full spec is in gis-api project memory (`~/.claude/projects/-home-orchestrator-Code-gis-api/memory/`):
|
||||||
|
- `project_architools_planhub_plan.md` — endpoints to add, in priority order
|
||||||
|
- `feedback_jwt_authentik_decision.md` — Authentik OIDC spec (scope mapping from LDAP groups)
|
||||||
|
- `reference_orchestrator_endpoints.md` — exact contracts for the 5 LAN endpoints to proxy
|
||||||
|
- `reference_cross_app_architecture.md` — the diagram + consumer responsibility matrix
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Revised plan (replaces architools claude's Faza 0..7)
|
||||||
|
|
||||||
|
### Faza A — Wait for gis-api gaps to close (managed in gis-api repo, ~3 days)
|
||||||
|
|
||||||
|
Driven from `~/Code/gis-api` in a separate session. NOT architools' work to do, but architools depends on it. Deliverables:
|
||||||
|
|
||||||
|
1. **PR1 (gis-api):** Authentik OIDC support alongside HS256 (`src/lib/auth.ts` issuer-based dual path). Add `tenant` claim. Eterra.live keeps working on HS256 during the overlap. Provisioning needed in Authentik:
|
||||||
|
- New OAuth2 application + provider with slug `gis-api`
|
||||||
|
- Property mappings for `enrichment_scope` derived from LDAP groups:
|
||||||
|
- `beletage-staff` → `enrichment_scope=full`, `is_beletage_group=true`
|
||||||
|
- `planhub-pro` → `full`
|
||||||
|
- `planhub-free` / no match → `basic`
|
||||||
|
- missing claim → REJECT
|
||||||
|
- `tenant` claim from `aud` (application slug)
|
||||||
|
2. **PR2 (gis-api):** 5 proxy endpoints + scope filter on `/parcela/:id` + CORS allowlist:
|
||||||
|
```
|
||||||
|
POST /api/v1/parcel/tech → orchestrator POST /api/v1/parcel/tech
|
||||||
|
POST /api/v1/parcel/units/fetch → orchestrator POST /api/v1/parcel-units/fetch
|
||||||
|
POST /api/v1/parcel/imm-apps → orchestrator POST /api/v1/imm-apps
|
||||||
|
POST /api/v1/building/tech → orchestrator POST /api/v1/building/tech
|
||||||
|
POST /api/v1/building/condo-owners → orchestrator POST /api/v1/building/condo-owners
|
||||||
|
```
|
||||||
|
All require `enrichment_scope >= basic`. Rewrite `correlationId` server-side to `${tenant}:${user_sub[:8]}:${requestId}`.
|
||||||
|
3. **PR3 (gis-api):** Per-tenant rate limit + `gis_meta.api_audit` migration.
|
||||||
|
|
||||||
|
When PR1+PR2 are green and deployed, signal back to this plan and proceed with Faza B.
|
||||||
|
|
||||||
|
### Faza B — ArchiTools secrets + Authentik client (½ day)
|
||||||
|
|
||||||
|
- In Authentik: create OAuth2 client for architools (slug `architools-app`). Configure redirect URI for its NextAuth flow.
|
||||||
|
- Infisical adds (prod env, path `/`):
|
||||||
|
- `AUTHENTIK_CLIENT_ID` / `AUTHENTIK_CLIENT_SECRET` / `AUTHENTIK_ISSUER` / `AUTHENTIK_JWKS_URL` for architools
|
||||||
|
- `GIS_API_URL=https://api.gis.ac`
|
||||||
|
- `NEXT_PUBLIC_MARTIN_URL=https://tiles.gis.ac`
|
||||||
|
- `NEXT_PUBLIC_PMTILES_URL=https://pmtiles.gis.ac/overview.pmtiles`
|
||||||
|
- NextAuth on architools: add Authentik OIDC provider. Store `access_token` in session. ⚠️ DO NOT mint HS256 JWTs server-side anymore — pass the Authentik `access_token` directly as Bearer to gis-api.
|
||||||
|
|
||||||
|
### Faza C — Rip-out (1 day, ONE PR, feature-flagged)
|
||||||
|
|
||||||
|
`NEXT_PUBLIC_USE_GIS_AC=1` flag controls cutover. With flag on:
|
||||||
|
|
||||||
|
- Geoportal map sources flip to `pmtiles.gis.ac` + `tiles.gis.ac`
|
||||||
|
- All `src/app/api/eterra/*` and `src/app/api/ancpi/*` route handlers replaced by thin wrappers that forward to api.gis.ac with the user's access_token
|
||||||
|
- All `src/modules/parcel-sync/**` and parcel-sync UI removed
|
||||||
|
- `src/config/{modules,navigation,flags}.ts` cleanup
|
||||||
|
|
||||||
|
With flag off: old behavior retained. **Do not drop Prisma tables (GisFeature, GisUat, GisSyncRun, GisSyncRule, CfExtract) in this PR.** Keep them as dead columns for 2-3 days post-cutover so rollback stays cheap. Drop in a follow-up PR after validation in prod.
|
||||||
|
|
||||||
|
Stop `martin` container on satra ONLY after flag-on is verified working in prod. The PMTiles webhook on satra:9876 — **does not exist**; the architools claude's plan had this wrong. The actual PMTiles trigger is the orchestrator cron at 03:00 EEST + admin button in eterra.live, hitting `gis-tippecanoe-builder` on **shop** at port 9876.
|
||||||
|
|
||||||
|
### Faza D — Thin client (1-2 days)
|
||||||
|
|
||||||
|
- `src/lib/gis-api-client.ts` — fetch wrapper with the Authentik access_token from session
|
||||||
|
- `gisApi.parcela.get(id)` → `GET /api/v1/parcela/:id`
|
||||||
|
- `gisApi.search(q, limit)` → `GET /api/v1/search`
|
||||||
|
- `gisApi.parcel.tech({siruta, cadastralRef, force})` → `POST /api/v1/parcel/tech`
|
||||||
|
- `gisApi.parcel.unitsFetch(...)` → `POST /api/v1/parcel/units/fetch`
|
||||||
|
- `gisApi.parcel.immApps(...)` → `POST /api/v1/parcel/imm-apps`
|
||||||
|
- `gisApi.building.tech(...)` → `POST /api/v1/building/tech`
|
||||||
|
- `gisApi.building.condoOwners(...)` → `POST /api/v1/building/condo-owners`
|
||||||
|
- `gisApi.enrichment.cf.list/get/create/patch/uploadPdf/getPdf/getCatalog` (mirror eterra.live's existing usage)
|
||||||
|
- NO separate `scope-mapper.ts` — gis-api resolves scope from Authentik claims itself.
|
||||||
|
- NO separate `gis-api-token.ts` — Authentik access_token is the JWT, NextAuth session already has it.
|
||||||
|
|
||||||
|
### Faza E — Geoportal rewrite (2 days)
|
||||||
|
|
||||||
|
- `map-viewer.tsx`: sources `pmtiles://pmtiles.gis.ac/overview.pmtiles` + Martin `tiles.gis.ac/{view}/{z}/{x}/{y}`. Drop satra Martin entirely.
|
||||||
|
- `search-bar.tsx` → `gisApi.search()`.
|
||||||
|
- `feature-info-panel.tsx` → `gisApi.parcela.get()` + (on missing enrichment) `gisApi.parcel.tech()` + `gisApi.parcel.unitsFetch()` for buildings.
|
||||||
|
- `basemap-switcher.tsx`: same config as eterra.live (liberty/dark/satellite/orto/topo50). Source files are public CSS/tile URLs; literal copy is OK.
|
||||||
|
- Decision for `boundary-check`, `cf-status`, `export`, `pad`, `piz`: if architools keeps them, rewrite as thin proxies to api.gis.ac equivalents (which may need new endpoints — defer that scoping until each is actually needed).
|
||||||
|
|
||||||
|
### Faza F — ePay / CF ordering (½ day)
|
||||||
|
|
||||||
|
- Architools UI for "Comandă extras CF" calls `gisApi.enrichment.cf.create({nrCadastral, type, ...})` directly.
|
||||||
|
- Status + download list: `gisApi.enrichment.cf.list({...})` and `getPdf(id)`.
|
||||||
|
- The `tenant=architools` claim in the access_token tells gis-api which billing account to attribute the order to (future-work for billing routing, but the field is in place from Faza A).
|
||||||
|
- Delete `src/app/api/ancpi/*` entirely.
|
||||||
|
|
||||||
|
### Faza G — Test E2E + deploy (½ day)
|
||||||
|
|
||||||
|
- `/geoportal` in architools renders tiles from gis.ac, click parcel → enrichment via api.gis.ac → live-fetch fallback triggers orchestrator.
|
||||||
|
- Verify zero references to `architools_postgres.GisFeature` remain in code AND that no DB query goes to the deprecated tables.
|
||||||
|
- Type-check + lint + build clean.
|
||||||
|
- Commit + push + Portainer redeploy.
|
||||||
|
- 24h grace period: old stack (Martin satra, deprecated parcel-sync code) kept around. After validation, follow-up PR drops Prisma tables + stops martin satra container.
|
||||||
|
|
||||||
|
### Faza H — Planhub (separate PR, post-architools)
|
||||||
|
|
||||||
|
- Same pattern. Different scope mapping: `planhub-free` → `basic`, `planhub-pro` → `full`.
|
||||||
|
- Already covered by gis-api's tenant claim from Faza A.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Specific corrections to the original plan
|
||||||
|
|
||||||
|
1. **`POST /api/internal/enrich` on eterra.live** — REMOVE. Use `api.gis.ac` directly.
|
||||||
|
2. **Bearer `$ARCHITOOLS_INTERNAL_TOKEN` static token** — REMOVE. Use Authentik OIDC access_token.
|
||||||
|
3. **`GIS_API_JWT_SECRET (shared)` in Infisical** — REMOVE. With Authentik OIDC, gis-api fetches JWKS from `https://auth.beletage.ro/application/o/gis-api/jwks/` — no shared secrets between apps.
|
||||||
|
4. **`scope-mapper.ts` in architools** — REMOVE. Scope is in the JWT claims; architools doesn't compute it.
|
||||||
|
5. **PMTiles webhook satra:9876** — does not exist. The builder is `gis-tippecanoe-builder` on **shop:9876**, fired by orchestrator cron + eterra.live admin button.
|
||||||
|
6. **Single-PR rip-out** — replace with feature-flagged migration (`NEXT_PUBLIC_USE_GIS_AC=1`), 2-3 day overlap, drop dead code in a separate follow-up PR after prod validation.
|
||||||
|
7. **eterralive-coordinator** terminology — drop. The pool is "owned" by `gis-sync-orchestrator` (LAN-only on shop:3101). gis-api is the auth/scope/audit gateway in front of it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
1. **Authentik OIDC client provisioning** — needs to be done first; until then, no architools migration. Marius or a sys-admin session needs to set up the OAuth2 provider in Authentik UI.
|
||||||
|
2. **Audit table schema** — does `gis_meta` already have an `api_audit` table? Probably no. Migration needed in same PR as proxy endpoints.
|
||||||
|
3. **Rate-limit defaults** — what's the cap per tenant? 100 req/min seems safe; configurable per tenant via Infisical JSON env. Confirm with Marius.
|
||||||
|
4. **Architools' `archi.*` schema writes** — currently architools writes Canvas, Job, RegistryAudit, RgiSearchTemplate directly via its own Prisma. Migration here is OPTIONAL for the cadastre work; could be done in a later sprint. Don't bundle it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Reference docs
|
||||||
|
|
||||||
|
- gis-api project memory: `~/.claude/projects/-home-orchestrator-Code-gis-api/memory/MEMORY.md`
|
||||||
|
- Plan 001 v2 master: `~/Code/ArchiTools/docs/plans/001-v2-central-gis-db-multi-app.md`
|
||||||
|
- Tile data sources / cache / gis.ac rule: `~/Code/busc-infra/references/TILE-DATA-SOURCES.md`
|
||||||
|
- Orchestrator workers + schema: `~/Code/busc-infra/references/SPRINT3.md`
|
||||||
|
- GIS stack infrastructure: `~/Code/busc-infra/references/GIS-STACK.md`
|
||||||
|
- Authentik configuration: `~/Code/busc-infra/references/AUTHENTIK.md`
|
||||||
|
- Sprint 2 (gis-api delivery): `~/Code/busc-infra/references/SPRINT2.md`
|
||||||
@@ -0,0 +1,257 @@
|
|||||||
|
# Plan 003 — ArchiTools cutover execution (post-PR1+PR2)
|
||||||
|
|
||||||
|
**Date:** 2026-05-17
|
||||||
|
**Author:** gis-api session (Claude, `/home/orchestrator/Code/gis-api`)
|
||||||
|
**Supersedes Faza A–H of:** `002-architools-thin-client-review.md`
|
||||||
|
**Starting brief for:** next ArchiTools session
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## TL;DR — what changed since 002 was written
|
||||||
|
|
||||||
|
PR1 (Authentik OIDC dual-support, `ad1825a` deployed 22:25) and PR2 (5 proxy endpoints + scope filter + CORS + rate limit + audit + RLS smoke test, `fc459e0` deployed 22:55) are both live on `api.gis.ac`. Migration 008-api-audit.sql applied to gis DB. RLS leak from 2026-05-17 patched; `relrowsecurity` enforced via CI smoke test.
|
||||||
|
|
||||||
|
**Skip Faza A entirely** — its full content is done.
|
||||||
|
|
||||||
|
**One pre-cutover gis-api blocker** discovered (PR2.1 below) — must ship before architools' first request hits the rate limiter or audit log starts collecting unreadable 40-char tenant strings.
|
||||||
|
|
||||||
|
Everything else in 002 (Faza B–G) is executable against the deployed api.gis.ac. Faza H (Planhub) deferred — separate PR after architools is stable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ PR2.1 — Tenant-slug translation (gis-api blocker)
|
||||||
|
|
||||||
|
**Status:** NOT YET SHIPPED. ~30 min of work in gis-api repo.
|
||||||
|
|
||||||
|
**Problem:** PR2 lifts `aud` → `claims.tenant` verbatim. For Authentik authorization_code AND client_credentials flows, `aud` is the OAuth client_id (40-char opaque string like `V59GMiYle87yd9VZOgUmdSmzYQALqNsKVAUR6QMi`), not the friendly slug `architools`. Two real consequences:
|
||||||
|
|
||||||
|
1. **Rate limiter mis-keyed:** `GIS_API_TENANT_RATE_LIMITS` env JSON uses friendly slugs (`{"architools":500,"planhub":120,...}`). Bucket lookup on `claims.tenant` = the 40-char client_id → miss → falls to `_default` (100 rpm). architools would silently run at 5× lower throughput than configured.
|
||||||
|
2. **Audit log unreadable:** `gis_meta."ApiAudit".tenant` stores the 40-char string. Ops queries (`SELECT … WHERE tenant = 'architools'`) return zero rows. Future billing routing has the same problem.
|
||||||
|
|
||||||
|
**Fix shape (small, surgical):**
|
||||||
|
|
||||||
|
- New env: `GIS_API_TENANT_MAP` — JSON `{"<client_id>":"<slug>", ...}`. Missing entry → fall back to verbatim aud. Missing env → no translation (current behavior).
|
||||||
|
- In `src/lib/auth.ts`, after `audToTenant(payload.aud)`, look up in the map.
|
||||||
|
- Unit test that adds a case to `tests/auth/oidc.test.ts` (or new `tenant-map.test.ts`).
|
||||||
|
- `.env.example` entry. Infisical `/gis-api` adds the secret.
|
||||||
|
|
||||||
|
**Who ships this:** gis-api session (next firing or this one if Marius approves). Architools cutover is gated on this being deployed first.
|
||||||
|
|
||||||
|
**Required values:** ArchiTools client_id (confirmed in user message: `V59GMiYle87yd9VZOgUmdSmzYQALqNsKVAUR6QMi` → `architools`). Planhub + eterra-live client_ids: TBD as those apps onboard.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Faza B — ArchiTools secrets + Authentik client (½ day)
|
||||||
|
|
||||||
|
| Aspect | Detail |
|
||||||
|
|---|---|
|
||||||
|
| **Owner** | ArchiTools session + Marius (Infisical) |
|
||||||
|
| **Repo** | ArchiTools (NextAuth config) + Infisical (secrets) + Authentik (no changes — provider pk=6 already exists with `gis-api enrichment scope` mapping attached, per `reference-authentik-provisioned-state` memory) |
|
||||||
|
| **gis-api involvement** | Zero. Just confirm: architools provider pk=6 in Authentik issues tokens with `iss = https://auth.beletage.ro/application/o/architools/`, `aud = V59GMiYle87yd9VZOgUmdSmzYQALqNsKVAUR6QMi`. **NOTE:** gis-api's current OIDC path validates against `GIS_API_AUTHENTIK_ISSUER = https://auth.beletage.ro/application/o/gis-api/` — architools tokens will NOT match this issuer and will fall through to HS256 path → 401. See **"Multi-issuer JWT support"** below — this is a second gis-api blocker that PR2.1 (or PR2.2) must address. |
|
||||||
|
|
||||||
|
**Infisical adds** (`/architools` folder, prod env):
|
||||||
|
- `AUTHENTIK_CLIENT_ID` — value from Authentik provider pk=6
|
||||||
|
- `AUTHENTIK_CLIENT_SECRET` — from provider pk=6
|
||||||
|
- `AUTHENTIK_ISSUER` = `https://auth.beletage.ro/application/o/architools/`
|
||||||
|
- `AUTHENTIK_JWKS_URL` = `https://auth.beletage.ro/application/o/architools/jwks/`
|
||||||
|
- `GIS_API_URL` = `https://api.gis.ac`
|
||||||
|
- `NEXT_PUBLIC_MARTIN_URL` = `https://tiles.gis.ac` *(verify infrastructure live first)*
|
||||||
|
- `NEXT_PUBLIC_PMTILES_URL` = `https://pmtiles.gis.ac/overview.pmtiles` *(same)*
|
||||||
|
|
||||||
|
**ArchiTools NextAuth changes:**
|
||||||
|
- Add Authentik OIDC provider. Request `scope: "openid profile email enrichment"` so the `enrichment_scope` claim is in the issued access_token (per the scope mapping attached to provider pk=6).
|
||||||
|
- Session callback stores `access_token` on the session object (not the JWT — different things; Authentik issues the JWT we forward).
|
||||||
|
- **Remove** any HS256 minting code (`gis-api-token.ts` and similar — per 002 corrections).
|
||||||
|
|
||||||
|
**Verification before moving to Faza C:**
|
||||||
|
- `architools` can mint an Authentik authorization_code token end-to-end (login flow works).
|
||||||
|
- Decoded token includes `enrichment_scope`, `is_beletage_group`, `org_ids` claims.
|
||||||
|
- `curl -H "Authorization: Bearer <token>" https://api.gis.ac/api/v1/me` returns 200 with those claims. **This curl will FAIL today** until multi-issuer support ships — see PR2.2 below.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ PR2.2 — Multi-issuer JWT support (second gis-api blocker)
|
||||||
|
|
||||||
|
**Status:** NOT YET SHIPPED.
|
||||||
|
|
||||||
|
**Problem:** PR1's `src/lib/auth.ts` validates `iss === GIS_API_AUTHENTIK_ISSUER` (the gis-api provider's issuer). architools tokens carry `iss = https://auth.beletage.ro/application/o/architools/` — different issuer, different JWKS endpoint. Current code falls through to HS256 → 401.
|
||||||
|
|
||||||
|
**Fix shape:**
|
||||||
|
- Multi-issuer config: env `GIS_API_ACCEPTED_ISSUERS` = JSON array `["https://auth.beletage.ro/application/o/gis-api/","https://auth.beletage.ro/application/o/architools/", …]`.
|
||||||
|
- For each accepted issuer, derive its JWKS URL via the well-known discovery path (`{iss}/.well-known/openid-configuration`) at boot, or accept a parallel `GIS_API_JWKS_URLS` JSON map keyed by issuer. The latter is simpler + avoids a discovery call at first-token-time.
|
||||||
|
- Cache one `createRemoteJWKSet` per issuer at module scope.
|
||||||
|
- Pre-parse `iss` from the token (already done) and route to the matching JWKS getter.
|
||||||
|
|
||||||
|
**Bundle PR2.1 + PR2.2** as one gis-api PR ("PR3" in the cutover flow): they touch the same file (`auth.ts`), share a config layout (per-issuer + per-tenant maps), and architools needs both before it can call.
|
||||||
|
|
||||||
|
**Who ships:** gis-api session, ~1 hour total.
|
||||||
|
|
||||||
|
**Required values (need to be in Infisical /gis-api):**
|
||||||
|
- `GIS_API_ACCEPTED_ISSUERS` = `["https://auth.beletage.ro/application/o/gis-api/","https://auth.beletage.ro/application/o/architools/"]` (add planhub+eterra-live later)
|
||||||
|
- `GIS_API_JWKS_URLS` = `{"https://auth.beletage.ro/application/o/gis-api/":"https://auth.beletage.ro/application/o/gis-api/jwks/","https://auth.beletage.ro/application/o/architools/":"https://auth.beletage.ro/application/o/architools/jwks/"}`
|
||||||
|
- `GIS_API_TENANT_MAP` = `{"V59GMiYle87yd9VZOgUmdSmzYQALqNsKVAUR6QMi":"architools"}` (PR2.1)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Faza C — Rip-out (1 day, feature-flagged)
|
||||||
|
|
||||||
|
| Aspect | Detail |
|
||||||
|
|---|---|
|
||||||
|
| **Owner** | ArchiTools session |
|
||||||
|
| **Repo** | ArchiTools only |
|
||||||
|
| **gis-api involvement** | None |
|
||||||
|
|
||||||
|
`NEXT_PUBLIC_USE_GIS_AC=1` flag controls cutover. Plan-as-written stands.
|
||||||
|
|
||||||
|
**One verification gis-api can pre-confirm:** CORS allowlist on api.gis.ac includes `architools.beletage.ro` (deployed in PR2 `src/lib/hono-app.ts`). Browser preflight from architools origin → 204 with `Access-Control-Allow-Origin` echo (smoke-tested 22:55).
|
||||||
|
|
||||||
|
**Do not drop Prisma tables** (GisFeature, GisUat, GisSyncRun, GisSyncRule, CfExtract) in this PR — keep as dead columns 2-3 days. Drop in follow-up PR after prod validation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Faza D — Thin client (1-2 days)
|
||||||
|
|
||||||
|
| Aspect | Detail |
|
||||||
|
|---|---|
|
||||||
|
| **Owner** | ArchiTools session |
|
||||||
|
| **Repo** | ArchiTools only |
|
||||||
|
| **gis-api involvement** | None — all endpoints exist as specified |
|
||||||
|
|
||||||
|
**Endpoint contract confirmation** (each verified against PR2 deploy):
|
||||||
|
|
||||||
|
| ArchiTools method | gis-api endpoint | Auth | Scope |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `gisApi.me()` | `GET /api/v1/me` | Bearer | any |
|
||||||
|
| `gisApi.parcela.get(id)` | `GET /api/v1/parcela/:id` | Bearer | `>= basic`; `none` → 403; `basic` redacts PROPRIETARI / PROPRIETARI_VECHI / NR_CF / DOC |
|
||||||
|
| `gisApi.search(q, limit)` | `GET /api/v1/search?q=&limit=` | Bearer | any |
|
||||||
|
| `gisApi.parcel.tech(body)` | `POST /api/v1/parcel/tech` | Bearer | `>= basic` |
|
||||||
|
| `gisApi.parcel.unitsFetch(body)` | `POST /api/v1/parcel/units/fetch` | Bearer | `>= basic` |
|
||||||
|
| `gisApi.parcel.immApps(body)` | `POST /api/v1/parcel/imm-apps` | Bearer | `>= basic` |
|
||||||
|
| `gisApi.building.tech(body)` | `POST /api/v1/building/tech` | Bearer | `>= basic` |
|
||||||
|
| `gisApi.building.condoOwners(body)` | `POST /api/v1/building/condo-owners` | Bearer | `>= basic` |
|
||||||
|
| `gisApi.enrichment.cf.list({...})` | `GET /api/v1/enrichment/cf?…` | Bearer | RLS-filtered |
|
||||||
|
| `gisApi.enrichment.cf.get(id)` | `GET /api/v1/enrichment/cf/:id` | Bearer | RLS-filtered |
|
||||||
|
| `gisApi.enrichment.cf.create(body)` | `POST /api/v1/enrichment/cf` | Bearer | RLS-owned write |
|
||||||
|
| `gisApi.enrichment.cf.patch(id, body)` | `PATCH /api/v1/enrichment/cf/:id` | Bearer | RLS-owned write |
|
||||||
|
| `gisApi.enrichment.cf.uploadPdf(id, buf)` | `POST /api/v1/enrichment/cf/:id/pdf` | Bearer + `Content-Type: application/pdf` | RLS-owned write |
|
||||||
|
| `gisApi.enrichment.cf.getPdf(id)` | `GET /api/v1/enrichment/cf/:id/pdf` | Bearer | RLS-filtered |
|
||||||
|
| `gisApi.enrichment.catalog(nrCadastral)` | `GET /api/v1/enrichment/catalog/:nrCadastral` | Bearer | `>= basic` |
|
||||||
|
|
||||||
|
**Proxy request payload shape** (parcel/tech, parcel/units/fetch, building/tech, building/condo-owners):
|
||||||
|
```ts
|
||||||
|
{ siruta: string /* /^\d{3,7}$/ */, cadastralRef: string /* 1..200 */, force?: boolean }
|
||||||
|
```
|
||||||
|
For `parcel/imm-apps` add `layerId?: "TERENURI_ACTIVE" | "CLADIRI_ACTIVE"`.
|
||||||
|
|
||||||
|
**Do NOT** pass `correlationId` from the client — gis-api rewrites it server-side. Anything you send is silently overwritten.
|
||||||
|
|
||||||
|
**Response shape:** orchestrator response forwarded verbatim. Per `reference-orchestrator-endpoints` memory: `{status: "ok", data: {...}}` on success or `{error: "...", code}` on failure.
|
||||||
|
|
||||||
|
**Rate limit headers** on every response: `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset` (seconds). On exhaustion: `429 { error: "rate_limited", retryAfterSec }`.
|
||||||
|
|
||||||
|
**No correlationId echoed** in proxy response today (see `project-audit-correlation-echo` memory). If architools wants to correlate, generate a client-side request ID and log it alongside; gis-api's `gis_meta."ApiAudit"` row has the server-side traceId in format `${tenant_slug}:${sub.slice(0,8)}:${reqId}` indexed by `(tenant, ts DESC)`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Faza E — Geoportal rewrite (2 days)
|
||||||
|
|
||||||
|
| Aspect | Detail |
|
||||||
|
|---|---|
|
||||||
|
| **Owner** | ArchiTools session |
|
||||||
|
| **Repo** | ArchiTools only |
|
||||||
|
| **Prerequisite to verify before starting** | `tiles.gis.ac` (Martin) and `pmtiles.gis.ac/overview.pmtiles` are live. NOT a gis-api concern — infrastructure on shop. The 002 brief assumed they exist; confirm with infra team before ripping out the satra Martin source. |
|
||||||
|
| **gis-api involvement** | Confirm via curl `curl -I https://tiles.gis.ac/` + `curl -I https://pmtiles.gis.ac/overview.pmtiles`. If either 404s, that's an infrastructure blocker for Faza E (not gis-api). |
|
||||||
|
|
||||||
|
`map-viewer.tsx`, `search-bar.tsx`, `feature-info-panel.tsx`, `basemap-switcher.tsx` rewrites stand as-written in 002.
|
||||||
|
|
||||||
|
For `feature-info-panel.tsx` calling `gisApi.parcel.tech()` on missing enrichment: that path requires `enrichment_scope >= basic`. Beletage staff (Arhitecti LDAP group) get `enrichment_scope=full` from the scope mapping → no friction. Anonymous / non-Arhitecti users get `none` → 403 → panel must handle gracefully (probably hide enrichment section + show "log in for details").
|
||||||
|
|
||||||
|
`boundary-check / cf-status / export / pad / piz`: defer scoping per 002. If architools keeps any, evaluate whether a thin proxy to api.gis.ac is enough OR if a new endpoint is needed. New endpoints = new gis-api PR — out of scope for this cutover.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Faza F — ePay / CF ordering (½ day)
|
||||||
|
|
||||||
|
| Aspect | Detail |
|
||||||
|
|---|---|
|
||||||
|
| **Owner** | ArchiTools session |
|
||||||
|
| **Repo** | ArchiTools only |
|
||||||
|
| **gis-api involvement** | None — endpoints already exist |
|
||||||
|
|
||||||
|
UI flow: "Comandă extras CF" button calls `gisApi.enrichment.cf.create({nrCadastral, type: "epay", …})`. Status + download list via `enrichment.cf.list` + `enrichment.cf.getPdf(id)`.
|
||||||
|
|
||||||
|
**Tenant claim for billing routing:** post-PR2.1, `claims.tenant === "architools"` (human-readable). The CfExtract row will be RLS-owned by the user's `sub`, but ops can attribute billing via the audit log + `tenant` join. Future billing PR will pick this up.
|
||||||
|
|
||||||
|
Delete `src/app/api/ancpi/*` entirely.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Faza G — Test E2E + deploy (½ day)
|
||||||
|
|
||||||
|
| Aspect | Detail |
|
||||||
|
|---|---|
|
||||||
|
| **Owner** | ArchiTools session |
|
||||||
|
| **Repo** | ArchiTools only |
|
||||||
|
| **gis-api involvement** | Read-only: watch container logs + `gis_meta."ApiAudit"` for first architools traffic |
|
||||||
|
|
||||||
|
Test plan:
|
||||||
|
1. `/geoportal` in architools renders tiles from `gis.ac` (no satra requests).
|
||||||
|
2. Click a parcel → `gisApi.parcela.get(id)` returns full enrichment (Beletage staff scope=full).
|
||||||
|
3. Click parcel with no enrichment → `gisApi.parcel.tech()` triggers orchestrator live-fetch → enriched response returns.
|
||||||
|
4. Place a CF order → `gisApi.enrichment.cf.create()` → row appears in `gis_meta."CfExtract"` with `userId = <Arhitecti user sub>`.
|
||||||
|
5. Audit log: `SELECT tenant, endpoint, statusCode, ts FROM gis_meta."ApiAudit" WHERE tenant='architools' ORDER BY ts DESC LIMIT 20;` shows architools traffic with human-readable tenant.
|
||||||
|
6. Rate limit: 600 rapid calls (architools is configured for 500 rpm) → ~500 succeed, rest 429 with `X-RateLimit-Reset` ≤ 60.
|
||||||
|
7. Zero references to `architools_postgres.GisFeature` remain in code.
|
||||||
|
8. Type-check + lint + build clean.
|
||||||
|
9. Portainer redeploy.
|
||||||
|
10. **24h grace period:** keep old stack (Martin satra, deprecated parcel-sync code) until validation. Follow-up PR drops dead Prisma tables + stops martin satra container.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prerequisites checklist (in execution order)
|
||||||
|
|
||||||
|
Mark these done before opening the architools session:
|
||||||
|
|
||||||
|
- [ ] **gis-api PR2.1 (tenant translation) + PR2.2 (multi-issuer JWT)** shipped + deployed on api.gis.ac. Verified with: mint an architools-issuer token via client_credentials → call `https://api.gis.ac/api/v1/me` → 200 + `claims.tenant === "architools"`.
|
||||||
|
- [ ] **Infisical `/architools`** populated: `AUTHENTIK_CLIENT_ID`, `AUTHENTIK_CLIENT_SECRET`, `AUTHENTIK_ISSUER`, `AUTHENTIK_JWKS_URL`, `GIS_API_URL`, `NEXT_PUBLIC_MARTIN_URL`, `NEXT_PUBLIC_PMTILES_URL`.
|
||||||
|
- [ ] **Infisical `/gis-api`** populated: `GIS_API_ACCEPTED_ISSUERS`, `GIS_API_JWKS_URLS`, `GIS_API_TENANT_MAP` (PR2.1/PR2.2 inputs).
|
||||||
|
- [ ] **tiles.gis.ac + pmtiles.gis.ac** live (curl 200 / non-404). Infra confirms.
|
||||||
|
- [ ] **architools-session Claude** has read this plan + `MEMORY.md` of `~/.claude/projects/-home-orchestrator-Code-gis-api/memory/`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Items resolved from 002's "Open questions"
|
||||||
|
|
||||||
|
| 002 question | Resolution |
|
||||||
|
|---|---|
|
||||||
|
| **1. Authentik OIDC client provisioning** | DONE. architools provider pk=6, scope mapping `41b23bc3-bdd8-4a61-b975-6e0eff56df72` attached. See `reference-authentik-provisioned-state`. |
|
||||||
|
| **2. Audit table schema exists?** | DONE. `gis_meta."ApiAudit"` created by busc-infra `0693457` (008-api-audit.sql). Mirrored in `prisma/schema.prisma` (`model ApiAudit`). |
|
||||||
|
| **3. Rate-limit defaults** | DONE. `GIS_API_TENANT_RATE_LIMITS` JSON env, defaults to 100 rpm. Deployed seed: `{"architools":500,"planhub":120,"eterra-live":300,"_default":100}`. ⚠️ but mis-keyed until PR2.1 ships. |
|
||||||
|
| **4. archi.* schema writes (Canvas, Job, etc.)** | DEFERRED. Out of scope for this cutover. Later sprint. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Items obsolete in 002
|
||||||
|
|
||||||
|
- **Faza A** entire content — done.
|
||||||
|
- **"PR3 (gis-api): Per-tenant rate limit + audit migration"** — done as part of actual PR2 (rolled together).
|
||||||
|
- **"`scope-mapper.ts` in architools"** correction — already noted; reaffirm: architools must NOT compute scope client-side.
|
||||||
|
- **"`GIS_API_JWT_SECRET (shared)` in Infisical"** correction — still valid; do not pull this into architools NextAuth.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What this gis-api session will produce next (if Marius approves)
|
||||||
|
|
||||||
|
**PR2.1 + PR2.2** (bundled into one gis-api commit): `src/lib/auth.ts` multi-issuer + tenant-map, `.env.example` adds, unit tests. ~50–80 LOC, ~1 hour including typecheck+build+push. Migration of architools onto api.gis.ac is gated on this landing.
|
||||||
|
|
||||||
|
After that, this gis-api session is done — architools cutover is owned by the architools session.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Reference
|
||||||
|
|
||||||
|
- gis-api memory: `~/.claude/projects/-home-orchestrator-Code-gis-api/memory/MEMORY.md`
|
||||||
|
- gis-api recent commits: `b34c58e` (PR1) → `ad1825a` (PR1.1) → `a43673a..fc459e0` (PR2 = 4 commits)
|
||||||
|
- busc-infra: `0693457` (migration 008)
|
||||||
|
- Authentik provider pks: gis-api pk=8, architools pk=6; scope mapping pk=41b23bc3-bdd8-4a61-b975-6e0eff56df72
|
||||||
|
- Live container: `gis-api` on shop, image at `fc459e0`, healthcheck `https://api.gis.ac/api/healthz`
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
# Plan 006 — ePay CF-Extract as a Multi-Tenant Service
|
||||||
|
|
||||||
|
**Status:** design (executable). **Author:** deep-dive 2026-06-04 + hardening 2026-06-05.
|
||||||
|
**Prereq reading:** plans 002/003 (architools thin-client cutover), `project_epay_cf_roadmap_2026_06` memory.
|
||||||
|
|
||||||
|
## Why this exists
|
||||||
|
|
||||||
|
The CF-extract capability (ANCPI ePay paid extracts + the free `cf-intern` copycf
|
||||||
|
circuit) will be offered beyond internal use — multi-tenant (ArchiTools, eterra-live,
|
||||||
|
Planhub, external paying customers). Today it runs as an **in-process queue inside
|
||||||
|
ArchiTools** (`src/modules/parcel-sync/services/epay-*`). That path is now hardened
|
||||||
|
(commit `f49fdb1`: cart hygiene, auth/IDOR gates, single-page fetch, parallel
|
||||||
|
downloads, recover-by-extractId) and is **billing-safe and correct for the internal
|
||||||
|
tool** — but it is the wrong *shape* for a service:
|
||||||
|
|
||||||
|
- Queue + ePay session are in-memory globals → die on redeploy mid-batch.
|
||||||
|
- One serial cart per process → no multi-tenant throughput.
|
||||||
|
- No catalog dedup on the paid path → the same parcel is paid for repeatedly.
|
||||||
|
- `EPAY_ORDERING_VIA_GIS_AC=false` because **gis-api `POST /enrichment/cf` inserts a
|
||||||
|
pending row that nothing fulfills** — the orchestrator has no ePay worker. Plan 003
|
||||||
|
Faza F ("endpoints already exist") is wrong: the fulfiller is the unwritten keystone.
|
||||||
|
|
||||||
|
This plan is the path from the hardened-internal state to a real service. Each phase
|
||||||
|
is independently shippable; do them in order, validate, then flip the flag per-tenant.
|
||||||
|
|
||||||
|
## Invariants carried over from the hardened internal path (do NOT regress)
|
||||||
|
|
||||||
|
These were learned the hard way (2026-06-04 incident, order 10009605). The new worker
|
||||||
|
MUST preserve every one:
|
||||||
|
|
||||||
|
1. **Submit is timeout-resilient.** A slow `EditCartSubmit` that ANCPI completes must
|
||||||
|
never be marked failed. Resolve the order via `findNewOrderId(previous, known)` which
|
||||||
|
never adopts a stale/known id. (`SUBMIT_TIMEOUT_MS`, today's fix.)
|
||||||
|
2. **Cart hygiene invariant.** ePay has ONE global cart per account; `EditCartSubmit`
|
||||||
|
checks out everything in it. After N adds a clean cart reports exactly N items — any
|
||||||
|
excess = orphan from a crash → wipe + abort, never submit a cart you didn't build.
|
||||||
|
3. **CF-number matching is authoritative; index fallback is `review`, not `completed`.**
|
||||||
|
4. **`%PDF` magic-byte check** on every download (expired session returns login HTML).
|
||||||
|
5. **Single-page order fetch** via `itemsPerPage` (5/page default silently drops docs).
|
||||||
|
6. **Recover is idempotent** (re-poll + re-download an already-paid order, no new charge).
|
||||||
|
|
||||||
|
## Phase A — DB-backed fulfiller worker (`eterra.cf-epay`) — THE KEYSTONE
|
||||||
|
|
||||||
|
A pg-boss worker in `gis-sync-orchestrator` (next to `enrichment-drainer`, cron 1–2 min).
|
||||||
|
**The CfExtract row IS the work item** — no in-memory queue.
|
||||||
|
|
||||||
|
- **Claim:** `SELECT … FROM gis_enrichment."CfExtract" WHERE status='pending' AND
|
||||||
|
type='epay' [AND account-compatible] ORDER BY "createdAt" FOR UPDATE SKIP LOCKED LIMIT N;
|
||||||
|
UPDATE → status='claimed', claimedAt=now()`. SKIP LOCKED → two instances never grab the
|
||||||
|
same rows.
|
||||||
|
- **State machine** (each transition = one UPDATE = a precise resumable marker):
|
||||||
|
`pending → claimed → cart → submitted_unconfirmed → polling → downloading → completed |
|
||||||
|
review | failed | cancelled`. Extend gis-api's `ExtractStatus` enum
|
||||||
|
(`gis-api/src/routes/enrichment.ts:9`) with `claimed`, `submitted_unconfirmed`, `review`.
|
||||||
|
- **Crash recovery:** a boot **reaper** requeues rows stuck in a transitional state past a
|
||||||
|
heartbeat TTL. `submitted_unconfirmed` rows are resolved via the recover pattern (find
|
||||||
|
the order at ANCPI, never re-charge). This structurally eliminates the in-memory-queue
|
||||||
|
orphan class (criticals C2).
|
||||||
|
- **Idempotent submit:** before `EditCartSubmit`, persist on the claimed rows the account's
|
||||||
|
current latest orderId + the intended `nrCadastral` set. On timeout/crash, resume
|
||||||
|
re-runs `findNewOrderId` against that snapshot — never adopts a stale id.
|
||||||
|
- Port the hardened `epay-client` here (see Phase G — shared package).
|
||||||
|
|
||||||
|
## Phase B — `epay_accounts` pool with one-batch-per-account lock
|
||||||
|
|
||||||
|
Mirror `gis_meta.eterra_accounts` (busc-infra migration 004): AES-256-GCM creds,
|
||||||
|
`status active/blocked/retired`, `blocked_reason`, `credits_cached`, optional hourly cap,
|
||||||
|
`in_flight_batch_id`.
|
||||||
|
|
||||||
|
- `pickEpayAccount`: `FOR UPDATE SKIP LOCKED`, but because ePay's cart is **global per
|
||||||
|
account**, atomically set `in_flight_batch_id` (status `busy`) so no second batch can
|
||||||
|
touch that account's cart. This is the structural fix for cart contamination (C1) in the
|
||||||
|
pooled world.
|
||||||
|
- Refuse to claim a batch larger than the account's cached credits. ePay credits are a
|
||||||
|
**hard consumable (real money)** — unlike the soft eTerra quota, the credit cap is
|
||||||
|
mandatory, not advisory.
|
||||||
|
|
||||||
|
## Phase C — Catalog dedup (largest recurring economic win)
|
||||||
|
|
||||||
|
`CfExtractCatalog` is written **only** on the `cf-intern` path today; nothing writes it
|
||||||
|
when a paid ePay order completes → a paid extract by tenant A is never "fresh" for tenant
|
||||||
|
B, so the 30-day money-saver is structurally unrealized.
|
||||||
|
|
||||||
|
- On ePay completion, `upsert CfExtractCatalog(nrCadastral, latestId,
|
||||||
|
expiresAt=documentDate+30d, isFresh=true)`.
|
||||||
|
- `POST /enrichment/cf/claim {nrCadastral}`: on a catalog hit, create a B-owned row
|
||||||
|
`type='catalog', status='completed', creditsUsed=0` pointing at the shared MinIO object
|
||||||
|
(or a copy). This turns today's 409 `catalog_hit` (`enrichment.ts:226`) into **instant,
|
||||||
|
free fulfillment**. RLS unchanged (B reads B's row). One paid extract serves every tenant
|
||||||
|
that needs that parcel within 30 days, at marginal zero ANCPI cost.
|
||||||
|
|
||||||
|
## Phase D — Credential model (tenant-policy-driven)
|
||||||
|
|
||||||
|
Store the strategy per-tenant; don't pick one globally:
|
||||||
|
|
||||||
|
- **Internal Beletage group** → pooled company accounts (Infisical, encrypted in
|
||||||
|
`epay_accounts`). Best batching + catalog sharing; per-credit attribution via audit.
|
||||||
|
- **External paying tenants** (eterra-live model) → dedicated per-tenant accounts so
|
||||||
|
credits/billing stay clean.
|
||||||
|
- Record `account_id` + `creditsUsed` on every `CfExtract` for attribution regardless.
|
||||||
|
- All three apps converge as thin callers of `POST /enrichment/cf` (Authentik multi-issuer
|
||||||
|
+ tenant claim already in place, `gis-api/src/lib/auth.ts`). Reuse eterra-live
|
||||||
|
`crypto.ts` (AES-256-GCM) + a 1-byte key-version prefix for rotation.
|
||||||
|
|
||||||
|
## Phase E — gis-api gaps for async consumption
|
||||||
|
|
||||||
|
1. **Completion webhook/SSE**, tenant-scoped + RLS-filtered (`GET /enrichment/cf/events`)
|
||||||
|
→ kills polling and the dead-Brevo dependency.
|
||||||
|
2. **Bulk-zip** `GET /enrichment/cf/zip?orderId=` streaming from MinIO (port the V3
|
||||||
|
streaming-zip approach).
|
||||||
|
3. `ExtractStatus` enum additions (see Phase A).
|
||||||
|
4. List filters `creditsUsed=0` / `type='catalog'` so the UI can label shared extracts.
|
||||||
|
|
||||||
|
## Phase F — Reversible migration, per-tenant flip
|
||||||
|
|
||||||
|
- **Phase 0 (now):** `EPAY_ORDERING_VIA_GIS_AC=false`, hardened legacy queue is the sole
|
||||||
|
fulfiller. `/api/ancpi/recover` stays as the manual safety net.
|
||||||
|
- **Phase 1:** deploy worker + pool + catalog-write; seed `epay_accounts` with ONLY the
|
||||||
|
Beletage account; flip the flag for `claims.tenant === 'architools'`.
|
||||||
|
- **Phase 2:** run both paths in parallel a grace window; reconcile on orderId (no double
|
||||||
|
charge).
|
||||||
|
- **Phase 3:** onboard external tenants with dedicated accounts; delete `epay-queue` /
|
||||||
|
`epay-client` / `epay-session-store` + `src/app/api/ancpi/*` from ArchiTools. The flag is
|
||||||
|
the kill-switch throughout.
|
||||||
|
|
||||||
|
## Phase G — Shared `epay-client` package (do regardless of phase)
|
||||||
|
|
||||||
|
ArchiTools and eterra-live each have a near-identical `epay-client.ts` that has **already
|
||||||
|
diverged dangerously**: ArchiTools got today's fixes; eterra-live got the method-internal
|
||||||
|
ports (commit `eterra-live d30128b`) but lacks cart-hygiene + the per-page parser refactor.
|
||||||
|
Extract `@beletage/epay-client` (natural home: `gis-sync-orchestrator`, which owns the
|
||||||
|
account pool) so a fix lands once. Until then, any epay-client change MUST be mirrored to
|
||||||
|
both repos in the same change.
|
||||||
|
|
||||||
|
## Known follow-ups not yet done
|
||||||
|
|
||||||
|
- eterra-live still lacks the cart-hygiene `numberOfItems` invariant (single-order flow
|
||||||
|
makes it lower-risk, but a crashed prior order can still orphan a cart row). Needs a
|
||||||
|
route-level touch + testing on that product before shipping.
|
||||||
|
- `BREVO_API_KEY` returns 401 "Key not found" → ArchiTools email notifications are dead;
|
||||||
|
the correct fix is the Phase E webhook, not patching Brevo. SMTP relay creds still work.
|
||||||
|
- ArchiTools `auth-options.ts` has pre-existing `react-hooks/rules-of-hooks` lint errors on
|
||||||
|
the `useGisAcFlag`/`useBasicPanelFlag` session calls (tolerated by `next build`).
|
||||||
@@ -59,6 +59,7 @@ export async function GET(req: Request) {
|
|||||||
minioPath: true,
|
minioPath: true,
|
||||||
documentDate: true,
|
documentDate: true,
|
||||||
completedAt: true,
|
completedAt: true,
|
||||||
|
status: true,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -71,7 +72,9 @@ export async function GET(req: Request) {
|
|||||||
for (let i = 0; i < ids.length; i++) {
|
for (let i = 0; i < ids.length; i++) {
|
||||||
const id = ids[i]!;
|
const id = ids[i]!;
|
||||||
const extract = extractMap.get(id);
|
const extract = extractMap.get(id);
|
||||||
if (!extract?.minioPath) continue;
|
// Skip rows without a file, and "review" rows (PDF present but the
|
||||||
|
// CF↔doc match is unverified — must not land in a "valid extracts" zip).
|
||||||
|
if (!extract?.minioPath || extract.status === "review") continue;
|
||||||
|
|
||||||
const dateForName = extract.documentDate ?? extract.completedAt ?? new Date();
|
const dateForName = extract.documentDate ?? extract.completedAt ?? new Date();
|
||||||
const d = new Date(dateForName);
|
const d = new Date(dateForName);
|
||||||
|
|||||||
@@ -15,7 +15,7 @@
|
|||||||
|
|
||||||
import { NextResponse } from "next/server";
|
import { NextResponse } from "next/server";
|
||||||
import { prisma } from "@/core/storage/prisma";
|
import { prisma } from "@/core/storage/prisma";
|
||||||
import { recoverBatch } from "@/modules/parcel-sync/services/epay-queue";
|
import { recoverBatch, getQueueStatus } from "@/modules/parcel-sync/services/epay-queue";
|
||||||
import { requireCfAccess } from "@/core/auth/cf-access";
|
import { requireCfAccess } from "@/core/auth/cf-access";
|
||||||
|
|
||||||
export const runtime = "nodejs";
|
export const runtime = "nodejs";
|
||||||
@@ -85,6 +85,22 @@ export async function GET(req: Request) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Cross-guard against the live batch queue: while a batch is processing,
|
||||||
|
// its rows sit at (orderId:null, status in cart/ordering/...) — exactly the
|
||||||
|
// orphan window the WHERE below matches. Recovering then would re-stamp a
|
||||||
|
// live batch's rows with THIS order's id (wrong PDF, status corruption).
|
||||||
|
// A genuinely-crashed batch leaves __epayQueueProcessing=false (reset on
|
||||||
|
// restart), so blocking here only defers against an actively-running queue.
|
||||||
|
if (getQueueStatus().processing) {
|
||||||
|
return NextResponse.json(
|
||||||
|
{
|
||||||
|
error:
|
||||||
|
"O comandă ePay este în curs de procesare. Așteaptă finalizarea înainte de recuperare.",
|
||||||
|
},
|
||||||
|
{ status: 409 },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
// Candidate rows: anything already tagged with this order that isn't
|
// Candidate rows: anything already tagged with this order that isn't
|
||||||
// terminal, PLUS recent orphaned rows (orderId:null) in a recoverable
|
// terminal, PLUS recent orphaned rows (orderId:null) in a recoverable
|
||||||
// state — the operator asserts they belong to this order.
|
// state — the operator asserts they belong to this order.
|
||||||
|
|||||||
@@ -254,6 +254,23 @@ export function CfOrderModal({
|
|||||||
const row = (data.orders ?? []).find((o) => o.id === id);
|
const row = (data.orders ?? []).find((o) => o.id === id);
|
||||||
if (row) {
|
if (row) {
|
||||||
const s = (row.status || "").toLowerCase();
|
const s = (row.status || "").toLowerCase();
|
||||||
|
// "review" rows DO have minioPath + documentName but the PDF
|
||||||
|
// may belong to another parcel (ambiguous CF↔doc match) — must
|
||||||
|
// be checked BEFORE the minioPath/documentName completion
|
||||||
|
// fallback, or it would short-circuit to "done" and hand the
|
||||||
|
// operator an unverified extract.
|
||||||
|
if (s === "review") {
|
||||||
|
setError(
|
||||||
|
"Comanda necesită verificare manuală (potrivire ambiguă document↔parcelă). Verifică în ParcelSync → ePay înainte de a folosi extrasul.",
|
||||||
|
);
|
||||||
|
setPhase("error");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (s === "failed" || s === "error") {
|
||||||
|
setError("Comanda a eșuat la ANCPI.");
|
||||||
|
setPhase("error");
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (
|
if (
|
||||||
s === "completed" ||
|
s === "completed" ||
|
||||||
s === "done" ||
|
s === "done" ||
|
||||||
@@ -264,11 +281,6 @@ export function CfOrderModal({
|
|||||||
setPhase("done");
|
setPhase("done");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
if (s === "failed" || s === "error") {
|
|
||||||
setError("Comanda a eșuat la ANCPI.");
|
|
||||||
setPhase("error");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch {
|
} catch {
|
||||||
|
|||||||
@@ -328,17 +328,35 @@ export function EpayTab() {
|
|||||||
* order, no new charge. For rows that failed at the download/poll
|
* order, no new charge. For rows that failed at the download/poll
|
||||||
* stage (the order exists at ANCPI but we never stored the PDF). -- */
|
* stage (the order exists at ANCPI but we never stored the PDF). -- */
|
||||||
const [retryingId, setRetryingId] = useState<string | null>(null);
|
const [retryingId, setRetryingId] = useState<string | null>(null);
|
||||||
|
const [retryNotice, setRetryNotice] = useState<string | null>(null);
|
||||||
const handleRetryDownload = async (order: CfExtractRecord) => {
|
const handleRetryDownload = async (order: CfExtractRecord) => {
|
||||||
setRetryingId(order.id);
|
setRetryingId(order.id);
|
||||||
|
setRetryNotice(null);
|
||||||
try {
|
try {
|
||||||
const res = await fetch(
|
const res = await fetch(
|
||||||
`/api/ancpi/recover?extractId=${encodeURIComponent(order.id)}`,
|
`/api/ancpi/recover?extractId=${encodeURIComponent(order.id)}`,
|
||||||
);
|
);
|
||||||
// 409 → the row has no orderId yet (never reached ANCPI); nothing to
|
const data = (await res.json().catch(() => ({}))) as {
|
||||||
// recover by row. Other errors surface on the next refresh.
|
error?: string;
|
||||||
await res.json().catch(() => ({}));
|
completed?: number;
|
||||||
|
attempted?: number;
|
||||||
|
};
|
||||||
|
if (!res.ok) {
|
||||||
|
// 409 (queue busy / no orderId yet), 404, 500 — tell the user.
|
||||||
|
setRetryNotice(data.error ?? `Reîncercare eșuată (${res.status}).`);
|
||||||
|
} else if ((data.completed ?? 0) > 0) {
|
||||||
|
setRetryNotice(
|
||||||
|
`Recuperat: ${data.completed}/${data.attempted ?? data.completed} extrase.`,
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
setRetryNotice(
|
||||||
|
"Nimic de recuperat — comanda nu există la ANCPI sau e deja finalizată.",
|
||||||
|
);
|
||||||
|
}
|
||||||
void fetchOrders(true);
|
void fetchOrders(true);
|
||||||
void fetchEpayStatus();
|
void fetchEpayStatus();
|
||||||
|
} catch {
|
||||||
|
setRetryNotice("Eroare rețea la reîncercare.");
|
||||||
} finally {
|
} finally {
|
||||||
setRetryingId(null);
|
setRetryingId(null);
|
||||||
}
|
}
|
||||||
@@ -911,6 +929,20 @@ export function EpayTab() {
|
|||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
{/* -- Retry notice ------------------------------------------- */}
|
||||||
|
{retryNotice && (
|
||||||
|
<div className="flex items-center justify-between gap-2 rounded-md border bg-muted/40 px-3 py-2 text-xs">
|
||||||
|
<span>{retryNotice}</span>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
className="text-muted-foreground hover:text-foreground"
|
||||||
|
onClick={() => setRetryNotice(null)}
|
||||||
|
>
|
||||||
|
✕
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
{/* -- Active orders indicator -------------------------------- */}
|
{/* -- Active orders indicator -------------------------------- */}
|
||||||
{hasActive && (
|
{hasActive && (
|
||||||
<div className="flex items-center gap-2 text-xs text-muted-foreground">
|
<div className="flex items-center gap-2 text-xs text-muted-foreground">
|
||||||
|
|||||||
@@ -16,7 +16,7 @@
|
|||||||
import { prisma } from "@/core/storage/prisma";
|
import { prisma } from "@/core/storage/prisma";
|
||||||
import { EpayClient } from "./epay-client";
|
import { EpayClient } from "./epay-client";
|
||||||
import { getEpayCredentials, updateEpayCredits } from "./epay-session-store";
|
import { getEpayCredentials, updateEpayCredits } from "./epay-session-store";
|
||||||
import { storeCfExtract } from "./epay-storage";
|
import { storeCfExtract, getNextFileIndex } from "./epay-storage";
|
||||||
import type { CfExtractCreateInput } from "./epay-types";
|
import type { CfExtractCreateInput } from "./epay-types";
|
||||||
|
|
||||||
/* ------------------------------------------------------------------ */
|
/* ------------------------------------------------------------------ */
|
||||||
@@ -308,7 +308,12 @@ async function processBatch(
|
|||||||
// leftover row from a previously-crashed batch would be paid for and
|
// leftover row from a previously-crashed batch would be paid for and
|
||||||
// attach the wrong PDF. We track our own basket ids for cleanup, and
|
// attach the wrong PDF. We track our own basket ids for cleanup, and
|
||||||
// bail the moment ANCPI reports more rows than we put in.
|
// bail the moment ANCPI reports more rows than we put in.
|
||||||
let addedCount = 0;
|
// cartCount tracks the rows actually in the cart (incremented on add,
|
||||||
|
// decremented only on a CONFIRMED delete) so the invariant stays correct
|
||||||
|
// even if a cleanup delete fails. cartDirty trips when a metadata-less
|
||||||
|
// row could not be confirmed-deleted → we must not submit.
|
||||||
|
let cartCount = 0;
|
||||||
|
let cartDirty = false;
|
||||||
for (let idx = 0; idx < items.length; idx++) {
|
for (let idx = 0; idx < items.length; idx++) {
|
||||||
const item = items[idx]!;
|
const item = items[idx]!;
|
||||||
const { extractId, input } = item;
|
const { extractId, input } = item;
|
||||||
@@ -316,14 +321,17 @@ async function processBatch(
|
|||||||
await updateStatus(extractId, "cart");
|
await updateStatus(extractId, "cart");
|
||||||
const { basketRowId, numberOfItems, itemIds } =
|
const { basketRowId, numberOfItems, itemIds } =
|
||||||
await client.addToCartDetailed(input.prodId ?? 14200);
|
await client.addToCartDetailed(input.prodId ?? 14200);
|
||||||
|
cartCount++;
|
||||||
|
|
||||||
// After N successful adds a clean cart reports exactly N items. More
|
// Right after the add, a clean cart reports exactly cartCount rows.
|
||||||
// than that = pre-existing junk (orphans from a crash). Never submit a
|
// More than that = pre-existing junk (orphans from a crash). Never
|
||||||
// cart we didn't fully build: wipe everything ANCPI listed and abort —
|
// submit a cart we didn't fully build: wipe everything ANCPI listed
|
||||||
// the next retry starts clean. No charge happens (we never submit).
|
// and abort — the next retry starts clean. No charge (we never submit).
|
||||||
if (numberOfItems > addedCount + 1) {
|
// (numberOfItems falls back to items.length, so an unexpected response
|
||||||
|
// shape degrades to "no excess detected" rather than a false abort.)
|
||||||
|
if (numberOfItems > cartCount) {
|
||||||
console.error(
|
console.error(
|
||||||
`[epay-queue] Dirty cart: expected ${addedCount + 1} rows, ANCPI reports ${numberOfItems}. Wiping + aborting batch.`,
|
`[epay-queue] Dirty cart: expected ${cartCount} rows, ANCPI reports ${numberOfItems}. Wiping + aborting batch.`,
|
||||||
);
|
);
|
||||||
const toWipe = itemIds.length
|
const toWipe = itemIds.length
|
||||||
? itemIds
|
? itemIds
|
||||||
@@ -338,7 +346,6 @@ async function processBatch(
|
|||||||
}
|
}
|
||||||
|
|
||||||
ourBasketIdsForCleanup.push(basketRowId);
|
ourBasketIdsForCleanup.push(basketRowId);
|
||||||
addedCount++;
|
|
||||||
item.basketRowId = basketRowId;
|
item.basketRowId = basketRowId;
|
||||||
await updateStatus(extractId, "cart", { basketRowId });
|
await updateStatus(extractId, "cart", { basketRowId });
|
||||||
|
|
||||||
@@ -379,14 +386,41 @@ async function processBatch(
|
|||||||
await updateStatus(extractId, "failed", {
|
await updateStatus(extractId, "failed", {
|
||||||
errorMessage: "Salvarea metadatelor în ePay a eșuat.",
|
errorMessage: "Salvarea metadatelor în ePay a eșuat.",
|
||||||
});
|
});
|
||||||
// Remove this metadata-less row from the cart so it can't be
|
item.basketRowId = undefined; // exclude from validItems regardless
|
||||||
// checked out and charged. Drop it from our tracking + batch.
|
// Remove this metadata-less row from the cart so it can't be checked
|
||||||
await client.deleteCartItem(basketRowId, idx);
|
// out and charged. Only drop it from tracking + decrement cartCount
|
||||||
|
// if ANCPI CONFIRMED the delete; otherwise the row is still in the
|
||||||
|
// cart, must stay in cleanup tracking, and the cart is now "dirty".
|
||||||
|
const deleted = await client.deleteCartItem(basketRowId, idx);
|
||||||
|
if (deleted) {
|
||||||
|
cartCount--;
|
||||||
ourBasketIdsForCleanup.pop();
|
ourBasketIdsForCleanup.pop();
|
||||||
addedCount--;
|
} else {
|
||||||
item.basketRowId = undefined;
|
// Undeletable metadata-less row → submitting now would check it out
|
||||||
|
// and charge for it. Refuse to submit this batch.
|
||||||
|
cartDirty = true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A metadata-less row we couldn't remove is still in the global cart;
|
||||||
|
// submitOrder would check out the WHOLE cart and charge for it. Abort
|
||||||
|
// (best-effort wipe) instead of submitting a cart we can't guarantee
|
||||||
|
// clean. The catch below would NOT fire on the success path, so this
|
||||||
|
// explicit guard is what stops the unintended charge.
|
||||||
|
if (cartDirty) {
|
||||||
|
console.error(
|
||||||
|
"[epay-queue] Cart has an undeletable metadata-less row — aborting before submit.",
|
||||||
|
);
|
||||||
|
await client.deleteCartItems(ourBasketIdsForCleanup);
|
||||||
|
for (const id of extractIds) {
|
||||||
|
await updateStatus(id, "failed", {
|
||||||
|
errorMessage:
|
||||||
|
"Coș ePay nu a putut fi curățat (ștergere neconfirmată) — comanda a fost oprită pentru a evita o plată fără metadate. Reîncearcă.",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
// Filter to only items that had successful metadata saves
|
// Filter to only items that had successful metadata saves
|
||||||
const validItems = items.filter((i) => i.basketRowId !== undefined);
|
const validItems = items.filter((i) => i.basketRowId !== undefined);
|
||||||
@@ -499,8 +533,14 @@ async function finalizeOrder(
|
|||||||
item: QueueItem;
|
item: QueueItem;
|
||||||
doc: (typeof downloadableDocs)[number];
|
doc: (typeof downloadableDocs)[number];
|
||||||
matchedByIndex: boolean;
|
matchedByIndex: boolean;
|
||||||
|
index: number;
|
||||||
};
|
};
|
||||||
const plans: Plan[] = [];
|
const plans: Plan[] = [];
|
||||||
|
// Per-cadastral file index, pre-allocated SEQUENTIALLY so two parallel
|
||||||
|
// tasks for the same cadastral never collide on the MinIO scan (which is
|
||||||
|
// a read-modify-write). Seed each distinct cadastral from MinIO once,
|
||||||
|
// then hand out 1,2,… within this batch.
|
||||||
|
const nextIndexByCad = new Map<string, number>();
|
||||||
for (let i = 0; i < validItems.length; i++) {
|
for (let i = 0; i < validItems.length; i++) {
|
||||||
const item = validItems[i]!;
|
const item = validItems[i]!;
|
||||||
const nrCF = item.input.nrCF ?? item.input.nrCadastral;
|
const nrCF = item.input.nrCF ?? item.input.nrCadastral;
|
||||||
@@ -525,12 +565,19 @@ async function finalizeOrder(
|
|||||||
});
|
});
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
plans.push({ item, doc, matchedByIndex });
|
|
||||||
|
const cad = item.input.nrCadastral;
|
||||||
|
let next = nextIndexByCad.get(cad);
|
||||||
|
if (next === undefined) next = await getNextFileIndex(cad);
|
||||||
|
nextIndexByCad.set(cad, next + 1);
|
||||||
|
|
||||||
|
plans.push({ item, doc, matchedByIndex, index: next });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Step 6: download + store in parallel (bounded). Each task is fully
|
// Step 6: download + store in parallel (bounded). Each task is fully
|
||||||
// self-contained so a failure on one row doesn't abort the others.
|
// self-contained so a failure on one row doesn't abort the others. The
|
||||||
await runWithConcurrency(plans, DOWNLOAD_CONCURRENCY, async ({ item, doc, matchedByIndex }) => {
|
// file index is pre-allocated above, so parallel stores never overwrite.
|
||||||
|
await runWithConcurrency(plans, DOWNLOAD_CONCURRENCY, async ({ item, doc, matchedByIndex, index: fileIndex }) => {
|
||||||
try {
|
try {
|
||||||
await updateStatus(item.extractId, "downloading", {
|
await updateStatus(item.extractId, "downloading", {
|
||||||
idDocument: doc.idDocument,
|
idDocument: doc.idDocument,
|
||||||
@@ -552,6 +599,7 @@ async function finalizeOrder(
|
|||||||
stare: finalStatus.status,
|
stare: finalStatus.status,
|
||||||
produs: "EXI_ONLINE",
|
produs: "EXI_ONLINE",
|
||||||
},
|
},
|
||||||
|
fileIndex,
|
||||||
);
|
);
|
||||||
|
|
||||||
if (!doc.dataDocument) {
|
if (!doc.dataDocument) {
|
||||||
|
|||||||
@@ -83,15 +83,25 @@ export function buildFileName(
|
|||||||
/**
|
/**
|
||||||
* Store a CF extract PDF in MinIO.
|
* Store a CF extract PDF in MinIO.
|
||||||
* Returns the MinIO path and file index.
|
* Returns the MinIO path and file index.
|
||||||
|
*
|
||||||
|
* Pass `explicitIndex` to skip the MinIO scan and use a caller-allocated
|
||||||
|
* index. Required when storing concurrently (parallel downloads): the scan
|
||||||
|
* is an unsynchronised read-modify-write, so two tasks for the same
|
||||||
|
* cadastral would both compute index 1 and the second putObject would
|
||||||
|
* silently overwrite the first. The caller pre-allocates distinct indices.
|
||||||
*/
|
*/
|
||||||
export async function storeCfExtract(
|
export async function storeCfExtract(
|
||||||
pdfBuffer: Buffer,
|
pdfBuffer: Buffer,
|
||||||
nrCadastral: string,
|
nrCadastral: string,
|
||||||
metadata: Record<string, string>,
|
metadata: Record<string, string>,
|
||||||
|
explicitIndex?: number,
|
||||||
): Promise<{ path: string; fileName: string; index: number }> {
|
): Promise<{ path: string; fileName: string; index: number }> {
|
||||||
await ensureAncpiBucket();
|
await ensureAncpiBucket();
|
||||||
|
|
||||||
const index = await getNextFileIndex(nrCadastral);
|
const index =
|
||||||
|
explicitIndex !== undefined
|
||||||
|
? explicitIndex
|
||||||
|
: await getNextFileIndex(nrCadastral);
|
||||||
const fileName = buildFileName(index, nrCadastral, new Date());
|
const fileName = buildFileName(index, nrCadastral, new Date());
|
||||||
// Store in subfolder per cadastral number
|
// Store in subfolder per cadastral number
|
||||||
const path = `parcele/${nrCadastral}/${fileName}`;
|
const path = `parcele/${nrCadastral}/${fileName}`;
|
||||||
|
|||||||
Reference in New Issue
Block a user