fix(parcel-sync): robust county population + local feature count in dropdown

- PATCH /api/eterra/uats: handle nested responses (unwrapArray), try multiple field names (extractName/extractCode), log sample UAT for debugging, match by code first then by name - GET /api/eterra/uats: include localFeatures count per SIRUTA via GisFeature groupBy query - Dropdown: show green badge with local feature count, county with dash - Add SKILLS.md for ParcelSync/eTerra/GIS module context Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 21:18:22 +02:00
parent 79750b2a4a
commit 431291c410
4 changed files with 399 additions and 51 deletions
@@ -0,0 +1,219 @@
+# ParcelSync / eTerra GIS — Skills & Context
+
+> This file is the **single source of truth** for any AI assistant working on
+> the ParcelSync module, eTerra integration, or GIS features. Read this FIRST
+> before touching any code in this area.
+
+---
+
+## Module Overview
+
+ParcelSync connects to Romania's **eTerra / ANCPI** national cadastral system
+to fetch, store, enrich, and export parcel (land + building) data. All GIS data
+lives in a local **PostGIS** database via Prisma ORM.
+
+**Key paths:**
+| Area | Path |
+|------|------|
+| Module root | `src/modules/parcel-sync/` |
+| Components | `src/modules/parcel-sync/components/parcel-sync-module.tsx` (~4100 lines, single file) |
+| Services | `src/modules/parcel-sync/services/` |
+| API routes | `src/app/api/eterra/` (20+ route files) |
+| Types | `src/modules/parcel-sync/types.ts` |
+| DB models | `prisma/schema.prisma` → `GisFeature`, `GisSyncRun`, `GisUat` |
+| Layer catalog | `services/eterra-layers.ts` — 23 layers, 4 categories |
+| Static UAT list | `public/uat.json` (~3000 entries, SIRUTA + name only) |
+
+---
+
+## Architecture
+
+### eTerra API Client (`services/eterra-client.ts`, ~1000 lines)
+
+- **Auth**: form-post login → JSESSIONID cookie jar (axios-cookiejar-support)
+- **Session cache**: keyed by credential hash, 9-minute TTL, auto-relogin on 401/redirect
+- **Two API surfaces**:
+  - **ArcGIS REST** (`/api/map/rest/{endpoint}/layer/{name}/query`) — spatial queries, geometry
+  - **Application API** (`/api/immovable/...`, `/api/immApps/...`, `/api/adm/nomen/...`) — business data
+- **Pagination**: `maxRecordCount=1000`, fallback page sizes (500, 200)
+- **Key methods**:
+  - `listLayer()` / `fetchAllLayer()` / `fetchAllLayerByWhere()` — ArcGIS layer queries
+  - `countLayer()` — feature count
+  - `getLayerFieldNames()` — discover available fields
+  - `searchImmovableByIdentifier()` — parcel search by cadastral number
+  - `fetchDocumentationData()` — CF, owners (partTwoRegs tree)
+  - `fetchImmovableParcelDetails()` — area, intravilan, use categories
+  - `fetchImmovableListByAdminUnit()` — all immovables in a UAT (paginated, Spring Boot Page format)
+  - `fetchCounties()` — all counties from nomenclature (`/api/adm/nomen/COUNTY/list`)
+  - `fetchAdminUnitsByCounty(nomenPk)` — UATs per county (`/api/adm/nomen/ADMINISTRATIVEUNIT/filterByParent/{pk}`)
+  - `fetchNomenByPk(pk)` — single nomenclature entry
+
+### Session Management (`services/session-store.ts`)
+
+- Global singleton (one shared session for the whole app)
+- Stores username + password in memory only
+- Tracks active job IDs (blocks disconnect while jobs run)
+- `getSessionCredentials()` — used by all API routes
+
+### Health Check (`services/eterra-health.ts`)
+
+- Pings `eterra.ancpi.ro` every 3 minutes
+- Detects maintenance by keywords in HTML response
+- Blocks login attempts when down
+- UI shows amber "Mentenanță" state
+
+---
+
+## Database Models
+
+### GisUat (UAT registry)
+```
+siruta     String  @id      — SIRUTA code (NOT eTerra nomenPk!)
+name       String           — UAT name (from uat.json)
+county     String?          — County name (populated via PATCH /api/eterra/uats)
+workspacePk Int?            — eTerra county workspace ID (= county nomenPk)
+```
+
+**CRITICAL**: `eTerra nomenPk ≠ SIRUTA`. The nomenclature API uses `nomenPk` internally,
+but our DB keys on SIRUTA codes from `uat.json`. Matching is done by name or by
+resolving WORKSPACE_ID from ArcGIS layer features.
+
+### GisFeature (parcel/building data)
+```
+id             String    @id @default(uuid())
+layerId        String    — e.g. "TERENURI_ACTIVE", "CLADIRI_ACTIVE"
+siruta         String    — UAT this feature belongs to
+objectId       Int       — eTerra OBJECTID (negative = no-geometry record)
+cadastralRef   String?   — National cadastral reference
+areaValue      Float?    — Area in sqm
+isActive       Boolean
+attributes     Json      — All ArcGIS attributes (WORKSPACE_ID, ADMIN_UNIT_ID, etc.)
+geometry       Json?     — EsriGeometry { rings: number[][][] }
+geometrySource String?   — "NO_GEOMETRY" for parcels without GIS geometry
+enrichment     Json?     — Scraped data: NR_CAD, NR_CF, PROPRIETARI, INTRAVILAN, etc.
+enrichedAt     DateTime? — When enrichment was fetched
+@@unique([layerId, objectId])
+```
+
+### GisSyncRun (sync history)
+```
+siruta, layerId, status, totalRemote, totalLocal, newFeatures, removedFeatures, etc.
+```
+
+---
+
+## Key Flows
+
+### 1. UAT Selection & Autocomplete
+- On mount: `GET /api/eterra/uats` → loads all UATs from DB (fast, no eTerra)
+- If DB empty: seeds via `POST /api/eterra/uats` from `public/uat.json`
+- Client-side filter: diacritics-insensitive, matches SIRUTA prefix or name/county substring
+- Dropdown shows: **name – county** + local feature count badge + SIRUTA code
+- On select: sets `siruta` + `workspacePk` state
+
+### 2. County Population (`PATCH /api/eterra/uats`)
+- **Phase 1**: UATs with `workspacePk` already → instant lookup via `fetchCounties()` map
+- **Phase 2**: Enumerate all counties → `fetchAdminUnitsByCounty()` per county → match by code then name
+- Auto-triggered on first eTerra connection when >50% UATs lack county
+- ~43 API calls total (1 counties + 42 per-county)
+- Name matching: NFD normalize, strip diacritics, uppercase
+
+### 3. Workspace Resolution
+- `workspacePk` = eTerra county identifier (= county nomenPk from nomenclature)
+- Resolved lazily: query 1 feature from TERENURI_ACTIVE → read WORKSPACE_ID attribute
+- Cached in-memory + persisted to GisUat.workspacePk
+- Resolution chain: explicit param → GisUat DB → ArcGIS layer query
+
+### 4. Layer Sync (`services/sync-service.ts`)
+- Background job: fetch all features from an ArcGIS layer for a SIRUTA
+- Uses UAT boundary geometry (from LIMITE_UAT) as spatial filter for dynamic layers
+- Stores features in GisFeature with full geometry
+- Progress tracking via `progress-store.ts` (2s polling)
+
+### 5. Enrichment Pipeline (`services/enrich-service.ts`)
+- Per-feature: hits eTerra `/api/immovable/list` to extract detailed data
+- Stored in `GisFeature.enrichment` JSONB:
+  - `NR_CAD`, `NR_CF`, `NR_CF_VECHI`, `NR_TOPO`
+  - `PROPRIETARI`, `PROPRIETARI_VECHI` (semicolon-separated)
+  - `SUPRAFATA`, `INTRAVILAN`, `CATEGORIE_FOLOSINTA`
+  - `HAS_BUILDING`, `BUILD_LEGAL`
+  - `ADRESA`, `SOLICITANT`
+
+### 6. Parcel Search (`POST /api/eterra/search`)
+- Input: SIRUTA + cadastral number(s)
+- Resolves workspace, then calls:
+  1. `searchImmovableByIdentifier()` — find immovable
+  2. `fetchDocumentationData()` — CF, owners (active/cancelled via nodeStatus tree)
+  3. `fetchImmovableParcelDetails()` — area, intravilan, use categories
+  4. `fetchImmAppsByImmovable()` → `fetchParcelFolosinte()` — application data
+- Returns `ParcelDetail[]` with full property info
+
+### 7. No-Geometry Sync (`services/no-geom-sync.ts`)
+- Finds eTerra immovables that have NO GIS geometry (no polygon in TERENURI_ACTIVE)
+- Cross-references immovable list against remote ArcGIS features
+- Stores as GisFeature with `geometry=null`, `objectId=-immovablePk`
+- Quality gate: must be active + hasLandbook + has identification/area
+
+### 8. Export
+- **GeoPackage** (`services/gpkg-export.ts`): EPSG:3844 → EPSG:4326 reprojection
+- **Export bundle** (`/api/eterra/export-bundle`): ZIP with CSV + GPKG
+- **Local export** (`/api/eterra/export-local`): from local DB
+
+---
+
+## API Routes Reference
+
+| Route | Method | Purpose |
+|-------|--------|---------|
+| `/api/eterra/uats` | GET | All UATs from DB (with local feature counts) |
+| `/api/eterra/uats` | POST | Seed DB from uat.json |
+| `/api/eterra/uats` | PATCH | Populate county from eTerra nomenclature |
+| `/api/eterra/login` | POST | Connect to eTerra |
+| `/api/eterra/session` | GET/DELETE | Session status / disconnect |
+| `/api/eterra/health` | GET | eTerra platform health |
+| `/api/eterra/search` | POST | Search parcels by cadastral number |
+| `/api/eterra/search-owner` | POST | Search by owner name |
+| `/api/eterra/layers` | GET | Available layers + field names |
+| `/api/eterra/count` | POST | Count features in layer |
+| `/api/eterra/sync` | POST | Start layer sync job |
+| `/api/eterra/sync-background` | POST | Background sync |
+| `/api/eterra/sync-status` | GET | Sync job progress |
+| `/api/eterra/progress` | GET | Generic job progress |
+| `/api/eterra/features` | GET | Query local features |
+| `/api/eterra/db-summary` | GET | Aggregate stats all UATs |
+| `/api/eterra/uat-dashboard` | GET | Per-UAT analytics |
+| `/api/eterra/export-bundle` | POST | Export ZIP (CSV + GPKG) |
+| `/api/eterra/export-layer-gpkg` | POST | Export single layer GPKG |
+| `/api/eterra/export-local` | POST | Export from local DB |
+| `/api/eterra/no-geom-scan` | POST | Scan for no-geometry parcels |
+| `/api/eterra/no-geom-debug` | POST | Debug no-geom data |
+| `/api/eterra/setup-postgis` | POST | Initialize PostGIS extensions |
+
+---
+
+## Critical Gotchas
+
+1. **nomenPk ≠ SIRUTA**: eTerra's internal nomenclature PKs are NOT SIRUTA codes. Never assume they're the same.
+2. **workspacePk = county nomenPk**: The WORKSPACE_ID from ArcGIS layers equals the county's nomenclature PK from `fetchCounties()`.
+3. **Session TTL**: eTerra sessions expire after ~10 minutes. Client has 9-minute cache TTL + auto-relogin.
+4. **Maintenance windows**: eTerra goes down regularly. Health check detects this and blocks operations.
+5. **ArcGIS maxRecordCount=1000**: Always paginate with `resultOffset`/`resultRecordCount`. Page size fallbacks: 1000 → 500 → 200.
+6. **Admin field auto-discovery**: Different layers use different field names for SIRUTA (ADMIN_UNIT_ID, SIRUTA, UAT_ID, etc.). Always use `findAdminField()` / `buildWhere()`.
+7. **LIMITE_UAT endpoint is "all"**: Unlike other layers (endpoint "aut"), LIMITE_UAT uses the "all" endpoint — no workspace context needed.
+8. **No-geometry parcels use negative objectId**: `objectId = -immovablePk` to avoid collision with real ArcGIS OBJECTIDs.
+9. **Spring Boot Page responses**: Some eTerra APIs return `{content: [...], totalPages: N}` instead of flat arrays. Always use `unwrapArray()` helper.
+10. **Geometry format**: EsriGeometry `{ rings: number[][][] }` in EPSG:3844 (Stereo70). Reproject to EPSG:4326 for GeoPackage export.
+11. **Owner tree parsing**: Documentation API returns owners as a tree (C→A→I→P nodes). Check `nodeStatus === -1` up the parent chain to detect radiated/cancelled owners.
+12. **Local UatEntry type**: The component defines its own `UatEntry` type at the top of `parcel-sync-module.tsx` — this shadows the one in `types.ts`. Keep both in sync.
+
+---
+
+## Test UAT
+
+**Feleacu** — SIRUTA `57582`, ~30k immovables, ~8k GIS features. Used as the primary test UAT.
+
+---
+
+## Last Updated
+
+2026-03-22 — Added county population via nomenclature API, local feature count in UAT dropdown.