fix(parcel-sync): robust county population + local feature count in dropdown
- PATCH /api/eterra/uats: handle nested responses (unwrapArray), try multiple field names (extractName/extractCode), log sample UAT for debugging, match by code first then by name - GET /api/eterra/uats: include localFeatures count per SIRUTA via GisFeature groupBy query - Dropdown: show green badge with local feature count, county with dash - Add SKILLS.md for ParcelSync/eTerra/GIS module context Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,219 @@
|
||||
# ParcelSync / eTerra GIS — Skills & Context
|
||||
|
||||
> This file is the **single source of truth** for any AI assistant working on
|
||||
> the ParcelSync module, eTerra integration, or GIS features. Read this FIRST
|
||||
> before touching any code in this area.
|
||||
|
||||
---
|
||||
|
||||
## Module Overview
|
||||
|
||||
ParcelSync connects to Romania's **eTerra / ANCPI** national cadastral system
|
||||
to fetch, store, enrich, and export parcel (land + building) data. All GIS data
|
||||
lives in a local **PostGIS** database via Prisma ORM.
|
||||
|
||||
**Key paths:**
|
||||
| Area | Path |
|
||||
|------|------|
|
||||
| Module root | `src/modules/parcel-sync/` |
|
||||
| Components | `src/modules/parcel-sync/components/parcel-sync-module.tsx` (~4100 lines, single file) |
|
||||
| Services | `src/modules/parcel-sync/services/` |
|
||||
| API routes | `src/app/api/eterra/` (20+ route files) |
|
||||
| Types | `src/modules/parcel-sync/types.ts` |
|
||||
| DB models | `prisma/schema.prisma` → `GisFeature`, `GisSyncRun`, `GisUat` |
|
||||
| Layer catalog | `services/eterra-layers.ts` — 23 layers, 4 categories |
|
||||
| Static UAT list | `public/uat.json` (~3000 entries, SIRUTA + name only) |
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### eTerra API Client (`services/eterra-client.ts`, ~1000 lines)
|
||||
|
||||
- **Auth**: form-post login → JSESSIONID cookie jar (axios-cookiejar-support)
|
||||
- **Session cache**: keyed by credential hash, 9-minute TTL, auto-relogin on 401/redirect
|
||||
- **Two API surfaces**:
|
||||
- **ArcGIS REST** (`/api/map/rest/{endpoint}/layer/{name}/query`) — spatial queries, geometry
|
||||
- **Application API** (`/api/immovable/...`, `/api/immApps/...`, `/api/adm/nomen/...`) — business data
|
||||
- **Pagination**: `maxRecordCount=1000`, fallback page sizes (500, 200)
|
||||
- **Key methods**:
|
||||
- `listLayer()` / `fetchAllLayer()` / `fetchAllLayerByWhere()` — ArcGIS layer queries
|
||||
- `countLayer()` — feature count
|
||||
- `getLayerFieldNames()` — discover available fields
|
||||
- `searchImmovableByIdentifier()` — parcel search by cadastral number
|
||||
- `fetchDocumentationData()` — CF, owners (partTwoRegs tree)
|
||||
- `fetchImmovableParcelDetails()` — area, intravilan, use categories
|
||||
- `fetchImmovableListByAdminUnit()` — all immovables in a UAT (paginated, Spring Boot Page format)
|
||||
- `fetchCounties()` — all counties from nomenclature (`/api/adm/nomen/COUNTY/list`)
|
||||
- `fetchAdminUnitsByCounty(nomenPk)` — UATs per county (`/api/adm/nomen/ADMINISTRATIVEUNIT/filterByParent/{pk}`)
|
||||
- `fetchNomenByPk(pk)` — single nomenclature entry
|
||||
|
||||
### Session Management (`services/session-store.ts`)
|
||||
|
||||
- Global singleton (one shared session for the whole app)
|
||||
- Stores username + password in memory only
|
||||
- Tracks active job IDs (blocks disconnect while jobs run)
|
||||
- `getSessionCredentials()` — used by all API routes
|
||||
|
||||
### Health Check (`services/eterra-health.ts`)
|
||||
|
||||
- Pings `eterra.ancpi.ro` every 3 minutes
|
||||
- Detects maintenance by keywords in HTML response
|
||||
- Blocks login attempts when down
|
||||
- UI shows amber "Mentenanță" state
|
||||
|
||||
---
|
||||
|
||||
## Database Models
|
||||
|
||||
### GisUat (UAT registry)
|
||||
```
|
||||
siruta String @id — SIRUTA code (NOT eTerra nomenPk!)
|
||||
name String — UAT name (from uat.json)
|
||||
county String? — County name (populated via PATCH /api/eterra/uats)
|
||||
workspacePk Int? — eTerra county workspace ID (= county nomenPk)
|
||||
```
|
||||
|
||||
**CRITICAL**: `eTerra nomenPk ≠ SIRUTA`. The nomenclature API uses `nomenPk` internally,
|
||||
but our DB keys on SIRUTA codes from `uat.json`. Matching is done by name or by
|
||||
resolving WORKSPACE_ID from ArcGIS layer features.
|
||||
|
||||
### GisFeature (parcel/building data)
|
||||
```
|
||||
id String @id @default(uuid())
|
||||
layerId String — e.g. "TERENURI_ACTIVE", "CLADIRI_ACTIVE"
|
||||
siruta String — UAT this feature belongs to
|
||||
objectId Int — eTerra OBJECTID (negative = no-geometry record)
|
||||
cadastralRef String? — National cadastral reference
|
||||
areaValue Float? — Area in sqm
|
||||
isActive Boolean
|
||||
attributes Json — All ArcGIS attributes (WORKSPACE_ID, ADMIN_UNIT_ID, etc.)
|
||||
geometry Json? — EsriGeometry { rings: number[][][] }
|
||||
geometrySource String? — "NO_GEOMETRY" for parcels without GIS geometry
|
||||
enrichment Json? — Scraped data: NR_CAD, NR_CF, PROPRIETARI, INTRAVILAN, etc.
|
||||
enrichedAt DateTime? — When enrichment was fetched
|
||||
@@unique([layerId, objectId])
|
||||
```
|
||||
|
||||
### GisSyncRun (sync history)
|
||||
```
|
||||
siruta, layerId, status, totalRemote, totalLocal, newFeatures, removedFeatures, etc.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Flows
|
||||
|
||||
### 1. UAT Selection & Autocomplete
|
||||
- On mount: `GET /api/eterra/uats` → loads all UATs from DB (fast, no eTerra)
|
||||
- If DB empty: seeds via `POST /api/eterra/uats` from `public/uat.json`
|
||||
- Client-side filter: diacritics-insensitive, matches SIRUTA prefix or name/county substring
|
||||
- Dropdown shows: **name – county** + local feature count badge + SIRUTA code
|
||||
- On select: sets `siruta` + `workspacePk` state
|
||||
|
||||
### 2. County Population (`PATCH /api/eterra/uats`)
|
||||
- **Phase 1**: UATs with `workspacePk` already → instant lookup via `fetchCounties()` map
|
||||
- **Phase 2**: Enumerate all counties → `fetchAdminUnitsByCounty()` per county → match by code then name
|
||||
- Auto-triggered on first eTerra connection when >50% UATs lack county
|
||||
- ~43 API calls total (1 counties + 42 per-county)
|
||||
- Name matching: NFD normalize, strip diacritics, uppercase
|
||||
|
||||
### 3. Workspace Resolution
|
||||
- `workspacePk` = eTerra county identifier (= county nomenPk from nomenclature)
|
||||
- Resolved lazily: query 1 feature from TERENURI_ACTIVE → read WORKSPACE_ID attribute
|
||||
- Cached in-memory + persisted to GisUat.workspacePk
|
||||
- Resolution chain: explicit param → GisUat DB → ArcGIS layer query
|
||||
|
||||
### 4. Layer Sync (`services/sync-service.ts`)
|
||||
- Background job: fetch all features from an ArcGIS layer for a SIRUTA
|
||||
- Uses UAT boundary geometry (from LIMITE_UAT) as spatial filter for dynamic layers
|
||||
- Stores features in GisFeature with full geometry
|
||||
- Progress tracking via `progress-store.ts` (2s polling)
|
||||
|
||||
### 5. Enrichment Pipeline (`services/enrich-service.ts`)
|
||||
- Per-feature: hits eTerra `/api/immovable/list` to extract detailed data
|
||||
- Stored in `GisFeature.enrichment` JSONB:
|
||||
- `NR_CAD`, `NR_CF`, `NR_CF_VECHI`, `NR_TOPO`
|
||||
- `PROPRIETARI`, `PROPRIETARI_VECHI` (semicolon-separated)
|
||||
- `SUPRAFATA`, `INTRAVILAN`, `CATEGORIE_FOLOSINTA`
|
||||
- `HAS_BUILDING`, `BUILD_LEGAL`
|
||||
- `ADRESA`, `SOLICITANT`
|
||||
|
||||
### 6. Parcel Search (`POST /api/eterra/search`)
|
||||
- Input: SIRUTA + cadastral number(s)
|
||||
- Resolves workspace, then calls:
|
||||
1. `searchImmovableByIdentifier()` — find immovable
|
||||
2. `fetchDocumentationData()` — CF, owners (active/cancelled via nodeStatus tree)
|
||||
3. `fetchImmovableParcelDetails()` — area, intravilan, use categories
|
||||
4. `fetchImmAppsByImmovable()` → `fetchParcelFolosinte()` — application data
|
||||
- Returns `ParcelDetail[]` with full property info
|
||||
|
||||
### 7. No-Geometry Sync (`services/no-geom-sync.ts`)
|
||||
- Finds eTerra immovables that have NO GIS geometry (no polygon in TERENURI_ACTIVE)
|
||||
- Cross-references immovable list against remote ArcGIS features
|
||||
- Stores as GisFeature with `geometry=null`, `objectId=-immovablePk`
|
||||
- Quality gate: must be active + hasLandbook + has identification/area
|
||||
|
||||
### 8. Export
|
||||
- **GeoPackage** (`services/gpkg-export.ts`): EPSG:3844 → EPSG:4326 reprojection
|
||||
- **Export bundle** (`/api/eterra/export-bundle`): ZIP with CSV + GPKG
|
||||
- **Local export** (`/api/eterra/export-local`): from local DB
|
||||
|
||||
---
|
||||
|
||||
## API Routes Reference
|
||||
|
||||
| Route | Method | Purpose |
|
||||
|-------|--------|---------|
|
||||
| `/api/eterra/uats` | GET | All UATs from DB (with local feature counts) |
|
||||
| `/api/eterra/uats` | POST | Seed DB from uat.json |
|
||||
| `/api/eterra/uats` | PATCH | Populate county from eTerra nomenclature |
|
||||
| `/api/eterra/login` | POST | Connect to eTerra |
|
||||
| `/api/eterra/session` | GET/DELETE | Session status / disconnect |
|
||||
| `/api/eterra/health` | GET | eTerra platform health |
|
||||
| `/api/eterra/search` | POST | Search parcels by cadastral number |
|
||||
| `/api/eterra/search-owner` | POST | Search by owner name |
|
||||
| `/api/eterra/layers` | GET | Available layers + field names |
|
||||
| `/api/eterra/count` | POST | Count features in layer |
|
||||
| `/api/eterra/sync` | POST | Start layer sync job |
|
||||
| `/api/eterra/sync-background` | POST | Background sync |
|
||||
| `/api/eterra/sync-status` | GET | Sync job progress |
|
||||
| `/api/eterra/progress` | GET | Generic job progress |
|
||||
| `/api/eterra/features` | GET | Query local features |
|
||||
| `/api/eterra/db-summary` | GET | Aggregate stats all UATs |
|
||||
| `/api/eterra/uat-dashboard` | GET | Per-UAT analytics |
|
||||
| `/api/eterra/export-bundle` | POST | Export ZIP (CSV + GPKG) |
|
||||
| `/api/eterra/export-layer-gpkg` | POST | Export single layer GPKG |
|
||||
| `/api/eterra/export-local` | POST | Export from local DB |
|
||||
| `/api/eterra/no-geom-scan` | POST | Scan for no-geometry parcels |
|
||||
| `/api/eterra/no-geom-debug` | POST | Debug no-geom data |
|
||||
| `/api/eterra/setup-postgis` | POST | Initialize PostGIS extensions |
|
||||
|
||||
---
|
||||
|
||||
## Critical Gotchas
|
||||
|
||||
1. **nomenPk ≠ SIRUTA**: eTerra's internal nomenclature PKs are NOT SIRUTA codes. Never assume they're the same.
|
||||
2. **workspacePk = county nomenPk**: The WORKSPACE_ID from ArcGIS layers equals the county's nomenclature PK from `fetchCounties()`.
|
||||
3. **Session TTL**: eTerra sessions expire after ~10 minutes. Client has 9-minute cache TTL + auto-relogin.
|
||||
4. **Maintenance windows**: eTerra goes down regularly. Health check detects this and blocks operations.
|
||||
5. **ArcGIS maxRecordCount=1000**: Always paginate with `resultOffset`/`resultRecordCount`. Page size fallbacks: 1000 → 500 → 200.
|
||||
6. **Admin field auto-discovery**: Different layers use different field names for SIRUTA (ADMIN_UNIT_ID, SIRUTA, UAT_ID, etc.). Always use `findAdminField()` / `buildWhere()`.
|
||||
7. **LIMITE_UAT endpoint is "all"**: Unlike other layers (endpoint "aut"), LIMITE_UAT uses the "all" endpoint — no workspace context needed.
|
||||
8. **No-geometry parcels use negative objectId**: `objectId = -immovablePk` to avoid collision with real ArcGIS OBJECTIDs.
|
||||
9. **Spring Boot Page responses**: Some eTerra APIs return `{content: [...], totalPages: N}` instead of flat arrays. Always use `unwrapArray()` helper.
|
||||
10. **Geometry format**: EsriGeometry `{ rings: number[][][] }` in EPSG:3844 (Stereo70). Reproject to EPSG:4326 for GeoPackage export.
|
||||
11. **Owner tree parsing**: Documentation API returns owners as a tree (C→A→I→P nodes). Check `nodeStatus === -1` up the parent chain to detect radiated/cancelled owners.
|
||||
12. **Local UatEntry type**: The component defines its own `UatEntry` type at the top of `parcel-sync-module.tsx` — this shadows the one in `types.ts`. Keep both in sync.
|
||||
|
||||
---
|
||||
|
||||
## Test UAT
|
||||
|
||||
**Feleacu** — SIRUTA `57582`, ~30k immovables, ~8k GIS features. Used as the primary test UAT.
|
||||
|
||||
---
|
||||
|
||||
## Last Updated
|
||||
|
||||
2026-03-22 — Added county population via nomenclature API, local feature count in UAT dropdown.
|
||||
@@ -67,6 +67,7 @@ type UatEntry = {
|
||||
name: string;
|
||||
county?: string;
|
||||
workspacePk?: number;
|
||||
localFeatures?: number;
|
||||
};
|
||||
|
||||
type SessionStatus = {
|
||||
@@ -1770,19 +1771,21 @@ export function ParcelSyncModule() {
|
||||
setSearchResults([]);
|
||||
}}
|
||||
>
|
||||
<span>
|
||||
<span className="flex items-center gap-1.5 min-w-0">
|
||||
<span className="font-medium">{item.name}</span>
|
||||
<span className="text-muted-foreground ml-1.5">
|
||||
({item.siruta})
|
||||
</span>
|
||||
{item.county && (
|
||||
<span className="text-muted-foreground">
|
||||
,{" "}
|
||||
–{" "}
|
||||
<span className="font-medium text-foreground/70">
|
||||
{item.county}
|
||||
</span>
|
||||
</span>
|
||||
)}
|
||||
{(item.localFeatures ?? 0) > 0 && (
|
||||
<span className="inline-flex items-center rounded-full bg-emerald-500/15 px-1.5 py-0.5 text-[10px] font-medium text-emerald-600 dark:text-emerald-400 shrink-0">
|
||||
{(item.localFeatures ?? 0).toLocaleString("ro")} local
|
||||
</span>
|
||||
)}
|
||||
</span>
|
||||
<span className="text-xs text-muted-foreground font-mono ml-2 shrink-0">
|
||||
{item.siruta}
|
||||
|
||||
@@ -7,6 +7,8 @@ export type UatEntry = {
|
||||
name: string;
|
||||
county?: string;
|
||||
workspacePk?: number;
|
||||
/** Number of GIS features synced locally for this UAT */
|
||||
localFeatures?: number;
|
||||
};
|
||||
|
||||
export type LayerSyncStatus = {
|
||||
|
||||
Reference in New Issue
Block a user