From 88754250a80f1900eb54f3a446662c958c0da835 Mon Sep 17 00:00:00 2001 From: AI Assistant Date: Mon, 23 Mar 2026 13:57:42 +0200 Subject: [PATCH] docs: update CLAUDE.md + SKILLS.md with ePay integration, performance fixes - ParcelSync version 0.6.0 with ePay CF extract ordering - ANCPI ePay in Current Integrations table - Static WORKSPACE_TO_COUNTY mapping documented - GisUat geometry select optimization documented - Feature count cache (5-min TTL) documented - ePay endpoint gotchas, auth flow, order flow - Cleaned outdated info, focused on actionable gotchas Co-Authored-By: Claude Opus 4.6 (1M context) --- CLAUDE.md | 29 ++- src/modules/parcel-sync/SKILLS.md | 378 ++++++++++++++---------------- 2 files changed, 204 insertions(+), 203 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 5847129..677ee0a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -113,7 +113,7 @@ legacy/ # Original HTML tools for reference | 12 | **Word Templates** | `/word-templates` | 0.1.0 | Template library, 8 categories, version tracking, .docx placeholder auto-detection | | 13 | **AI Chat** | `/ai-chat` | 0.2.0 | Multi-provider (OpenAI/Claude/Ollama/demo), **project linking via Tag Manager**, provider status badge | | 14 | **Hot Desk** | `/hot-desk` | 0.1.1 | 4 desks, week-ahead calendar, room layout (window+door proportioned), reserve/cancel | -| 15 | **ParcelSync** | `/parcel-sync` | 0.5.0 | eTerra ANCPI integration, **PostGIS database**, background sync, 23-layer catalog, enrichment pipeline, owner search, **per-UAT analytics dashboard**, **health check + maintenance detection** | +| 15 | **ParcelSync** | `/parcel-sync` | 0.6.0 | eTerra ANCPI integration, **PostGIS database**, background sync, 23-layer catalog, enrichment pipeline, owner search, **per-UAT analytics dashboard**, **health check + maintenance detection**, **ANCPI ePay CF extract ordering** (batch orders, MinIO PDF storage, dedup protection, credit tracking), **static WORKSPACE_TO_COUNTY mapping**, **GisUat geometry select optimization**, **feature count cache (5-min TTL)** | | 16 | **Visual Copilot** | `/visual-copilot` | 0.1.0 | AI-powered image analysis — **developed in separate repo** (`https://git.beletage.ro/gitadmin/vim`), placeholder in ArchiTools, will be merged as module later | ### Registratura — Legal Deadline Tracking (Termene Legale) @@ -214,6 +214,9 @@ The ParcelSync module connects to Romania's national eTerra/ANCPI cadastral syst - **Owner search**: DB-first (ILIKE on enrichment JSON) with eTerra API fallback - **Per-UAT dashboard**: SQL aggregates (area stats, intravilan/extravilan, land use, top owners), CSS-only visualizations (donut ring, bar charts) - **Health check** (`eterra-health.ts`): pings `eterra.ancpi.ro` every 3min, detects maintenance by keywords in HTML response, blocks login when down, UI shows amber "Mentenanță" state +- **ANCPI ePay CF extract ordering**: batch orders via `epay-client.ts`, PDF storage to MinIO, dedup protection (queue + API level), credit tracking +- **Static county mapping**: `WORKSPACE_TO_COUNTY` in `county-refresh.ts` — 42 verified entries, preferred over unreliable nomenclature API +- **Performance**: GisUat queries use `select` to exclude geometry column; feature counts cached 5-min TTL - **Test UAT**: Feleacu (SIRUTA 57582, ~30k immovables, ~8k GIS features) Key files: @@ -224,8 +227,15 @@ Key files: - `services/enrich-service.ts` — Enrichment pipeline (FeatureEnrichment type) - `services/eterra-health.ts` — Health check singleton, maintenance detection - `services/session-store.ts` — Server-side session management -- `components/parcel-sync-module.tsx` — Main UI (~4100 lines), 4 tabs (Export/Layers/Search/DB) +- `services/epay-client.ts` — ePay HTTP client (login, cart, metadata, submit, poll, download) +- `services/epay-queue.ts` — Batch queue with dedup protection +- `services/epay-storage.ts` — MinIO storage helpers for CF extract PDFs +- `services/epay-counties.ts` — County index mapping (eTerra county name → ePay alphabetical index 0-41) +- `app/api/eterra/session/county-refresh.ts` — Static `WORKSPACE_TO_COUNTY` mapping, LIMITE_UAT geometry refresh +- `components/parcel-sync-module.tsx` — Main UI (~4100 lines), 5 tabs (Export/Layers/Search/DB/Extrase CF) - `components/uat-dashboard.tsx` — Per-UAT analytics dashboard (CSS-only charts) +- `components/epay-tab.tsx` — CF extract ordering tab +- `components/epay-connect.tsx` — ePay connection widget --- @@ -353,6 +363,20 @@ src/modules// - **Never hardcode timeouts too low** — eTerra 1000-feature geometry pages can take 60-90s; default is 120s - **CookieJar + axios-cookiejar-support** required for eTerra auth (JSESSIONID tracking) - **Page size fallbacks**: if 1000 fails, retry with 500, then 200 +- **WORKSPACE_TO_COUNTY is the authoritative county mapping** — static 42-entry map in `county-refresh.ts`, preferred over `fetchCounties()` which 404s intermittently +- **GisUat.geometry is huge** — always use Prisma `select` to exclude it in list queries; forgetting this turns 50ms into 5+ seconds +- **Feature counts are expensive** — cached in global with 5-min TTL in UATs route; returns stale data while refreshing + +### ANCPI ePay Rules + +- **ePay county IDs = eTerra WORKSPACE_IDs** (CLUJ=127, ALBA=10) — zero discovery calls needed +- **ePay UAT IDs = SIRUTA codes** — use `GisUat.workspacePk` + `siruta` directly +- **EpayJsonInterceptor uses form-urlencoded** (NOT JSON body) — `reqType=nomenclatorUAT&countyId=127` +- **saveProductMetadataForBasketItem uses multipart/form-data** (form-data npm package) +- **Document IDs are HTML-encoded** in ShowOrderDetails — `"idDocument":47301767` must be decoded before JSON parse +- **ePay auth is OpenAM** — gets `AMAuthCookie`, then navigate to `http://` (not https) for JSESSIONID +- **MinIO metadata must be ASCII** — strip diacritics from values before storing +- Env vars: `ANCPI_USERNAME`, `ANCPI_PASSWORD`, `ANCPI_BASE_URL`, `ANCPI_LOGIN_URL`, `ANCPI_DEFAULT_SOLICITANT_ID`, `MINIO_BUCKET_ANCPI` ### Before Pushing @@ -386,6 +410,7 @@ src/modules// | **ManicTime Sync** | ✅ Implemented | `/api/manictime` — bidirectional Tags.txt sync, needs SMB mount | | **NAS Paths** | ✅ Active | `\\newamun` (10.10.10.10), drives A/O/P/T, hostname+IP fallback, `src/config/nas-paths.ts` | | **eTerra ANCPI** | ✅ Active | ParcelSync module, `eterra-client.ts`, health check + maintenance detection | +| **ANCPI ePay** | ✅ Active | CF extract ordering, `epay-client.ts`, MinIO PDF storage, batch queue + dedup, `/api/ancpi/*` routes | | **PostGIS** | ✅ Active | `GisFeature` model, geometry storage, spatial queries, used by ParcelSync | | **Email Notifications** | ✅ Implemented | Brevo SMTP daily digest, `/api/notifications/digest` + `/preferences`, N8N cron trigger | | **N8N automations** | ✅ Active (digest cron) | Daily digest cron `0 8 * * 1-5`, Bearer token auth, future: backups, workflows | diff --git a/src/modules/parcel-sync/SKILLS.md b/src/modules/parcel-sync/SKILLS.md index 9db4c25..4fdfd1f 100644 --- a/src/modules/parcel-sync/SKILLS.md +++ b/src/modules/parcel-sync/SKILLS.md @@ -1,28 +1,26 @@ # ParcelSync / eTerra GIS — Skills & Context -> This file is the **single source of truth** for any AI assistant working on -> the ParcelSync module, eTerra integration, or GIS features. Read this FIRST -> before touching any code in this area. +> Single source of truth for AI assistants working on ParcelSync, eTerra, or ANCPI ePay. --- ## Module Overview -ParcelSync connects to Romania's **eTerra / ANCPI** national cadastral system -to fetch, store, enrich, and export parcel (land + building) data. All GIS data -lives in a local **PostGIS** database via Prisma ORM. +ParcelSync connects to Romania's **eTerra / ANCPI** cadastral system to fetch, store, enrich, and export parcel data. GIS data lives in **PostGIS** via Prisma ORM. CF extract ordering uses **ANCPI ePay** (separate auth, separate API). **Key paths:** | Area | Path | |------|------| | Module root | `src/modules/parcel-sync/` | -| Components | `src/modules/parcel-sync/components/parcel-sync-module.tsx` (~4100 lines, single file) | -| Services | `src/modules/parcel-sync/services/` | -| API routes | `src/app/api/eterra/` (20+ route files) | -| Types | `src/modules/parcel-sync/types.ts` | -| DB models | `prisma/schema.prisma` → `GisFeature`, `GisSyncRun`, `GisUat` | +| Main component | `components/parcel-sync-module.tsx` (~4100 lines) | +| Services | `services/` (eterra-client, sync, enrich, epay-*, etc.) | +| API routes — eTerra | `src/app/api/eterra/` (20+ routes) | +| API routes — ePay | `src/app/api/ancpi/` (7 routes) | +| Types | `types.ts` | +| DB models | `prisma/schema.prisma` — `GisFeature`, `GisSyncRun`, `GisUat`, `CfExtract` | | Layer catalog | `services/eterra-layers.ts` — 23 layers, 4 categories | -| Static UAT list | `public/uat.json` (~3000 entries, SIRUTA + name only) | +| Static UAT list | `public/uat.json` (~3000 entries, SIRUTA + name) | +| County mapping | `src/app/api/eterra/session/county-refresh.ts` — `WORKSPACE_TO_COUNTY` (static, verified) | --- @@ -30,145 +28,211 @@ lives in a local **PostGIS** database via Prisma ORM. ### eTerra API Client (`services/eterra-client.ts`, ~1000 lines) -- **Auth**: form-post login → JSESSIONID cookie jar (axios-cookiejar-support) -- **Session cache**: keyed by credential hash, 9-minute TTL, auto-relogin on 401/redirect +- **Auth**: form-post login, JSESSIONID cookie jar (axios-cookiejar-support) +- **Session cache**: keyed by credential hash, 9-min TTL, auto-relogin on 401/redirect - **Two API surfaces**: - **ArcGIS REST** (`/api/map/rest/{endpoint}/layer/{name}/query`) — spatial queries, geometry - **Application API** (`/api/immovable/...`, `/api/immApps/...`, `/api/adm/nomen/...`) — business data - **Pagination**: `maxRecordCount=1000`, fallback page sizes (500, 200) -- **Key methods**: - - `listLayer()` / `fetchAllLayer()` / `fetchAllLayerByWhere()` — ArcGIS layer queries - - `countLayer()` — feature count - - `getLayerFieldNames()` — discover available fields - - `searchImmovableByIdentifier()` — parcel search by cadastral number - - `fetchDocumentationData()` — CF, owners (partTwoRegs tree) - - `fetchImmovableParcelDetails()` — area, intravilan, use categories - - `fetchImmovableListByAdminUnit()` — all immovables in a UAT (paginated, Spring Boot Page format) - - `fetchCounties()` — all counties from nomenclature (`/api/adm/nomen/COUNTY/list`) - - `fetchAdminUnitsByCounty(nomenPk)` — UATs per county (`/api/adm/nomen/ADMINISTRATIVEUNIT/filterByParent/{pk}`) - - `fetchNomenByPk(pk)` — single nomenclature entry +- **Key methods**: `listLayer()`, `fetchAllLayer()`, `fetchAllLayerByWhere()`, `countLayer()`, `searchImmovableByIdentifier()`, `fetchDocumentationData()`, `fetchImmovableParcelDetails()`, `fetchImmovableListByAdminUnit()`, `fetchCounties()`, `fetchAdminUnitsByCounty(nomenPk)` ### Session Management (`services/session-store.ts`) -- Global singleton (one shared session for the whole app) -- Stores username + password in memory only -- Tracks active job IDs (blocks disconnect while jobs run) -- `getSessionCredentials()` — used by all API routes +Global singleton, stores username + password in memory, tracks active job IDs (blocks disconnect while running). ### Health Check (`services/eterra-health.ts`) -- Pings `eterra.ancpi.ro` every 3 minutes -- Detects maintenance by keywords in HTML response -- Blocks login attempts when down -- UI shows amber "Mentenanță" state +Pings `eterra.ancpi.ro` every 3min, detects maintenance by keywords in HTML, blocks login when down, UI shows amber state. + +### County Resolution + +**Primary method**: Static `WORKSPACE_TO_COUNTY` mapping in `county-refresh.ts` — 42 entries, maps WORKSPACE_ID to county name. Populated on first eTerra login via LIMITE_UAT layer query. + +**Legacy method**: `fetchCounties()` exists in eterra-client but the nomenclature API (`/api/adm/nomen/COUNTY/list`) is unreliable (404s intermittently). The PATCH `/api/eterra/uats` route still uses it as fallback. Prefer the static mapping. --- ## Database Models -### GisUat (UAT registry) +### GisUat ``` -siruta String @id — SIRUTA code (NOT eTerra nomenPk!) -name String — UAT name (from uat.json) -county String? — County name (populated via PATCH /api/eterra/uats) -workspacePk Int? — eTerra county workspace ID (= county nomenPk) +siruta String @id — SIRUTA code (NOT eTerra nomenPk!) +name String +county String? +workspacePk Int? — eTerra county WORKSPACE_ID +geometry Json? — LIMITE_UAT boundary polygon (EPSG:3844) +areaValue Float? +lastUpdatedDtm String? ``` -**CRITICAL**: `eTerra nomenPk ≠ SIRUTA`. The nomenclature API uses `nomenPk` internally, -but our DB keys on SIRUTA codes from `uat.json`. Matching is done by name or by -resolving WORKSPACE_ID from ArcGIS layer features. +**PERFORMANCE CRITICAL**: `geometry` column stores huge polygon data. ALWAYS use `select` to exclude it in list queries. The GET `/api/eterra/uats` route uses `select: { siruta: true, name: true, county: true, workspacePk: true }` — never `findMany()` without select. -### GisFeature (parcel/building data) +**Feature counts cache**: The groupBy query for per-UAT feature counts is expensive (~25s). The UATs route caches it in a global with 5-min TTL, returning stale data while refreshing in background. + +### GisFeature ``` id String @id @default(uuid()) -layerId String — e.g. "TERENURI_ACTIVE", "CLADIRI_ACTIVE" -siruta String — UAT this feature belongs to +layerId String — "TERENURI_ACTIVE", "CLADIRI_ACTIVE", etc. +siruta String objectId Int — eTerra OBJECTID (negative = no-geometry record) -cadastralRef String? — National cadastral reference -areaValue Float? — Area in sqm +cadastralRef String? +areaValue Float? isActive Boolean -attributes Json — All ArcGIS attributes (WORKSPACE_ID, ADMIN_UNIT_ID, etc.) +attributes Json — ArcGIS attributes (WORKSPACE_ID, ADMIN_UNIT_ID, etc.) geometry Json? — EsriGeometry { rings: number[][][] } geometrySource String? — "NO_GEOMETRY" for parcels without GIS geometry -enrichment Json? — Scraped data: NR_CAD, NR_CF, PROPRIETARI, INTRAVILAN, etc. -enrichedAt DateTime? — When enrichment was fetched +enrichment Json? — NR_CAD, NR_CF, PROPRIETARI, INTRAVILAN, etc. +enrichedAt DateTime? @@unique([layerId, objectId]) ``` -### GisSyncRun (sync history) +### GisSyncRun ``` siruta, layerId, status, totalRemote, totalLocal, newFeatures, removedFeatures, etc. ``` +### CfExtract (ANCPI ePay orders) +``` +id String @id @default(uuid()) +orderId String? — ePay orderId (shared across batch items, NOT unique) +basketRowId Int? +nrCadastral String +nrCF String? +siruta String? +judetIndex Int — ePay county index (0-41, alphabetical) +judetName String +uatId Int — ePay UAT ID = SIRUTA code +uatName String +prodId Int @default(14200) +solicitantId String @default("14452") — Beletage persoana juridica +status String @default("pending") — pending|queued|cart|searching|ordering|polling|downloading|completed|failed|cancelled +epayStatus String? +idDocument Int? +documentName String? +documentDate DateTime? +minioPath String? — parcele/{nrCadastral}/{idx}_Extras CF_{nrCadastral} - {DD-MM-YYYY}.pdf +minioIndex Int? +creditsUsed Int @default(1) +immovableId String? +immovableType String? +measuredArea String? +legalArea String? +address String? +gisFeatureId String? +version Int @default(1) — increments on re-order +expiresAt DateTime? — 30 days after documentDate +supersededById String? +requestedBy String? +errorMessage String? +pollAttempts Int @default(0) +``` + --- -## Key Flows +## Critical Gotchas -### 1. UAT Selection & Autocomplete -- On mount: `GET /api/eterra/uats` → loads all UATs from DB (fast, no eTerra) -- If DB empty: seeds via `POST /api/eterra/uats` from `public/uat.json` -- Client-side filter: diacritics-insensitive, matches SIRUTA prefix or name/county substring -- Dropdown shows: **name – county** + local feature count badge + SIRUTA code -- On select: sets `siruta` + `workspacePk` state +1. **nomenPk is NOT SIRUTA**: eTerra's internal nomenclature PKs differ from SIRUTA codes. Never assume they're the same. +2. **WORKSPACE_ID = county ID everywhere**: Both eTerra ArcGIS and ePay use the same WORKSPACE_IDs for counties (CLUJ=127, ALBA=10, etc.). The static `WORKSPACE_TO_COUNTY` mapping is the authoritative source. +3. **ePay county IDs = WORKSPACE_IDs, ePay UAT IDs = SIRUTA codes**: Zero discovery calls needed for ordering — use `GisUat.workspacePk` + `siruta` directly. +4. **GisUat.geometry is huge**: Always `select` only needed fields. Forgetting this turns a 50ms query into 5+ seconds. +5. **Feature counts are cached**: 5-min TTL global cache in UATs route. Returns stale data while refreshing. +6. **Session TTL**: eTerra expires after ~10min, client has 9-min cache + auto-relogin. +7. **Maintenance windows**: eTerra goes down regularly. Health check detects and blocks. +8. **ArcGIS maxRecordCount=1000**: Always paginate. Fallback: 1000 -> 500 -> 200. +9. **Admin field auto-discovery**: Different layers use different SIRUTA field names. Use `findAdminField()` / `buildWhere()`. +10. **LIMITE_UAT uses endpoint "all"**: No workspace context needed (unlike other layers using "aut"). +11. **No-geometry parcels**: `objectId = -immovablePk` to avoid collision with real ArcGIS OBJECTIDs. +12. **Spring Boot Page responses**: Some APIs return `{content: [...]}` instead of arrays. Use `unwrapArray()`. +13. **Geometry in EPSG:3844**: Reproject to EPSG:4326 for GeoPackage export. +14. **Owner tree parsing**: Documentation API returns tree (C->A->I->P). Check `nodeStatus === -1` up parent chain for cancelled owners. -### 2. County Population (`PATCH /api/eterra/uats`) -- **Phase 1**: UATs with `workspacePk` already → instant lookup via `fetchCounties()` map -- **Phase 2**: Enumerate all counties → `fetchAdminUnitsByCounty()` per county → match by code then name -- Auto-triggered on first eTerra connection when >50% UATs lack county -- ~43 API calls total (1 counties + 42 per-county) -- Name matching: NFD normalize, strip diacritics, uppercase +--- -### 3. Workspace Resolution -- `workspacePk` = eTerra county identifier (= county nomenPk from nomenclature) -- Resolved lazily: query 1 feature from TERENURI_ACTIVE → read WORKSPACE_ID attribute -- Cached in-memory + persisted to GisUat.workspacePk -- Resolution chain: explicit param → GisUat DB → ArcGIS layer query +## ANCPI ePay — CF Extract Ordering -### 4. Layer Sync (`services/sync-service.ts`) -- Background job: fetch all features from an ArcGIS layer for a SIRUTA -- Uses UAT boundary geometry (from LIMITE_UAT) as spatial filter for dynamic layers -- Stores features in GisFeature with full geometry -- Progress tracking via `progress-store.ts` (2s polling) +### Overview -### 5. Enrichment Pipeline (`services/enrich-service.ts`) -- Per-feature: hits eTerra `/api/immovable/list` to extract detailed data -- Stored in `GisFeature.enrichment` JSONB: - - `NR_CAD`, `NR_CF`, `NR_CF_VECHI`, `NR_TOPO` - - `PROPRIETARI`, `PROPRIETARI_VECHI` (semicolon-separated) - - `SUPRAFATA`, `INTRAVILAN`, `CATEGORIE_FOLOSINTA` - - `HAS_BUILDING`, `BUILD_LEGAL` - - `ADRESA`, `SOLICITANT` +Orders CF extracts (Carte Funciara) from ANCPI ePay (`epay.ancpi.ro`). Separate auth from eTerra. Uses prepaid credits (1 per extract). -### 6. Parcel Search (`POST /api/eterra/search`) -- Input: SIRUTA + cadastral number(s) -- Resolves workspace, then calls: - 1. `searchImmovableByIdentifier()` — find immovable - 2. `fetchDocumentationData()` — CF, owners (active/cancelled via nodeStatus tree) - 3. `fetchImmovableParcelDetails()` — area, intravilan, use categories - 4. `fetchImmAppsByImmovable()` → `fetchParcelFolosinte()` — application data -- Returns `ParcelDetail[]` with full property info +### Critical ID Mapping -### 7. No-Geometry Sync (`services/no-geom-sync.ts`) -- Finds eTerra immovables that have NO GIS geometry (no polygon in TERENURI_ACTIVE) -- Cross-references immovable list against remote ArcGIS features -- Stores as GisFeature with `geometry=null`, `objectId=-immovablePk` -- Quality gate: must be active + hasLandbook + has identification/area +- **ePay county IDs = eTerra WORKSPACE_IDs** (CLUJ=127, ALBA=10, etc.) +- **ePay UAT IDs = SIRUTA codes** (Cluj-Napoca=54975, Floresti=57706) +- Zero discovery needed: use `GisUat.workspacePk` as countyId, `siruta` as uatId +- ePay also uses alphabetical county indices 0-41 for some endpoints (mapped via `epay-counties.ts`) -### 8. Export -- **GeoPackage** (`services/gpkg-export.ts`): EPSG:3844 → EPSG:4326 reprojection -- **Export bundle** (`/api/eterra/export-bundle`): ZIP with CSV + GPKG -- **Local export** (`/api/eterra/export-local`): from local DB +### Auth Flow (OpenAM) + +1. POST `https://oassl.ancpi.ro/openam/UI/Login` with `IDToken1={user}&IDToken2={pass}` (form-urlencoded) +2. Response sets `AMAuthCookie` (NOT `iPlanetDirectoryPro`) +3. Navigate to `http://epay.ancpi.ro:80/epay/LogIn.action` (**HTTP, not HTTPS!**) for JSESSIONID +4. Session TTL: ~1 hour + +### Order Flow (per batch) + +1. **AddToCartOrWishListFromPost.action** x N -> basketRowIds +2. **EpayJsonInterceptor.action** x N -> `reqType=saveProductMetadataForBasketItem` + `productMetadataJSON` +3. **EditCartSubmit.action** -> `goToCheckout=true` (ONE submit for ALL items) +4. **CheckoutConfirmationSubmit.action** -> confirms order +5. Poll **ShowOrderDetails.action?orderId=...** -> parse documents from HTML-encoded JSON +6. **DownloadFile.action?typeD=4&id=...** -> PDF download + +### ePay Endpoint Gotchas + +1. **EpayJsonInterceptor uses form-urlencoded** (NOT JSON): `reqType=nomenclatorUAT&countyId=127` +2. **saveProductMetadataForBasketItem uses multipart/form-data** (form-data npm package) +3. **CF/CAD values use `stringValues[0]`** (array!), not `stringValue` +4. **Document IDs are HTML-encoded** in ShowOrderDetails: `"idDocument":47301767` -> must decode before parsing +5. **DownloadFile sends Content-Type: application/pdf in the REQUEST** (not just response) +6. **EditCartSubmit returns 200** (not redirect) — Angular app does client-side redirect to CheckoutConfirmation +7. **SearchEstate needs `identificator`/`judet`/`uat`** (not `identifier`/`countyId`/`uatId`), plus internal IDs +8. **MinIO metadata must be ASCII** — strip diacritics (`Floresti` not `Florești`) +9. **fetchCounties() is unreliable** — the nomenclature API 404s intermittently. Use static `WORKSPACE_TO_COUNTY` mapping instead. + +### Dedup Protection + +- **Queue level**: batch key = sorted cadastral numbers, 60s dedup window +- **API level**: optional `nonce` field, 60s idempotency cache +- **Test endpoint**: 30s dedup on hardcoded test parcels + +### Key Files + +| File | Purpose | +|------|---------| +| `services/epay-client.ts` | HTTP client (login, cart, metadata, submit, poll, download) | +| `services/epay-queue.ts` | Batch queue with dedup | +| `services/epay-storage.ts` | MinIO storage helpers | +| `services/epay-counties.ts` | County index mapping (eTerra county name -> ePay alphabetical index) | +| `services/epay-session-store.ts` | Session singleton | +| `services/epay-types.ts` | TypeScript types | +| `components/epay-connect.tsx` | Connection widget | +| `components/epay-order-button.tsx` | Per-parcel order button | +| `components/epay-tab.tsx` | Full "Extrase CF" tab | +| `api/ancpi/session/` | Connect/disconnect | +| `api/ancpi/order/` | Create batch orders | +| `api/ancpi/orders/` | List all extracts | +| `api/ancpi/credits/` | Credit balance | +| `api/ancpi/download/` | Stream PDF from MinIO | +| `api/ancpi/download-zip/` | Batch ZIP download | +| `api/ancpi/test/` | Diagnostic endpoint (temporary) | + +### Credentials + +- Env vars: `ANCPI_USERNAME`, `ANCPI_PASSWORD`, `ANCPI_BASE_URL`, `ANCPI_LOGIN_URL`, `ANCPI_DEFAULT_SOLICITANT_ID` +- MinIO bucket: `ancpi-documente` (env: `MINIO_BUCKET_ANCPI`) --- ## API Routes Reference +### eTerra Routes (`/api/eterra/`) + | Route | Method | Purpose | |-------|--------|---------| -| `/api/eterra/uats` | GET | All UATs from DB (with local feature counts) | +| `/api/eterra/uats` | GET | All UATs from DB (with cached feature counts, excludes geometry) | | `/api/eterra/uats` | POST | Seed DB from uat.json | -| `/api/eterra/uats` | PATCH | Populate county from eTerra nomenclature | -| `/api/eterra/login` | POST | Connect to eTerra | +| `/api/eterra/uats` | PATCH | Populate county from eTerra nomenclature (fallback) | +| `/api/eterra/login` | POST | Connect to eTerra (triggers county-refresh on first login) | | `/api/eterra/session` | GET/DELETE | Session status / disconnect | | `/api/eterra/health` | GET | eTerra platform health | | `/api/eterra/search` | POST | Search parcels by cadastral number | @@ -189,114 +253,26 @@ siruta, layerId, status, totalRemote, totalLocal, newFeatures, removedFeatures, | `/api/eterra/no-geom-debug` | POST | Debug no-geom data | | `/api/eterra/setup-postgis` | POST | Initialize PostGIS extensions | ---- +### ANCPI ePay Routes (`/api/ancpi/`) -## Critical Gotchas - -1. **nomenPk ≠ SIRUTA**: eTerra's internal nomenclature PKs are NOT SIRUTA codes. Never assume they're the same. -2. **workspacePk = county nomenPk**: The WORKSPACE_ID from ArcGIS layers equals the county's nomenclature PK from `fetchCounties()`. -3. **Session TTL**: eTerra sessions expire after ~10 minutes. Client has 9-minute cache TTL + auto-relogin. -4. **Maintenance windows**: eTerra goes down regularly. Health check detects this and blocks operations. -5. **ArcGIS maxRecordCount=1000**: Always paginate with `resultOffset`/`resultRecordCount`. Page size fallbacks: 1000 → 500 → 200. -6. **Admin field auto-discovery**: Different layers use different field names for SIRUTA (ADMIN_UNIT_ID, SIRUTA, UAT_ID, etc.). Always use `findAdminField()` / `buildWhere()`. -7. **LIMITE_UAT endpoint is "all"**: Unlike other layers (endpoint "aut"), LIMITE_UAT uses the "all" endpoint — no workspace context needed. -8. **No-geometry parcels use negative objectId**: `objectId = -immovablePk` to avoid collision with real ArcGIS OBJECTIDs. -9. **Spring Boot Page responses**: Some eTerra APIs return `{content: [...], totalPages: N}` instead of flat arrays. Always use `unwrapArray()` helper. -10. **Geometry format**: EsriGeometry `{ rings: number[][][] }` in EPSG:3844 (Stereo70). Reproject to EPSG:4326 for GeoPackage export. -11. **Owner tree parsing**: Documentation API returns owners as a tree (C→A→I→P nodes). Check `nodeStatus === -1` up the parent chain to detect radiated/cancelled owners. -12. **Local UatEntry type**: The component defines its own `UatEntry` type at the top of `parcel-sync-module.tsx` — this shadows the one in `types.ts`. Keep both in sync. +| Route | Method | Purpose | +|-------|--------|---------| +| `/api/ancpi/session` | POST/DELETE | Connect/disconnect ePay | +| `/api/ancpi/credits` | GET | Credit balance | +| `/api/ancpi/order` | POST | Create batch CF extract order | +| `/api/ancpi/orders` | GET | List all extracts from DB | +| `/api/ancpi/download` | GET | Stream single PDF from MinIO | +| `/api/ancpi/download-zip` | POST | Batch ZIP download | +| `/api/ancpi/test` | POST | Diagnostic test endpoint | --- ## Test UAT -**Feleacu** — SIRUTA `57582`, ~30k immovables, ~8k GIS features. Used as the primary test UAT. - ---- - -## ANCPI ePay — CF Extract Ordering - -### Overview - -Orders CF extracts (Carte Funciară) from ANCPI ePay (epay.ancpi.ro). -Separate auth system from eTerra. Uses credits (prepaid, 1 per extract). - -### Critical Discovery: ID Mapping - -- **ePay internal county IDs = eTerra WORKSPACE_IDs** (CLUJ=127, ALBA=10, etc.) -- **ePay UAT IDs = SIRUTA codes** (Cluj-Napoca=54975, Florești=57706) -- This means ZERO discovery calls needed — use GisUat.workspacePk + siruta directly - -### Auth Flow (OpenAM) - -1. GET login page with `module=SelfRegistration&goto=http://epay.ancpi.ro:80/epay/LogIn.action` -2. POST credentials (`IDToken1`, `IDToken2`) → gets `AMAuthCookie` (NOT `iPlanetDirectoryPro`) -3. Navigate to `http://epay.ancpi.ro:80/epay/LogIn.action` (HTTP, not HTTPS!) for JSESSIONID -4. Session TTL: ~1 hour - -### Order Flow (4 requests per batch) - -1. `POST AddToCartOrWishListFromPost.action` × N → basketRowIds -2. `POST EpayJsonInterceptor.action` (multipart) × N → `reqType=saveProductMetadataForBasketItem` + `productMetadataJSON` -3. `POST EditCartSubmit.action` → `goToCheckout=true` (ONE submit for ALL items) -4. `GET CheckoutConfirmationSubmit.action` → confirms order -5. Poll `ShowOrderDetails.action?orderId=...` → parse documents from HTML-encoded JSON (`"` → `"`) -6. `POST DownloadFile.action?typeD=4&id=...` with `Content-Type: application/pdf` in REQUEST header - -### ePay Endpoint Gotchas - -1. **EpayJsonInterceptor uses form-urlencoded** (NOT JSON): `reqType=nomenclatorUAT&countyId=127` -2. **saveProductMetadataForBasketItem uses multipart/form-data** (form-data npm package) -3. **CF/CAD use `stringValues[0]`** (array!), not `stringValue` -4. **Document IDs are HTML-encoded** in ShowOrderDetails: `"idDocument":47301767` → decode first -5. **DownloadFile sends Content-Type: application/pdf in the REQUEST** (not response) -6. **EditCartSubmit returns 200** (not redirect) — Angular does client-side redirect to CheckoutConfirmation -7. **SearchEstate needs `identificator`/`judet`/`uat`** (not `identifier`/`countyId`/`uatId`), plus requires internal IDs -8. **MinIO metadata must be ASCII** — strip diacritics from values (`Florești` → `Floresti`) - -### Dedup Protection - -- **Queue level**: batch key = sorted cadastral numbers, 60s dedup window -- **API level**: optional `nonce` field, 60s idempotency cache -- **Test endpoint**: 30s dedup on hardcoded test parcels - -### Key Files - -| File | Purpose | -|------|---------| -| `services/epay-client.ts` | HTTP client (login, cart, metadata, submit, poll, download) | -| `services/epay-queue.ts` | Batch queue with dedup | -| `services/epay-storage.ts` | MinIO storage helpers | -| `services/epay-counties.ts` | County index mapping (legacy, not needed for ordering) | -| `services/epay-session-store.ts` | Session singleton | -| `services/epay-types.ts` | TypeScript types | -| `components/epay-connect.tsx` | Connection widget | -| `components/epay-order-button.tsx` | Per-parcel order button | -| `components/epay-tab.tsx` | Full "Extrase CF" tab | -| `api/ancpi/session/` | Connect/disconnect | -| `api/ancpi/order/` | Create batch orders | -| `api/ancpi/orders/` | List all extracts | -| `api/ancpi/credits/` | Credit balance | -| `api/ancpi/download/` | Stream PDF from MinIO | -| `api/ancpi/test/` | Diagnostic/test endpoint (temporary) | - -### DB Model: CfExtract - -- `orderId` — shared across batch items (NOT unique) -- `nrCadastral`, `siruta`, `judetName`, `uatName` — parcel identity -- `status` — pending|queued|cart|ordering|polling|downloading|completed|failed|cancelled -- `minioPath` — `parcele/{nrCadastral}/{idx}_Extras CF_{nrCadastral} - {DD-MM-YYYY}.pdf` -- `expiresAt` — 30 days after documentDate -- `version` — increments on re-order for same cadastral number - -### Credentials - -- Env vars: `ANCPI_USERNAME`, `ANCPI_PASSWORD` (hardcoded in docker-compose for Portainer CE) -- Default solicitant: ID `14452` (Beletage persoană juridică) -- MinIO bucket: `ancpi-documente` +**Feleacu** -- SIRUTA `57582`, ~30k immovables, ~8k GIS features. --- ## Last Updated -2026-03-23 — ANCPI ePay CF extract ordering: full backend + UI + dedup protection. +2026-03-23 -- ANCPI ePay CF extract ordering, performance fixes (GisUat select, feature count cache), static WORKSPACE_TO_COUNTY mapping.