gitadmin/ArchiTools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update tile evaluation + monitoring + add geoportal improvement mega prompt

Browse Source 
- TILE-SERVER-EVALUATION.md: updated to reflect current architecture (PMTiles z0-z18)
- MODULE-MAP.md: added PMTiles + tile-cache to Geoportal section
- Monitor: timeout increased to 90 min for z18 builds, description updated
- Added PROMPT-GEOPORTAL-IMPROVE.md with mega prompt for future sessions
  (includes MLT check, mvt-rs evaluation prompt, operational commands)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
AI Assistant
2026-03-28 18:38:53 +02:00
parent 870e1bd4c2
commit ee86af6183
4 changed files with 209 additions and 157 deletions
Download Patch File Download Diff File Expand all files Collapse all files
geoportal/TILE-SERVER-EVALUATION.md
+84 -150
View File
@@ -138,179 +138,113 @@ Changes applied:
4. **PostGIS view geometry type**: Martin logs `UNKNOWN GEOMETRY TYPE` for all views — this is normal for nested views (`SELECT * FROM parent_view`). Views don't register specific geometry types in `geometry_columns`. Does not affect tile generation or property inclusion.
### Phase 2A: nginx Tile Cache
### Phase 2A: nginx Tile Cache — DONE (2026-03-27)
**Impact**: 10-100x faster on repeat requests, zero PostGIS load for cached tiles.
**Effort**: ~2 hours.
#### Implementation Prompt
Changes applied:
- `nginx/tile-cache.conf`: proxy_cache config with 2GB cache zone, 7-day TTL, stale serving
- `tile-cache.Dockerfile`: bakes nginx config into custom image (Portainer CE pattern)
- `docker-compose.yml`: `tile-cache` container, Martin no longer exposed on host
- Gzip passthrough (Martin already compresses), browser caching via Cache-Control headers
- CORS headers for cross-origin tile requests
```
Add an nginx reverse proxy cache in front of Martin for tile serving in ArchiTools.
### Phase 2B: PMTiles — DONE (2026-03-27)
Context:
- Martin serves tiles at http://martin:3000 (container name: martin, internal port 3000)
- Martin is built from martin.Dockerfile (COPY martin.yaml into image)
- Traefik proxies external traffic to ArchiTools at tools.beletage.ro
- Current tile URL pattern: https://tools.beletage.ro/tiles/{source}/{z}/{x}/{y}
- NEXT_PUBLIC_MARTIN_URL=https://tools.beletage.ro/tiles
- Portainer CE deploys from Gitea repo — files must be in git, not just on host
**Impact**: Sub-10ms overview tiles, zero PostGIS load for z0-z18.
Requirements:
1. Create an nginx container `tile-cache` in docker-compose.yml
2. nginx config: proxy_cache for all paths with:
- Cache zone: 2GB max, keys in shared memory
- Cache valid: 200 responses for 1 hour
- Stale serving on error/timeout
- Cache-Control headers passed through
- CORS headers for tiles (Access-Control-Allow-Origin: *)
- Gzip/brotli passthrough (Martin already compresses)
3. Route Martin traffic through tile-cache:
- tile-cache listens on port 3010 (host) -> 80 (container)
- tile-cache proxies to http://martin:3000
- Martin removes its host port mapping (only accessible via tile-cache)
4. Volume for persistent cache across container restarts
5. IMPORTANT: nginx config must be baked into a custom image (same pattern as martin.Dockerfile)
because Portainer CE cannot mount files from the repo. Create nginx/tile-cache.conf and
a tile-cache.Dockerfile.
Changes applied:
- `scripts/rebuild-overview-tiles.sh`: ogr2ogr export (3844->4326) + tippecanoe generation
- PMTiles archive: z0-z18, ~1-2 GB, includes all terenuri, cladiri, UATs, and administrativ layers
- `map-viewer.tsx`: pmtiles:// protocol registered on MapLibre, hybrid source switching
- MinIO bucket `tiles` with public read + CORS for Range Requests
- N8N webhook trigger for rebuild (via monitor page)
- Monitor page (`/monitor`): rebuild + warm-cache actions with live status polling
Do NOT change the frontend NEXT_PUBLIC_MARTIN_URL — keep the same external URL.
Build with `npx next build` to verify zero errors.
```
### Phase 2C: MLT Format — DEFERRED
### Phase 2B: PMTiles for UAT Overview Layers
Martin v1.4 advertises MLT support, but it cannot generate MLT from PostGIS live queries.
MLT generation requires pre-built tile archives (tippecanoe does not output MLT either).
No actionable path until Martin or tippecanoe adds MLT output from PostGIS sources.
**Impact**: Sub-10ms overview tiles, zero PostGIS load for z0-z12.
**Effort**: ~4-6 hours.
#### Implementation Prompt
```
Implement PMTiles pre-generation for UAT overview layers in ArchiTools Geoportal.
Context:
- PostGIS at 10.10.10.166:5432, database architools_db, user architools_user
- Views: gis_uats_z0, gis_uats_z5, gis_uats_z8, gis_uats_z12, gis_administrativ
- All geometries EPSG:3844 (Stereo70), need reprojection to 4326 for tippecanoe
- MinIO at 10.10.10.166:9002 (API) / 9003 (console), bucket for tiles
- Frontend: MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx
- Portainer CE deploys from Gitea — any config files must be baked into Docker images
Requirements:
1. Create `scripts/rebuild-overview-tiles.sh`:
- Export each view with ogr2ogr: -f FlatGeobuf -s_srs EPSG:3844 -t_srs EPSG:4326
- Generate combined PMTiles with tippecanoe:
- --layer per view, --minimum-zoom=0, --maximum-zoom=14
- --detect-shared-borders (critical for adjacent UAT polygons)
- --hilbert for compression
- Atomic upload to MinIO: upload as overview_new.pmtiles, then rename to overview.pmtiles
- Cleanup temp files
2. Create Dockerfile for tippecanoe build container (or use ghcr.io/felt/tippecanoe)
3. Add `tippecanoe` service to docker-compose.yml (one-shot, for manual/cron runs)
4. Configure MinIO:
- Create bucket `tiles` with public read
- CORS: Allow GET/HEAD from tools.beletage.ro, expose Range/Content-Range headers
5. Update map-viewer.tsx:
- npm install pmtiles
- Register pmtiles:// protocol on MapLibre
- Add PMTiles source for overview layers (z0-z14)
- Keep existing Martin sources for detail layers (z14+)
- Set zoom breakpoints: PMTiles below z14, Martin above z14
6. Add N8N webhook trigger or cron for nightly rebuild after weekend deep sync
The UAT overview layers change rarely (only when new UATs are synced).
Parcel/building layers stay on Martin for live data freshness.
Build with `npx next build` to verify zero errors.
Read CLAUDE.md for project conventions before starting.
```
### Phase 2C: MLT Format Testing
**Impact**: 6x smaller tiles, 4x faster client decode.
**Effort**: ~1 hour to test.
#### Implementation Prompt
```
Test MLT (MapLibre Tiles) format on one layer in ArchiTools Geoportal.
Context:
- Martin v1.4.0 running with config baked in via martin.Dockerfile
- MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx
- Test layer: gis_terenuri (largest layer, ~250K features)
Requirements:
1. Research how Martin v1.4 serves MLT format:
- Check if it's automatic via Accept header or needs config
- Check Martin docs for MLT serving configuration
2. Update map-viewer.tsx to request MLT for one source (gis_terenuri):
- Add `encoding: "mlt"` to the vector source definition if MapLibre supports it
- Or configure via source URL parameter if Martin expects it
3. Test and measure:
- Compare tile sizes: MVT vs MLT for same tile coordinates
- Compare decode time in browser DevTools Network tab
- Check that all properties (cadastral_ref, area_value, etc.) survive MLT encoding
- Check label rendering still works
4. If MLT works correctly, apply to all Martin sources
If issues found, document them and revert to MVT
This is experimental — keep MVT as fallback. Do not break existing functionality.
Build with `npx next build` to verify zero errors.
```
### Phase 2D: mvt-rs Evaluation (Future — Multi-Tenant)
### Phase 2D: mvt-rs Evaluation — FUTURE (Multi-Tenant)
**Impact**: Built-in auth, admin UI, per-layer access control.
**Effort**: 1-2 days for evaluation + migration.
#### Implementation Prompt
Reserved for when external client access to the geoportal is needed.
mvt-rs (v0.16.2+, Rust, Salvo framework) provides per-layer auth and admin UI.
---
## Phase 3: Current Architecture (as of 2026-03-27)
Full tile-serving pipeline in production:
```
Evaluate mvt-rs as a replacement for Martin in ArchiTools Geoportal for multi-tenant deployment.
PostGIS (EPSG:3844)
|
+--> Martin v1.4.0 (live MVT from 9 PostGIS views)
| |
| +--> tile-cache (nginx reverse proxy, 2GB disk, 7d TTL)
| |
| +--> Traefik (tools.beletage.ro/tiles)
|
+--> ogr2ogr (3844->4326) + tippecanoe (z0-z18)
|
+--> PMTiles archive (~1-2 GB)
|
+--> MinIO bucket "tiles" (HTTP Range Requests)
|
+--> MapLibre (pmtiles:// protocol)
```
Context:
- Current: Martin v1.4.0 serving 9 PostGIS sources (views in EPSG:3844)
- Martin config baked into image via martin.Dockerfile
- Goal: Expose geoportal to external clients with per-layer access control
- mvt-rs repo: https://github.com/mvt-proj/mvt-rs (v0.16.2+, Rust, Salvo framework)
**Hybrid strategy**:
- PMTiles serves pre-generated overview tiles (all zoom levels, all layers)
- Martin serves live detail tiles (real-time PostGIS data)
- nginx tile-cache sits in front of Martin to absorb repeat requests
- Rebuild triggered via N8N webhook from the `/monitor` page
Requirements:
---
1. Deploy mvt-rs as a Docker container alongside Martin (don't replace yet)
- Use same DATABASE_URL
- Map to different port (e.g., 3011)
## Operational Commands
2. Configure mvt-rs with equivalent layers to martin.yaml:
- gis_uats_z0/z5/z8/z12, gis_administrativ, gis_terenuri, gis_cladiri
- gis_terenuri_status, gis_cladiri_status
- All with EPSG:3844, explicit properties
### Rebuild PMTiles
3. Test:
- Do all properties appear in MVT output? (especially cadastral_ref on gis_cladiri)
- Performance comparison: curl timing for 10 representative tiles vs Martin
- Admin UI: create test user, assign layer permissions
- Cache: configure disk cache, measure cold vs warm tile times
Trigger from the Monitor page (`/monitor` -> "Rebuild PMTiles" button), which sends a webhook to N8N.
N8N runs `scripts/rebuild-overview-tiles.sh` on the server.
4. Document findings:
- Property inclusion: pass/fail per layer
- Performance delta vs Martin
- Admin UI capabilities and limitations
- Missing features vs Martin (PMTiles, MLT, etc.)
Manual rebuild (SSH to 10.10.10.166):
```bash
cd /path/to/architools
bash scripts/rebuild-overview-tiles.sh
```
5. Decision matrix: when to switch from Martin to mvt-rs
### Warm nginx Cache
Do NOT modify the production Martin setup. This is a parallel evaluation only.
Trigger from the Monitor page (`/monitor` -> "Warm Cache" button).
Pre-loads frequently accessed tiles into the nginx disk cache.
### Purge nginx Tile Cache
```bash
docker exec tile-cache rm -rf /var/cache/nginx/tiles/*
docker exec tile-cache nginx -s reload
```
### Restart Martin (after PostGIS view changes)
```bash
docker restart martin
```
Martin caches source schema at startup — must restart after DDL changes to pick up new columns.
### Check PMTiles Status
```bash
# Check file size and last modified in MinIO
docker exec minio mc stat local/tiles/overview.pmtiles
```
---