docs(geoportal): update evaluation + skills with deployment lessons learned

- Portainer CE volume mount pitfall (silent empty directory creation)
- Martin Docker tag format change at v1.0 (v prefix dropped)
- UNKNOWN GEOMETRY TYPE log is normal for views
- Bake-into-image pattern for config files in Portainer deployments
- Updated all implementation prompts with Portainer-safe instructions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
AI Assistant
2026-03-27 11:41:54 +02:00
parent 675b1e51dd
commit 67f3237761
2 changed files with 125 additions and 36 deletions
+83 -26
View File
@@ -4,7 +4,7 @@
ArchiTools Geoportal serves vector tiles (MVT) from PostgreSQL 16 + PostGIS 3 via Martin. ArchiTools Geoportal serves vector tiles (MVT) from PostgreSQL 16 + PostGIS 3 via Martin.
Data: ~330K GIS features (parcels, buildings, admin boundaries) in EPSG:3844 (Stereo70), growing to 1M+. Data: ~330K GIS features (parcels, buildings, admin boundaries) in EPSG:3844 (Stereo70), growing to 1M+.
Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted. Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted via Portainer CE.
--- ---
@@ -12,7 +12,7 @@ Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
1. Martin v0.15.0 was running in **auto-discovery mode** — the existing `martin.yaml` config was never mounted 1. Martin v0.15.0 was running in **auto-discovery mode** — the existing `martin.yaml` config was never mounted
2. Building labels (`cadastral_ref`) missing from MVT tiles despite the view exposing them 2. Building labels (`cadastral_ref`) missing from MVT tiles despite the view exposing them
3. Performance concerns at scale (330K 1M+ features) 3. Performance concerns at scale (330K -> 1M+ features)
--- ---
@@ -23,12 +23,12 @@ Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
| Aspect | Detail | | Aspect | Detail |
|---|---| |---|---|
| Root cause | `martin.yaml` not mounted in docker-compose — Martin ran in auto-discovery mode | | Root cause | `martin.yaml` not mounted in docker-compose — Martin ran in auto-discovery mode |
| Fix | Mount config + upgrade v0.15 v1.4.0 | | Fix | Bake config into custom image via Dockerfile + upgrade v0.15 -> v1.4.0 |
| Performance | Fastest tile server benchmarked (2-3x faster than #2 Tegola) | | Performance | Fastest tile server benchmarked (2-3x faster than #2 Tegola) |
| EPSG:3844 | Native support via `default_srid: 3844` | | EPSG:3844 | Native support via `default_srid: 3844` |
| New in v1.4 | ZSTD compression, MLT format, materialized views, better logging | | New in v1.4 | ZSTD compression, MLT format, materialized views, better logging |
**Status: IMPLEMENTED** — docker-compose.yml updated, building labels activated. **Status: IMPLEMENTED AND VERIFIED IN PRODUCTION** (2026-03-27)
### 2. pg_tileserv (CrunchyData) ### 2. pg_tileserv (CrunchyData)
@@ -39,7 +39,7 @@ Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
| Performance | 2-3x slower than Martin (Rechsteiner benchmark) | | Performance | 2-3x slower than Martin (Rechsteiner benchmark) |
| EPSG:3844 | Supported (auto-reprojects via ST_Transform) | | EPSG:3844 | Supported (auto-reprojects via ST_Transform) |
| Killer feature | Function-based sources (full SQL tile functions) | | Killer feature | Function-based sources (full SQL tile functions) |
| Dealbreaker | View extent estimation bug (#156) affects all our views, development stagnant | | Dealbreaker | View extent estimation bug (#156) affects all our views, development stagnant (last release Feb 2025) |
**Verdict: NO** — slower, buggy with views, stagnant development. **Verdict: NO** — slower, buggy with views, stagnant development.
@@ -67,7 +67,7 @@ Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
| Aspect | Detail | | Aspect | Detail |
|---|---| |---|---|
| 330K features | ~270 MB uncompressed, 800 MB1.4 GB browser memory | | 330K features | ~270 MB uncompressed, 800 MB-1.4 GB browser memory |
| Browser impact | 10-30s main thread freeze, mobile crash | | Browser impact | 10-30s main thread freeze, mobile crash |
| Pan/zoom | Full re-fetch on every viewport change, flickering | | Pan/zoom | Full re-fetch on every viewport change, flickering |
| Viable range | Only at zoom 16+ with <500 features in viewport | | Viable range | Only at zoom 16+ with <500 features in viewport |
@@ -116,12 +116,27 @@ Source: [github.com/FabianRechsteiner/vector-tiles-benchmark](https://github.com
## Implementation Roadmap ## Implementation Roadmap
### Phase 1: Martin Fix (DONE) ### Phase 1: Martin Fix DONE (2026-03-27)
Changes applied: Changes applied:
- `docker-compose.yml`: Martin v0.15 → v1.4.0, config mounted, command changed to `--config` - `martin.Dockerfile`: custom image that COPY-s `martin.yaml` into `/config/`
- `martin.yaml`: Comment updated to reflect v1.4 - `docker-compose.yml`: Martin v0.15 -> v1.4.0, build from Dockerfile, `--config` flag
- `map-viewer.tsx`: Building labels layer activated (`cladiriLabel` at minzoom 16) - `martin.yaml`: comment updated to reflect v1.4
- `map-viewer.tsx`: building labels layer activated (`cladiriLabel` at minzoom 16)
#### Deployment Lessons Learned
1. **Docker image tag format changed at v1.0**: old tags use `v` prefix (`v0.15.0`), new tags do not (`1.4.0`). The tag `ghcr.io/maplibre/martin:v1.4.0` does NOT exist — correct is `ghcr.io/maplibre/martin:1.4.0`.
2. **Portainer CE volume mount pitfall**: volume `./martin.yaml:/config/martin.yaml:ro` fails because Portainer deploys only the docker-compose.yml content, not the full git repo. Docker silently creates an empty directory instead of failing. Solution: bake config into a custom image with a 2-line Dockerfile:
```dockerfile
FROM ghcr.io/maplibre/martin:1.4.0
COPY martin.yaml /config/martin.yaml
```
3. **Martin config format is stable**: YAML format unchanged from v0.15 to v1.4 — `postgres.tables`, `connection_string`, `auto_publish`, `properties` map all work identically. No migration needed.
4. **PostGIS view geometry type**: Martin logs `UNKNOWN GEOMETRY TYPE` for all views — this is normal for nested views (`SELECT * FROM parent_view`). Views don't register specific geometry types in `geometry_columns`. Does not affect tile generation or property inclusion.
### Phase 2A: nginx Tile Cache ### Phase 2A: nginx Tile Cache
@@ -134,14 +149,16 @@ Changes applied:
Add an nginx reverse proxy cache in front of Martin for tile serving in ArchiTools. Add an nginx reverse proxy cache in front of Martin for tile serving in ArchiTools.
Context: Context:
- Martin serves tiles at http://martin:3000 (container name: martin, port 3010 on host) - Martin serves tiles at http://martin:3000 (container name: martin, internal port 3000)
- Martin is built from martin.Dockerfile (COPY martin.yaml into image)
- Traefik proxies external traffic to ArchiTools at tools.beletage.ro - Traefik proxies external traffic to ArchiTools at tools.beletage.ro
- Current tile URL pattern: https://tools.beletage.ro/tiles/{source}/{z}/{x}/{y} - Current tile URL pattern: https://tools.beletage.ro/tiles/{source}/{z}/{x}/{y}
- NEXT_PUBLIC_MARTIN_URL=https://tools.beletage.ro/tiles - NEXT_PUBLIC_MARTIN_URL=https://tools.beletage.ro/tiles
- Portainer CE deploys from Gitea repo — files must be in git, not just on host
Requirements: Requirements:
1. Create an nginx container `tile-cache` in docker-compose.yml 1. Create an nginx container `tile-cache` in docker-compose.yml
2. nginx config: proxy_cache for /tiles/* with: 2. nginx config: proxy_cache for all paths with:
- Cache zone: 2GB max, keys in shared memory - Cache zone: 2GB max, keys in shared memory
- Cache valid: 200 responses for 1 hour - Cache valid: 200 responses for 1 hour
- Stale serving on error/timeout - Stale serving on error/timeout
@@ -149,14 +166,13 @@ Requirements:
- CORS headers for tiles (Access-Control-Allow-Origin: *) - CORS headers for tiles (Access-Control-Allow-Origin: *)
- Gzip/brotli passthrough (Martin already compresses) - Gzip/brotli passthrough (Martin already compresses)
3. Route Martin traffic through tile-cache: 3. Route Martin traffic through tile-cache:
- tile-cache listens on port 3010 (replace Martin's host port) - tile-cache listens on port 3010 (host) -> 80 (container)
- tile-cache proxies to http://martin:3000 - tile-cache proxies to http://martin:3000
4. Add cache purge endpoint or script for post-sync invalidation - Martin removes its host port mapping (only accessible via tile-cache)
5. Volume for persistent cache across container restarts 4. Volume for persistent cache across container restarts
5. IMPORTANT: nginx config must be baked into a custom image (same pattern as martin.Dockerfile)
Files to modify: because Portainer CE cannot mount files from the repo. Create nginx/tile-cache.conf and
- docker-compose.yml (add tile-cache service, adjust martin ports) a tile-cache.Dockerfile.
- Create nginx/tile-cache.conf
Do NOT change the frontend NEXT_PUBLIC_MARTIN_URL — keep the same external URL. Do NOT change the frontend NEXT_PUBLIC_MARTIN_URL — keep the same external URL.
Build with `npx next build` to verify zero errors. Build with `npx next build` to verify zero errors.
@@ -173,11 +189,12 @@ Build with `npx next build` to verify zero errors.
Implement PMTiles pre-generation for UAT overview layers in ArchiTools Geoportal. Implement PMTiles pre-generation for UAT overview layers in ArchiTools Geoportal.
Context: Context:
- PostGIS at 10.10.10.166:5432, database architools_db - PostGIS at 10.10.10.166:5432, database architools_db, user architools_user
- Views: gis_uats_z0, gis_uats_z5, gis_uats_z8, gis_uats_z12, gis_administrativ - Views: gis_uats_z0, gis_uats_z5, gis_uats_z8, gis_uats_z12, gis_administrativ
- All geometries EPSG:3844 (Stereo70), need reprojection to 4326 for tippecanoe - All geometries EPSG:3844 (Stereo70), need reprojection to 4326 for tippecanoe
- MinIO at 10.10.10.166:9002, bucket for tiles - MinIO at 10.10.10.166:9002 (API) / 9003 (console), bucket for tiles
- Frontend: MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx - Frontend: MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx
- Portainer CE deploys from Gitea — any config files must be baked into Docker images
Requirements: Requirements:
@@ -225,7 +242,7 @@ Read CLAUDE.md for project conventions before starting.
Test MLT (MapLibre Tiles) format on one layer in ArchiTools Geoportal. Test MLT (MapLibre Tiles) format on one layer in ArchiTools Geoportal.
Context: Context:
- Martin v1.4.0 running with config at martin.yaml - Martin v1.4.0 running with config baked in via martin.Dockerfile
- MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx - MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx
- Test layer: gis_terenuri (largest layer, ~250K features) - Test layer: gis_terenuri (largest layer, ~250K features)
@@ -264,6 +281,7 @@ Evaluate mvt-rs as a replacement for Martin in ArchiTools Geoportal for multi-te
Context: Context:
- Current: Martin v1.4.0 serving 9 PostGIS sources (views in EPSG:3844) - Current: Martin v1.4.0 serving 9 PostGIS sources (views in EPSG:3844)
- Martin config baked into image via martin.Dockerfile
- Goal: Expose geoportal to external clients with per-layer access control - Goal: Expose geoportal to external clients with per-layer access control
- mvt-rs repo: https://github.com/mvt-proj/mvt-rs (v0.16.2+, Rust, Salvo framework) - mvt-rs repo: https://github.com/mvt-proj/mvt-rs (v0.16.2+, Rust, Salvo framework)
@@ -299,7 +317,22 @@ Do NOT modify the production Martin setup. This is a parallel evaluation only.
## Key Technical Details ## Key Technical Details
### Martin v1.4.0 Config (validated compatible) ### Martin v1.4.0 Deployment Architecture
```
Gitea repo (martin.yaml + martin.Dockerfile)
-> Portainer CE builds custom image: FROM martin:1.4.0, COPY martin.yaml
-> Container starts with --config /config/martin.yaml
-> Reads DATABASE_URL from environment
-> Serves 9 PostGIS view sources on port 3000
-> Host maps 3010:3000
-> Traefik proxies tools.beletage.ro/tiles -> host:3010
```
**Critical**: Do NOT use volume mounts for config files in Portainer CE stacks.
Always bake configs into custom images via Dockerfile COPY.
### Martin Config (validated compatible v0.15 through v1.4)
The `martin.yaml` at project root defines 9 sources with explicit properties. The `martin.yaml` at project root defines 9 sources with explicit properties.
Config format unchanged from v0.15 to v1.4 — no migration needed. Config format unchanged from v0.15 to v1.4 — no migration needed.
@@ -309,13 +342,21 @@ Key config features used:
- `default_srid: 3844` — all sources use Stereo70 - `default_srid: 3844` — all sources use Stereo70
- `properties:` map per source — explicit column name + PostgreSQL type - `properties:` map per source — explicit column name + PostgreSQL type
- `minzoom/maxzoom` per source — controls tile generation range - `minzoom/maxzoom` per source — controls tile generation range
- `bounds: [20.2, 43.5, 30.0, 48.3]` — approximate Romania extent
### Docker Image Tag Convention
Martin changed tag format at v1.0:
- Pre-1.0: `ghcr.io/maplibre/martin:v0.15.0` (with `v` prefix)
- Post-1.0: `ghcr.io/maplibre/martin:1.4.0` (no `v` prefix)
- Also available: `latest`, `nightly`
### PostGIS View Chain ### PostGIS View Chain
``` ```
GisFeature table (Prisma) gis_features view gis_terenuri / gis_cladiri / gis_administrativ GisFeature table (Prisma) -> gis_features view -> gis_terenuri / gis_cladiri / gis_administrativ
gis_terenuri_status / gis_cladiri_status (with JOINs) -> gis_terenuri_status / gis_cladiri_status (with JOINs)
GisUat table gis_uats_z0/z5/z8/z12 (with ST_SimplifyPreserveTopology) GisUat table -> gis_uats_z0/z5/z8/z12 (with ST_SimplifyPreserveTopology)
``` ```
### MapLibre Layer Architecture ### MapLibre Layer Architecture
@@ -325,14 +366,30 @@ Sources (Martin): gis_uats_z0, z5, z8, z12, administrativ, terenuri, cladiri
Layers per source: fill + line + label (where applicable) Layers per source: fill + line + label (where applicable)
Selection: Separate highlight layers on terenuri source Selection: Separate highlight layers on terenuri source
Drawing: GeoJSON source for freehand/rect polygon Drawing: GeoJSON source for freehand/rect polygon
Building labels: cladiriLabel layer, cadastral_ref at minzoom 16
``` ```
--- ---
## Deployment Pitfalls (Discovered During Implementation)
1. **Portainer CE does not expose repo files to containers at runtime.** Volume mounts like `./file.conf:/etc/file.conf:ro` fail silently — Docker creates an empty directory. Always bake config files into custom images via Dockerfile COPY.
2. **Martin Docker tag format change at v1.0.** `v1.4.0` does not exist, `1.4.0` does. Always check [ghcr.io/maplibre/martin](https://github.com/maplibre/martin/pkgs/container/martin) for actual tags.
3. **Martin logs `UNKNOWN GEOMETRY TYPE` for PostGIS views.** This is normal — nested views don't register geometry types in `geometry_columns`. Does not affect functionality.
4. **Martin auto-discovery mode is unreliable for property inclusion.** Always use explicit config with `auto_publish: false` and per-source `properties:` definitions.
5. **Martin caches source schema at startup.** After PostGIS view DDL changes (e.g., adding columns to gis_features), Martin must be restarted to pick up new columns.
---
## References ## References
- [Martin Documentation](https://maplibre.org/martin/) - [Martin Documentation](https://maplibre.org/martin/)
- [Martin Releases](https://github.com/maplibre/martin/releases) - [Martin Releases](https://github.com/maplibre/martin/releases)
- [Martin Container Registry](https://github.com/maplibre/martin/pkgs/container/martin)
- [Vector Tiles Benchmark (Rechsteiner 2025)](https://github.com/FabianRechsteiner/vector-tiles-benchmark) - [Vector Tiles Benchmark (Rechsteiner 2025)](https://github.com/FabianRechsteiner/vector-tiles-benchmark)
- [PMTiles Specification](https://github.com/protomaps/PMTiles) - [PMTiles Specification](https://github.com/protomaps/PMTiles)
- [tippecanoe (Felt)](https://github.com/felt/tippecanoe) - [tippecanoe (Felt)](https://github.com/felt/tippecanoe)
+42 -10
View File
@@ -65,11 +65,18 @@ postgres:
area: float8 area: float8
``` ```
### Docker image tags
Martin changed tag format at v1.0:
- Pre-1.0: `ghcr.io/maplibre/martin:v0.15.0` (with `v` prefix)
- Post-1.0: `ghcr.io/maplibre/martin:1.4.0` (no `v` prefix)
### Docker deployment ### Docker deployment
**If your orchestrator has access to the full repo** (docker-compose CLI, Docker Swarm with repo checkout):
```yaml ```yaml
martin: martin:
image: ghcr.io/maplibre/martin:v1.4.0 image: ghcr.io/maplibre/martin:1.4.0
command: ["--config", "/config/martin.yaml"] command: ["--config", "/config/martin.yaml"]
environment: environment:
- DATABASE_URL=postgresql://user:pass@host:5432/db - DATABASE_URL=postgresql://user:pass@host:5432/db
@@ -79,6 +86,28 @@ martin:
- "3010:3000" - "3010:3000"
``` ```
**If using Portainer CE or any system that only sees docker-compose.yml** (not full repo):
Volume mounts for repo files fail silently — Docker creates an empty directory instead.
Bake config into a custom image:
```dockerfile
# martin.Dockerfile
FROM ghcr.io/maplibre/martin:1.4.0
COPY martin.yaml /config/martin.yaml
```
```yaml
martin:
build:
context: .
dockerfile: martin.Dockerfile
command: ["--config", "/config/martin.yaml"]
environment:
- DATABASE_URL=postgresql://user:pass@host:5432/db
ports:
- "3010:3000"
```
### Custom SRID handling ### Custom SRID handling
Martin handles non-4326/3857 SRIDs natively. Set `default_srid` globally or `srid` per source. Martin reprojects to Web Mercator (3857) internally for tile envelope calculations. Your PostGIS spatial indexes on the source SRID are used correctly. Martin handles non-4326/3857 SRIDs natively. Set `default_srid` globally or `srid` per source. Martin reprojects to Web Mercator (3857) internally for tile envelope calculations. Your PostGIS spatial indexes on the source SRID are used correctly.
@@ -170,12 +199,15 @@ map.addSource('overview', {
## Common Pitfalls ## Common Pitfalls
1. **Martin auto-discovery drops properties** — always use explicit config with `auto_publish: false` 1. **Martin auto-discovery drops properties** — always use explicit config with `auto_publish: false`
2. **Nested views lose SRID metadata** — cast geometry: `geom::geometry(Geometry, 3844)` 2. **Martin Docker tag format changed at v1.0**`v0.15.0` (with v) but `1.4.0` (without v). Check actual tags at ghcr.io.
3. **GisUat.geometry is huge** — always `select` to exclude in list queries 3. **Portainer CE volume mounts fail silently** — Docker creates empty directory instead of file. Bake configs into images via Dockerfile COPY.
4. **Low-zoom tiles scan entire dataset** — use zoom-dependent simplified views 4. **Martin logs `UNKNOWN GEOMETRY TYPE` for views** — normal for nested views, does not affect tile generation
5. **No tile cache by default** — add nginx/Varnish in front of any tile server 5. **Nested views lose SRID metadata** — cast geometry: `geom::geometry(Geometry, 3844)`
6. **tippecanoe requires WGS84** — reproject from custom SRID before generating PMTiles 6. **GisUat.geometry is huge** — always `select` to exclude in list queries
7. **PMTiles not incrementally updatable**full rebuild required on data change 7. **Low-zoom tiles scan entire dataset**use zoom-dependent simplified views
8. **Tegola doesn't support custom SRIDs** — only 3857/4326, requires ST_Transform everywhere 8. **No tile cache by default** — add nginx/Varnish in front of any tile server
9. **pg_tileserv `ST_Estimated_Extent` fails on views** — use materialized views or function layers 9. **tippecanoe requires WGS84** — reproject from custom SRID before generating PMTiles
10. **Martin caches source schema at startup** — restart after view DDL changes 10. **PMTiles not incrementally updatable** — full rebuild required on data change
11. **Tegola doesn't support custom SRIDs** — only 3857/4326, requires ST_Transform everywhere
12. **pg_tileserv `ST_Estimated_Extent` fails on views** — use materialized views or function layers
13. **Martin caches source schema at startup** — restart after view DDL changes