gitadmin/ArchiTools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(geoportal): update evaluation + skills with deployment lessons learned

Browse Source 
- Portainer CE volume mount pitfall (silent empty directory creation)
- Martin Docker tag format change at v1.0 (v prefix dropped)
- UNKNOWN GEOMETRY TYPE log is normal for views
- Bake-into-image pattern for config files in Portainer deployments
- Updated all implementation prompts with Portainer-safe instructions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
AI Assistant
2026-03-27 11:41:54 +02:00
parent 675b1e51dd
commit 67f3237761
2 changed files with 125 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
geoportal/TILE-SERVER-EVALUATION.md
+83 -26
View File
@@ -4,7 +4,7 @@
ArchiTools Geoportal serves vector tiles (MVT) from PostgreSQL 16 + PostGIS 3 via Martin.
Data: ~330K GIS features (parcels, buildings, admin boundaries) in EPSG:3844 (Stereo70), growing to 1M+.
Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted via Portainer CE.
---
@@ -12,7 +12,7 @@ Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
1. Martin v0.15.0 was running in **auto-discovery mode** — the existing `martin.yaml` config was never mounted
2. Building labels (`cadastral_ref`) missing from MVT tiles despite the view exposing them
3. Performance concerns at scale (330K → 1M+ features)
3. Performance concerns at scale (330K -> 1M+ features)
---
@@ -23,12 +23,12 @@ Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
| Aspect | Detail |
|---|---|
| Root cause | `martin.yaml` not mounted in docker-compose — Martin ran in auto-discovery mode |
| Fix | Mount config + upgrade v0.15 → v1.4.0 |
| Fix | Bake config into custom image via Dockerfile + upgrade v0.15 -> v1.4.0 |
| Performance | Fastest tile server benchmarked (2-3x faster than #2 Tegola) |
| EPSG:3844 | Native support via `default_srid: 3844` |
| New in v1.4 | ZSTD compression, MLT format, materialized views, better logging |
**Status: IMPLEMENTED** — docker-compose.yml updated, building labels activated.
**Status: IMPLEMENTED AND VERIFIED IN PRODUCTION** (2026-03-27)
### 2. pg_tileserv (CrunchyData)
@@ -39,7 +39,7 @@ Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
| Performance | 2-3x slower than Martin (Rechsteiner benchmark) |
| EPSG:3844 | Supported (auto-reprojects via ST_Transform) |
| Killer feature | Function-based sources (full SQL tile functions) |
| Dealbreaker | View extent estimation bug (#156) affects all our views, development stagnant |
| Dealbreaker | View extent estimation bug (#156) affects all our views, development stagnant (last release Feb 2025) |
**Verdict: NO** — slower, buggy with views, stagnant development.
@@ -67,7 +67,7 @@ Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted.
| Aspect | Detail |
|---|---|
| 330K features | ~270 MB uncompressed, 800 MB–1.4 GB browser memory |
| 330K features | ~270 MB uncompressed, 800 MB-1.4 GB browser memory |
| Browser impact | 10-30s main thread freeze, mobile crash |
| Pan/zoom | Full re-fetch on every viewport change, flickering |
| Viable range | Only at zoom 16+ with <500 features in viewport |
@@ -116,12 +116,27 @@ Source: [github.com/FabianRechsteiner/vector-tiles-benchmark](https://github.com
## Implementation Roadmap
### Phase 1: Martin Fix (DONE)
### Phase 1: Martin Fix — DONE (2026-03-27)
Changes applied:
- `docker-compose.yml`: Martin v0.15 → v1.4.0, config mounted, command changed to `--config`
- `martin.yaml`: Comment updated to reflect v1.4
- `map-viewer.tsx`: Building labels layer activated (`cladiriLabel` at minzoom 16)
- `martin.Dockerfile`: custom image that COPY-s `martin.yaml` into `/config/`
- `docker-compose.yml`: Martin v0.15 -> v1.4.0, build from Dockerfile, `--config` flag
- `martin.yaml`: comment updated to reflect v1.4
- `map-viewer.tsx`: building labels layer activated (`cladiriLabel` at minzoom 16)
#### Deployment Lessons Learned
1. **Docker image tag format changed at v1.0**: old tags use `v` prefix (`v0.15.0`), new tags do not (`1.4.0`). The tag `ghcr.io/maplibre/martin:v1.4.0` does NOT exist — correct is `ghcr.io/maplibre/martin:1.4.0`.
2. **Portainer CE volume mount pitfall**: volume `./martin.yaml:/config/martin.yaml:ro` fails because Portainer deploys only the docker-compose.yml content, not the full git repo. Docker silently creates an empty directory instead of failing. Solution: bake config into a custom image with a 2-line Dockerfile:
```dockerfile
FROM ghcr.io/maplibre/martin:1.4.0
COPY martin.yaml /config/martin.yaml
```
3. **Martin config format is stable**: YAML format unchanged from v0.15 to v1.4 — `postgres.tables`, `connection_string`, `auto_publish`, `properties` map all work identically. No migration needed.
4. **PostGIS view geometry type**: Martin logs `UNKNOWN GEOMETRY TYPE` for all views — this is normal for nested views (`SELECT * FROM parent_view`). Views don't register specific geometry types in `geometry_columns`. Does not affect tile generation or property inclusion.
### Phase 2A: nginx Tile Cache
@@ -134,14 +149,16 @@ Changes applied:
Add an nginx reverse proxy cache in front of Martin for tile serving in ArchiTools.
Context:
- Martin serves tiles at http://martin:3000 (container name: martin, port 3010 on host)
- Martin serves tiles at http://martin:3000 (container name: martin, internal port 3000)
- Martin is built from martin.Dockerfile (COPY martin.yaml into image)
- Traefik proxies external traffic to ArchiTools at tools.beletage.ro
- Current tile URL pattern: https://tools.beletage.ro/tiles/{source}/{z}/{x}/{y}
- NEXT_PUBLIC_MARTIN_URL=https://tools.beletage.ro/tiles
- Portainer CE deploys from Gitea repo — files must be in git, not just on host
Requirements:
1. Create an nginx container `tile-cache` in docker-compose.yml
2. nginx config: proxy_cache for /tiles/* with:
2. nginx config: proxy_cache for all paths with:
- Cache zone: 2GB max, keys in shared memory
- Cache valid: 200 responses for 1 hour
- Stale serving on error/timeout
@@ -149,14 +166,13 @@ Requirements:
- CORS headers for tiles (Access-Control-Allow-Origin: *)
- Gzip/brotli passthrough (Martin already compresses)
3. Route Martin traffic through tile-cache:
- tile-cache listens on port 3010 (replace Martin's host port)
- tile-cache listens on port 3010 (host) -> 80 (container)
- tile-cache proxies to http://martin:3000
4. Add cache purge endpoint or script for post-sync invalidation
5. Volume for persistent cache across container restarts
Files to modify:
- docker-compose.yml (add tile-cache service, adjust martin ports)
- Create nginx/tile-cache.conf
- Martin removes its host port mapping (only accessible via tile-cache)
4. Volume for persistent cache across container restarts
5. IMPORTANT: nginx config must be baked into a custom image (same pattern as martin.Dockerfile)
because Portainer CE cannot mount files from the repo. Create nginx/tile-cache.conf and
a tile-cache.Dockerfile.
Do NOT change the frontend NEXT_PUBLIC_MARTIN_URL — keep the same external URL.
Build with `npx next build` to verify zero errors.
@@ -173,11 +189,12 @@ Build with `npx next build` to verify zero errors.
Implement PMTiles pre-generation for UAT overview layers in ArchiTools Geoportal.
Context:
- PostGIS at 10.10.10.166:5432, database architools_db
- PostGIS at 10.10.10.166:5432, database architools_db, user architools_user
- Views: gis_uats_z0, gis_uats_z5, gis_uats_z8, gis_uats_z12, gis_administrativ
- All geometries EPSG:3844 (Stereo70), need reprojection to 4326 for tippecanoe
- MinIO at 10.10.10.166:9002, bucket for tiles
- MinIO at 10.10.10.166:9002 (API) / 9003 (console), bucket for tiles
- Frontend: MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx
- Portainer CE deploys from Gitea — any config files must be baked into Docker images
Requirements:
@@ -225,7 +242,7 @@ Read CLAUDE.md for project conventions before starting.
Test MLT (MapLibre Tiles) format on one layer in ArchiTools Geoportal.
Context:
- Martin v1.4.0 running with config at martin.yaml
- Martin v1.4.0 running with config baked in via martin.Dockerfile
- MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx
- Test layer: gis_terenuri (largest layer, ~250K features)
@@ -264,6 +281,7 @@ Evaluate mvt-rs as a replacement for Martin in ArchiTools Geoportal for multi-te
Context:
- Current: Martin v1.4.0 serving 9 PostGIS sources (views in EPSG:3844)
- Martin config baked into image via martin.Dockerfile
- Goal: Expose geoportal to external clients with per-layer access control
- mvt-rs repo: https://github.com/mvt-proj/mvt-rs (v0.16.2+, Rust, Salvo framework)
@@ -299,7 +317,22 @@ Do NOT modify the production Martin setup. This is a parallel evaluation only.
## Key Technical Details
### Martin v1.4.0 Config (validated compatible)
### Martin v1.4.0 Deployment Architecture
```
Gitea repo (martin.yaml + martin.Dockerfile)
-> Portainer CE builds custom image: FROM martin:1.4.0, COPY martin.yaml
-> Container starts with --config /config/martin.yaml
-> Reads DATABASE_URL from environment
-> Serves 9 PostGIS view sources on port 3000
-> Host maps 3010:3000
-> Traefik proxies tools.beletage.ro/tiles -> host:3010
```
**Critical**: Do NOT use volume mounts for config files in Portainer CE stacks.
Always bake configs into custom images via Dockerfile COPY.
### Martin Config (validated compatible v0.15 through v1.4)
The `martin.yaml` at project root defines 9 sources with explicit properties.
Config format unchanged from v0.15 to v1.4 — no migration needed.
@@ -309,13 +342,21 @@ Key config features used:
- `default_srid: 3844` — all sources use Stereo70
- `properties:` map per source — explicit column name + PostgreSQL type
- `minzoom/maxzoom` per source — controls tile generation range
- `bounds: [20.2, 43.5, 30.0, 48.3]` — approximate Romania extent
### Docker Image Tag Convention
Martin changed tag format at v1.0:
- Pre-1.0: `ghcr.io/maplibre/martin:v0.15.0` (with `v` prefix)
- Post-1.0: `ghcr.io/maplibre/martin:1.4.0` (no `v` prefix)
- Also available: `latest`, `nightly`
### PostGIS View Chain
```
GisFeature table (Prisma) → gis_features view → gis_terenuri / gis_cladiri / gis_administrativ
→ gis_terenuri_status / gis_cladiri_status (with JOINs)
GisUat table → gis_uats_z0/z5/z8/z12 (with ST_SimplifyPreserveTopology)
GisFeature table (Prisma) -> gis_features view -> gis_terenuri / gis_cladiri / gis_administrativ
-> gis_terenuri_status / gis_cladiri_status (with JOINs)
GisUat table -> gis_uats_z0/z5/z8/z12 (with ST_SimplifyPreserveTopology)
```
### MapLibre Layer Architecture
@@ -325,14 +366,30 @@ Sources (Martin): gis_uats_z0, z5, z8, z12, administrativ, terenuri, cladiri
Layers per source: fill + line + label (where applicable)
Selection: Separate highlight layers on terenuri source
Drawing: GeoJSON source for freehand/rect polygon
Building labels: cladiriLabel layer, cadastral_ref at minzoom 16
```
---
## Deployment Pitfalls (Discovered During Implementation)
1. **Portainer CE does not expose repo files to containers at runtime.** Volume mounts like `./file.conf:/etc/file.conf:ro` fail silently — Docker creates an empty directory. Always bake config files into custom images via Dockerfile COPY.
2. **Martin Docker tag format change at v1.0.** `v1.4.0` does not exist, `1.4.0` does. Always check [ghcr.io/maplibre/martin](https://github.com/maplibre/martin/pkgs/container/martin) for actual tags.
3. **Martin logs `UNKNOWN GEOMETRY TYPE` for PostGIS views.** This is normal — nested views don't register geometry types in `geometry_columns`. Does not affect functionality.
4. **Martin auto-discovery mode is unreliable for property inclusion.** Always use explicit config with `auto_publish: false` and per-source `properties:` definitions.
5. **Martin caches source schema at startup.** After PostGIS view DDL changes (e.g., adding columns to gis_features), Martin must be restarted to pick up new columns.
---
## References
- [Martin Documentation](https://maplibre.org/martin/)
- [Martin Releases](https://github.com/maplibre/martin/releases)
- [Martin Container Registry](https://github.com/maplibre/martin/pkgs/container/martin)
- [Vector Tiles Benchmark (Rechsteiner 2025)](https://github.com/FabianRechsteiner/vector-tiles-benchmark)
- [PMTiles Specification](https://github.com/protomaps/PMTiles)
- [tippecanoe (Felt)](https://github.com/felt/tippecanoe)
geoportal/skill-vector-tile-serving.md
+42 -10
View File
@@ -65,11 +65,18 @@ postgres:
area: float8
```
### Docker image tags
Martin changed tag format at v1.0:
- Pre-1.0: `ghcr.io/maplibre/martin:v0.15.0` (with `v` prefix)
- Post-1.0: `ghcr.io/maplibre/martin:1.4.0` (no `v` prefix)
### Docker deployment
**If your orchestrator has access to the full repo** (docker-compose CLI, Docker Swarm with repo checkout):
```yaml
martin:
image: ghcr.io/maplibre/martin:v1.4.0
image: ghcr.io/maplibre/martin:1.4.0
command: ["--config", "/config/martin.yaml"]
environment:
- DATABASE_URL=postgresql://user:pass@host:5432/db
@@ -79,6 +86,28 @@ martin:
- "3010:3000"
```
**If using Portainer CE or any system that only sees docker-compose.yml** (not full repo):
Volume mounts for repo files fail silently — Docker creates an empty directory instead.
Bake config into a custom image:
```dockerfile
# martin.Dockerfile
FROM ghcr.io/maplibre/martin:1.4.0
COPY martin.yaml /config/martin.yaml
```
```yaml
martin:
build:
context: .
dockerfile: martin.Dockerfile
command: ["--config", "/config/martin.yaml"]
environment:
- DATABASE_URL=postgresql://user:pass@host:5432/db
ports:
- "3010:3000"
```
### Custom SRID handling
Martin handles non-4326/3857 SRIDs natively. Set `default_srid` globally or `srid` per source. Martin reprojects to Web Mercator (3857) internally for tile envelope calculations. Your PostGIS spatial indexes on the source SRID are used correctly.
@@ -170,12 +199,15 @@ map.addSource('overview', {
## Common Pitfalls
1. **Martin auto-discovery drops properties** — always use explicit config with `auto_publish: false`
2. **Nested views lose SRID metadata** — cast geometry: `geom::geometry(Geometry, 3844)`
3. **GisUat.geometry is huge** — always `select` to exclude in list queries
4. **Low-zoom tiles scan entire dataset** — use zoom-dependent simplified views
5. **No tile cache by default** — add nginx/Varnish in front of any tile server
6. **tippecanoe requires WGS84** — reproject from custom SRID before generating PMTiles
7. **PMTiles not incrementally updatable** — full rebuild required on data change
8. **Tegola doesn't support custom SRIDs** — only 3857/4326, requires ST_Transform everywhere
9. **pg_tileserv `ST_Estimated_Extent` fails on views** — use materialized views or function layers
10. **Martin caches source schema at startup** — restart after view DDL changes
2. **Martin Docker tag format changed at v1.0** — `v0.15.0` (with v) but `1.4.0` (without v). Check actual tags at ghcr.io.
3. **Portainer CE volume mounts fail silently** — Docker creates empty directory instead of file. Bake configs into images via Dockerfile COPY.
4. **Martin logs `UNKNOWN GEOMETRY TYPE` for views** — normal for nested views, does not affect tile generation
5. **Nested views lose SRID metadata** — cast geometry: `geom::geometry(Geometry, 3844)`
6. **GisUat.geometry is huge** — always `select` to exclude in list queries
7. **Low-zoom tiles scan entire dataset** — use zoom-dependent simplified views
8. **No tile cache by default** — add nginx/Varnish in front of any tile server
9. **tippecanoe requires WGS84** — reproject from custom SRID before generating PMTiles
10. **PMTiles not incrementally updatable** — full rebuild required on data change
11. **Tegola doesn't support custom SRIDs** — only 3857/4326, requires ST_Transform everywhere
12. **pg_tileserv `ST_Estimated_Extent` fails on views** — use materialized views or function layers
13. **Martin caches source schema at startup** — restart after view DDL changes