67f3237761
- Portainer CE volume mount pitfall (silent empty directory creation) - Martin Docker tag format change at v1.0 (v prefix dropped) - UNKNOWN GEOMETRY TYPE log is normal for views - Bake-into-image pattern for config files in Portainer deployments - Updated all implementation prompts with Portainer-safe instructions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
401 lines
16 KiB
Markdown
401 lines
16 KiB
Markdown
# Tile Server Evaluation — ArchiTools Geoportal (March 2026)
|
|
|
|
## Context
|
|
|
|
ArchiTools Geoportal serves vector tiles (MVT) from PostgreSQL 16 + PostGIS 3 via Martin.
|
|
Data: ~330K GIS features (parcels, buildings, admin boundaries) in EPSG:3844 (Stereo70), growing to 1M+.
|
|
Frontend: MapLibre GL JS 5.21, Next.js 16, Docker self-hosted via Portainer CE.
|
|
|
|
---
|
|
|
|
## Problem Statement
|
|
|
|
1. Martin v0.15.0 was running in **auto-discovery mode** — the existing `martin.yaml` config was never mounted
|
|
2. Building labels (`cadastral_ref`) missing from MVT tiles despite the view exposing them
|
|
3. Performance concerns at scale (330K -> 1M+ features)
|
|
|
|
---
|
|
|
|
## Solutions Evaluated (7 options + emerging tech)
|
|
|
|
### 1. Martin (Fix + Upgrade) — WINNER
|
|
|
|
| Aspect | Detail |
|
|
|---|---|
|
|
| Root cause | `martin.yaml` not mounted in docker-compose — Martin ran in auto-discovery mode |
|
|
| Fix | Bake config into custom image via Dockerfile + upgrade v0.15 -> v1.4.0 |
|
|
| Performance | Fastest tile server benchmarked (2-3x faster than #2 Tegola) |
|
|
| EPSG:3844 | Native support via `default_srid: 3844` |
|
|
| New in v1.4 | ZSTD compression, MLT format, materialized views, better logging |
|
|
|
|
**Status: IMPLEMENTED AND VERIFIED IN PRODUCTION** (2026-03-27)
|
|
|
|
### 2. pg_tileserv (CrunchyData)
|
|
|
|
| Aspect | Detail |
|
|
|---|---|
|
|
| Architecture | Go binary, zero-config, delegates ST_AsMVT to PostGIS |
|
|
| Property control | Auto from schema + URL `?properties=` parameter |
|
|
| Performance | 2-3x slower than Martin (Rechsteiner benchmark) |
|
|
| EPSG:3844 | Supported (auto-reprojects via ST_Transform) |
|
|
| Killer feature | Function-based sources (full SQL tile functions) |
|
|
| Dealbreaker | View extent estimation bug (#156) affects all our views, development stagnant (last release Feb 2025) |
|
|
|
|
**Verdict: NO** — slower, buggy with views, stagnant development.
|
|
|
|
### 3. Tegola (Go-based)
|
|
|
|
| Aspect | Detail |
|
|
|---|---|
|
|
| Architecture | Go, TOML config, explicit per-layer SQL |
|
|
| Performance | 2nd in benchmarks, but 2-3x slower than Martin |
|
|
| Built-in cache | File, S3/MinIO, Redis — with seed/purge CLI |
|
|
| EPSG:3844 | **NOT SUPPORTED** (only 3857/4326) — requires ST_Transform in every query |
|
|
| Killer feature | Built-in tile seeding and cache purging |
|
|
|
|
**Verdict: NO** — EPSG:3844 not supported, dealbreaker for our data.
|
|
|
|
### 4. t-rex (Rust-based)
|
|
|
|
| Aspect | Detail |
|
|
|---|---|
|
|
| Status | **Abandoned/unmaintained** — no releases since 2023 |
|
|
|
|
**Verdict: NO** — dead project.
|
|
|
|
### 5. GeoJSON Direct from Next.js API
|
|
|
|
| Aspect | Detail |
|
|
|---|---|
|
|
| 330K features | ~270 MB uncompressed, 800 MB-1.4 GB browser memory |
|
|
| Browser impact | 10-30s main thread freeze, mobile crash |
|
|
| Pan/zoom | Full re-fetch on every viewport change, flickering |
|
|
| Viable range | Only at zoom 16+ with <500 features in viewport |
|
|
|
|
**Verdict: NO** — does not scale beyond ~20K features.
|
|
|
|
### 6. PMTiles (Pre-generated)
|
|
|
|
| Aspect | Detail |
|
|
|---|---|
|
|
| Architecture | Single-file tile archive, HTTP Range Requests, no server needed |
|
|
| Performance | ~5ms per tile (vs 200-2000ms for Martin on low-zoom) |
|
|
| Property control | tippecanoe gives explicit include/exclude per property |
|
|
| Update strategy | Full rebuild required (~3-7 min for 330K features) |
|
|
| EPSG:3844 | Requires reprojection to 4326 via ogr2ogr before tippecanoe |
|
|
| MinIO serving | Yes — direct HTTP Range Requests with CORS |
|
|
|
|
**Verdict: YES as hybrid complement** — excellent for static UAT overview layers (z0-z12), Martin for live detail.
|
|
|
|
### 7. Emerging Solutions
|
|
|
|
| Solution | Status | Relevance |
|
|
|---|---|---|
|
|
| **mvt-rs** (Rust) | v0.16.2, active | Admin UI, auth per layer, cache — good for multi-tenant |
|
|
| **MLT format** | Stable Jan 2026 | 6x compression, 4x faster decode — Martin v1.3+ supports it |
|
|
| **BBOX** | Maturing | Similar to Tegola performance, unified raster+vector |
|
|
| **DuckDB tiles** | Early | Not PostGIS replacement, interesting for GeoParquet |
|
|
| **FlatGeobuf** | Stable | Good for <100K features, not a tile replacement |
|
|
|
|
---
|
|
|
|
## Benchmark Reference (Rechsteiner, April 2025)
|
|
|
|
| Rank | Server | Language | Relative Speed |
|
|
|---|---|---|---|
|
|
| 1 | **Martin** | Rust | 1x (fastest) |
|
|
| 2 | Tegola | Go | 2-3x slower |
|
|
| 3 | BBOX | Rust | ~same as Tegola |
|
|
| 4 | pg_tileserv | Go | ~4x slower |
|
|
| 5 | TiPg | Python | Slower |
|
|
| 6 | ldproxy | Java | 4-70x slower |
|
|
|
|
Source: [github.com/FabianRechsteiner/vector-tiles-benchmark](https://github.com/FabianRechsteiner/vector-tiles-benchmark)
|
|
|
|
---
|
|
|
|
## Implementation Roadmap
|
|
|
|
### Phase 1: Martin Fix — DONE (2026-03-27)
|
|
|
|
Changes applied:
|
|
- `martin.Dockerfile`: custom image that COPY-s `martin.yaml` into `/config/`
|
|
- `docker-compose.yml`: Martin v0.15 -> v1.4.0, build from Dockerfile, `--config` flag
|
|
- `martin.yaml`: comment updated to reflect v1.4
|
|
- `map-viewer.tsx`: building labels layer activated (`cladiriLabel` at minzoom 16)
|
|
|
|
#### Deployment Lessons Learned
|
|
|
|
1. **Docker image tag format changed at v1.0**: old tags use `v` prefix (`v0.15.0`), new tags do not (`1.4.0`). The tag `ghcr.io/maplibre/martin:v1.4.0` does NOT exist — correct is `ghcr.io/maplibre/martin:1.4.0`.
|
|
|
|
2. **Portainer CE volume mount pitfall**: volume `./martin.yaml:/config/martin.yaml:ro` fails because Portainer deploys only the docker-compose.yml content, not the full git repo. Docker silently creates an empty directory instead of failing. Solution: bake config into a custom image with a 2-line Dockerfile:
|
|
```dockerfile
|
|
FROM ghcr.io/maplibre/martin:1.4.0
|
|
COPY martin.yaml /config/martin.yaml
|
|
```
|
|
|
|
3. **Martin config format is stable**: YAML format unchanged from v0.15 to v1.4 — `postgres.tables`, `connection_string`, `auto_publish`, `properties` map all work identically. No migration needed.
|
|
|
|
4. **PostGIS view geometry type**: Martin logs `UNKNOWN GEOMETRY TYPE` for all views — this is normal for nested views (`SELECT * FROM parent_view`). Views don't register specific geometry types in `geometry_columns`. Does not affect tile generation or property inclusion.
|
|
|
|
### Phase 2A: nginx Tile Cache
|
|
|
|
**Impact**: 10-100x faster on repeat requests, zero PostGIS load for cached tiles.
|
|
**Effort**: ~2 hours.
|
|
|
|
#### Implementation Prompt
|
|
|
|
```
|
|
Add an nginx reverse proxy cache in front of Martin for tile serving in ArchiTools.
|
|
|
|
Context:
|
|
- Martin serves tiles at http://martin:3000 (container name: martin, internal port 3000)
|
|
- Martin is built from martin.Dockerfile (COPY martin.yaml into image)
|
|
- Traefik proxies external traffic to ArchiTools at tools.beletage.ro
|
|
- Current tile URL pattern: https://tools.beletage.ro/tiles/{source}/{z}/{x}/{y}
|
|
- NEXT_PUBLIC_MARTIN_URL=https://tools.beletage.ro/tiles
|
|
- Portainer CE deploys from Gitea repo — files must be in git, not just on host
|
|
|
|
Requirements:
|
|
1. Create an nginx container `tile-cache` in docker-compose.yml
|
|
2. nginx config: proxy_cache for all paths with:
|
|
- Cache zone: 2GB max, keys in shared memory
|
|
- Cache valid: 200 responses for 1 hour
|
|
- Stale serving on error/timeout
|
|
- Cache-Control headers passed through
|
|
- CORS headers for tiles (Access-Control-Allow-Origin: *)
|
|
- Gzip/brotli passthrough (Martin already compresses)
|
|
3. Route Martin traffic through tile-cache:
|
|
- tile-cache listens on port 3010 (host) -> 80 (container)
|
|
- tile-cache proxies to http://martin:3000
|
|
- Martin removes its host port mapping (only accessible via tile-cache)
|
|
4. Volume for persistent cache across container restarts
|
|
5. IMPORTANT: nginx config must be baked into a custom image (same pattern as martin.Dockerfile)
|
|
because Portainer CE cannot mount files from the repo. Create nginx/tile-cache.conf and
|
|
a tile-cache.Dockerfile.
|
|
|
|
Do NOT change the frontend NEXT_PUBLIC_MARTIN_URL — keep the same external URL.
|
|
Build with `npx next build` to verify zero errors.
|
|
```
|
|
|
|
### Phase 2B: PMTiles for UAT Overview Layers
|
|
|
|
**Impact**: Sub-10ms overview tiles, zero PostGIS load for z0-z12.
|
|
**Effort**: ~4-6 hours.
|
|
|
|
#### Implementation Prompt
|
|
|
|
```
|
|
Implement PMTiles pre-generation for UAT overview layers in ArchiTools Geoportal.
|
|
|
|
Context:
|
|
- PostGIS at 10.10.10.166:5432, database architools_db, user architools_user
|
|
- Views: gis_uats_z0, gis_uats_z5, gis_uats_z8, gis_uats_z12, gis_administrativ
|
|
- All geometries EPSG:3844 (Stereo70), need reprojection to 4326 for tippecanoe
|
|
- MinIO at 10.10.10.166:9002 (API) / 9003 (console), bucket for tiles
|
|
- Frontend: MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx
|
|
- Portainer CE deploys from Gitea — any config files must be baked into Docker images
|
|
|
|
Requirements:
|
|
|
|
1. Create `scripts/rebuild-overview-tiles.sh`:
|
|
- Export each view with ogr2ogr: -f FlatGeobuf -s_srs EPSG:3844 -t_srs EPSG:4326
|
|
- Generate combined PMTiles with tippecanoe:
|
|
- --layer per view, --minimum-zoom=0, --maximum-zoom=14
|
|
- --detect-shared-borders (critical for adjacent UAT polygons)
|
|
- --hilbert for compression
|
|
- Atomic upload to MinIO: upload as overview_new.pmtiles, then rename to overview.pmtiles
|
|
- Cleanup temp files
|
|
|
|
2. Create Dockerfile for tippecanoe build container (or use ghcr.io/felt/tippecanoe)
|
|
|
|
3. Add `tippecanoe` service to docker-compose.yml (one-shot, for manual/cron runs)
|
|
|
|
4. Configure MinIO:
|
|
- Create bucket `tiles` with public read
|
|
- CORS: Allow GET/HEAD from tools.beletage.ro, expose Range/Content-Range headers
|
|
|
|
5. Update map-viewer.tsx:
|
|
- npm install pmtiles
|
|
- Register pmtiles:// protocol on MapLibre
|
|
- Add PMTiles source for overview layers (z0-z14)
|
|
- Keep existing Martin sources for detail layers (z14+)
|
|
- Set zoom breakpoints: PMTiles below z14, Martin above z14
|
|
|
|
6. Add N8N webhook trigger or cron for nightly rebuild after weekend deep sync
|
|
|
|
The UAT overview layers change rarely (only when new UATs are synced).
|
|
Parcel/building layers stay on Martin for live data freshness.
|
|
|
|
Build with `npx next build` to verify zero errors.
|
|
Read CLAUDE.md for project conventions before starting.
|
|
```
|
|
|
|
### Phase 2C: MLT Format Testing
|
|
|
|
**Impact**: 6x smaller tiles, 4x faster client decode.
|
|
**Effort**: ~1 hour to test.
|
|
|
|
#### Implementation Prompt
|
|
|
|
```
|
|
Test MLT (MapLibre Tiles) format on one layer in ArchiTools Geoportal.
|
|
|
|
Context:
|
|
- Martin v1.4.0 running with config baked in via martin.Dockerfile
|
|
- MapLibre GL JS 5.21 in src/modules/geoportal/components/map-viewer.tsx
|
|
- Test layer: gis_terenuri (largest layer, ~250K features)
|
|
|
|
Requirements:
|
|
|
|
1. Research how Martin v1.4 serves MLT format:
|
|
- Check if it's automatic via Accept header or needs config
|
|
- Check Martin docs for MLT serving configuration
|
|
|
|
2. Update map-viewer.tsx to request MLT for one source (gis_terenuri):
|
|
- Add `encoding: "mlt"` to the vector source definition if MapLibre supports it
|
|
- Or configure via source URL parameter if Martin expects it
|
|
|
|
3. Test and measure:
|
|
- Compare tile sizes: MVT vs MLT for same tile coordinates
|
|
- Compare decode time in browser DevTools Network tab
|
|
- Check that all properties (cadastral_ref, area_value, etc.) survive MLT encoding
|
|
- Check label rendering still works
|
|
|
|
4. If MLT works correctly, apply to all Martin sources
|
|
If issues found, document them and revert to MVT
|
|
|
|
This is experimental — keep MVT as fallback. Do not break existing functionality.
|
|
Build with `npx next build` to verify zero errors.
|
|
```
|
|
|
|
### Phase 2D: mvt-rs Evaluation (Future — Multi-Tenant)
|
|
|
|
**Impact**: Built-in auth, admin UI, per-layer access control.
|
|
**Effort**: 1-2 days for evaluation + migration.
|
|
|
|
#### Implementation Prompt
|
|
|
|
```
|
|
Evaluate mvt-rs as a replacement for Martin in ArchiTools Geoportal for multi-tenant deployment.
|
|
|
|
Context:
|
|
- Current: Martin v1.4.0 serving 9 PostGIS sources (views in EPSG:3844)
|
|
- Martin config baked into image via martin.Dockerfile
|
|
- Goal: Expose geoportal to external clients with per-layer access control
|
|
- mvt-rs repo: https://github.com/mvt-proj/mvt-rs (v0.16.2+, Rust, Salvo framework)
|
|
|
|
Requirements:
|
|
|
|
1. Deploy mvt-rs as a Docker container alongside Martin (don't replace yet)
|
|
- Use same DATABASE_URL
|
|
- Map to different port (e.g., 3011)
|
|
|
|
2. Configure mvt-rs with equivalent layers to martin.yaml:
|
|
- gis_uats_z0/z5/z8/z12, gis_administrativ, gis_terenuri, gis_cladiri
|
|
- gis_terenuri_status, gis_cladiri_status
|
|
- All with EPSG:3844, explicit properties
|
|
|
|
3. Test:
|
|
- Do all properties appear in MVT output? (especially cadastral_ref on gis_cladiri)
|
|
- Performance comparison: curl timing for 10 representative tiles vs Martin
|
|
- Admin UI: create test user, assign layer permissions
|
|
- Cache: configure disk cache, measure cold vs warm tile times
|
|
|
|
4. Document findings:
|
|
- Property inclusion: pass/fail per layer
|
|
- Performance delta vs Martin
|
|
- Admin UI capabilities and limitations
|
|
- Missing features vs Martin (PMTiles, MLT, etc.)
|
|
|
|
5. Decision matrix: when to switch from Martin to mvt-rs
|
|
|
|
Do NOT modify the production Martin setup. This is a parallel evaluation only.
|
|
```
|
|
|
|
---
|
|
|
|
## Key Technical Details
|
|
|
|
### Martin v1.4.0 Deployment Architecture
|
|
|
|
```
|
|
Gitea repo (martin.yaml + martin.Dockerfile)
|
|
-> Portainer CE builds custom image: FROM martin:1.4.0, COPY martin.yaml
|
|
-> Container starts with --config /config/martin.yaml
|
|
-> Reads DATABASE_URL from environment
|
|
-> Serves 9 PostGIS view sources on port 3000
|
|
-> Host maps 3010:3000
|
|
-> Traefik proxies tools.beletage.ro/tiles -> host:3010
|
|
```
|
|
|
|
**Critical**: Do NOT use volume mounts for config files in Portainer CE stacks.
|
|
Always bake configs into custom images via Dockerfile COPY.
|
|
|
|
### Martin Config (validated compatible v0.15 through v1.4)
|
|
|
|
The `martin.yaml` at project root defines 9 sources with explicit properties.
|
|
Config format unchanged from v0.15 to v1.4 — no migration needed.
|
|
|
|
Key config features used:
|
|
- `auto_publish: false` — only explicitly listed sources are served
|
|
- `default_srid: 3844` — all sources use Stereo70
|
|
- `properties:` map per source — explicit column name + PostgreSQL type
|
|
- `minzoom/maxzoom` per source — controls tile generation range
|
|
- `bounds: [20.2, 43.5, 30.0, 48.3]` — approximate Romania extent
|
|
|
|
### Docker Image Tag Convention
|
|
|
|
Martin changed tag format at v1.0:
|
|
- Pre-1.0: `ghcr.io/maplibre/martin:v0.15.0` (with `v` prefix)
|
|
- Post-1.0: `ghcr.io/maplibre/martin:1.4.0` (no `v` prefix)
|
|
- Also available: `latest`, `nightly`
|
|
|
|
### PostGIS View Chain
|
|
|
|
```
|
|
GisFeature table (Prisma) -> gis_features view -> gis_terenuri / gis_cladiri / gis_administrativ
|
|
-> gis_terenuri_status / gis_cladiri_status (with JOINs)
|
|
GisUat table -> gis_uats_z0/z5/z8/z12 (with ST_SimplifyPreserveTopology)
|
|
```
|
|
|
|
### MapLibre Layer Architecture
|
|
|
|
```
|
|
Sources (Martin): gis_uats_z0, z5, z8, z12, administrativ, terenuri, cladiri
|
|
Layers per source: fill + line + label (where applicable)
|
|
Selection: Separate highlight layers on terenuri source
|
|
Drawing: GeoJSON source for freehand/rect polygon
|
|
Building labels: cladiriLabel layer, cadastral_ref at minzoom 16
|
|
```
|
|
|
|
---
|
|
|
|
## Deployment Pitfalls (Discovered During Implementation)
|
|
|
|
1. **Portainer CE does not expose repo files to containers at runtime.** Volume mounts like `./file.conf:/etc/file.conf:ro` fail silently — Docker creates an empty directory. Always bake config files into custom images via Dockerfile COPY.
|
|
|
|
2. **Martin Docker tag format change at v1.0.** `v1.4.0` does not exist, `1.4.0` does. Always check [ghcr.io/maplibre/martin](https://github.com/maplibre/martin/pkgs/container/martin) for actual tags.
|
|
|
|
3. **Martin logs `UNKNOWN GEOMETRY TYPE` for PostGIS views.** This is normal — nested views don't register geometry types in `geometry_columns`. Does not affect functionality.
|
|
|
|
4. **Martin auto-discovery mode is unreliable for property inclusion.** Always use explicit config with `auto_publish: false` and per-source `properties:` definitions.
|
|
|
|
5. **Martin caches source schema at startup.** After PostGIS view DDL changes (e.g., adding columns to gis_features), Martin must be restarted to pick up new columns.
|
|
|
|
---
|
|
|
|
## References
|
|
|
|
- [Martin Documentation](https://maplibre.org/martin/)
|
|
- [Martin Releases](https://github.com/maplibre/martin/releases)
|
|
- [Martin Container Registry](https://github.com/maplibre/martin/pkgs/container/martin)
|
|
- [Vector Tiles Benchmark (Rechsteiner 2025)](https://github.com/FabianRechsteiner/vector-tiles-benchmark)
|
|
- [PMTiles Specification](https://github.com/protomaps/PMTiles)
|
|
- [tippecanoe (Felt)](https://github.com/felt/tippecanoe)
|
|
- [MLT Format Announcement](https://maplibre.org/news/2026-01-23-mlt-release/)
|
|
- [mvt-rs](https://github.com/mvt-proj/mvt-rs)
|
|
- [pg_tileserv](https://github.com/CrunchyData/pg_tileserv)
|
|
- [Tegola](https://github.com/go-spatial/tegola)
|
|
- [Serving Vector Tiles Fast (Spatialists)](https://spatialists.ch/posts/2025/04/05-serving-vector-tiles-fast/)
|