gitadmin/ArchiTools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(parcel-sync): incremental sync, smart export, auto-refresh + weekend deep sync

Browse Source 
Sync Incremental:
- Add fetchObjectIds (returnIdsOnly) to eterra-client — fetches only OBJECTIDs in 1 request
- Add fetchFeaturesByObjectIds — downloads only delta features by OBJECTID IN (...)
- Rewrite syncLayer: compare remote IDs vs local, download only new features
- Fallback to full sync for first sync, forceFullSync, or delta > 50%
- Reduces sync time from ~10 min to ~5-10s for typical updates

Smart Export Tab:
- Hero buttons detect DB freshness — use export-local (instant) when data is fresh
- Dynamic subtitles: "Din DB (sync acum Xh)" / "Sync incremental" / "Sync complet"
- Re-sync link when data is fresh but user wants forced refresh
- Removed duplicate "Descarca din DB" buttons from background section

Auto-Refresh Scheduler:
- Self-contained timer via instrumentation.ts (Next.js startup hook)
- Weekday 1-5 AM: incremental refresh for existing UATs in DB
- Staggered processing with random delays between UATs
- Health check before processing, respects eTerra maintenance

Weekend Deep Sync:
- Full Magic processing for 9 large municipalities (Cluj, Bistrita, TgMures, etc.)
- Runs Fri/Sat/Sun 23:00-04:00, round-robin intercalated between cities
- 4 steps per city: sync terenuri, sync cladiri, import no-geom, enrichment
- State persisted in KeyValueStore — survives restarts, continues across nights
- Email status report at end of each session via Brevo SMTP
- Admin page at /wds: add/remove cities, view progress, reset
- Hint link on export tab pointing to /wds

API endpoints:
- POST /api/eterra/auto-refresh — N8N-compatible cron endpoint (Bearer token auth)
- GET/POST /api/eterra/weekend-sync — queue management for /wds page

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
AI Assistant
2026-03-26 20:50:34 +02:00
parent 8f65efd5d1
commit 3b456eb481
11 changed files with 1929 additions and 128 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/modules/parcel-sync/components/tabs/export-tab.tsx
+102 -86
View File
@@ -15,7 +15,9 @@ import {
Clock,
ArrowDownToLine,
AlertTriangle,
Moon,
} from "lucide-react";
import Link from "next/link";
import { Button } from "@/shared/components/ui/button";
import { Badge } from "@/shared/components/ui/badge";
import { Card, CardContent } from "@/shared/components/ui/card";
@@ -141,6 +143,20 @@ export function ExportTab({
const dbTotalFeatures = dbLayersSummary.reduce((sum, l) => sum + l.count, 0);
const allFresh =
dbLayersSummary.length > 0 && dbLayersSummary.every((l) => l.isFresh);
const hasData = dbTotalFeatures > 0;
const canExportLocal = allFresh && hasData;
const oldestSyncDate = dbLayersSummary.reduce(
(oldest, l) => {
if (!l.lastSynced) return oldest;
if (!oldest || l.lastSynced < oldest) return l.lastSynced;
return oldest;
},
null as Date | null,
);
const progressPct =
exportProgress?.total && exportProgress.total > 0
? Math.round((exportProgress.downloaded / exportProgress.total) * 100)
@@ -649,52 +665,85 @@ export function ExportTab({
)}
{/* Hero buttons */}
{sirutaValid && session.connected ? (
<div className="grid gap-3 sm:grid-cols-2">
<Button
size="lg"
className="h-auto py-4 text-base bg-zinc-900 hover:bg-zinc-800 text-white dark:bg-zinc-100 dark:text-zinc-900 dark:hover:bg-zinc-200"
disabled={exporting}
onClick={() => void handleExportBundle("base")}
>
{exporting && exportProgress?.phase !== "Detalii parcele" ? (
<Loader2 className="mr-2 h-5 w-5 animate-spin" />
) : (
<FileDown className="mr-2 h-5 w-5" />
)}
<div className="text-left">
<div className="font-semibold">
Descarcă Terenuri și Clădiri
{sirutaValid && (session.connected || canExportLocal) ? (
<div className="space-y-2">
<div className="grid gap-3 sm:grid-cols-2">
<Button
size="lg"
className="h-auto py-4 text-base bg-zinc-900 hover:bg-zinc-800 text-white dark:bg-zinc-100 dark:text-zinc-900 dark:hover:bg-zinc-200"
disabled={exporting || downloadingFromDb}
onClick={() =>
canExportLocal
? void handleDownloadFromDb("base")
: void handleExportBundle("base")
}
>
{(exporting || downloadingFromDb) &&
exportProgress?.phase !== "Detalii parcele" ? (
<Loader2 className="mr-2 h-5 w-5 animate-spin" />
) : canExportLocal ? (
<Database className="mr-2 h-5 w-5" />
) : (
<FileDown className="mr-2 h-5 w-5" />
)}
<div className="text-left">
<div className="font-semibold">
Descarcă Terenuri și Clădiri
</div>
<div className="text-xs opacity-70 font-normal">
{canExportLocal
? `Din DB (sync ${oldestSyncDate ? relativeTime(oldestSyncDate) : "recent"})`
: hasData
? "Sync incremental + GPKG"
: "Sync complet + GPKG"}
</div>
</div>
<div className="text-xs opacity-70 font-normal">
Sync + GPKG (din cache dacă e proaspăt)
</div>
</div>
</Button>
</Button>
<Button
size="lg"
className="h-auto py-4 text-base bg-teal-700 hover:bg-teal-600 text-white dark:bg-teal-600 dark:hover:bg-teal-500"
disabled={exporting}
onClick={() => void handleExportBundle("magic")}
>
{exporting && exportProgress?.phase === "Detalii parcele" ? (
<Loader2 className="mr-2 h-5 w-5 animate-spin" />
) : (
<Sparkles className="mr-2 h-5 w-5" />
)}
<div className="text-left">
<div className="font-semibold">Magic</div>
<div className="text-xs opacity-70 font-normal">
Sync + îmbogățire (CF, proprietari, adresă) + GPKG + CSV
<Button
size="lg"
className="h-auto py-4 text-base bg-teal-700 hover:bg-teal-600 text-white dark:bg-teal-600 dark:hover:bg-teal-500"
disabled={exporting || downloadingFromDb}
onClick={() =>
canExportLocal
? void handleDownloadFromDb("magic")
: void handleExportBundle("magic")
}
>
{(exporting || downloadingFromDb) &&
exportProgress?.phase === "Detalii parcele" ? (
<Loader2 className="mr-2 h-5 w-5 animate-spin" />
) : (
<Sparkles className="mr-2 h-5 w-5" />
)}
<div className="text-left">
<div className="font-semibold">Magic</div>
<div className="text-xs opacity-70 font-normal">
{canExportLocal
? `GPKG + CSV din DB (sync ${oldestSyncDate ? relativeTime(oldestSyncDate) : "recent"})`
: "Sync + îmbogățire (CF, proprietari, adresă) + GPKG + CSV"}
</div>
</div>
</Button>
</div>
{canExportLocal && session.connected && (
<div className="text-center">
<button
type="button"
className="text-xs text-muted-foreground hover:text-foreground underline transition-colors"
disabled={exporting}
onClick={() => void handleExportBundle("base")}
>
<RefreshCw className="inline h-3 w-3 mr-1 -mt-0.5" />
Re-sincronizează de pe eTerra
</button>
</div>
</Button>
)}
</div>
) : (
<Card>
<CardContent className="py-12 text-center text-muted-foreground">
{!session.connected ? (
{!session.connected && !canExportLocal ? (
<>
<Wifi className="h-10 w-10 mx-auto mb-3 opacity-30" />
<p>Conectează-te la eTerra pentru a activa exportul.</p>
@@ -1222,54 +1271,6 @@ export function ExportTab({
</div>
)}
{/* Row 3: Download from DB buttons */}
{dbTotalFeatures > 0 && (
<div className="grid gap-2 sm:grid-cols-2">
<Button
variant="outline"
size="sm"
className="h-auto py-2.5 justify-start"
disabled={downloadingFromDb}
onClick={() => void handleDownloadFromDb("base")}
>
{downloadingFromDb ? (
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
) : (
<Database className="mr-2 h-4 w-4" />
)}
<div className="text-left">
<div className="text-xs font-semibold">
Descarcă din DB — Bază
</div>
<div className="text-[10px] opacity-60 font-normal">
GPKG terenuri + clădiri (instant, fără eTerra)
</div>
</div>
</Button>
<Button
variant="outline"
size="sm"
className="h-auto py-2.5 justify-start border-teal-200 dark:border-teal-800 hover:bg-teal-50 dark:hover:bg-teal-950/30"
disabled={downloadingFromDb}
onClick={() => void handleDownloadFromDb("magic")}
>
{downloadingFromDb ? (
<Loader2 className="mr-2 h-4 w-4 animate-spin text-teal-600" />
) : (
<Sparkles className="mr-2 h-4 w-4 text-teal-600" />
)}
<div className="text-left">
<div className="text-xs font-semibold">
Descarcă din DB — Magic
</div>
<div className="text-[10px] opacity-60 font-normal">
GPKG + CSV + raport calitate (instant)
</div>
</div>
</Button>
</div>
)}
{!session.connected && dbTotalFeatures === 0 && (
<p className="text-xs text-muted-foreground ml-6">
Conectează-te la eTerra pentru a porni sincronizarea fundal,
@@ -1280,6 +1281,21 @@ export function ExportTab({
</Card>
)}
{/* Weekend Deep Sync hint */}
<div className="flex items-center gap-2 text-xs text-muted-foreground">
<Moon className="h-3.5 w-3.5 shrink-0" />
<span>
Municipii mari cu Magic complet?{" "}
<Link
href="/wds"
className="underline hover:text-foreground transition-colors"
>
Weekend Deep Sync
</Link>
{" "}— sincronizare automata Vin/Sam/Dum noaptea.
</span>
</div>
{/* Background sync progress */}
{bgJobId && bgProgress && bgProgress.status !== "unknown" && (
<Card