initial: split from gov-agreg — vreau.digital standalone platform
Moved from gov-agreg/src/pages/achizitii/* to root (drop prefix). - 22 pages migrated, 127 files total - All internal links: /achizitii/X → /X (176 occurrences fixed) - AchizitiiLayout subnav rewritten: /X paths, top-right link to vreaudigital.ro hub - BaseLayout new (vreau.digital branding, OG tags, site URL) - astro.config.mjs: site https://vreau.digital, server output (was static) - docker-compose: port 5096 (vreaudigital is 5095), container vreau-digital - deploy.sh: paths /opt/vreau-digital, log /var/log/vreau-digital-deploy.log Backend shared with gov-agreg: - PostgreSQL satra (same schemas: seap, firms, anaf, anre, ...) - Photon, Martin tiles - Infisical /vreaudigital path (DATABASE_URL etc. shared) build: PASS (npx astro check 0 errors, npm run build 5s vite + 10s server)
This commit is contained in:
@@ -0,0 +1,84 @@
|
||||
# WSP Daily Sync — Deployment on satra (Docker)
|
||||
|
||||
The WSP scraper deploys as a Docker container on satra. The container exits
|
||||
after each run; a cron entry triggers it daily at 06:00.
|
||||
|
||||
## One-time setup
|
||||
|
||||
### 1. Sync code + cert from orchi
|
||||
```bash
|
||||
rsync -av --exclude='.venv' --exclude='__pycache__' --exclude='*.log' \
|
||||
/home/orchestrator/Code/gov-agreg/services/seap-scraper/ \
|
||||
satra:/opt/vreaudigital/services/seap-scraper/
|
||||
```
|
||||
|
||||
### 2. Create env file on satra
|
||||
```bash
|
||||
ssh satra "sudo mkdir -p /opt/wsp && sudo chown bulibasa:bulibasa /opt/wsp && chmod 700 /opt/wsp"
|
||||
|
||||
# From orchi (don't echo values to logs):
|
||||
( echo "DATABASE_URL=$(ssh satra 'grep ^DATABASE_URL /opt/architools/.env | cut -d= -f2-' | sed 's|?schema=[^&]*||;s|?$||')" && \
|
||||
source ~/Code/claude-dotfiles/load-infisical-path.sh /seap >/dev/null 2>&1 && \
|
||||
echo "SEAP_USER=$SEAP_USER" && \
|
||||
echo "SEAP_PASS=$SEAP_PASS" && \
|
||||
echo "SEAP_CERT_KEY=$SEAP_CERT_KEY" \
|
||||
) | ssh satra "tee /opt/wsp/.env >/dev/null && chmod 600 /opt/wsp/.env"
|
||||
```
|
||||
|
||||
### 3. Build the image on satra
|
||||
```bash
|
||||
ssh satra 'cd /opt/vreaudigital/services/seap-scraper && \
|
||||
docker compose -f wsp-docker-compose.yml --env-file /opt/wsp/.env build'
|
||||
```
|
||||
|
||||
### 4. Test run (manual)
|
||||
```bash
|
||||
ssh satra 'cd /opt/vreaudigital/services/seap-scraper && \
|
||||
docker compose -f wsp-docker-compose.yml --env-file /opt/wsp/.env run --rm wsp-incremental \
|
||||
python -m wsp.runner status'
|
||||
```
|
||||
|
||||
### 5. Install cron entry
|
||||
```bash
|
||||
ssh satra 'echo "0 6 * * * bulibasa cd /opt/vreaudigital/services/seap-scraper && \
|
||||
docker compose -f wsp-docker-compose.yml --env-file /opt/wsp/.env run --rm wsp-incremental \
|
||||
>> /var/log/wsp-incremental.log 2>&1" | sudo tee /etc/cron.d/wsp-incremental'
|
||||
ssh satra 'sudo chmod 644 /etc/cron.d/wsp-incremental'
|
||||
```
|
||||
|
||||
## Manual operation
|
||||
|
||||
### Check status
|
||||
```bash
|
||||
ssh satra 'cd /opt/vreaudigital/services/seap-scraper && \
|
||||
docker compose -f wsp-docker-compose.yml --env-file /opt/wsp/.env run --rm wsp-incremental \
|
||||
python -m wsp.runner status'
|
||||
```
|
||||
|
||||
### Run incremental for one op
|
||||
```bash
|
||||
ssh satra 'cd /opt/vreaudigital/services/seap-scraper && \
|
||||
docker compose -f wsp-docker-compose.yml --env-file /opt/wsp/.env run --rm wsp-incremental \
|
||||
python -m wsp.runner incremental SU_CaNotices'
|
||||
```
|
||||
|
||||
### Refresh materialized views (after sync)
|
||||
```bash
|
||||
ssh satra 'docker exec architools_postgres psql -U architools_user -d architools_db \
|
||||
-c "SELECT seap.refresh_wsp_views()"'
|
||||
```
|
||||
|
||||
## Backfill (one-time, large)
|
||||
|
||||
Run from **orchi** (5 workers, 12 months, 1-2h):
|
||||
```bash
|
||||
. /tmp/wsp_env.sh && cd ~/Code/gov-agreg/services/seap-scraper && \
|
||||
./.venv/bin/python -m wsp.runner backfill SU_CaNotices --start 2025-05-06 --end 2026-05-06 --workers 5
|
||||
```
|
||||
|
||||
Or from satra container:
|
||||
```bash
|
||||
ssh satra 'cd /opt/vreaudigital/services/seap-scraper && \
|
||||
docker compose -f wsp-docker-compose.yml --env-file /opt/wsp/.env run --rm wsp-incremental \
|
||||
python -m wsp.runner backfill SU_CaNotices --start 2025-05-06 --end 2026-05-06 --workers 5'
|
||||
```
|
||||
Reference in New Issue
Block a user