ctde_v0.run_lag_t3¶
T3 connectivity-mechanism test — does the soft dual hold a GLOBAL-λ₂ target @32²/10?
The question: how high can we push REAL, GLOBAL connectivity (team algebraic connectivity λ₂ ≥ a target θ) WITHOUT collapsing coverage — and does the learned dual actually do the work?
Why the target is GLOBAL λ₂ and not per-agent degree (the correction): Connectivity is topological — the whole team in ONE connected piece. A spread-out spanning tree/chain is connected (λ₂ > 0) with most agents holding only 1-2 neighbours. Penalising low degree would fight the very spreading coverage needs (and degree ≥ 1 per agent does not even guarantee one component — the team can fragment into pairs). So the constraint is on the true global λ₂. It is computed by the TRAINER from all positions (centralised training) and enters only through the REWARD; the agents never observe it and act on local info at execution (CTDE). NO hard action mask ever (hard mask makes connectivity unbreakable → kills the resilience study).
The mechanism (thermostat): a per-step reward penalty -λ · relu(θ - λ₂) is subtracted; the
policy (primal) descends the PPO loss on that shaped reward, and the dual λ (dual ascent) rises
whenever the target is violated until connectivity reaches θ. We sweep the target θ so it BINDS
(a slack target never fires λ → tests nothing) and to trace the coverage↔connectivity frontier.
MECHANISM (3, ladder of adaptiveness): soft_lambda — λ is a FIXED constant (brittle baseline: one weight for all scales). lagrangian — adaptive dual ascent λ←relu(λ+lr·v); auto-tunes but MONOTONE (only ratchets up). pid_lagrangian — PID dual (Stooke 2020); responsive — P/D let λ rise AND fall, tracks θ stably. TARGET θ (the λ₂ floor, swept to bind): 0.5 (real-connectivity bar) · 0.7 (demanding).
ARMS (5): soft_th5 : soft_lambda · θ=0.5 (fixed-weight baseline, penalty=1.0) lag_th5 : lagrangian · θ=0.5 · lr=0.05 lag_th7 : lagrangian · θ=0.7 · lr=0.05 pid_th5 : pid_lagrangian · θ=0.5 (default PID gains) pid_th7 : pid_lagrangian · θ=0.7 (default PID gains) Fixed: 32²/10 · comm_r 5 · collision-mask on · frontier-attn · role split (expl_relay) · up-weighted coverage (--w-coverage 3) · cover_r 0 · conn-signal global_lambda2 · the FULL 7-channel SLAM occupancy KB belief (--sense-walls --sense-free --boundary: free/occupied/unknown + occ_frontier + boundary; was 5-channel coverage-trail).
Per seed: 5 runs. Run-dir
We log the two curves that ARE the test — the dual λ(t) (dual_lambda_next) and the realised
connectivity λ₂(t) (dual_violation / connectivity_real; emitted by ppo.py ~1147-1151) land in
each run's history.json — so we can SEE whether λ actually rises to enforce θ and whether λ₂
converges to it.
GRADE: the coverage↔connectivity_real frontier (does forcing λ₂→θ collapse coverage?) + does the dual track θ. Expectation: pid holds θ more cleanly than monotone lagrangian; both beat fixed soft_lambda. If λ climbs but λ₂ never reaches θ, local info isn't enough → relay / λ₂-estimator.
# see the whole plan first (no training):
JAX_PLATFORMS=cpu PYTHONPATH=.:../../../FiedlerValueEstimation /Users/bijanmehr/Project.Zymera/zymera_lab/.venv/bin/python -m ctde_v0.run_lag_t3 --dry-run --seeds 3
# the real batch on balthar (drop JAX_PLATFORMS so it uses the GPU), 3 at a time:
PYTHONPATH=.:../../../FiedlerValueEstimation ~/ZymeraLab/.venv/bin/python -m ctde_v0.run_lag_t3 --out runs/lag_t3 --seeds 3 --iters 2000 --rollouts 16 --jobs 3
===>>> This launcher LAUNCHES REAL TRAINING; run it deliberately. <<<===
Unit(uid, run_dir, seed, extra)
One train_ctde run: id, run-dir, seed, the extra flags. FRESH — no warm-start, no deps.
Source code in experiments/ctde_v0/run_lag_t3.py
99 100 101 102 103 104 105 106 107 | |