ctde_v0.run_critic_t4¶
T4 — count-invariant critic A/B, a FROM-SCRATCH + ZERO-SHOT-TRANSFER test.
Belief: the fixed spec now carries the FULL 7-channel SLAM occupancy KB as the baseline belief
(--sense-walls --sense-free --boundary: free/occupied/unknown + occ_frontier + boundary),
so every arm/rung reads occupancy rather than the old 5-channel coverage-trail belief.
Does a FIXED-N conv critic degrade where a count-invariant DeepSets critic holds, when
the SAME team is scaled up? The conv Critic reads the global central_obs (grid-invariant
via same-padding conv + global-average-pool), so it LOADS across rungs — but its value
predictions are learned against a fixed agent count; the DeepSets critics pool per-agent
encodings permutation- and count-invariantly, so they should transfer better as N grows.
Three ARMS, identical in everything but the critic architecture (the exact Wire-phase flag): conv (control) --critic-arch conv conv over the global central_obs (v0 baseline) setpool --critic-arch setpool DeepSets, mean agent-pool setattn --critic-arch setattn DeepSets, attention agent-pool
TWO phases:
(1) TRAIN — every scale FROM SCRATCH (no warm-start): 16²/4, 24²/6, 32²/10 each trained independently per arm, per seed. 3 arms × 3 seeds × 3 scales = 27 independent trainings. The 16² run is the transfer eval's SOURCE; the 24²/32² runs are the in-distribution REFERENCE the zero-shot transfer numbers are read against (trained-at-that-scale ceiling). The --critic-arch flag is forwarded to every run so the checkpoint arch always matches.
Uses the SAME bounded-parallel scheduler as the obstacle launcher
(run_obstacle_overnight._schedule): a run with a model.eqx is skipped; every unit is
independent (no --init-from deps); --jobs runs up to N at once.
(2) ZERO-SHOT TRANSFER EVAL — the transfer test proper. For each arm × seed, take the 16²-TRAINED checkpoint (ladder rung 0) and evaluate it, WITHOUT any further training, at 16²/4 (in-distribution anchor), 24²/6 and 32²/10. Two metrics per target scale: - value-prediction error: the critic's v_team vs the Monte-Carlo discounted return-to-go (bootstrap-free ground truth). Reported as RMSE and as NRMSE (RMSE / std(returns)) — the NRMSE is the cross-rung-comparable number, since the raw return magnitude differs between 16²/4 and 32²/10. - downstream coverage: the final-step coverage the transplanted (actor, critic) achieves at the target scale (does the policy still sweep, zero-shot). The hypothesis: conv's NRMSE blows up at 24²/6 and 32²/10 while setpool/setattn hold.
This launcher SHELLS OUT to python -m ctde_v0.train_ctde for training and runs the eval
IN-PROCESS (loads each checkpoint via ppo.init_state_from_checkpoint + a fresh rollout).
CPU-friendly; keep --iters / --rollouts in the train_ctde budget.
# see the whole plan (train + eval), launch nothing:
JAX_PLATFORMS=cpu PYTHONPATH=.:../../../FiedlerValueEstimation /Users/bijanmehr/Project.Zymera/zymera_lab/.venv/bin/python -m ctde_v0.run_critic_t4 --dry-run
# the real thing (drop JAX_PLATFORMS on balthar for GPU), 2 trainings at a time:
PYTHONPATH=.:../../../FiedlerValueEstimation ~/ZymeraLab/.venv/bin/python -m ctde_v0.run_critic_t4 --out runs/critic_t4 --seeds 3 --iters 2000 --jobs 2
===>>> This launcher LAUNCHES REAL TRAINING; run it deliberately. <<<===
Unit(uid, run_dir, rung, seed, extra, init_from_dir, needs_dir)
One warm-start-ladder rung (a train_ctde subprocess). Mirrors the scheduler's Unit interface (run_dir/uid/ready()/cmd()/proc/init_from) but bakes T4's _FIXED + the arm's --critic-arch into cmd() so the obstacle launcher's _FIXED never leaks in.
Source code in experiments/ctde_v0/run_critic_t4.py
111 112 113 114 115 116 117 118 119 | |