ctde_v0.config

config

Config for the GROUNDED CTDE v0 agent — full §5-style nested schema.

This mirrors EXPERIMENT_PLAN.md §5 ("every run carries one") and captures EVERY knob of the grounded TeamBlue agent (agent_architecture.md): the LPAC-style backbone + GNN-KB aggregator, the multi-level goal-pointer → L1-controller action stack, the decentralized local-Fiedler λ̂₂ head, the centralized critic, the mission-safety mechanism, the coverage+connectivity reward weights, and the regularization. Nothing the agent does is left implicit — a run is fully documented by its saved config.

The schema is a tree of small frozen dataclasses (World, Backbone, ActionHead, MissionSafety, Reward, Connectivity, Loss, Trainer, Regularization) hung off the top-level :class:CTDEConfig. to_dict / :func:from_dict round-trip the whole tree to/from JSON so train_ctde.py can save it next to each run and rebuild it.

Defaults = the v0 slice the task specifies: comm-coverage at 16×16 / 4 agents, 100-step horizon, MAPPO-CTDE, LPAC backbone with max aggregation, a goal-pointer head over a 9-candidate offset stencil + greedy controller, action-mask mechanism, coverage(1)+connectivity(2) reward.

World dataclass

World(recipe='comm-coverage', grid=16, n_agents=4, comm_r=5, sense_r=1, cover_r=0, sense_walls=True, sense_free=False, boundary=False, n_obstacles=0, spawn_radius=2, horizon=100, terrain='open', rooms=3, pillar_spacing=4, pillar_size=2, occlusion=False, occlusion_c=None)

zymera comm-coverage recipe + per-rung overrides (absolute cells).

Backbone dataclass

Backbone(type='lpac', depth=2, width=64, mp_rounds=2, norm='layer', agg='max', heads=4, message_content='learned', recurrence='feedforward', position_ground=False)

LPAC-style backbone: CNN local-perception -> GAP -> per-agent feature, then a GNN message-passing KB over the in-range comm graph.

• type — "lpac" (CNN+GNN). Only value for v0.
• depth — conv layers in the local-perception CNN.
• width — conv channels / latent width = belief dim.
• mp_rounds — GNN message-passing rounds over the comm graph (KB fusion).
• norm — "layer" (LayerNorm on the belief) | "none".
• agg — neighbour aggregator: "mean" | "max" (default) | "multihead". NORMALIZED / size-invariant by construction (never raw-sum).
• heads — attention heads when agg == "multihead".
• message_content — the I2 "message design" dial: WHAT each agent puts in its comm message BEYOND the learned msg feature transform (an extra per-edge channel the receiver fuses alongside the learned messages):
  • "learned" (default): the message is msg(feats) exactly — v0 behaviour, byte-unchanged (the update Linear stays 2W -> W).
  • "edge_distance": append the (comm_r-normalized) sender→receiver Chebyshev distance per edge, summarized to each receiver as its [mean, min] neighbour distance — so the receiver knows HOW FAR each neighbour is. The distance (env_utils.kb_distance) is normalized by comm_r (in [0,1]) so the signal is SCALE-INVARIANT (a model trained @16²/4 reads it the same @32²/10).
  • "index": append a per-sender IDENTITY signal — a FIXED sinusoidal embedding of the sender's NORMALIZED index i/N (nets._index_signal), mean-pooled over neighbours, so a receiver can tell its neighbours apart. Fixed (not a learned per-N table) and a function of i/N -> AGENT-COUNT-INVARIANT (the same function maps any team size). The extra channel widens the update Linear; the comm-graph distance is threaded from the rollout into the backbone only for the non-default modes (the 'learned' path ignores it -> byte-identical to v0).
• recurrence— per-agent temporal memory over the belief (the recurrence axis):
  • "feedforward" (default): the heads read the per-step belief z directly — v0 behaviour, byte-unchanged. The GRU is still BUILT (stable param surface) but never used.
  • "recurrent": an eqx.nn.GRUCell (W -> W) carries a per-agent hidden state h ACROSS the 100-step episode (h_t = GRUCell(z_t, h_{t-1}), reset to zeros at episode start); EVERY head reads h_t instead of z_t so the agent remembers its own trajectory / coverage history (relevant to dispersal OVER TIME). The hidden threads through BOTH the rollout (scan carry) and the PPO loss (recomputed along each trajectory under the current params, BPTT via a per-episode scan; the recurrent path minibatches over EPISODES, keeping each 100-step sequence intact). Width = width (the belief dim).

ActionHead dataclass

ActionHead(kind='goal_pointer', K=9, stride=3, controller='greedy', planner='wavefront', explorer_tool='goal_head', relay_tool='lambda2_anchor', compass='off')

Multi-level (L3 goal -> L1 controller) action stack — NO direct moves.

• kind — "goal_pointer": pick 1 of K candidate relative waypoints.
• K — number of candidate offsets in the stencil (9 = center + 8 compass dirs; if a different K is given the stencil is the first K of [center, N, E, S, W, NE, SE, SW, NW]).
• stride — cells from the agent to each compass candidate (absolute, so the goal geometry is scale-invariant).
• controller — the L1 controller that turns the chosen goal into a 1-step move:
  • "greedy" (default): Chebyshev-descent toward the goal, emitting ONLY env-valid moves (STAY fallback). The sim still sees 1-step moves; the 100-step budget is unchanged. v0 behaviour, byte-unchanged.
  • "navfield": a nav-field + reactive controller — a navigation field is planned toward the goal (see planner) and the agent descends it with reactive local avoidance. Only consulted when controller == 'navfield'.
  • "mvprop": the LEARNED MVProp planner (t5lab) — a frozen, distilled value field is flooded toward the goal and the agent ascends it (argmax value) with the same reactive collision veto + STAY fallback. The planner module is passed to ppo.train(..., mvplanner=); the RL loss never touches it (frozen). This is the "exact previous architecture, greedy -> planner" swap.
• planner — the nav-field planner used ONLY when controller == 'navfield' (inert under the default greedy controller): "wavefront" (default) | "bfs" | "fmm" (fast-marching) | "astar".
• explorer_tool — how the EXPLORER picks its goal sector (the L4 "disperse" skill / I2 explorer-tool axis):
  • "goal_head" (default): the goal-pointer logits come from the belief z ALONE (nets.Actor.goal_head) — v0 behaviour, byte-unchanged.
  • "frontier_attn": a learned frontier-attention module (nets.FrontierAttn) ADDS a per-sector bias to the goal logits, pulling the goal toward the compass sector with the most UNCOVERED ground (the agent's own known channel = frontier). It gives the explorer an EXPLICIT frontier-seeking mechanism instead of relying on the reward to discover dispersal from a clustered spawn. The module is ALWAYS built (a stable param surface) but only contributes under this value; PPO still samples the goal (the bias never argmaxes). Only the EXPLORER role uses it — relays run the λ̂₂-anchor controller and discard the goal regardless, so when role_picker == 'off' every (explorer) agent uses the tool and when it is on relays are unaffected. Scale-invariant: K fixed, the sector features are normalized fractions, so a model trained @16²/4 transfers.
• relay_tool — which RELAY controller the relay role calls (the I2 relay-tool axis / agent_architecture.md "Relay tool"). Only the relay role uses it; explorers are unaffected, and with role_picker == 'off' every agent is an explorer so this knob is inert:
  • "lambda2_anchor" (default): the existing controller.relay_move — each relay actively takes the env-valid move that MAXIMIZES its local soft-degree (λ̂₂-anchor), i.e. it climbs connectivity every step. v0/I1 behaviour, byte-unchanged.
  • "hold": controller.relay_hold_move — a low-energy STATIC BEACON. The relay STAYS put (keeps the bridge from where it stands) UNLESS staying would leave it isolated (soft-degree below a floor), in which case it takes the single valid move that best re-establishes a neighbour. "Don't wander, just hold the connection" vs the anchor's active connectivity-climbing.
• compass — append a small SCALE-INVARIANT directional feature to the per-agent belief z BEFORE the heads (the I2 compass feature / agent_architecture.md "compass"):
  • "off" (default): z unchanged -> byte-identical to the pre-compass actor. The compass module is STILL built (stable param surface) but never used.
  • "on": nets.Compass ADDS a gated projection of two soft K-sector DIRECTIONS — the GATHER direction (toward the centroid of in-range teammates, the neighbors channel) and the EXPLORE direction (toward the nearest uncovered cell, 1 - known) — to z, giving every head an explicit navigation cue beyond the CNN's local view. Directions only (no distances / absolute coords) -> scale-invariant: a model trained @16²/4 transfers.

MissionSafety dataclass

MissionSafety(mechanism='action_mask', conn_signal='global_lambda2', degree_target=1.0, min_lambda2=0.001, lambda_init=0.0, lambda_lr=0.05, constraint_threshold=None, pid_kp=1.0, pid_ki=0.01, pid_kd=0.1)

Connectivity / mission-safety enforcement — the swept mechanism axis.

• mechanism — the connectivity-enforcement mechanism (I1b extends the set):
  • "action_mask" (default): forbid goal candidates whose greedy first move would drop true λ₂ below min_lambda2 (a hard local guardrail).
  • "soft_lambda": no masking; a FIXED-weight λ·penalty term is added to the reward instead (Reward.soft_lambda_penalty).
  • "lagrangian": an ADAPTIVE penalty — a dual variable λ ≥ 0 (carried in the train state) is dual-ascended on the realized connectivity violation so the policy LEARNS to hold the graph (Lagrangian-PPO).
  • "pid_lagrangian": same violation, but λ comes from a PID controller (Stooke et al. 2020, "Responsive Safety in RL") for smoother dual dynamics (carries integral + prev-error in the train state). The two adaptive mechanisms read a CTDE training-time true-λ₂ signal; only they activate the dual-variable state (action_mask / soft_lambda are byte- unchanged from I1).
• conn_signal — the SIGNAL SOURCE the penalty mechanisms read, ORTHOGONAL to mechanism (I1c adds this axis; every mechanism × conn_signal combo is valid, action_mask alone ignores it since it masks actions, no penalty):
  • "global_lambda2" (default): the I1b signal — a GLOBAL scalar (true team λ₂ vs the floor) broadcast IDENTICALLY to all N agents, so no single agent knows it is the one stretching the bridge. Default keeps I1b byte-unchanged.
  • "local_edge_margin": a LOCAL, PER-AGENT signal — each agent's own soft-degree shortfall (env_utils.local_edge_margin), positive only for agents drifting toward the edge of comm range (anticipatory, partial-obs- native), ≈0 for agents comfortably in the interior. Not broadcast.
• degree_target — the per-agent soft-degree floor the "local_edge_margin" signal charges the shortfall against (p_i = relu(degree_target − soft_deg_i)).
• min_lambda2 — the connectivity floor the action_mask defends.
• lambda_init — initial dual variable λ for the adaptive mechanisms.
• lambda_lr — dual-ascent step size for the "lagrangian" mechanism.
• constraint_threshold — the connectivity floor τ the violation v = relu(τ − mean_rollout(true λ₂)) is measured against (global_lambda2); under local_edge_margin the violation is instead v = mean_i p_i, the rollout-mean per-agent margin shortfall. None -> reuse the locked grading threshold Connectivity.threshold (do NOT invent a second floor); resolve via :meth:CTDEConfig.constraint_threshold.
• pid_kp / pid_ki / pid_kd — PID gains for "pid_lagrangian".

Reward dataclass

Reward(kind='extrinsic', w_coverage=1.0, w_connectivity=2.0, w_collision=-4.0, soft_lambda_penalty=1.0, normalized=False, barrier_weight=0.0, barrier_a=None, barrier_M=None, barrier_p=2.0, barrier_cap=50.0)

Coverage + connectivity reward, composed in the experiment from the env's UNWEIGHTED per-term magnitudes (reward engineering stays here).

Defaults = zymera DEFAULT_TERMS weights (coverage 1 / connectivity 2 / collision -4). soft_lambda_penalty is the λ scale used only when MissionSafety.mechanism == 'soft_lambda'.

Connectivity-FLOOR barrier ("Hyper-Singularity") — a STANDALONE, config-knobbed reward term (env_utils.connectivity_barrier) that COMPOSES with every other connectivity mechanism (conn_signal / mechanism); it does NOT replace any. A capped one-sided wall on each agent's nearest-neighbour Chebyshev distance: EXACTLY 0 inside barrier_a (silent in the safe zone), an explosive-but-FINITE rise as a link nears barrier_M (the break range), saturating at barrier_cap at/past barrier_M. barrier_weight is the formula's k; at its DEFAULT 0 the term is OFF / exactly 0 and the composed reward is byte-unchanged.

• barrier_weight — k; the term's weight. 0 (default) -> OFF (no-op).
• barrier_a — launch point (0 below it). None -> world.comm_r * 0.6.
• barrier_M — the wall / break range. None -> world.comm_r (link breaks at comm range). Both resolve via the barrier_a / barrier_M accessors on :class:CTDEConfig (one source of truth for comm_r, like MissionSafety.constraint_threshold).
• barrier_p — explosion power on the (M - x) denominator.
• barrier_cap — the finite "almost-infinity" ceiling the wall saturates at.

Connectivity dataclass

Connectivity(estimator='fiedler_local_poweriter', grade_on='true_lambda2', threshold=0.001, real_threshold=0.5, trade_off_lambda=None, lambda2_sharp=2.0, estimator_iters=8)

The locked connectivity metric + agent signal (EXPERIMENT_PLAN §1).

Loss dataclass

Loss(ppo_clip=0.2, aux_beta=0.1, aux_loss='mse', huber_delta=0.1, vf_coef=0.5, credit='shared', remat=False)

PPO + auxiliary λ₂ supervision knobs.

• credit — the advantage credit-assignment scheme:
  • "shared" (default): the team-mean reward feeds one GAE advantage broadcast to every agent — v0 behaviour, byte-unchanged.
  • "difference": the EXACT submodular per-agent difference reward (Wolpert–Tumer D_i = team value − team-value-without-i, computed exactly on the submodular coverage objective) replaces the shared reward per agent, paying disjoint sweeping and starving redundant floods. Distinct from the top-level credit == 'agent' axis (which uses each agent's own per-agent reward).

Trainer dataclass

Trainer(kind='mappo', lr=0.0003, gamma=0.99, gae_lambda=0.95, clip=0.2, ppo_epochs=4, minibatches=4, max_grad_norm=0.5)

The optimizer (MAPPO-CTDE).

Regularization dataclass

Regularization(degree_reg=0.001, entropy_coef=0.01, weight_decay=0.0001, dropout=0.0)

All regularizers, each gated behind a knob.

• degree_reg — SizeShiftReg-style weight on the variance of per-node aggregated degree statistics across the batch (guards GNN size-transfer); small default.
• entropy_coef — entropy bonus on the goal policy.
• weight_decay — AdamW decoupled weight decay.
• dropout — dropout rate on the belief (0 = off).

CTDEConfig dataclass

CTDEConfig(world=World(), backbone=Backbone(), action_head=ActionHead(), mission_safety=MissionSafety(), reward=Reward(), connectivity=Connectivity(), loss=Loss(), trainer=Trainer(), regularization=Regularization(), role_picker='off', reward_anti_overlap='off', anti_overlap_weight=1.0, collision_mask='off', warmstart_noise=0.0, critic_mode='central', critic_arch='conv', diversity_residual='off', fork_groups=1, selector='off', flock='scripted', congestion='off', congestion_weight=0.5, explore_infogain='off', info_gain_weight=0.1, credit='shared', scale='16x16/4', iters=50, rollouts_per_iter=8, seed=0, ckpt_path=None)

constraint_threshold property

constraint_threshold

Connectivity floor τ the Lagrangian mechanisms measure the violation against. mission_safety.constraint_threshold if set, else the locked grading threshold connectivity.threshold (one floor, not a second).

barrier_a property

barrier_a

Resolved barrier launch point a: reward.barrier_a if set, else world.comm_r * 0.6 (one source of truth for comm_r).

barrier_M property

barrier_M

Resolved barrier wall / break range M: reward.barrier_M if set, else world.comm_r (the link breaks at comm range).

from_dict

from_dict(d)

Rebuild a full config tree from a plain dict (e.g. loaded JSON).

Source code in experiments/ctde_v0/config.py

def from_dict(d: dict) -> CTDEConfig:
    """Rebuild a full config tree from a plain dict (e.g. loaded JSON)."""
    top_scalar = {f.name for f in dataclasses.fields(CTDEConfig)} - set(_LEAF_TYPES)
    kw: dict[str, Any] = {}
    for name, cls in _LEAF_TYPES.items():
        if name in d:
            kw[name] = _build_leaf(cls, d[name])
    for name in top_scalar:
        if name in d:
            kw[name] = d[name]
    return CTDEConfig(**kw)

flat_schema

flat_schema(cfg)

Flatten the config tree to block.field -> value (for human listing/logs).

Source code in experiments/ctde_v0/config.py

def flat_schema(cfg: CTDEConfig) -> dict:
    """Flatten the config tree to ``block.field -> value`` (for human listing/logs)."""
    out: dict[str, Any] = {}
    for k, v in cfg.to_dict().items():
        if isinstance(v, dict):
            for kk, vv in v.items():
                out[f"{k}.{kk}"] = vv
        else:
            out[k] = v
    return out