Troubleshooting
Symptom-indexed fixes for the issues that actually block RL training
Failure modes that recur when operating SWE-Lego-RL, distilled to symptom → root cause → fix. The pages here cover the ones you are most likely to hit; the full index at the bottom lists every incident write-up in the repo, including the ones too situational to promote to a page.
Most config-level problems never get this far
PREFLIGHT_ONLY=1 catches the mistakes that would otherwise cost a multi-hour run —
parser mismatch, topology that doesn't add up, R3 without the new verl tree, no
reachable image source. Run it first; see Preflight.
Quick index by symptom
The run won't start, or dies on step 0
| Symptom | Page |
|---|---|
response_mask must contain at least one valid token on the first step | Tool-call parser mismatch |
import torch / import vllm costs ~9 s on every launch | venv on a network FS |
Every trial fails env_setup with empty trajectories | Preflight §6 |
Pods end in ImagePullBackOff on a fresh cluster | Image source |
Training runs, but learns nothing
| Symptom | Page |
|---|---|
Every reward is 0.0 though the grader clearly runs | Reward always zero |
| Pod startup creeps 30 s → 100 s+ over hours; throughput tanks | cgroup / memcg leak |
| First validation hangs forever; no traceback, no dead node | Fully-async val deadlock |
Reward flat while grad_norm is a fraction of baseline | training_env/fully-async-veomni-reward-flat-three-causes.md |
val / eval numbers look wrong
| Symptom | Page |
|---|---|
num_turns is 0 across the whole val pass | Low val scores §3 |
| Only ~170 of 500 val tasks get scored each time | Low val scores §1 |
| Offline eval disagrees with the reported val number | Low val scores §4 |
Standalone val_only eval emits garbled output, even for the base model | Offline eval garbled output |
Cluster and sandbox
| Symptom | Page |
|---|---|
| Remote Docker won't start (FlashInfer / Ray env / docker SDK / compose) | Remote Docker pitfalls |
| nvme at 100% util with no pods being created; load 30+ | Kyverno I/O storm |
Pods stuck ContainerCreating; uv installs storm the disk | Pod-startup disk bottleneck |
Pages by area
- Training startup — model/serving config issues that surface the moment a run starts.
- Rewards & throughput — the reward signal is zero, or the host degrades over hours.
- Fully-async val deadlock — the first validation hangs and never returns.
- Low val scores that are not the model — four infrastructure causes of a low val score, and how to re-score a finished run.
- Offline eval garbled output — a standalone eval emits garbled output because of the colocated serving path.
- Sandbox backends — Docker and Kubernetes runtime issues.
Full write-up index
Every incident write-up lives under troubleshooting/ in the repo, filed by the
subsystem that owned the bug. Pages above are distillations; these are the originals,
with the raw evidence.
training_env/ — trainer, proxy, scaffold
| File | What it covers |
|---|---|
tool-call-parser-mismatch.md | response_mask all-zero crash from a parser that doesn't match the model |
local-venv-cache.md | caching the venv on local NVMe to kill the cold-import tax |
litellm-rollout-replica-discovery-race.md | half the rollout GPUs idle because replica discovery stopped at the first gap (legacy LiteLLM path) |
fully-async-val-deadlock-doValidate-blocks-eventloop.md | first val hangs forever; do_validate blocks the event loop |
fully-async-veomni-reward-flat-three-causes.md | reward won't move: lr=0, gradient starvation, seq-TIS collapse |
221-node-oom-disconnect-fullyasync-deadlock.md | node OOM from unbounded env pods → permanently deadlocked async run |
eval-zero-garbage-output-colocated-vllm-moe.md | colocated vLLM dummy+sleep/wake mis-maps fused MoE weights |
reward_drop_analysis_v6_20260612.md | the v6 reward-drop investigation (trajectory filter, sequence drift) |
veomni-engine-setup-and-run-20260610.md | VeOmni engine install + run runbook for qwen3_5_moe |
r3-rollout-routing-replay-fsdp-20260608.md | R3 (rollout routing replay) for Qwen3.5-MoE on FSDP |
r3-multiturn-routing-loss-20260608.md | routing values lost across multi-turn replay |
r3-routing-coverage-rootcause-fix-20260610.md | routing-coverage root cause and fix |
r3-fsdp-v0.8.0-validated-20260610.md | R3 on verl v0.8.0 FSDP, validated end to end |
r3-fsdp-on-v0.8.0-port-handoff-20260610.md | the port/handoff notes for that migration |
r3-fsdp-qwen3moe-enablement-20260616.md | enabling R3 for qwen3_moe on the FSDP engine |
r3-dev-shm-sigbus-exhaustion-20260616.md | routed-experts buffers exhaust /dev/shm → SIGBUS at vLLM startup |
ohsdk-agent-name-vs-import-path-20260613.md | OH-SDK all-dummy smoke: agent_name vs import_path |
anthropic-proxy-token-fidelity.md | the token-fidelity rework behind the in-process proxy |
cc-fidelity-debug-20260604.md | Claude Code token-fidelity debugging log |
cc-prefix-mismatch-subagent-routing-20260605.md | prefix mismatch, sub-agent session collisions, three-layer fix |
cc-context-overflow-retry-storm-20260608.md | 128k context overflow → SDK retry storm; 500 → 400 fast-fail |
openhands-ai-qwen3coder-description-param-20260615.md | model emits a parameter's description as an argument |
pangu92b-vs-qwen35a3b-reward-trend-20260721.md | systematic comparison of a flat vs rising reward curve |
harbor/ — task execution, verifier, grading
| File | What it covers |
|---|---|
swe-rebench-v2-reward-zero.md | build-time git clone fails in an egress-less pod → all-zero reward |
swerebench-v2-passed-match-grading.md | passed_match exact grading: a correct fix still scores 0 |
val-false-zero-diagnosis-and-offline-regrade.md | the four val fake-zero causes + the offline regrade procedure |
val-failures-pythonsafepath-test-sqlite-20260717.md | PYTHONSAFEPATH ↔ test_sqlite side effects; cross-run comparison method |
reward-hacking-git-history-leak-and-fix.md | agents recovering the fix from git history; the single-commit rebuild fix |
harbor-env-setup-slowdown.md | env_setup time growing linearly (the cgroup/memcg leak) |
k8s/ — cluster and admission
| File | What it covers |
|---|---|
kyverno-webhook-oom-pod-stuck-val-collapse.md | admission-controller OOM blocks pod creation → fake val collapse |
kyverno-reports-controller-issue.md | reports-controller I/O storm hammering etcd |
k8s-sandbox-disk-fix.md | uv-cache write storm; tmpfs mutation policy |
k8s-health-check-indicators.md | the health baseline to check first |
k8s-inline-build-for-selfmade-tasks.md | inline build for self-made SWE tasks |
egress-sidecar-netadmin-bash-bug-20260717.md | the egress-isolation sidecar silently no-oped (no bash in the image) |
k8s-1.32-multi-node-setup.md | multi-node cluster deployment guide |
k8s_upgrade_to_1_31.md | single-node upgrade record |
nydus/ — image distribution
| File | What it covers |
|---|---|
nydus-local-mirror-deploy.md | full local mirror-registry deployment SOP |
nydus-image-cache-on-cpfs.md | cross-node image cache on CPFS |
shared-registry-insecure-trust-and-gotchas.md | insecure trust, probing, autostart — the shared-registry gotchas |
docker/ — the Docker backend
| File | What it covers |
|---|---|
remote-docker-backend-setup.md | using a remote Docker daemon instead of K8s |
remote-docker-troubleshooting-20260527.md | the four startup gotchas, in order |
data_collection/ — datasets and inference
| File | What it covers |
|---|---|
openswe-golden-patch-leak-in-verifier.md | instances that apply the golden patch inside the verifier → reward always 1, zero gradient |
openswe_infer_gpu_idle_longtail_barrier.md | inference GPU idling from a batch barrier plus env-start long tail |
swerebench_filtered_eval_train_plan.md | the filtered eval/train selection plan |
webui/
| File | What it covers |
|---|---|
dashboard-deployment.md | deploying the dashboard, including the Cloudflare Pages path |
Adding a write-up
File it under the subsystem that owned the bug, name it
<symptom>-<date>.md, and lead with symptom → root cause → fix. Add a row here; if
it recurs across runs or clusters, promote it to a page in this section.