Skip to content

Allow WANDB_ENTITY override via caller environment #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dhadfieldmenell wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Allow WANDB_ENTITY override via caller environment #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fcb943c
Allow WANDB_ENTITY to be overridden by the caller's environment
dhadfieldmenell Jul 20, 2026
34c8859
Remove reward-based gating from self-distillation sample inclusion
dhadfieldmenell Jul 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion training/verl_training.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ export PYTHONBUFFERED=1
# export RAY_DEBUG=1
ulimit -c 0

export WANDB_ENTITY="sample-efficient-rlvr" # team
export WANDB_ENTITY="${WANDB_ENTITY:-sample-efficient-rlvr}" # team (override by pre-setting WANDB_ENTITY)
export EXPERIMENT=${1:-"experiment"}
CONFIG_NAME=${2:-"ppo_trainer"}
export TASK=${3:-"datasets/ttcs/lasgroup_verifiable-corpus_math-ai_math500_1000"}
Expand Down
15 changes: 9 additions & 6 deletions verl/trainer/ppo/ray_trainer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -770,12 +770,15 @@ def _build_teacher_message(i: int) -> list[dict]:
for i in range(batch_size)
]

# self_distillation_mask is True if sample has a solution OR feedback is used (i.e., will get a reprompted message)
self_distillation_mask = torch.tensor(
[solution_strs[i] is not None or feedback_used[i] for i in range(batch_size)],
dtype=torch.float32,
device=device
)
# Every sample participates in self-distillation, regardless of whether the
# task's reward function marked it "successful". Reward is not a training
# signal in this loss (see compute_self_distillation_loss, which never takes
# advantages/reward as input) — it was previously only used to gate which
# samples got a demonstration-based reprompt, which conflates "did the
# reward function like this rollout" with "should this sample be distilled."
# Those are unrelated once the reward function is just a placeholder/proxy
# rather than the actual training objective.
self_distillation_mask = torch.ones(batch_size, dtype=torch.float32, device=device)

uids = set(batch.non_tensor_batch["uid"])
num_with_feedback_available = sum(1 for f in feedback_list if f is not None)
Expand Down