Skip to content

feat: adapt Agent-R1 to verl 0.8.0#100

Open
lqzxt wants to merge 2 commits into
AgentR1:mainfrom
lqzxt:feat/verl-0.8.0
Open

feat: adapt Agent-R1 to verl 0.8.0#100
lqzxt wants to merge 2 commits into
AgentR1:mainfrom
lqzxt:feat/verl-0.8.0

Conversation

@lqzxt

@lqzxt lqzxt commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adapt Agent-R1 to verl==0.8.0 by migrating training, reward, rollout, and worker integration to the updated verl APIs.

Changes

  • Switch actor/critic execution to the unified model engine worker path and remove legacy FSDP/DataParallel actor/critic overrides.
  • Update PPO trainer logic to use tensor-dict mini-batch updates, extract_reward, and the new reward config namespace.
  • Replace legacy agent loop server manager usage with LLMServerClient and GlobalRequestLoadBalancer.
  • Update reward loop imports and reward model config paths for verl 0.8.0.
  • Ensure rollout weight sync calls update_weights(mode="naive") before waking vLLM replicas.
  • Update recipe agent flow type hints/imports for the new LLM client.
  • Add a GSM8K GRPO launch script.
  • Update README and installation docs from verl==0.7.0 to verl==0.8.0.

@lqzxt lqzxt requested a review from 0russwest0 July 6, 2026 09:59
@gemini-code-assist

Copy link
Copy Markdown

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant