Tag: policy-gradient

Blog Post·2025-06-20·13 min read

GRPO Variants: From GRPO to DAPO, Dr. GRPO, and Beyond

A systematic comparison of GRPO and its descendants — CLIP-DAPO, CISCO, DAPO, Dr. GRPO, GDPO, REINFORCE++ — what each fixes, what trade-offs each makes, and when to use which.

grpo rl-for-llms llm-training alignment reinforce policy-gradient

Blog Post·2025-06-20·13 min read

GRPO Variants: From GRPO to DAPO, Dr. GRPO, and Beyond

A systematic comparison of GRPO and its descendants — CLIP-DAPO, CISCO, DAPO, Dr. GRPO, GDPO, REINFORCE++ — what each fixes, what trade-offs each makes, and when to use which.

grpo rl-for-llms llm-training alignment reinforce policy-gradient

Blog Post·2024-06-19·6 min read

Policy Gradients: The Math Behind RLHF

The policy gradient theorem lets you differentiate through a reward signal you can't backprop through. Here's the derivation and why it works.

rl policy-gradient llm-training

Blog Post·2024-06-19·6 min read

Policy Gradients: The Math Behind RLHF

The policy gradient theorem lets you differentiate through a reward signal you can't backprop through. Here's the derivation and why it works.

rl policy-gradient llm-training