Direct Preference Optimization
10-20-2026
Use this note to derive the DPO objective and compare its reference-model, data, and optimization tradeoffs with RLHF and online reinforcement learning.
10-20-2026
Use this note to derive the DPO objective and compare its reference-model, data, and optimization tradeoffs with RLHF and online reinforcement learning.