RLHF
RLHF is a reinforcement learning method that uses human feedback as a reward, while RLVR is a reinforcement learning method that uses verifiable correct answers as a reward; both are used to align LLM outputs with human expectations.
## The Technology That Transforms LLMs from "Smart" to "Usable" An LLM that has completed pre-training possesses vast knowledge, but is difficult to use as-is. It may generate continuations of text rather than answering questions, or produce harmful content. Alignment is the process of transforming this "smart but unwieldy" state into "smart and user-friendly," and RLHF is its core technology. ## RLHF: Judged by Humans In RLHF (Reinforcement Learning from Human Feedback), human annotators compare multiple outputs from a model and evaluate which is better. A reward model is trained on that evaluation data, and the LLM is then adjusted through reinforcement learning to obtain higher rewards. The reason ChatGPT and Claude can deliver "conversational" responses is a result of RLHF. However, challenges remain. Human evaluation is costly, prone to subjective inconsistency, and difficult to scale. The problem of reward hacking—where responses that appear plausible but are actually incorrect receive high ratings—has also been noted. ## RLVR: Restricted to Tasks with Verifiable Answers RLVR (Reinforcement Learning with Verifiable Rewards) is a method that gained attention in 2025 through DeepSeek-R1. It is limited to tasks where correctness can be mechanically verified—such as mathematical proofs or code execution results—and rewards are assigned without human evaluation. Because no human subjectivity is involved, reward noise is low, and large volumes of feedback can be generated at low cost. On benchmarks covering mathematics, coding, and formal logic, accuracy improvements surpassing those of RLHF have been reported. Specific algorithms such as GRPO and DPO belong to this paradigm. ## Which Should You Use? The two approaches are not mutually exclusive. RLVR is efficient for verifiable tasks (code generation, mathematics, fact verification), while RLHF remains necessary for tasks where "there is no single correct answer," such as creative writing or conversational quality. In practice, hybrid approaches combining both are becoming increasingly common.
Related Terms
RRF
RRF (Reciprocal Rank Fusion) is a scoring method that integrates ranking results returned by multiple retrieval methods. By summing the reciprocal ranks from each method, it enables the fusion of different scoring systems without normalization.
LoRA
LoRA (Low-Rank Adaptation) is a technique that inserts low-rank delta matrices into the weight matrices of large language models and trains only those deltas, enabling fine-tuning by adding approximately 0.1–1% of the total model parameters.
RAG (Retrieval-Augmented Generation)
RAG (Retrieval-Augmented Generation) is a technique that improves the accuracy and currency of responses by retrieving relevant information from external knowledge sources and appending the results to the input of an LLM.
