Reinforcement learning trains agents from reward rather than from labels, and the field now runs two economies. The first is language post-training, where RLHF (Reinforcement Learning from Human Feedback) turned human preference data into the alignment step every frontier lab runs . The second is the classic agentic line: Q-learning and its deep successors, which learned 49 Atari games from pixels . Between them sit policy gradient methods, reward modeling and a widening gap in who knows which. Employers rarely advertise which economy their seat belongs to, and that omission is the source of most failed reinforcement learning searches, because the two populations rarely even read the same papers.
Challenges in Reinforcement Learning Recruiting
RLHF (Reinforcement Learning from Human Feedback) turned alignment into a training pipeline
RLHF (Reinforcement Learning from Human Feedback) converted a research technique into a three-stage industrial process: supervised fine-tuning on demonstrations, reward model training on preference comparisons, then policy optimization against the reward with a KL penalty to stop drift . The datasets show the industrial character: about 13,000 prompts for fine-tuning, 33,000 for reward model training, and 31,000 for the policy stage, with labelers ranking four to nine outputs per prompt . InstructGPT's 1.3B policy beat the 175B GPT-3 in human preference, which told every lab that the pipeline mattered more than the raw scale . The roles that fell out of it are post-training engineers: rollout infrastructure, preference data logistics, and the reward model as a first-class artifact. None of that resembles classic control-theory reinforcement learning, and the CVs do not overlap either.
Policy gradient methods split PPO infrastructure from GRPO research
Policy gradient methods have split into two working cultures. PPO, the standard for years, trains a value function alongside the policy and carries its memory and compute overhead . GRPO, the algorithm behind DeepSeek-R1, drops the critic, samples a group of outputs per question, and computes advantages from the group's own reward statistics . DeepSeek then pushed the split further: R1-Zero skipped supervised fine-tuning entirely and trained on rule-based correctness and format rewards, which demanded rollout machinery at a scale most teams will never touch . The R1 pipeline added layers on top, cold-start fine-tuning, reasoning-oriented RL, rejection sampling, that exist precisely because pure RL alone leaves gaps the lab chose to close with data . A candidate fluent in PPO hyperparameters has not built GRPO rollout infrastructure, and a GRPO researcher may never have tuned a value function. The brief should say which, because the interview questions for the two cultures barely intersect.
Reward modeling hires on preference data quality, not loss curves
Reward modeling lives and dies upstream of the loss function. InstructGPT's reward model trained on comparisons from about 33,000 prompts ranked by labelers, and the pipeline was deliberately iterated: more comparison data from the current policy, a new reward model, a new policy . The paper is equally explicit about what that means: the resulting model aligns to the stated preferences of a specific group of labelers, not to any broader notion of human values . Everything downstream inherits the quality of those preference pairs, which is why the scarce skill is not training the model but running the annotation program: rater instructions, disagreement resolution, and the decision of whose preferences count. Candidates from research backgrounds rarely own that logistics layer, and candidates from annotation vendors rarely own the training. The role needs both, and neither half is visible on a standard ML CV, which is how reward modeling seats stay open while reinforcement learning stays fashionable.
Direct preference optimization (DPO) collapsed the pipeline into one loss
Direct preference optimization (DPO) compressed RLHF's three stages into a single classification objective on preference pairs, no explicit reward model and no sampling from the policy inside the loop, while matching or beating PPO-based RLHF on sentiment control, summarization and dialogue . The derivation is a change of variables: the policy implicitly becomes the reward, and the Bradley-Terry preference model is optimized directly against the pairs . The hiring effect was a reshuffling of what matters. Teams no longer need to staff a full PPO stack to do preference training, which moved the scarce resource from RL infrastructure toward preference dataset curation: choosing the pairs, managing the reference policy, and watching for the degenerate behaviors DPO papers discuss . An RLHF engineer who has never run DPO is missing the cheapest tool in the box; a DPO practitioner who has never seen a reward model cannot debug the harder cases, and the harder cases are where production post-training actually lives.
Q-learning still carries the robotics branch of the field
The agentic side of reinforcement learning still runs on value-based methods. The deep Q-network paired Q-learning with experience replay to play 49 Atari games from raw pixels, beating prior methods on 43 of them and exceeding 75% of a professional human tester's score on more than half . AlphaZero generalized the line with self-play and search: starting from random play and knowing nothing but the game rules, it beat the strongest engines in chess, shogi and Go, searching roughly 60,000 positions per second where Stockfish searched 60 million, and it taught itself chess in about nine hours . Robotics and control hiring draws from this lineage: simulators, exploration, sparse rewards, real-world transfer. A post-training engineer from the language side shares almost no tooling with this population, and the reverse is equally true. The discipline name covers both; the skills never did, and neither did the employers, which is how the same search ends up reviewing two completely different candidate pools that both call themselves reinforcement learning engineers.
Reward modeling claims fail at the reward hacking question
Reward modeling is the discipline's most gamed layer, so the interview should play that game. Ask how the candidate detected reward hacking on their own runs: what the policy learned to exploit, which KL coefficient held it back, whether length or format was being over-weighted, and how the reward model was retrained when it drifted . Ask what they would change about a preference dataset whose raters keep disagreeing, and why . The answers separate people who have watched a training loss curve from people who have watched a policy misbehave. The cost of a miss is the standard one in this field: a model that maximizes the proxy and not the intent, discovered after it shipped. Post-training seats cannot absorb that mistake, because the fix is another training run and another round of preference data, and every round of preference data is itself an annotation program someone has to run.
References
- Training Language Models to Follow Instructions with Human Feedback — OpenAI (arXiv 2203.02155). (accessed 2026-09-28)
- Human-Level Control Through Deep Reinforcement Learning — Nature. (accessed 2026-09-28)
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — Nature. (accessed 2026-09-28)
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model — Stanford University (arXiv 2305.18290). (accessed 2026-09-28)
- A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go Through Self-Play — Science. (accessed 2026-09-28)