5.1 Core RL Concepts
Interview-ready field notes
The 20-second answer
Reinforcement learning learns a policy by trial and error: an agent acts in an environment and seeks to maximize long-term reward.
Core ideas to retain
- State is the situation, action is the choice, reward is immediate feedback, and a policy maps states to action probabilities.
- Return is discounted future reward: . decides how much the future matters.
- A value function estimates expected return; a Q-function estimates expected return after a specific action.
- Exploration tries uncertain actions to learn; exploitation chooses actions that currently look best.
Interview / OA rule
The Bellman idea is the core OA tool: current value equals immediate reward plus discounted value of what follows. It turns a long-horizon problem into recursive updates.
One good written resource
OpenAI Spinning Up — Key Concepts — read this after the video when you want a clearer mental model, not more pages of notes.
Most asked
Interview questions to practise aloud
Each answer is the level of detail expected for a strong fundamentals round.
What is the difference between reward and return?
Reward is immediate feedback at one step; return is the discounted sum of future rewards from a point in time.
Value function vs Q-function?
$V(s)$ scores a state under a policy; $Q(s,a)$ scores taking an action in that state, then following the policy.
Exploration vs exploitation?
Exploration gathers information about uncertain actions; exploitation uses current knowledge for reward. A useful agent needs both.