Broad intuition · Study map 5 of 85
Optimization beyond gradients
Suggested preparation: Probability, uncertainty, and causality
Picture it
When a score is available but a useful derivative is not, you can still search. Evolutionary methods keep and mutate promising candidates. Bayesian optimization builds a cheap model of an expensive experiment and chooses what to try next.
What to understand
- Black-box objectives, random search, evolutionary search
- Exploration versus exploitation in hyperparameter tuning
Follow the references
You understand it when…
If training takes a day, why might the next experiment be chosen for what it teaches you rather than its predicted score?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.