615.19 -- Simulated Annealing PDF
Preview 615.19 -- Simulated Annealing
Simulated Annealing Biostatistics 615/815 Lecture 19 Scheduling Need to pick a date for mid-term (cid:122) Default date is December 20, 2006 (cid:122) We could have it earlier… (cid:122) • For example, on December 12, 2006? What do you prefer? (cid:122) So far … “Greedy” optimization methods (cid:122) • Can get trapped at local minima • Outcome might depend on starting point Examples: (cid:122) • Golden Search • Nelder-Mead Simplex Optimization • E-M Algorithm Today … Simulated Annealing (cid:122) Markov-Chain Monte-Carlo method (cid:122) Designed to search for global minimum (cid:122) among many local minima The Problem Most minimization strategies find the (cid:122) nearest local minimum Standard strategy (cid:122) • Generate trial point based on current estimates • Evaluate function at proposed location • Accept new value if it improves solution The Solution We need a strategy to find other minima (cid:122) This means, we must sometimes select (cid:122) new points that do not improve solution How? (cid:122) Annealing One manner in which crystals are formed (cid:122) Gradual cooling of liquid … (cid:122) • At high temperatures, molecules move freely • At low temperatures, molecules are "stuck" If cooling is slow (cid:122) • Low energy, organized crystal lattice formed Simulated Annealing Analogy with thermodynamics (cid:122) Incorporate a temperature parameter into the (cid:122) minimization procedure At high temperatures, explore parameter space (cid:122) At lower temperatures, restrict exploration (cid:122) Markov Chain Start with some sample (cid:122) • A set of mixture parameters Propose a change (cid:122) • Edit mixture parameters in some way Decide whether to accept change (cid:122) • Decision is based on relative probabilities of two outcomes Simulated Annealing Strategy Consider decreasing series of temperatures (cid:122) For each temperature, iterate these steps: (cid:122) • Propose an update and evaluate function • Accept updates that improve solution • Accept some updates that don't improve solution • Acceptance probability depends on “temperature” parameter If cooling is sufficiently slow, the global minimum (cid:122) will be reached
Description:The list of books you might like
