SA国外课件1

-2
System Energy
-4
-6
-8
-10
0ቤተ መጻሕፍቲ ባይዱ
50
100
150 Iteration Number
200
250
300
9
Statistical Mechanics
10
The Analogy
• Statistical Mechanics: The behavior of systems with many degrees of freedom in thermal equilibrium at a finite temperature.
• Define energy function for “atom” sample problem
N
Ei
mgyi
potential energy j i
xi
xj
2
yi
yj
1 2 2
kinetic energy
N
E R
i 1
Ei ri
Absolute and relative position of each atom contributes to Energy
H q, q, t
i
qi pi
L q, q, t
Hamiltonian
H
T V
kinetic potential Energy
Etot
Energy of configuration = Objective function value of design
14
Energy sample problem
– This ground state corresponds to the minimum of the cost function in an optimization problem.
11
Sample Atom Configuration
Original Configuration
Atom Configuration - Sample Problem 7 6 5 4 7 6 5
Simulated Annealing
A Basic Introduction
Lecture 10
Olivier de Weck, Ph.D.
Massachusetts Institute of Technology
1
Outline
• Heuristics • Background in Statistical Mechanics
-3 -3 -2 -1 0 x 1 2 3
8
“peaks” convergence
• Initially ~ nearly random search • Later ~ gradient search
2 current configuration new best configuration 0 SA convergence history
• Combinatorial Optimization: Finding the minimum of a given function depending on many variables. • Analogy: If a liquid material cools and anneals too quickly, then the material will solidify into a sub-optimal configuration. If the liquid material cools slowly, the crystals within the material will solidify optimally into a state of minimum energy (i.e. ground state).
• Unlike gradient-based methods in a convex design space, heuristics are NOT guaranteed to find the true global optimal solution in a single objective problem, but should find many good solutions (the mathematician's answer vs. the engineer’s answer) • Heuristics are good at dealing with local optima without getting stuck in them while searching for the global optimum.
3
Heuristics
4
What is a Heuristic?
• A Heuristic is simply a rule of thumb that hopefully will find a good answer. • Why use a Heuristic?
– Heuristics are typically used to solve complex (large, nonlinear, nonconvex (i.e. contain local minima)) multivariate combinatorial optimization problems that are difficult to solve to optimality.
15
Compute Energy of Config. A
• Energy of initial configuration
7 Atom Configuration - Sample Problem
E1 1 10 1 E2 1 10 2 E3 1 10 4 E4 1 10 3
5 5 18 20
18 5 5 5
ri with i=1,..,4 r1 1 , r2 1 3 , r3 2 4 , r4 4 5 3
– Specify a slot for each atom
ri with i=1,..,4 R 1 12 19 23
13
Energy of a state
• Each state (configuration) has an energy level associated with it
Perturbed Configuration
Atom Configuration - Sample Problem
3
4
3
2 1 0 -1 -1
4
2 1
3
2 1 0 -1 -1
4
3 1 2
0
1
2
3
4
5
6
7
0
1
2
3
4
5
6
7
E=133.67 Energy of original (configuration)
– Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P., “Optimization by Simulated Annealing,” Science, Volume 220, Number 4598, 13 May 1983, pp. 671680.
7
MATLAB® “peaks” function
T=1
T=1 - E avg=38.1017 14000 12000
12000
12000
10000
10000
10000
• • • • • • Difficult due to plateau at z=0, local maxima SAdemo1 xo=[-2,-2] Maximum Optimum at x*=[ 0.012, 1.524] z*=8.0484
Peaks 3 2 1 0
y
-1
-2
Start Point
P {r}
exp
E ({r}) k BT
Boltzmann probability depends on energy and temperature
17
Boltzmann Collapse at low T
T=100
T=100 - E avg=100.1184
T=10
T=10 - E avg=58.9245 14000
5
Types of Heuristics
• Heuristics Often Incorporate Randomization • Two Special Cases of Heuristics
– Construction Methods
• Must first find a feasible solution and then improve it.
– Improvement Methods
• Start with a feasible solution and just try to improve it.
• 3 Most Common Heuristic Techniques
– – – – Simulated Annealing Genetic Algorithms Tabu Search New Methods: Particle Swarm Optimization, etc…
Number of configurations
P=# of slots=25 N=# of atoms =4 NR=6,375,600
What is the likelihood that a particular configuration will exist in a large ensemble of configurations?
E=109.04 Perturbing = move a random atom to a new random (unoccupied) slot 12
Configurations
• Mathematically describe a configuration
– Specify coordinates of each “atom”
6
Origin of Simulated Annealing (SA)
• Definition: A heuristic technique that mathematically mirrors the cooling of a set of atoms to a state of minimum energy. • Origin: Applying the field of Statistical Mechanics to the field of Combinatorial Optimization (1983) • Draws an analogy between the cooling of a material (search for minimum energy state) and the solving of an optimization problem. • Original Paper Introducing the Concept
– Atom Configuration Problem – Metropolis Algorithm
• Simulated Annealing Algorithm • Sample Problems and Applications • Summary
2
Learning Objectives
• Review background in Statistical Mechanics: configuration, ensemble, entropy, heat capacity • Understand the basic assumptions and steps in Simulated Annealing (SA) • Be able to transform design problems into a combinatorial optimization problem suitable to SA • Understand strengths and weaknesses of SA
20 5 2 2
20.95 26.71 47.89 38.12
6 5 4 3 2 1 0 -1 -1
3 4 2 1
0 1 2 3 4 5 6 7
Total Energy Configuration A: E({rA})= 133.67
16
Boltzmann Probability
NR P! P N !