OPTIMIZATION-

SIMULATED ANNEALING
 OPTIMIZATION-SIMULATED ANNEALING

CHAPTER 1

INTRODUCTION

1.1 OPTIMIZATION
It is a selection of best element (with regard to some criteria) from some set of available
alternatives. In the simplest case, an optimization problem consist of maximizing or minimizing
a real function by choosing input values from within an allowed set and computing the value of
function. The classical optimization techniques are useful in finding the optimum solution or
unconstrained maxima or minima of continuous and differentiable functions. These are analytical
methods and make use of differential calculus in locating the optimum solution. The classical
methods have limited scope in practical applications as some of them involve objective functions
which are not continuous and un-differentiable. Yet, the study of these classical techniques of
optimization form a basis for developing most of the numerical techniques that have evolved into
advanced techniques more suitable to todays practical problems.

1.1.1 Types of Optimization Algorithms

o Mathematical Algorithms
Simplex (LP), BFGS (NLP), B&B (DP)
o Drawbacks of Mathematical Algorithms
LP: Too Ideal (All Linear Functions)
NLP: Not for Discrete Var. or Complex Fn., Feasible Initial
Vector, Local Optima
DP: Exhaustive Enumeration, Wrong Direction
o Meta-Heuristic Algorithms
GA, SA, TS, ACO, PSO,

MECHANICAL ENGINEERING Page 2

 OPTIMIZATION-SIMULATED ANNEALING

1.2 SIMULATED ANNEALING (SA)

It is a random-search technique which exploits an analogy between the way in which a metal
cools and freezes into a minimum energy crystalline structure (the annealing process) and the
search for a minimum in a more general system; it forms the basis of an optimization technique
for combinatorial and other problems. Simulated annealing was developed in 1983 to deal with
highly nonlinear problems. SA approaches the global maximization problem similarly to using a
bouncing ball that can bounce over mountains from valley to valley. It begins at a high
"temperature" which enables the ball to make very high bounces, which enables it to bounce
over any mountain to access any valley, given enough bounces. As the temperature declines the
ball cannot bounce so high and it can also settle to become trapped in relatively small ranges of
valleys. A generating distribution generates possible valleys or states to be explored. An
acceptance distribution is also defined, which depends on the difference between the function
value of the present generated valley to be explored and the last saved lowest valley. The
acceptance distribution decides probabilistically whether to stay in a new lower valley or to
bounce out of it. All the generating and acceptance distributions depend on the temperature. It
has been proved that by carefully controlling the rate of cooling of the temperature, SA can find
the global optimum. However, this requires infinite time. Fast annealing and very fast simulated
reannealing (VFSR) or adaptive simulated annealing (ASA) are each in turn exponentially faster
and overcome this problem.

MECHANICAL ENGINEERING Page 3

 OPTIMIZATION-SIMULATED ANNEALING

CHAPTER 2

STUDY OF SIMULATED ANNEALING

2.1 METHOD

Step 1: Initialize Start with a random initial placement. Initialize a very high temperature.

Step 2: Move Perturb the placement through a defined move.

Step 3: Calculate score calculate the change in the score due to the move made.

Step 4: Choose Depending on the change in score, accept or reject the move. The probe of
acceptance depending on the current temperature.

Step 5: Update and repeat Update the temperature value by lowering the temperature. Go back
to Step 2.

The process is done until Freezing Point is reached.

Fig.2.1:- Hill Climbing

COOLING SCHEDULE

MECHANICAL ENGINEERING Page 4

 OPTIMIZATION-SIMULATED ANNEALING

Fig.2.1.1:- Cooling Schedule

MECHANICAL ENGINEERING Page 5

 OPTIMIZATION-SIMULATED ANNEALING

2.2 ALGORITHM

Algorithm SIMULATED-ANNEALING

Begin

temp = INIT-TEMP;

place = INIT-PLACEMENT;

while (temp > FINAL-TEMP) do

while (inner_loop_criterion = FALSE) do

new_place = PERTURB(place);

C = COST (new_place) - COST (place);

if (C < 0) then

place = new_place;

else if (RANDOM (0,1) > e-(C/temp)) then

place = new_place;

temp = SCHEDULE (temp);

MECHANICAL ENGINEERING Page 6

 OPTIMIZATION-SIMULATED ANNEALING

CHAPTER 3

CASE STUDY HEAT EXCHANGER

3.1 GENERAL

Owing to the wide utilization of heat exchangers in industrial processes, their cost
minimization is an important target for both designers and users. Traditional design approaches
are based on iterative procedures which gradually change the design and geometric parameters to
satisfy a given heat duty and constraints. Although well proven, this kind of approach is time
consuming and may not lead to cost effective design as no cost criteria are explicitly accounted
for. The present study explores the use of non-traditional optimization technique called simulated
annealing (SA), for design optimization of shell and tube heat exchangers from an economic
point of view. The optimization procedure involves the selection of the major geometric
parameters such as tube diameters, tube length, baffle spacing, number of tube passes, tube
layout, type of head, baffle cut, etc., and minimization of total annual cost is considered as the
design target. The presented simulated annealing technique is simple in concept, few in
parameters and easy for implementations. Furthermore, the SA algorithm explores the good
quality solutions quickly, giving the designer more degrees of freedom in the final choice with
respect to traditional methods. The methodology takes into account the geometric and
operational constraints typically recommended by design codes. Three different case studies are
presented to demonstrate the effectiveness and accuracy of proposed algorithm. The SA
approach is able to reduce the total cost of the heat exchanger compared to cost obtained by the
previously reported GA approach.

MECHANICAL ENGINEERING Page 7

 OPTIMIZATION-SIMULATED ANNEALING

3.2 THE OPTIMAL HEAT EXCHANGER DESIGN PROBLEM

The procedure for optimal heat exchanger design includes the following step:
a. Estimation of the exchanger heat transfer area based on the required duty and other design
specifications assuming a set of design variable.
b. Evaluation of the capital investment, operating cost and the objective function.
c. Utilization of the optimization algorithm to select a new set of values for the design variables.
d. Iterations of the previous steps until a minimum of the objective function is found.
The entire process is schematized in Figure 1. User input. Following parameters are required as a
user defined input to calculate the heat exchanger area:
1. Mass flow rate and inlet/outlet temperatures of shell side and tube side fluids.
2. Thermo physical properties of both fluids, e.g., density, viscosity, heat capacity, thermal
conductivity.
3. Fouling resistances, R foul, shell and R foul, tube.

MECHANICAL ENGINEERING Page 8

 OPTIMIZATION-SIMULATED ANNEALING

3.3 CALCULATION SEQUENCE

1. Assume overall heat transfer coefficient based on type of shell and tube side fluid.
2. Based on the actual values of the design specifications and on the current values of the
optimization variables, the exchanger design routine determines the values of the shell
side and tube side heat transfer coefficients, the overall exchanger area, the number of
tubes, the shell diameter and tube side and shell side flow velocities, thus defining all
constructive details of the exchanger satisfying the assigned thermal duty specifications.
3. Overall actual heat transfer coefficient is then calculated and compared with the assumed
values of U.
4. If the difference between assumed and actual values is within 0.5%, then algorithm goes
to the next step. Otherwise it assigns new Uassumed = Ucalc and iterates the step 2 until
the Uassumed and Ucalc values are within 0.5% agreement.
5. The computed values of flow velocities and the constructive details of the exchanger
structure are then used to evaluate the cost objective function. The optimization
algorithm, based on the value of the objective function, updates the trial values of the
optimization variables which are then passed on to the design routine to define a new
architecture of the heat exchanger. The process is iterated until a minimum of the
objective function is found or a prescribed convergence criterion is met.

MECHANICAL ENGINEERING Page 9

 OPTIMIZATION-SIMULATED ANNEALING

CHAPTER 4

CASE STUDY- THE TRAVELLING SALESMEN PROBLEM

4.1 GENERAL

The TSP is one of the major success stories for optimization. Decades of research into
optimization techniques, combined with the continuing rapid growth in computer speeds and
memory capacities, have led to one new record after another. Over the past 15 years, the record
for the largest nontrivial TSP instance solved to optimality has increased from 318 cities.

4.2 PROCESS

In the traveling salesman problem, or TSP, we are given a set {c 1 ,c 2 , . . . ,cN} ofNcities
and for each pair {c i ,c j} of distinct cities a distance d(c i ,c j ). Our goal is to find an ordering
of the cities that minimizes the quantity
i 1

d(c (i) ,c (i 1 ) ) d(c (N) ,c ( 1 ) )

N-1.

This quantity is referred to as the tour length, since it is the length of the tour a salesman
would make when visiting the cities in the order specified by the permutation, returning at the
end to the initial city. We shall concentrate in this chapter on the symmetric TSP, in which the
distances satisfy d(c i ,c j ) = d(c j ,c i ) for 1 i, j N.
The symmetric traveling salesman problem has many applications, from VLSI chip N
fabrication [Korte, 1988] to X-ray crystallography [Bland & Shallcross, 1989], and a long
history, for which see Lawler, Lenstra, Rinnooy Kan, and Shmoys [1985]. It is NPhard [Garey &
Johnson, 1979] and so any algorithm for finding optimal tours must have a worst-case running
time that grows faster than any polynomial (assuming the widely believed conjecture that P
NP). This leaves researchers with two alternatives: either look for heuristics that merely find
near-optimal tours, but do so quickly, or attempt to develop optimization algorithms that work
well on real-world, rather than worst-case instances. Because of its simplicity and
applicability (or perhaps simply because of its intriguing name), the TSP has for decades served

MECHANICAL ENGINEERING Page 10

 OPTIMIZATION-SIMULATED ANNEALING

as an initial proving ground for new ideas related to both these alternatives. These new ideas
include most of the local search variants covered in this book, which makes the TSP an ideal
subject for a case study. In addition, the new ideas include many of the important advances in the
related area of optimization algorithms, and to keep our discussions of local search in
perspective, let us begin by noting the impressive recent progress in this latter domain.

MECHANICAL ENGINEERING Page 11

 OPTIMIZATION-SIMULATED ANNEALING

CHAPTER 5

CASE STUDY MECHANICAL FAILURE

5.1 GENERAL

The computational method of simulated annealing was applied to the problem of predicting
crack paths during mechanical failure of a solid containing a sharp flaw. The simulated annealing
algorithm described in the present research compared alternative crack paths using a normalized
strain energy density function based on the cumulative strain energy density for all area elements
along the crack path. The initial crack path chosen was co-linear with the existing sharp flaw
since this crack path would be expected to be the most likely crack path in a homogeneous
material of uniform properties. The simulated annealing algorithm was used to examine a
minimum of 60 and maximum of 60,000 alternative crack paths in the material. Alternative crack
paths were identified with a forward-bias, random walk algorithm. The simulated annealing
algorithm proved to be very effective in examining a range of alternative solutions in a window
60 angular width from the initial (co-linear) crack path. In addition, the algorithm was likely
to identify a more probable crack path as the number of solutions examined increased.

5.2 PROCESS

Many studies have focused on the mechanical properties of brittle materials [1-3], and, in
particular, calculation of a critical applied stress as a prediction of mechanical failure. Also of
concern for micro structural design is the 2 crack path in the presence of flaws or multiple
phases. However, prediction of crack paths has been prohibitive due to the large number of
possible crack paths and the resulting computational overhead along with the need to account for
the probabilistic nature of material properties. Simulated annealing has been shown effective in
optimization of multivariate discrete systems [4-23], and has been successfully applied to the
design of electrical circuits, manufacturing operations, scheduling, atomic structures, and
architectural layout. This paper describes the application of simulated annealing to the problem
of mechanical failure, and in particular, identification of probable crack paths in a solid
containing a sharp flaw. The task of finding the probable crack path is a problem of optimization

MECHANICAL ENGINEERING Page 12

 OPTIMIZATION-SIMULATED ANNEALING

if the probability of occurrence of any individual crack path can be quantified using a
mathematical function. In general, the mathematical function optimized in simulated annealing
describes the optimum case as a minimum in the mathematical function. Any such mathematical
function must consider both the boundary conditions (applied loads) and material properties, and
must return a single quantitative assessment. Optimization problems have been traditionally
solved with Heuristic approaches or iterative approaches. The Heuristic approach divides a
complex problem into simpler subsets, optimizes each subset, and combines the optimized
subsets into the optimum solution for the complex problem. While computationally efficient, this
approach can provide misleading solutions if synergistic effects between subsets are not properly
identified. An Iterative approach simply computes some small change to the system, calculates
movement of the system either towards or away from an optimum, and eliminates from
consideration any changes that move the system away from an optimum. An

Iterative approach is computationally intensive for complex multivariable systems, but often
misses a global optimum if the algorithm identifies a local optimum. Simulated Annealing has
been reported to be more effective at finding global optima since it permits some short range
motion away from a local optimum in search of an alternative solution. Simulated annealing
permits movement of the system away from local minima consistent with a Boltzmann
probability distribution:
P e(-E/kT)
Where: P = probability of some change occurring.
E = thermodynamic energy required for some change.
k = Boltzmanns constant.
T = absolute temperature of the system.
Any change in the system has a finite probability of occurring, and the magnitude of the
probability is related to the magnitude of the thermodynamic energy required for the change and
the magnitude of the absolute temperature of the system. If the change in the system results in a
small increase in the systems energy, it is probable (high Boltzmann probability) that the system
would change.
If the change in the system results in a large increase in the systems energy, it is improbable
(low Boltzmann probability) that the system would change.

MECHANICAL ENGINEERING Page 13

 OPTIMIZATION-SIMULATED ANNEALING

CHAPTER 6

6.1 APPLICATION

Circuit partitioning and placement.

Strategy scheduling for capital products with complex product structure.
Umpire scheduling in US Open Tennis tournament!
Event-based learning situations.
Basic Problems
o Traveling salesman
o Graph partitioning
o Matching problems
o Graph coloring
o Scheduling
Engineering
o VLSI design
Placement
Routing
Array logic minimization
Layout
o Facilities layout
o Image processing
o Code design in information theory

6.2 MOTIVATION

Annealing in metals.
Heat the solid state metal to a high temperature.
Cool it down very slowly according to a specific schedule.

MECHANICAL ENGINEERING Page 14

 OPTIMIZATION-SIMULATED ANNEALING

If the heating temperature is sufficiently high to ensure random state and the cooling
process is slow enough to ensure thermal equilibrium, then the atoms will place
themselves in a pattern that corresponds to the global energy minimum of a perfect
crystal.

6.3 HOW TO COPE WITH DISADVANTAGES

Repeat algorithm many times with different initial configurations

Use information gathered in previous runs

Use a more complex Generation Function to jump out of local optimum

Use a more complex Evaluation Criterion that accepts sometimes (randomly) also
solutions away from the (local) optimum

MECHANICAL ENGINEERING Page 15

 OPTIMIZATION-SIMULATED ANNEALING

CHAPTER 7

CONCLUSION

Simulated Annealing algorithms are usually better than greedy algorithms, when it comes
to problems that have numerous locally optimum solutions.

Simulated Annealing is not the best solution to circuit partitioning or placement. Network
flow approach to solving these problems functions much faster.

Simulated Annealing guarantees a convergence upon running sufficiently large number

of iterations.

Studying the ways in which this was done for the TSP may well be a good source of ideas
about how best to accomplish it in other domains.

MECHANICAL ENGINEERING Page 16

 OPTIMIZATION-SIMULATED ANNEALING

REFERENCES

1. E. H. L. AARTS AND J. KORST, Boltzmann machines for traveling salesman problems,

1. European J. Operational Res. 39 (1989), 79-95.
2. E. H. L. AARTS, J. KORST, AND P. J. M. VAN LAARHOVEN, A quantitative analysis
of the simulated annealing algorithm: A case study for the traveling salesman problem, J.
Statistical Physics 50 (1988), 189-206.
3. E. H. L. AARTS AND P. J. M. VAN LAARHOVEN, simulated annealing: An
introduction, Statistical Neerlandica 43 (1985), 31-52.
4. S. Kirkpatrick, C.D. Gelatt, Jr., and M.P. Vecchi, Science, 220, 671 (1983).
5. B.A. Berg and T. Neuhaus, Phys. Lett., B267, 249 (1991).

MECHANICAL ENGINEERING Page 17