Name: ___________________________

Signature: ___________________________ Theory

CEE 5290/COM S 5722/ORIE 5340 Heuristic Methods for Optimization Fall 2009 Final 12/10/2009

(Show your work so the grader can read and understand what you have done. Clearly state any assumptions you make and give any equations (if any) you are using.) You may have your lecture notes, your own handwritten notes, copies of your homework and posted homework solutions. You cannot have copies of old exams, copies of the textbook or any papers. Note you are asked to write on the last page and turn it in with your exam book. 1. (10 points) Assume you want to minimize f(x) for x in a set D. The global minimum of f(x) occurs at a point z . You have an algorithm A that searches for the global minimum of f(x). Let xi be the current value of the solution in the ith iteration . So A generates a series {xi} Assume A reaches the global solution on the 1000th iteration. However, you dont know this x1000 is z ( the global optimal solution), and you continue to run the algorithm forever so there is an infinite series {xi}. The algorithm is such that the sequence of points {xi} does not get within a distance r of three points y1, y2, y3 in D. Which of the following is true about algorithm A: a) Algorithm A converges to the global optimum of G for any continuous function G. (True, False or Not enough information given to answer) FALSE b) Algorithm A is a local optimization algorithm that converges to one local optimum (which could possibly but not necessarily be a global optimum) of G for any continuous function G. (True, False or Not enough information given to answer) not enough INFORmation Explain your answer to a) and b). Algorithm A is not dense so does not necessarily converge.,since it is not dense. ANSWER:

Theorem (Torn and Zilinska (1989)

An algorithm converges to the global minimum f for any continuous f if and only if the sequence of points visited by the algorithm is dense in D.

2. ( 10 Points) Multi Level Single Linkage The figure below shows the progress of an Multi Level Single Linkage algorithm that is seeking the global minimum of f(x). Explain your answers below.

a) Explain what a start point is. It is the initial point for a search by a local optimization meOD. b) Assume points A and B are start points from previous iterations and C is a start point for the current iteration. Give a rank order of the three values of the objective function at the points marked C, D, E , and the rank order of the objective function at the points C and B or indicate that there is not enough information to indicate the order. (For example, you could say something like f(C)> f(D) and f(C) > f(E), but I do not have enough information to know the ranking between D and E or between C and B. Note that is an example and is probably not the right answer.) CALL ME IF YOU HAVE QUESTIONS ABOUT The answers here. c) What is the next set of calculations to be done after this picture (before generating more sample points) ? Why are the next set of calculations done? Do the LOCAL OPTImizatioN SEARch. 3. (15 points) GA schema Consider a binary strings of length 11 and consider the following schemas.

1*********1, 11*********, *****111*** and **1***1**0*. a) Under crossover with uniform crossover site selection calculate a lower limit on the probability of these schemass surviving crossover. b) Calculate the probability of surviving mutation if the probability of mutation is 0.9 at a single bit position. c) Now calculate the probability of these schemass surviving both crossover and mutation (assuming crossover and mutation occur independently of each other).

a) 0, 9/10, 8/10,3/10 b) Probability of survival of a single bit =0.1 Exact (1-p(m))^o(H) Approx 1- o(H)*p(m) 0.8, 0.8, 0.7, 0.7 c) 0, 0.72, 0.56, 0.21

4. (15 Points) Simulated Annealing Assume you have an analysis of simulated annealing and you find that the stationary distribution of a 5 state system for fixed T is (exp(-8/T), exp(-5/T), exp(-8/T), exp(-10/T))/(M(T) Where M(T) = [exp(-8/T) + exp(-5/T) + exp(-8/T) + exp(-10/T)]

Does this stationary distribution converge as T goes to 0? What is the probability of being in each of the 4 states in the limit as T goes to 0? How many states are optimal? Identify all the optimal states. ANS (see page 64 of text) Limit goes to (0,1,0,0), optimal solution is state 2