Sie sind auf Seite 1von 4

An Improved Genetic Algorithm and its Blending Application with Neural Network

YAN Tai-shan
School of Information and Communication Engineering Hunan Institute of Science and Technology, 414000

Yueyang, Hunan, China Yantaishan163@163.com


AbstractIn order to overcome the limitation such as

premature convergence and low global convergence speed of standard genetic algorithm, an improved genetic algorithm named adaptive genetic algorithm simulating human reproduction mode is proposed. The genetic operators of this algorithm include selection operator, help operator, adaptive crossover operator and adaptive mutation operator. The genetic individuals sex feature, age feature and consanguinity feature are considered. Two individuals with opposite sex can reproduce the next generation if they are distant consanguinity individuals and their age is allowable. Based on this improved genetic algorithm, a thoroughly evolutionary neural network algorithm named IGA-BP algorithm is proposed. In IGA-BP algorithm, genetic algorithm is used firstly to evolve and design the structure, the initial weights and thresholds, the training ratio and momentum factor of neural network roundly. Then, training samples are used to search for the optimal solution by the evolutionary neural network. IGA-BP algorithm was used in pattern recognition example of Gray code. The illustrational results showed that IGA-BP algorithm was better than traditional neural network algorithm in both speed and precision of convergence, and its validity was proved.
KeywordsAdaptive genetic algorithm; Human reproduction mode; Evolutionary neural network; BP algorithm; Pattern recognition

I.

INTRODUCTION

Genetic algorithm is a stochastic global searching and optimizing algorithm which is used to solve complicated problems. It simulates the evolutionary course of natural selection and natural inheritance of biology circles. And it is provided with many advantages such as simple searching method, strong robustness, global parallel searching, and be suitable to solve the problems with large scale etc.. However, genetic algorithm also has its own shortages such as lower convergence speed, larger number of repeated times, and local optimal solutions etc. in solving the complicated optimization problem and multi-peak function optimization problem. Some academic research proved that the simple genetic algorithm couldnt reach the global optimal convergence state sometimes. In recent years, many scholars have carried on a

fruitful improvement towards genetic algorithm[1-14]. The actual genetic algorithm and its all kinds of improved algorithms have elevated the global convergence rate and optimization efficiency. But it needs further research and development, further more, simple genetic algorithm and many improved genetic algorithms only simulates natural evolution process simply and considers human own reproduction mode less. There-for, we use human own reproduction mode for reference, an adaptive genetic algorithm simulating human reproduction mode is proposed in this paper. Neural network is a complicated network system that can realize parallel disposing and nonlinear transformation for information by simulating the way of human cerebral nerves to dispose information. In all kinds of neural network algorithms, BP algorithm is a typical delegate. BP algorithm belongs to the grads descending algorithm essentially, it is easy for BP algorithm to get into local minimum and the speed of convergence is slow. In order to avoid conventional BP algorithms getting into local minimum and speed up its training and reduce the convergence time, genetic algorithm is usually used to evolve and design the initial parameters of neural network in the course of training. In this way, it is easy to achieve the global solution and improve the performance of neural network. Many scholars have done fruitful works in this area[15-19]. However, most of the existing design methods of neural network optimization are aiming at the structure and initial weights of network, and ignore the training ratio, momentum factor and initial thresholds. Actually, when a neural network is designed, the initial weights and thresholds are produced stochastically, and the training ratio, momentum factor and structure of network are usually gained depend on human experience. Structure, initial weights and thresholds, training ratio and momentum factor influence the training speed and disposing ability of neural network directly. Only optimize them roundly can the whole performance of network be improved. Aiming at the shortage of former research, the improved genetic algorithm is used to evolve and design the structure, initial weights and thresholds, the training ratio and momentum factor of neural network roundly. And then, training samples are used to optimize weights and thresholds in the evolved neural network. A thoroughly evolutionary

978-1-4244-5874-5/10/$26.00 2010 IEEE

neural network BP algorithm based on improved genetic algorithm was proposed. II. ADAPTIVE GENETIC ALGORITHM SIMULATING HUMAN REPRODUCTION MODE (HRAGA)

of individual sex is 0 or 1, 0 may be used to denote female individuals, and 1 may be used to denote male individuals. C. Judging individual age The age of the individuals who will enter mating pool is judged after selection operation in HRAGA. Suppose that the maximal permitted age is N when individuals mate. The individual age of initial population P (0) is ni (0) =1, and individual age will be add one when the population evolves one times. The individual age ni (t ) of evolved new population will be noted after an evolution. If ni (t ) <N, the individual may be put into the mating pool. Generally, the maximal permitted age N is not bigger then 90 percent of the maximal evolution generation in order to prevent the individuals without reproduction ability from evolving. D. Mating and examining individual consanguinity In HRAGA, individuals with same sex are not permitted to mate. That is to say a male individual can only mate with a female individual. Further more, individuals mate in order of their excellence sequence, namely male individuals that lined up according to the excellence sequence mate with female individuals that lined up according to the excellence sequence one by one. That is advantageous for genetic algorithm to elevate the speed of finding out the global optimal solution and boost up the global convergence ability. In order to avoid near relative reproduction, two opposite sex individuals still need to carry on an examination on individual difference. If two individuals are equal or the difference between their corresponding binary values is one, they can't cross and it is necessary to revise them. Here, the high bit value of individual code with low fitness should be modified by the different value of the high bit value of individual code with high fitness. Thus, far consanguinity reproduction between individuals can be ensured and elevate the efficiency of genetic algorithm. E. Genetic operator Selection operator The selection method that lines up individual order between two generations is adopted to select optimal individuals in HRAGA. Re-line up all male individuals and female individuals of father generation and child generation respectively, then choose the anterior N/2 excellent individuals from male individuals set and female individuals set respectively as objects of crossover operation according to the population size N in allow of the age scope. Thus, not only the effective mating of individuals in crossover operation can be ensured, but also the excellent individuals of every generation can be reserved and the ill individuals may be washed out. 2) Help operator In order to reinforce the ability of algorithm to jump out local optimal solutions and increase the speed of convergence, a help operator is used in HRAGA to help individuals according to the probability of help. Help operation is located after selecting operation and before mating operation. Under the condition that the individuals fitness doesnt reduce, its key operation is to change the individuals gene whose value is 0 according to the probability of help. 1)

A. General Idea Human reproduction mode is essentially different from that of other animals. So, the advancement of human quality and human civilization is superior to other animals. Firstly, in order to improve the quality of population, laws make explicit provision to marry age. Certainly, individual bearing ability will disappear naturally after a certain age. Secondly, human reproduction mode is a strict far consanguinity reproduction, but in the animal kingdom, "incest" phenomenon takes place usually. In order to avoid near consanguinity reproduction, corresponding laws are established to restrict human reproduction. Thirdly, human are able to adopt all kinds of effective measures actively to help offspring individuals including little embryos in order to ensure their healthy growth. And other animals only conform to environment passively. Human reproduction mode is used for reference and an adaptive genetic algorithm simulating human reproduction mode (HRAGA) is proposed. The genetic individuals of this algorithm are separated into male individuals and female individuals, the age feature and consanguinity feature are fused into individuals. Two individuals with opposite sex can reproduce the next generation if they are distant consanguinity individuals and their age is allowable. The genetic operators include selection operator, help operator, adaptive crossover operator and adaptive mutation operator. Fig.1 illustrates the working flow of HRAGA.

Figure 1.

The working flow of HRAGA

B.

Coding Method

Binary system is used to code individuals in HRAGA. The individual code is consisted of two parts, the first of which is the code of individual exhibition model and the second of which is the code of individual sex. Where, the code

3) Adaptive crossover operator and adaptive mutation operator Under binary coding mode, one-point crossover is a very effective crossover operation, it is adopted in HRAGA. Mutation means to choose some individuals stochastically from population under a small probability of mutation, then choose one bit or some bits of the chosen individuals stochastically and overturn their numbers, namely turn 1 to 0 or turn 0 to 1. In the course of crossover and mutation, the probability of crossover and the probability of mutation are adjusted adaptively according to the following formulas:
Pc max Pc min , Pc min + f / f avg 1 + exp( ( )) Pc = f max f avg Pc max , Pm max Pm min , Pm min + f f avg 1 + exp( ( )) Pm = f max f avg Pm max , f / f avg f / f avg
f f avg f f avg

In IGA-BP algorithm, the improved genetic algorithm is firstly used to evolve and design the structure, initial weights and thresholds, training ratio and momentum factor of neural network roundly, and a better searching space is found out in the solution space. And then BP algorithm is used to search for the optimal solution in the small searching area. B. Code Scheme All of the structure, weights and thresholds, training ratio and momentum factor of neural network need to be designed are regarded as a set of chromosomes, code them by binary system. Every individual code comprises five parts: the first part is the code of weights, the second part is the code of thresholds, the third part is the code of connection state between nodes, the fourth part is the code of training ratio and the fifth part is the code of momentum factor. Fig.2 denotes the structure of individual code.
Weights Thresholds Connection Training state ratio Fig.2. The structure of individual code Momentum factor

Where, Pc min and Pc max denotes the lower limit and the upper limit of the probability of crossover respectively, f max denotes the maximal fitness of the population, f avg denotes the average fitness of the population, f / denotes the higher fitness of the two crossover individuals, f denotes the fitness of mutation individual. From formula 1 2 we can see that the probability and , of crossover and the probability of mutation are adjusted 1 nonlinearly according to the function f ( x) = 1 + exp( x ) between the average fitness and the maximal fitness. When the fitness of the majority of individuals is near and the average fitness is close to the maximal fitness, the probability of crossover and the probability of mutation will be elevated. So, the individuals nearby the maximal fitness are preserved as many as possible. III. THOROUGHLY EVOLUTIONARY NEURAL NETWORK BP ALGORITHM BASED ON IMPROVED GENETIC ALGORITHM(IGA-BP)

C. Fitness Function The fitness function may be defined as follows: 1 (5) f =


R +1

Where, R = 1 e(i ) 2 e(i ) = y (i ) y m (i ) l denotes the


l
i =1

number of training samples, y (i) denotes networks factual output, y m (i ) denotes networks expected output, e(i ) denotes the error between networks factual output and its expected output. D. Working flow of IGA-BP Fig.3 illustrates the working flow of IGA-BP algorithm.

A. Basic Principle BP algorithm is a kind of training algorithm with tutor, it is virtually a kind of simple static algorithm for seeking the optimal by grads descending. Weight w(n) is modified only according to the direction of negative grads at time of n , and the former experience is not considered. Generally, an additional momentum is added. This method means to add an item in proportion to the former variety to current variety of all weights and thresholds, and the new weights and thresholds will be produced according to the reverse transmission. The regulating formulas with additional momentum of weights and thresholds are defined as: w jt (k + 1) = (1 mc) d tk b j + mc w jt (k ) (3)
t ( k + 1) = (1 mc ) d tk + mc t ( k )

Fig.3. The working flow of IGA-BP algorithm

IV.

APPLICATION OF IGA-BP IN PATTERN RECOGNITION

(4)

Where, k denotes training times, denotes training ratio, mc denotes momentum factor, 0 < < 1 0 mc < 1 .

We take the pattern recognition targets in table 1 as examples, and simulate them by IGA-BP algorithm. In the given pattern recognition targets, the first bit of input data is a noise bit that has nothing to do with output. Output data is the Gray code of the last four bits of corresponding input data[20].
TABLE . PATTERN RECOGNITION TARGETS

Input 00000 00010 00100 00110 01000 01010 01100 01110

Output Input 0000 10000 0011 10010 0110 10100 0101 10110 1100 11000 1111 11010 1010 11100 1001 11110

Output Input 0000 00001 0011 00011 0110 00101 0101 00111 1100 01001 1111 01011 1010 01101 1001 01111

Output 0001 0010 0111 0100 1101 1110 1011 1000

Input Output 10001 0001 10011 0010 10101 0111 10111 0100 11001 1101 11011 1110 11101 1011 11111 1000

area. A thoroughly evolutionary BP neural network algorithm based on improved genetic algorithm (IGA-BP algorithm) was formed in this way. IGA-BP algorithm was used in pattern recognition example of Gray code. The experimental results verified that the convergence speed and precision of IGA-BP algorithm was better than that of conventional BP algorithm. REFERENCES
[1] [2] [3] Nicol N Schraudolph, Richard K B. Dynamic Parameter Encoding for Genetic Algorithms [J]. Machine L earning, 1992, 6 (1) : 18. Fogel B D. An Introduction to Simulated Evolutionary Optimization[J]. IEEE Transactions on Neural Networks, 1994, 5 (1) : 414. WEN Guang-hui, WANG Ming-xu, GUO Si-cong. A New GA Coding Scheme for Solving TSP[J].Science Technology and Engineering, 2006, 6(2): 206208 LIU Ying. Research on Fitness Function in Genetic Algorithm[J]. Journal of Lanzhou Polytechnic College, 2006,13(3):14 Zhang Si-cai Zhang Fang-xiao. A Modified Method To Fitness Function Of Genetic Algorithms[J]. Computer Applications and Software, 2006,23(2):108110 TIAN Dong-ping. A Study of Crossover Operators in Genetic Algorithms[J]. Journal of Inner Mongolia Normal University (Natural Science Edition),2007,36(3):308-314 Khadiza Tahera, Raafat N. Ibrahim, Paul B. Lochert. GADYM - A Novel Genetic Algorithm in Mechanical Design Problems[J]. Journal of Universal Computer Science, 2008,14(15):2566-2581 WANG Zeng - qiang , ZENG Bi. Research on Selecting Crossover Individuals in Genetic Algorithms[J]. Journal of Shantou University ( Natural Science),2005,20(4):5558 XIONG Wei-qing LIU Ming-da ZHANG Shao-yu. A Genetic Algorithm with Sex Character[J]. Computer Engineering, 2005,31(1):165166 Li He-cheng Wang Yu-ping. Genetic Algorithm with a Hybrid Crossover Operator and its Convergence[J]. Computer engineering and application,2006,16:2224 CAI Liang-wei , L I Xia. Improvement on crossover operation of genetic algorithms[J]. Systems Engineering and Elect ronics, 2006,28(6):925 928 Kuang Hangyu, Jin Jing, Su Yong. Improving Crossover and Mutation for Adaptive Genetic Algorithm[J]. Computer engineering and application,2006,12:9396 WU Jian-sheng , NONGJi-fu etc. An Analysis of the Convergence of Genetic Algorithm[J]. Journal of Guangxi Teachers Education University (Natural Science Edition), 2004,21(1):4852 WANG Li. An Unified Convergence Criterion for Genetic Algorithms [J].Technique s of Automation &Applications, 2004,23(6):1619 HE Fangguo, Qi Huan. Back propagation neural network based on modified genetic algorithm and its application[J]. Journal of Huazhong Normal University(Nat . Sci. ) , 2007, 41(1):51-54 Li Ying, Xu Tao, Xing Wei. Optimization for Neural Network Based on Evolved Genetic Algorithm[J]. Journal of Changchun University of Technology, 2006(3):4850 Deng Zheng-Hong, Hu Qing, Zhen Yu-Shan.Two-time Training Algorithm of Neural Network Based on Genetic Algorithm[J]. Microelectronics and Computer, 2005(3):252254 Dam M, Saraf D N. Design of neural networks using genetic algorithm for online property estimation of crude fractionator products[J].Computers and Chemical Engineering, 2006, 30 (4):722-729. Mu A-Hua, Zhou Shao-Lei, Liu Qing-Zhi, Xu Jin. Using Genetic Algorithm to Improve BP Training algorithm[J]. Computer emulation , 2005(2): 150151 Zhou Ming, Sun Shu-Dong: Genetic algorithm: theory and application[M]. National defensive industry publishing press, Beijing 1999

Twenty-eight samples are selected from table 1 to train neural network by standard BP algorithm, BP algorithm with additional momentum and IGA-BP algorithm respectively. In the course of training, the error level is designed as MSE10 . After trained 66 times by IGA-BP algorithm, the network arrived at its convergence and MSE =0.000010. After trained 1501 times by BP Algorithm with additional momentum, the network arrived at its convergence and MSE =0.000996. And after trained 3462 times by standard BP Algorithm, the network arrived at its convergence and MSE =0.001000. Fig.4 gives the connection curve between the number of iteration and MSE of these algorithms.
0.1 0.09 0.08 0.07 0.06 MSE 0.05 0.04 0.03 0.02 0.01 0 IGA-BP algorithm BP algorithm with additional momentum Standard BP algorithm

[4] [5]

[6]

[7]

[8]

[9]

[10]

500

1000

1500 2000 Iteration times

2500

3000

3500

[11]

Fig.4. The connection curves between the number of iteration and MSE of three algorithms

[12]

It is obvious that IGA-BP algorithm is better than standard BP algorithm and BP algorithm with additional momentum whether looked from the speed of convergence or looked from the precision when given the same number of iteration. Four samples that have not been trained are selected from table 1 for neural networks to recognize after training. The recognition results of the neural network trained by IGA-BP algorithm are shown in table 2. Distinctly, table 2 shows that the recognition results are very exact.
TABLE . PATTERN RECOGNITION RESULTS Input Factual Output 10001 0.001825 0.000630 0.000522 0.993675 01110 0.999568 0.000329 0.000098 0.999826 01011 0.999875 0.999128 0.999623 0.000583 11010 0.999304 0.998950 0.999612 0.999910

[13]

[14] [15]

[16]

[17]

[18]

V.

CONCLUSION
[19]

An improved genetic algorithm named adaptive genetic algorithm simulating human reproduction mode (HRAGA) was proposed in this paper. By right of the powerful ability of this improved genetic algorithm to search for the optimal in global space, it was used to evolve and design the structure, the initial weights and thresholds, the training rate and momentum factor of BP neural network, and orientated a better searching area in the solving space. Then, BP algorithm was used to search the optimal solution in the small searching

[20]

Das könnte Ihnen auch gefallen