A hybrid system composed of neural networks and genetic algorithms

Dumitrescu Mihaela
University of Pitesti, Romania e-mail: monydum2006@yahoo.com

Abstract Neural networks are known for their ability to recognize patterns of noisy, complex data and to estimate a nonlinear relationship between them. But the design of neural networks is very complex because it works on the principle of "black box". For this reason, the application of genetic algorithms in neural networks with hybrid systems can improve a network's ability to make predictions. Hybrid systems involve the use of combined techniques, issues and different models in order to achieve the overall performance better than those offered by each solution considered separately. Projections made by hybrid systems have smaller errors and constructed systems are able to automatically select the variables necessary to function effectively. This is possible due to the principle of selective biological function of genetic algorithms. They select from a large population of neural networks the best generations, made their exchange of elements and even mutations to get the most advanced networks. For an evolving and performance changes economic environment are necessary tools that can help make faster optimal decisions and increase the business efficiency. Keywords- genetic algorithms; hybrid systems; neural network; business forecasting

I. INTRODUCTION For any participant in economic activity, the dynamics of different phenomena or market indicators is essential. For this reason we need fast and efficient ways to conduct economic forecasts. An analysis technique is based on data for previous periods and can be achieved using neural networks. They have the ability to learn from the evolution of prices in a market, for example, to estimate their future values. But it is very difficult to effective conceive a neural network because of the "hidden" specific way of work. Thus, in neural network design can be used technology of genetic algorithms to get better results. Genetic algorithms can be used to find an optimal architecture for a neural network to design an optimal training algorithm control parameters and optimization. II. REASONS FOR USING HYBRID SYSTEMS The motivation of using intelligent hybrid systems is simple: although various intelligent techniques (neural networks, genetic algorithms, fuzzy logic, evolutionary computation, expert systems) offers encouraging results in solving problems with high degree of customization, complex problems can be solved through a single intelligent techniques. For example, neural networks are successfully used in making economic forecasts, but can not provide an explanation regarding the decision. To compare the performance of intelligent systems is necessary to define their properties. Knowledge acquisition represents a crucial stage in the development of intelligent systems. As a process, involves the interpretation and representation of knowledge for a specific area. In the case of expert systems, requires a long time, is expensive and sometimes unreliable. From this point of view, techniques such as neural networks or genetic algorithms that can learn directly from the input shook, have certain advantages. Given the distributed way of knowledge representation, neural networks has some advantages in this respect. Even if some of the neurons are no longer operational, the remaining neural network will work independently and provide to some extent the right response. Fuzzy systems deal with the fragility of a specific approach to knowledge representation and reasoning methods. Knowledge is represented by fuzzy sets that allow the introduction of vague boundaries between concepts. This representation of data, in conjunction with fuzzy reasoning mechanism allows processing of incorrect or partially correct data. The existence of processes of information processing, parallel and low level, respectively sequential and high level was highlighted in human behavior by different researchers. Expert systems have proved plausible models to describe high-level cognitive tasks (eg, generating and understanding language). In contrast, neural network have complementary

properties: pattern recognition model is suitable, for example in visual processing tasks, but are unable to model sequential high-level cognitive tasks. Possibility of providing to the user explanations reasoning the process is an important feature of intelligent systems. Explanations are necessary to validate by the user of solutions generated by intelligent systems, for example, the automatic generation of medical diagnostics. In fuzzy logic, the final decision is generated by aggregating the final decision making all the rules of inference based on existing rules. In such systems to obtain the chain of inference is not easy, but the rules of the form "if-then" are easily understood by the user. Genetic algorithms, in particular, for the classification systems can build models of reasoning in the form of rules. As with expert systems, it is possible to follow the chain of inferences and provide a degree of explanation about the thinking process. In contrast to the systems mentioned above, neural networks are not generally able to offer explanations, be cause knowledge are not explicit, knowledge are transferred in the form of distributed weights the entire neural network. III. ARTIFICIAL NEURAL NETWORKS Artificial neural networks are computer reproduction of the human nervous system (Michalaewicz, Schmidt, Michalewicz & Chiriac, 2007; Gorunescu, 2011). An artificial neural network contains several layers of artificial neurons: input layer, hidden layer and output layer. They are connected and to build a neural network architecture is a complex activity. A neural network model can be applied to economic forecasting using back-propagation method. The algorithm backpropagation neural network consists of several stages. First, you select a network input vector. It may comprise a series of values specific for an economic indicator. These values can be represented by observations in discrete time (days, weeks, quarters) or can be obtained in continuous time data. Thus, with a time series : , where tT and TR refers to time, knowing the past values of the phenomenon studied: 1 , 2 , 3 ,..., t , we can use as input vector for neural network as: x i ={ x 1 , x 2 , , x n }, to estimate future values of its , h>1. The first stage involves sending the signal before the network the following calculations includes: u
j
t +h

=a 0 j +

v k = b 0k +

b
j =1

i =1 J

ij

xi ,

(1)

jk

yj ,

(2) (3)

y j =g(u j ) , j= 1, , J,

(4) z k =g(v k ) , k= 1, , K. Next establish the mean squared error recorded:

1 N k n=1 k =1 ( z kn t kn ) 2 E= 2 NK
w
m +1

(5)

After the error is calculated, weights are adjusted to improve network performance: = wm-d ,
N m

d =

x
n =1

y
n

(6)

These steps are repeated until the error is less than a predetermined value or after a certain number of iterations. The advantage of neural network in making predictions in various areas is that it can be used although there is no predefined relationship between input data and output data. The way it works allows him to find a correlation between the two categories of data. To improve results using neural networks in making predictions, we appeal to another technology: genetic algorithms.

IV. MODELS EVALUATION There are a variety of methods to achieve different prediction models, the problem is choosing the right one. To solve this problem it is necessary for different models to be evaluated and compared. In principle, the best method is one that provides the lowest prediction error. In reality, we face some difficulties. First, the data available is not large enough. Secondly, the performance of a method of training data may be different from performance on another set of data. It is important to develop a forecasting model that works well on new data. Another problem is that the forecasting methods that give different results require different techniques of error measurement. Also, bear in mind the cost of possible errors. When cases are classified into two categories, for example "right" or "fraud" there may be two types of errors: false-positive errors when the result is incorrectly shown as "correct" give falsenegative, when the result is classified as "fraud", in fact correct. The cost of these errors is different. When a transaction is properly classified as fraud, is a small cost to verify the transaction. On the other hand, the classification of a fraudulent transaction to be fair is a much higher cost, especially if it has great value. To do better with changes that occur with each new case indicated is the selection of a forecast model with a high degree of generality. Such a model allows a better adjusting to changes in the database. V. GENETIC ALGORITHMS IN BUILDING ARTIFICIAL NEURAL NETWORKS A method of building artificial neural networks architectures automatically using genetic algorithms is described below. It contains two adaptive processes: genetic search through the data entry window, the forecast horizon, network architecture space and control parameters to select the best performers on the one hand and back propagation learning in individual networks to evaluate the selected architectures. The method starts with an initial population of randomly generated networks. The networks will increase and hidden nodes will be inserted into the network as the population evolves. Input nodes receive a random combination of input data from an input window that takes data from the data file and transmit network training. Node selects a random output from a forecasting horizon exit window and uses a data item associated with the target values . Both windows provide training data given random sequence. Initial population of networks pass the first generation of evolution. As in real biological system, the cycles of learning are outlined according to population trends. Each learning cycle involves the entire population of learning networks with input-output pairs offered by input-output windows. Network output is compared with target data set (expected output) and the connection weights are adjusted to achieve the desired output, thus minimizing errors. At the end of each cycle of learning, networks are classified according to predetermined criteria such as their ability to generalize. The weaker networks will be excluded from the population and most advanced networks will be retained to form a new reproduction cycle. Their output nodes become hidden nodes for next generation networks that store the knowledge that the previous generation acquired. Occasionally, mutations are introduced to ensure that those networks will not return to the initial process of learning and development. One way to make changes is to modify the network weights with the poor performance of the input data or change the forecast horizon. After each cycle of evolution, a better picture of the network is preserved. The image includes the input data, the forecast horizon, models of evolution and weights so that the best network pass in next generation networks with the newly formed population. Picture of the best network remains intact until there is another network with better performance. VI. NEURAL NETWORK OPTIMIZATION WITH GENETIC ALGORITHMS A hybrid system consisting of neural networks and genetic algorithms has the ability to create networks and find the best architecture for predicting a phenomenon (Taniar 2008, Koza, 1992, Fyfe,1996). It builds an initial population of networks from which to start crossing. These networks are evaluated using a function (referred to as FE). Only a percentage of initial networks will be retained for a subsequent cross called crossover rate. Another process, the mutation involves addition of the mutual exchange of genes and change them to some extent (percentage) is called mutation rate. By crosses are obtained successive new generations until reaching an optimum network architecture. Generation 0 Initial population of networks is represented by the networks from which to start crossing. Will be kept only two networks to assess the best as shown in Figure1 .

Input 7 3 15 20

Neurons number in layer 1 6 4 14 17

Neurons number in layer 1 5 4 14 0

Output 2 1 1 2

Constant 1 0 0 1

FE 95 80 90 81

Figure 1. Table with initial population of networks Generation 1 Selected networks will be transformed by changing some genes (properties) of these. This can be done without the mutation, as shown in Figure 2. Also, the transformation by changing of gene can be done with mutation when in addition to the exchange of genes is done an amendment of them with a certain amount, as shown in Figure 3. The purpose for the mutation is getting more numerous populations without reaching the initial population.

Input

3 20 20 3

Neurons number in layer 1 4 17 4 17

Neurons number in layer 1 4 0 4 0

Output

Constant

FE

Network type 80 81 75 85 parent parent new new

1 2 1 2

0 1 0 1

Figure 2. Table with selected networks transformed by changing some genes (properties)

Input

3 20 21 4

Neurons number in layer 1 4 17 4 17

Neurons number in layer 1 4 0 4 0

Output

Constant

FE

Network type 80 81 78 83 parent parent new new

1 2 1 2

0 1 0 1

Figure 3. Table with mutation for getting more numerous populations without reaching the initial population

Thus, one can see that, as it made crossing the network, are selected those with the best evaluation, as shown in the table below.

Gene ration 1 2 2 with mutation 1

FE 80 75 78

FE2 81 80 80

Figure 4. The selected networks with the best evaluation

Crossing continues with other genes (next hidden layer of neurons, etc.) but it should be noted that in real applications is based on much larger populations and used computers with high-speed computing. VII. CONCLUSIONS Using genetic algorithms is a general procedure based on genetics and natural selection and its power lies in the fact that the population of parents resulted in a new generation networks with chances to take the best features. Thus, network performance population increase from generation to generation. Genetic algorithms can be used both to find the best network architecture and to find the right combination of inputs, the best prediction horizon and the most effective weight. Among the technology falls is that it operate only directly on numeric variables. As a result, any non-numeric variable in the data set must be converted into numeric and analyzed before use in training the network. In complex problems, the user is forced to solve a compromise between increasing the number of hidden neurons, which can lead to very long training and to accept a simpler topology, associated with less accurate solutions. For datasets with many attributes, the use of neural networks becomes unfeasible. Determining the number of hidden neurons for complex classification problems, can only be made experimental, which on one hand increases substantially the search time for optimal classification model, on the other hand leaves the quality of analysis results to depend on the experience user. The absence of a descriptive component generated by a neural network model makes the model development phase of training to lack of transparency for the user. Because of these characteristics, the technique is often compared to a "black box". Another limitation of the artificial neural networks is the needed for a long time for training and the need for a relatively large number of instances in training set. Artificial neural networks technique has several significant advantages, however. Thus, once trained, the network can make predictions for new instances rapidly. This feature makes the neural networks are successfully used in problems that require real-time response. Using artificial neural networks and genetic algorithms is not always a perfect solution but in many cases can lead to an acceptable solution with minimum consumption of resources and time.

References Agrawal R., Srikant R., (1995) Mining sequential patterns, Proc.11th ICDE, Taipei, Taiwan, March Andone I., Mockler R., Dologite D., ugui A., (2001) The Development of inteligent systems in economy, Editura Economic, Cleanu C., Tiponu V., Neuronal Networks. Application, (paper achieved during the programme called TEMPUS ACJEP 13438-98) Fyfe, C., (1996) Artificial Neural Networks, Departament of Computing and Information Systems, The University of Paisley, 71-90. Gorunescu, F. , (2011) Data Mining Concepts, Models and Techniques, Springer-Verlag Berlin Heideberg, 191-248, 289-313. Koza, J. (1992) Genetic Programming, Massachusetts Institute of Tehnology, 17-63. Michalaewicz, Y., Schmidt, M., Michalewicz, M., Chiriac, C., (2007). Adaptive Business Inteligence, Springer-Verlag Berlin Heideberg, 131-150. Taniar, D., (2008) Data Mining and Knowledge Discovery Tehnologies, IGI Publishing Hershey-New York, 98. Velupillay K., (2000 ) Computable Economics, Oxford University Press Inc. New York.