Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

1 of 35 pages

Pythia The Neural Network designer Introduction

Pythia is a program for the development and design of Neural Networks. Neural Networks are used to detect hidden relations in a set of patterns, e.g. stock market data or weather data. Pythia es un programa para el desarrollo y diseo de redes neuronales. Redes neuronales se utilizan para detectar relaciones ocultas en un conjunto de patrones, por ejemplo, mercado de valores de datos o de datos meteorolgicos. Pythia features Backpropagation Networks. The network parameters (Weights) are initially set to random value. During the Training phase the actual output of the network is compared with the desired output and the error propagated back toward the input of the network. Pythia caractersticas backpropagation Redes. Los parmetros de red ("Pesos") inicialmente establecido en valor aleatorio. Durante la "fase de formacin" la produccin efectiva de la red se compara con la salida deseada y el error de propagacin hacia la entrada de la red. A Neural Network has two phases, commonly referred to as the Training phase and the Reproduction phase. During the training phase sample data containing both inputs and desired outputs are processed to optimize the networks output, meaning to minimize the deviation Una red neuronal tiene dos fases, comnmente conocida como la "fase de formacin" y la "fase de la reproduccin". Durante la fase de entrenamiento que contiene tanto datos de ejemplo - las entradas y salidas deseada - se procesan para optimizar la salida de la red, es decir, para reducir al mnimo la desviacin (OUTPUTDATA OUTPUTNET)2
OUTPUTDATA is the output value in the training data, OUTPUTNET is the output value provided by reproducing the input data with the network.

OUTPUTDATA es el valor de salida de datos en la formacin, OUTPUTNET es el valor de salida por la reproduccin de los datos de entrada con la red.

During the reproduction phase the networks parameters are not changed anymore and the network is used for the reproduction of input data in order to predict suitable output data. Durante la "fase de la reproduccin" de la red no se cambian los parmetros y la red ya se utiliza para la

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

2 of 35 pages

reproduccin de datos de entrada con el fin de "predecir" una salida adecuada de datos.

Picture 1 A typical Backpropagation network Picture 1 shows a typical Backpropagation Network. It has 2 inputs and 1 output. It contains two layers (levels) of neurons, level 1 with 2 neurons and level 2 with 1 neuron. Figura 1 muestra una tpica red backpropagation. Dispone de 2 entradas y 1 salida. Contiene dos capas (niveles) de las neuronas, de nivel 1 con 2 neuronas y el nivel 2 en contra y 1 neurona. In Backpropagation Networks each neuron has one output and as many inputs as neurons in the previous level. En Redes backpropagation cada neurona tiene una salida y el mayor nmero de entradas como las neuronas en el nivel anterior. Each network input is connected to every neuron in the first level. Each neuron output is connected to every neuron in the next level. Cada red est conectada a la entrada de cada neurona en el primer nivel. Cada neurona est conectada a la salida de cada neurona en el siguiente nivel. The Networks output is the output of the last levels neurons. La Red de la salida es la salida del ltimo nivel de las neuronas.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

3 of 35 pages

Each neurons output is calculated as Cada uno de las neuronas de salida se calcula como On = F(Ik * Wkn)
k O ist the neurons output, n is the number of the neuron, Ik are the neurons inputs, k is the number of inputs, Wkn are the neurons weights. F is the Fermi function 1/(1+Exp(-4*(x-0.5)))

Picture 2 Sample calculation of a Network

The activity of N1 is calculated as A = (1*0.249733)+(0*-0.233776) = 0.249733 The output of N1 is calculated as O = Fermi(A)= 1/(1+Exp(-4*(0.249733-0.5))) = 0.268731 The outputs of N1 and N2 are the inputs for the calculation of N3

The network is processed from the left to the right. Picture 2 shows a sample how the output of a Neural Network is calculated from the input. La red es la transformacin de la izquierda a la derecha. Figura 2 muestra un ejemplo de cmo la salida de una red neuronal se calcula a partir de la entrada.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

4 of 35 pages

Pythia allows you to import data from different file formats or from spreadsheet programs like Microsoft Excel. You can design and train Neural Networks. Both, data and networks can easily be stored on disk. Pythia le permite importar datos de diferentes formatos de archivo o de programas de hoja de clculo como Microsoft Excel. Usted puede disear y entrenar redes neuronales. Tanto, los datos y las redes pueden ser fcilmente almacenado en el disco. A special feature of Pythia is the Evolutionary Optimizer that automatically generates suitable networks for a given training data set. It uses evolutionary algorithms for the selection and generation of neural networks. Una caracterstica especial de Pythia es el Optimizador evolutivo que genera automticamente las redes adecuadas para un determinado conjunto de datos de formacin. Utiliza algoritmos evolutivos para la seleccin y la generacin de redes neuronales. In order to get familiar with Pythia we recommend to work through the two examples we discuss below. Con el fin de familiarizarse con Pythia recomendamos trabajar a travs de los dos ejemplos se discuten a continuacin.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

5 of 35 pages

Ejemplo 1 Problema XOR Un ejemplo clsico para aprender redes neuronales es el problema XOR. La red se supone que debe aprender el patrn: Input1 Input2 Output Pattern 1 0 0 0 Pattern 2 0 1 1 Pattern 3 1 0 1 Pattern 4 1 1 0 Tabla 1: Tabla Lgica XOR

La red que necesitamos disear tiene 2 entradas y 1 salida.

Figura 1: Tabla de Patrones XOR En primer lugar tenemos que obtener los patrones de formacin (0,0,1), (0,1,1), (1, 0, 1) y (1,1,0) en Pythia. La forma ms fcil es usar un programa de hoja de clculo como Excel. Introduzca el patrn como se muestra en la Figura 1. Seleccionar columnas A hasta C y filas de 1 a 4 con el ratn. Presione CTRL-INS, o presionar el botn derecho y luego seleccione Copiar en el men desplegable.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

6 of 35 pages

Los datos se encuentran en el portapapeles de Windows. El siguiente paso es llamar Pythia. Pythia inicialmente se ver como se muestra en la Figura 5.

Figura 2: Men del Pythia Seleccione EDITPASTE CELLS o simplemente presione SHIFT-INSERT.

Figura 3: Ventana de Dialogo de Copia

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

7 of 35 pages

Usted recibir un cuadro de dilogo como se muestra en la Figura 3. Deje el campo delimitador (TAB) y los campos por registro (3). Pegar a los nuevos criterios que es correcta, porque queremos crear un nuevo patrn establecido. Observe que los productos no est establecido en 1 por defecto. Si la red se supone que tiene algo ms que una salida, aqu se debe especificar. Finalmente pulse el botn OK. Figura 4 muestra Pythia despus de pegar el modelo de datos establecido.

Figura 4: Los criterios que los datos se han pegado en una tabla nueva

Ves las columnas I1, I2 y O1. Las columnas O1 (NET) y (SQ DV) estn vacos porque se llenan ms tarde por la red neuronal ahora vamos a disear. Es una buena idea guardar los criterios que con el nombre de "XOR.PAT" ahora.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

8 of 35 pages

Figura 5: Creacin de una Red Neuronal Para crear una nueva red neuronal hacer los siguientes pasos: Seleccionar NETCREATE NET Introduzca el nmero de entradas de su red tendr (2) Introduzca el nmero de neuronas en el nivel 1. Seleccionamos 2. Introduzca el nmero de neuronas en el nivel 2. Dado que este es el ltimo nivel se define, es igual al nivel de salida. Por lo tanto, seleccione 1. Finalmente pulse el botn OK.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

9 of 35 pages

Figura 6: Red Neuronal Creada Ahora trate de reproducir el modelo establecido con la nueva red.

Figura 7: Mostrar Patrones Seleccione NET REPRO PATTERN SET, o pulse el icono que se indica en la figura 7 Como puede ver en la Figura 7 los resultados son pobres tranquila. La columna O1 (NET) debe ser similar a O1. La columna de la derecha ms SQ DV le informa sobre el cuadrado de la desviacin y se define como SQ DVi = (O1i O1(NET)i)2
i is the pattern number Example: (0-0.071094)2 = 0.005054

Es la meta de la fase de entrenamiento para reducir al mnimo estos valores. Si nos fijamos en los resultados en la columna O1, ver que la salida es 0 1. En consecuencia una O1 (NET) de un valor de <0,5 ser suficiente para 0 y un valor de ms de 0,5 ser suficiente para 1. Esto significa que la desviacin para cada patrn se reproduce debe ser <0,5, es decir, la plaza de esta desviacin debe ser <0,25.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

10 of 35 pages

Figura 8: Set up a learn plan

Vamos a configurar un plan de aprender. Seleccionar NETLEARN PATTERN SET Figura 11 muestra la ventana de dilogo para la creacin del plan de aprender. Por favor, deje los valores por defecto, con la excepcin de * desviacin. Este punto de verificacin e introduzca el valor 0,25. Presione el botn OK.

Si cada desviacin es inferior a 0,25 de la red es lo suficientemente entrenado. Esto probablemente no es el caso de que la primera vez que intentarlo. La razn es que no todos los valores a partir de la neurona causa del peso de esta red para obtener el patrn establecido. Por favor introduzca nuevos pesos al azar antes de intentar la formacin de nuevo. Usted puede hacer esto ya sea marcando la casilla Establecer pesos al azar o por seleccin de la correspondiente punto del men de la barra de men principal (NETSET RANDOM WEIGHTS). Hacer la formacin de nuevo. Repita hasta que la desviacin sea inferior a 0,25. Despus de alcanzar este objetivo la red es lo suficientemente capacitado, a fin de predecir el resultado correcto de las posibles entradas (0,0), (0,1), (1,0) y (1,1). Puede guardar la red en el disco ahora.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

11 of 35 pages

Cabe mencionar que los criterios que XOR.PAT sera adquirida por una gran red neuronal en el primer juicio. Sin embargo, hay varias razones para preferir una red lo ms pequeo posible. Una de las razones es el rendimiento de la red. Un reducido se calcula uno ms rpido. La segunda, an ms importante razn, es la capacidad de las redes de la abstraccin. Una gran red podra ser capaz de aprender las pautas de formacin, pero puede fallar en algo diferente de datos durante la fase de reproduccin.

Example 2 Stock Market Prediction

Ejemplo 2 - Prediccin del Mercado de Valores It must be stated clearly: No Neural Network can ever predict a market crash or any point gain or loss due to actual economical or political events you dont know in advance. But a Neural Network might be able discover relations between stock market data that are not obvious. Una buena aplicacin para la red neuronal es la prediccin del mercado de valores. Mantenemos nuestro siguiente ejemplo muy simple y se limita a mostrar la forma de preparar el mercado de valores para la entrada de datos en una red neuronal. Resultados deben estar siempre sujeto a la interpretacin y siempre es el riesgo para los inversores apostar dinero real en ella. Debe sealarse claramente: Una red neuronal no puede predecir la cada de un mercado o cualquier punto de ganancia o la prdida real debido a los acontecimientos polticos o econmicos que no saben de antemano. Sin embargo, una red neuronal podra descubrir las relaciones entre los datos del mercado de valores que no son evidentes. Nuestro ejemplo examinar el Dow Jones Industrial ndice en 1999.

Date Open High Low Close Volume 1/4/99 9184.01 9350.33 9122.47 9184.27 883 1/5/99 9184.78 9338.74 9182.98 9311.19 779 1/6/99 9315.42 9562.22 9315.42 9544.97 986 1/7/99 9542.14 9542.14 9426.02 9537.76 857 1/8/99 9538.28 9647.96 9525.41 9643.32 940 1/11/99 9643.32 9643.32 9532.61 9619.89 816

: : Picture 12 DJI open, high, low, close and volume in 1999 La figura 12 muestra algunos datos DJI para 1999. Nuestra idea es que puede haber una relacin entre el da de hoy abiertos, altos, bajos, estrechos y el volumen y la maana del cierre. Utilizando los valores directamente como entrada a nuestra red no es aconsejable porque

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

12 of 35 pages

nuestro inters no es ms que el porcentaje de descenso y ganar. Por lo tanto, obtener nuestros datos de entrada de los datos originales de la siguiente manera:
Open(t) High(t) Low(t) Close(t) Volume(t) = % change day t-1 day t = % rel. to Open = % rel. to Open = % change day t-1 day t = % change day t-1 day t = (Open(t)-Open(t-1))/Open(t-1)*100 = (High(t)-Open(t))/Open(t)*100 = (Low(t)-Open(t))/Open(t)*100 = (Close(t)-Close(t-1))/Close(t-1)*100 = (Volume(t)-Volume(t-1))/Volume(t-1)*100

La salida se define como

Close(t+1) = % change day t day t+1 = (Close(t+1)-Close(t))/Close(t)*100

Las frmulas se pueden introducir fcilmente en una tabla de hoja de clculo y se ver como se muestra en la imagen 13:

Date

Open

High

Low

Close

Volume

Open

High

Low

Close

Volume Close+1 0.030932 1.381928 2.510742 -0.07554 1.106759 -0.24297 -1.50948 -1.32057 -2.44536 2.407868 0.157057

12/31/98 9274.12 9287.77 9181.43 9181.43 1/4/99 9184.01 9350.33 9122.47 9184.27 1/5/99 9184.78 9338.74 9182.98 9311.19 1/6/99 9315.42 9562.22 9315.42 9544.97 1/7/99 9542.14 9542.14 9426.02 9537.76 1/8/99 9538.28 9647.96 9525.41 9643.32 1/11/99 9643.32 9643.32 9532.61 9619.89 1/12/99 9618.86 9620.15 9451.77 9474.68 1/13/99 9471.34 9471.34 9213.1 9349.56 1/14/99 9349.56 9359.08 9087.72 9120.93 1/15/99 9127.16 9342.61 9127.16 9340.55

755 -0.50274 883 -0.97163 779 0.008384 986 1.422353 857 2.433814 940 -0.04045 816 1.101247 794 -0.25365 935 -1.53365 798 -1.28577 921 -2.37872

0.147184 -0.99945 1.676251 2.649371

-1.005 27.10438 -11.778

1.810974 -0.67008 0.030932 16.95364 -0.0196 1.381928 0 2.510742 26.57253

0 -1.21692 -0.07554 -13.0832 1.149893 -0.13493 1.106759 9.684947 0 -1.14805 -0.24297 -13.1915 0.013411 -1.73711 -1.50948 -2.69608 0 -2.72654 -1.32057 17.75819 0.101823 -2.80056 -2.44536 -14.6524 2.360537 0 2.407868 15.41353

Picture 13 DJI data modified for input into a Neural Network

Ahora puede seleccionar y copiar el derecho 6 columnas y las pega en Pythia. Tambin puede simplemente cargar el archivo de ejemplo que viene con DOW99I.PAT Pythia. Nuestro enfoque ser el de formar a una red con los datos de la 1 mitad del ao 1999. Con la red de formacin a continuacin vamos a examinar la salida para la reproduccin de la 2 mitad de 1999. Nuestra red neuronal espera de entrada y salida de cualquier valor para estar entre 0 y 1. Por lo tanto, el patrn de juegos debe ser normalizado antes de ser transformados por la red. La normalizacin se calcula: N(i) = (i - low) / (high - low)
i is the input (or output value)

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

13 of 35 pages

low is the minimum possible value high is the maximum possible value

Esto normalmente se hace automticamente, pero debemos tomar el tiempo para examinar los parmetros de normalizacin ahora. Seleccione PATTERNOPTIONS. Para nuestro ejemplo actual que vea las opciones como se muestra en el Cuadro 14. INPUT1 por ejemplo, cuenta con valores posibles entre -2,378722 y 2,846741. Supongamos que usted tiene que normalizar el valor I1 = 0.5. Que calcular:
N(0.5) = (0.5 (-2.378722)) / (2.846741- (2.378722)) = 0.55090276

Picture 14 Options for pattern sets Once you create a new pattern set Pythia calculates the lowest and the highest value for each column. You can change these values in the option menu if you like. If you train a network with a certain pattern set you must choose the same normalization parameters for a pattern set you are going to reproduce with this network.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

14 of 35 pages

The display format parameters only specifies how the number are displayed in the table. Una vez que haya creado un nuevo modelo establecido Pythia calcula el ms bajo y el ms alto valor para cada columna. Puede cambiar estos valores en el men de opciones si lo desea. Si un tren de la red con un determinado patrn de conjunto que debe elegir los mismos parmetros de la normalizacin para establecer un patrn que se va a reproducir con esta red. El formato de visualizacin de parmetros slo especifica la forma en que el nmero se muestran en la tabla.

Now lets design an appropriate network. Create a new network with the values (5,5,7,1). This network has the required 5 inputs and 1 output. Select NET LEARN PATTERN SET. Leave the default values except the repetition count, which we set to 3000. Press the Ok button Ahora el diseo de una red adecuada. Crear una nueva red con los valores (5,5,7,1). Esta red tiene el 5 entradas y 1 salida. Seleccionar NET APRENDER PLAN CONJUNTO. Deje los valores predeterminados, salvo la repeticin contar, que nos fijamos para 3000. Presione el botn OK

The training pattern set will now be processed 3000 times. Finally the trained network automatically reproduces the inputs of the pattern set. The results can be examined in Picture 15. Have a look at the columns O1 and NET(O1). As you remember, NET(O1) is the output generated by the Neural Network, O1 is the output we would like to get. The values describe the percentage drop or gain between two days close values. Our network at least has learned some of the patterns. Especially if our network makes a significant statement (lets say it predicts a gain or drop of 0.6% or more) it is usually right with the direction the DJI moves. El modelo de formacin establecido ser procesada 3000 veces. Por ltimo, la red de formacin reproduce automticamente los insumos del modelo establecido.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

15 of 35 pages

Los resultados se pueden examinar en el Cuadro 15. Echa un vistazo a las columnas O1 y NET (O1). Como recordarn, NET (O1) es la salida generada por la red neuronal, O1 es la salida que nos gustara conseguir. Los valores que describen el porcentaje de disminucin o aumento de entre dos das de cierre de los valores. Nuestra red ha aprendido, al menos, algunos de los patrones. Especialmente si nuestra red de manera significativa declaracin (digamos que predice un aumento o descenso del

Picture 15 The network after the training 0,6% o ms) por lo general es la derecha con la direccin se mueve el DJI. Now lets test the same network with the 2nd half year data. The network has not been trained with these. So they are completely unknown to the network. Please load dow99ii.pat into Pythia. Note that the normalization options are the same for dow99i.pat and dow99ii.pat.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

16 of 35 pages

Reproduce this pattern set.

You see that this time the outcome is very poor.

Ahora vamos a probar la misma red con el 2 semestre de datos. La red no se ha entrenado con estos. Por lo que son totalmente desconocidos para la red. Por favor, carga dow99ii.pat en Pythia. Tenga en cuenta que la normalizacin de las opciones son las mismas para dow99i.pat y dow99ii.pat. reproducir este patrn establecido. Usted ve que esta vez el resultado es muy pobre.

Picture 16 Reproducing the 2nd half 1999 data generates poor results

Next step would be to modify the model, usually by adding input parameters like interest rates, overseas market indices, currency ratio etc.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

17 of 35 pages

El siguiente paso sera la de modificar el modelo, por lo general mediante la adicin de parmetros de entrada como los tipos de inters, ndices de los mercados de ultramar, etc ratio de moneda

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

18 of 35 pages

Evolutionary Optimizer

Picture 17 The Evolutionary Optimizer The Evolutionary Optimizer is a tool for generating Neural Networks. The only thing you must provide is the training pattern set. Lets see how the Evolutionary Optimizer generates a network for the XOR sample: El Optimizador evolutiva es una herramienta para la generacin de Redes Neuronales. Lo nico que usted debe proporcionar es el patrn de formacin establecidos. Vamos a ver cmo el Evolutiva Optimizador genera una red para la XOR muestra: Load the pattern set XOR.PAT into Pythia. Select NET EVOLUTIONARY OPTIMIZER Cargar el modelo establecido en XOR.PAT Pythia. Seleccione NET EVOLUTIONARY Optimizador You will see a dialog window as shown in Picture 17. Uncheck the deviation field Change the *deviation field to 0.25 Change the # neurons field to 3

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

19 of 35 pages

Press the Ok button

The Evolutionary Optimization will start processing now. It will stop if the Goals to achieve are true, meaning Ver una ventana de dilogo como se muestra en la Figura 17. Desmarca la desviacin campo Cambie el campo * desviacin a 0,25 Cambie el campo a las neuronas # 3 Presione el botn OK Optimizacin evolutiva de la transformacin se iniciar ahora. Se detendr si los objetivos a alcanzar son verdaderas, es decir, *deviation < 0.25 and # neurons <= 3 However, if will perform max. 1000 evolution steps. You can find further explanations of the options in the reference part of this manual.

Sin embargo, si se realizan mx. 1000 evolucin pasos. Puede encontrar ms explicaciones de las opciones en la parte de referencia de este manual.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

20 of 35 pages

Picture 18 The Evolutionary Optimizer generated a network After a couple of generations the optimizer finally found a suitable network in line 11. This network has 3 neurons and the max. deviation for any pattern is 0.078666. Click the checkbox on the left of this line and press the Ok button. You now have a network that is sufficiently trained for the XOR problem. Despus de un par de generaciones el optimizador encontrado una red adecuada en la lnea 11. Esta red tiene 3 neuronas y el mx. cualquier desviacin de patrn es 0,078666. Haga clic en la casilla a la izquierda de esta lnea y pulse el botn OK. Ahora tiene una red que tiene la suficiente formacin para el problema XOR.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

21 of 35 pages

Reference Referencias
The reference part of this manual will describe in detail the menu items and the settings in the various dialog boxes. La referencia de este manual se describen en detalle los elementos del men y los ajustes en los diversos cuadros de dilogo. The Main Window Pythias main windows contains three areas: The pattern set area on the left side The network area on the right side A log window on the bottom

The pattern and the network area can contain any number of pattern sets or networks. Any operation a user selects will process the currently selected pattern set or network. The log windows displays the last transactions. LA VENTANA PRINCIPAL Pythia principales ventanas contiene tres reas: El patrn establecido en la zona izquierda El rea de red en el lado derecho Una ventana de registro en la parte inferior El patrn y el rea de red puede contener cualquier nmero de conjuntos de patrones o redes. Cualquier operacin de un usuario selecciona el proceso que actualmente seleccionado patrn establecido o de la red. El registro muestra ventanas de la ltima transaccin. FILE EXIT This terminates Pythia. PATTERN READ PATTERN SET This menu item reads a previously saved pattern set into pythia. Pattern set files have the extension .PAT

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

22 of 35 pages

PATTERN IMPORT PATTERN SET This menu item imports a data file as a pattern set into pythia. The data files format is assumed to be ASCII. After choosing a file name a dialog box as shown in Picture 19 will pop up. A preview window displays the first lines of the file. You can further specify a field delimiter, the number of fields per record and the number of outputs.

Picture 19 Import pattern file dialog

Pythia tries to determine the field delimiter and the field per record value automatically. Usually you wont have to change these. The number of outputs field default to 1. You must change this if your data contain more than one output. Finally press the Ok button. Your new pattern set will be display on the left side of Pythias main windows. Note: During import some values will set automatically. These settings are the Normalization parameters and the Display format. Normalization: The High of each input or output is set to the maximum value of the referred column. The Low of each input or output is set to the minimum value of the referred column. Display format: The display for each column is set to (9,6), meaning the values will be displayed by a 9 character string with 6 characters after the decimal point. PATTERN SAVE PATTERN SET This menu item allows you to save the currently selected pattern set. Any special settings like Normalization and Display format parameters are stored along with the pattern set.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

23 of 35 pages

PATTERN CLOSE PATTERN SET This menu item closes the currently selected pattern set. The user will be prompted to save this pattern set if it was modified.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

24 of 35 pages

PATTERN OPTIONS This menu item allows you to change certain settings for the pattern set. Normalization: A Neural Network expects any input and output value to be between 0 and 1. Therefore the pattern sets must be normalized before being processed by the network. The normalization is calculated: N(i) = (i - low) / (high - low)
i is the input (or output value) low is the minimum possible value high is the maximum possible value

The dialog box shown in Picture 20 allows you change the normalization parameters.

Picture 20 Options for pattern sets

The initial parameters are calculated when the pattern set was first created, either by importing or by pasting. They are set to the lowest and highest value found in a column. The low..high range should span any possible value. However, if your data set contain any runaways you should either exclude these from you data or ignore them by manually adjusting the normalization parameters. You can change values in the option window by simply entering the new value into the cell. Alternatively, you can make use of the following operations: Push this button to specify a new low value for all columns. Push this button to specify a new high value for all columns. Restore the default values (new calculation as if the pattern set was just created). Discard changes. Note: If you train a network with a certain pattern set you must choose the same normalization parameters for another pattern set you are going to reproduce with this network. The display format parameters only specifies how the number are displayed in the table.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

25 of 35 pages

Display format: The display format parameters specify how the numbers are displayed in the table. By default the display for each column is set to (9,6), meaning the values will be displayed by a 9 character string with 6 characters after the decimal point. Use the following operations to change the look of the numbers: Push this button to specify a new width for all columns. Push this button to specify a new decimal value for all columns. Restore the default values (9,6). Discard changes. PATTERN TOGGLE VIEW Select this command to toggle between the original and the normalized form of the pattern set. You recognize the currently select view on the tab of the pattern set. The notation N(xxx.pat) means normalized, xxx.pat indicates native (original).

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

26 of 35 pages

NET CREATE NET This menu item is used to create a new Neural Network. Enter the new networks desired topology into the dialog box display in Picture 21. Begin with the number of inputs the new network is supposed to have. Continue with the levels (usually 2 or 3 levels). Keep in mind that the last described level is the output level, meaning the number of neurons in the last level must match the number of neurons in the output level. Pressing the Sample button automatically creates a sample topology that is able to process the currently selected pattern set (if there is one).

Picture 21 Create a new Neural Network

Finally press the Ok button. The new created Neural Net is now displayed in the network area of Pythias main window. Picture 22 shows how this will look like.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

27 of 35 pages

Picture 22 A new created Neural Network

NET READ NET This menu item reads a previously saved network into pythia. Network files have the extension .NN NET CLOSE NET This menu item closes the currently selected network. The user will be prompted to save this network if it was modified. NET SAVE NET This menu item allows you to save the currently selected network. The network topology and weights are stored to a file with the extension .NN.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

28 of 35 pages

NET EVOLUTIONARY OPTIMIZER The Evolutionary Optimizer is a tool for generating Neural Networks. The only thing you must provide is the training pattern set. Selecting this menu item pops up a dialog box as displayed in Picture 23. How the Evolutionary Optimizer works (the following brief description assumes parameters to be set as shown in Picture 23) The Evolutionary Optimizer initially creates a generation containing 50 randomly created networks. Each network within this generation will be trained shortly and its fitness determined according to the parameters in Goals to achieve. Then a new generation of networks will be created from the old one according to the following procedure: Two parent networks will be chosen out of the old generation. The selection algorithm will choose networks with a high fitness by a higher probability. Two children networks will be created from the two parent networks. With the probability of 0.2 the two children networks will be crossed over. This means they will swap level with each other. Example: Child 1: (2,2,1) (2,2 1) Child 2: (2,6,6,1) (2,6 6,1) st Compose the 1 part of child 1 with the 2nd part of child 2 to (2,2,6,1) Compose the 1st part of child 2 with the 2nd part of child 1 to (2,6,1) The children have been crossed over to (2,2,6,1) and (2,6,1). The children now will be mutated with a probability of 0.04. Mutation means insertion or deletion of a level, insertion or deletion of a neuron into a level or change of weights. The two parent network will now be checked if they belong to the 10 fittest of the old generation. If they do they will be mutated and rolled over into the new generation.

The selection continues until the new generation has 50 members too. After completion the new generation will be evaluated. The Evolutionary Optimizer is considered ready as soon as it found a network with a fitness of 100. Otherwise it continues for 1000 generations, what might take hours or days to compute. You always can stop the process if you do not see any progress.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

29 of 35 pages

Options There are lots of options you can finetune the optimizers work with. 1st ancestor Neural Net: Specify a template network the first generation is created from or allow to generate a new random one with create new. Pattern set to learn: Specify the pattern set you want to generate a network for. This pattern set must be loaded previously. Check the option Mix pattern randomly if you want to overcome a given but unwanted sort order in the pattern set. Note that a pattern sets sort order influences the training phase of the network. Goals to achieve: Here you can specify what the network should be optimized for. There are three goals possible: Optimize for medium deviation ( deviation) Optimize for max. deviation within the pattern set (*deviation) Optimize for size (# neurons)

Picture 23 The Evolutionary Optimizer

Tag or untag the checkboxes on the left side to determine if the certain goal shall play a role at all. If a goal is tagged specify the value you want to push the network below. Specify the contribution a goal will make to the overall fitness. The example in Picture 23 does not care about the medium deviation, but wants the max. deviation to be below 0.25 and the network size below or equal 3. Both checked goals will contribute 50% to the overall fitness of an evolutionary created network (1:1 = 50:50). If you would want the network size to have an importance of only 20% you

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

30 of 35 pages

would have to enter the value 4 and 1 into the contribution fields for max. deviation and size. Evolutionary algorithms settings: Different options influence the optimization process Population size: number of networks created per generation. Evolution steps: max. number of generations before stop. Mutation rate: probability for a network to be modified during rollover to a new generation. Cross over rate: probability for a child network to be crossed over with another child network # fittest/Generation: number of networks that are rolled over to a new generation as they are. Any other networks of one generation are discarded. However, discarded network still might become parent network. Modify fittest: if checked, the rolled over networks are modified with the given mutation rate.

Pressing the Ok button starts the evolutionary process. You can watch the progress in the progress window shown in Picture 24 During optimization you always can stop the process by pressing the Stop button. The colored circle right of the checkbox show you the quality of a network. Red means no goal achieved at all, yellow means some goals achieved and green means all goals achieved. If there is any green network the optimization terminates. The column Topology describes the levels of the network. The column Neurons describes the number of neurons the network owns. The column dev shows the networks medium square deviation on pattern set reproduction. The column *dev shows the networks max square deviation on pattern set reproduction. The column Fitness describes the networks fitness with a number between 0 and 100.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

31 of 35 pages

Picture 24 The Evolutionary Optimizer generated a network When the optimization stops (by either finding a network with a fitness of 100 or stopping manually) you can choose one or more network to be move into the network area of Pythias main window. The moved networks are ready for use. NET REPRO PATTERN SET This menu item causes the currently selected network to reproduce the currently selected pattern set. The net output columns and the deviation column will be refreshed. Note: You can only choose this command if the currently selected network is compatible to the currently selected pattern set. Compatibility means same number of inputs and same number of outputs.

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

32 of 35 pages

NET LEARN PATTERN SET Choosing this menu item pops up a dialog box (Picture 25) that allows you to specify a learn plan for the training of the currently selected network with the currently selected pattern set. Options Neuronal Network to train: Specify the network you want to train. This network must be compatible with the pattern set and must have been loaded into the network area of the main window. Pattern set to learn: Specify the pattern set you want to train the network with. This pattern set must be loaded previously. Check the option Mix pattern randomly if you want to overcome a given but unwanted sort order in the pattern set. Note that a pattern sets sort order influences the training of the network. Set weights randomly: Check this box to set the networks weights to initial random values. This might be necessary because sometimes the given weight values do not lead to a suitable network even if this is possible. Train until: Declare the cancel criteria of the learn plan here. You can connect the criteria with each other by the AND or the OR operator. The criteria for exiting the learn plan are: Repetition Medium deviation ( deviation) Max. deviation within the pattern set (*deviation) Time passed

Picture 25 Set up a learn plan

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

33 of 35 pages

Tag or untag the checkboxes on the left side to determine if the certain criterion shall play a role at all. In the example in Picture 25 the learn plan will stop as soon as either the repetition count reaches 1000 or the max deviation is below 0.25 or 300 seconds have passed. Learn rate: The learn rate specifies how fast an error is propagated backwards. A large value accelerates the learning, but might cause the network to overshoot the mark. Choose a value between 0.1 and 0.5. Tag Automatically adjust if you want Pythia to adjust the learn rate during the training automatically. (Note: this feature is not implemented yet). Finally: Tag Reproduce pattern set if you want to get the pattern set reproduced after the training. This is equivalent with the command NET REPRO PATTERN SET. Check show results in native form to get the pattern set shown in their original form instead of the normalized form. This is equivalent with the command PATTERN TOGGLE VIEW. NET REPRODUCE SINGLE This menu item causes the currently selected network to reproduce a single pattern. Manually enter the inputs, delimited by a , . The number of data you need to enter equals the number of inputs of the network. Furthermore, you need to check if the data entered will be interpreted normalized or original. Picture 26 Reproduce a single pattern

Picture 27 Result of the single pattern reproduction

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

34 of 35 pages

The result of this operation will be displayed in Pythias log window on the bottom of the main window.

NET LEARN SINGLE This menu item causes the currently selected network to perform one learn step for a single pattern. Manually enter the inputs and outputs, delimited by a , . The number of data you need to enter equals the number of inputs plus the number of outputs of the network.

Picture 28 Perform one learn step for a single pattern

Furthermore, you need to check if the data entered will be interpreted normalized or original. The result of this operation will be displayed in Pythias log window on the bottom of the main window. NET SET RANDOM WEIGHTS Choose this menu item to reset a networks weights to random values. This might be necessary because sometimes the given weight values do not lead to a suitable network even if this is possible. Picture 30 Set new random weights You need to enter the low..high range for the new random weights. These are usually between 1 and 1. Picture 29 Result of a single learn step

Pythia The Neural Network Designer 2000 by Runtime Software

- 11/24/13 - 12:21 PM

35 of 35 pages

NET SET LEARN RATE Choose this menu item to change a networks learn rate. The learn rate specifies how fast an error is propagated backwards. A large value accelerates the learning, but might cause the network to overshoot the mark. Choose a value between 0.1 and 0.5. EDIT COPY CELLS This command copies a pattern sets selected cells into the windows clipboard. From there you can paste the data into other applications, e.g. text editors or MS Excel. EDIT PASTE CELLS This command pastes cells from the Windows clipboard into Pythia. You will get a dialog box as shown in Picture 31. Field delimiter: This specifies the delimiter between single fields. Fields per record: Specifies the number of fields in each record. Paste to new Pattern Set: Select this options if you intend to Picture 31 Paste dialog paste the clipboard into a whole new pattern set. You must specify the number of inputs too. Paste to selected Pattern Set: Select this option if you want to paste the clipboard into an existing pattern set. You can either paste at the current position or append. Finally press the Ok button.