Beruflich Dokumente
Kultur Dokumente
Environmental Sciences
Exploration of Artificial Intelligence Techniques
Valliappa.Lakshmanan@noaa.gov
lakshman@ou.edu 1
Data Driven Methods
Common AI techniques
lakshman@ou.edu 2
What is AI?
lakshman@ou.edu 3
AI vs. humans
AI applications built on Aristotlean logic
Induction, semantic queries, system of logic
Human reasoning involves more than just induction
lakshman@ou.edu 4
Successful AI applications
lakshman@ou.edu 5
Data Driven Methods
Common AI techniques
lakshman@ou.edu 6
Fuzzy logic
lakshman@ou.edu 7
Fuzzy logic example
Source: Matlab fuzzy logic toolbox tutorial
http://www.mathworks.com/access/helpdesk/help/toolbox/fuzzy/fp350.html
lakshman@ou.edu 8
Advantages of fuzzy logic
Simple to create
Not much need for data or ground truth
Logic tends to be easy to program
lakshman@ou.edu 10
Neural Networks
lakshman@ou.edu 11
A example neural network
Diagram from:
http://www.codeproject.com/useritems/GA_ANN_XOR.asp
lakshman@ou.edu 12
Advantages of neural networks
Can approximate any smooth function
The three-layer neural network
Can yield true probabilities
If output node is a sigmoid node
Not hard to train
Training process is well understood
Fast in operations
Training is slow, but once trained, the network can
calculate the output for a set of inputs quite fast
Easy to implement
Just a sum of exponential functions
lakshman@ou.edu 13
Disadvantages of neural networks
A black box
The final set of weights yields no insights
Magnitude of weights doesnt mean much
lakshman@ou.edu 14
Recap:
Fuzzy logic
Humans provide the rules
Not optimal
Neural network
Humans can not understand system
Optimal
Middle ground?
Genetic Algorithms
Decision Trees
lakshman@ou.edu 15
Genetic algorithms
In genetic algorithms
One fixes the model (rule base, equations,
class of functions, etc.)
Optimize the parameters to model on
training data set
Use optimal set of parameters for unknown
cases
lakshman@ou.edu 16
An example genetic algorithm
Sources:
http://tx.technion.ac.il/~edassau/web/genetic_algorithms.htm
http://cswww.essex.ac.uk/research/NEC/
lakshman@ou.edu 17
Advantages of genetic algorithms
lakshman@ou.edu 18
Disadvantages of genetic algorithms
lakshman@ou.edu 19
Decision trees
Can automatically build decision trees from
known data Root
Prune trees
30 50
Select thresholds
Choose operators
Disadvantages
Piece-wise linear, so typically less T < 10C T > 10C
skilled than neural networks 20 15 10 35
Large decision trees are effectively a
blackbox
Can not do regression, only
classification Z > 45 Z < 45 V<5 V>5
Advantages: 18 2 2 13 82 2 33
Fast to train
New advances: bagged, boosted
decision trees approach skill of neural
networks, but are no longer fast to train
lakshman@ou.edu 20
Radial Basis Functions
Diagram from: A. W. Jayawardena & D. Achela K. Fernando 1998: Use of Radial Basis Function Type Artificial
Neural Networks for Runoff Simulation, Computer-Aided Civil and Infrastructure Engineering 13:2
lakshman@ou.edu 21
Data Driven Methods
Common AI techniques
lakshman@ou.edu 22
Typical data-driven application
Features
AI application
in run-time f(features)
Result
lakshman@ou.edu 23
What is the role of the data?
Validation
Test known model
Technique:
Difference between model output and ground truth helps to validate the
model
Calibration
Find parameters to model with desired structure
Technique:
Tuned fuzzy logic method
Genetic algorithms
Induction
Find model and parameters from just data
Technique:
Neural network methods, bagged/boosted decision trees, support vector
machines, etc.
lakshman@ou.edu 24
What is the problem to solve?
Do you have a bunch of data and want to:
Estimate an unknown parameter from it?
True rainfall based on radar observations?
Amount of liquid content from in-situ measurements of
temperature, pressure, etc?
Regression
Classify what the data correspond to?
A water surge?
A temperature inversion?
A boundary?
Classification
Regression and classification arent that different
Classification: estimate probability of an event
A function from 0-1
lakshman@ou.edu 25
Which AI technique?
Do you have expert knowledge?
Humans have a model in their head? Should the final f() be
understandable?
Create fuzzy logic rules from experts reasoning
Aggregate the individual fuzzy logic rules
Can tune the fuzzy rules based on data
Using regression, decision trees or neural networks for RMS error criterion
Genetic algorithms for error criteria like ROC, economic cost, etc.
Many times the original rules are just fine
Do you already know the model?
A power-law relationship? Gaussian? Quadratic? Rules?
Just need to find parameters to this model?
If linear, just use linear regression
If non-linear: use genetic algorithms
Use continuous GAs
Both of these can be used for regression (therefore, also
classification)
lakshman@ou.edu 26
Which AI technique (contd.)
Do you know nothing about the data?
Not the suspected equation/model (GA)?
Not the suspected rules (fuzzy logic)?
Use a AI technique that supplies its equations/rules
black box.
For classification, use:
Bagged decision trees or Support Vector Machines
If output is probabilistic, remember to apply Platt scaling
Summary statistics on bagged DTs can help answer why
Neural Networks
For regression, use:
Neural networks
lakshman@ou.edu 27
Where do your data come from?
Observed data
Compute features
Choose AI technique
The 4 choices in the previous two slides
Simulated data:
Example: trying to replicate a very complex model
Throw randomly-generated data at model
Compute features
Choose AI technique:
GA for parametric approximations
NN when you dont know how to approximate
lakshman@ou.edu 28
Where do you get your inputs?
What type of data do you have?
Individual observations?
Sample them (choose at random) and use directly
Sparse observations in a time series?
Generate time-based features (1D moving windows)
Signal processing features from time series
Data from remotely sensed 2D grids?
Generate image-based features using convolution filters
Do you need:
Pixel-based regression/classification?
Use convolution features directly
Object-based regression/classification?
Identify regions using region growing
Use region-aggregate features
lakshman@ou.edu 29
Typical data-driven application
Observed data
Signal/image processing;sampling
Features
normalize/create chromosome/
determine confidences
FzLogic/GenAlg/NN/DecTree
f()
Platt method/region-average/threshold
A data-driven application
in run-time
Result
lakshman@ou.edu 30
Data Driven Methods
Common AI techniques
lakshman@ou.edu 31
Preprocessing
Often can not use pixel data directly
Too much data, too highly correlated
May need to segment pixels into objects and use features
computed on the objects
Different data sets may not be collocated
Need to interpolate to line them up
Mapping, objective analysis
Noise in data may need to be reduced
Smoothing
Present statistic of data, rather than data itself
Features need to be extracted from data
Human experts often good source of ideas on signatures to
extract from data
lakshman@ou.edu 32
Postprocessing
Theoutput of an expert system may be grid
point by grid point
May need to provide output on objects
Storms, forests, etc.
Can average outputs over objects pixels
May need probabilistic output
Scale output of maximum marginal techniques
Use a sigmoid function
Called Platt scaling
lakshman@ou.edu 33
Summary
What is Artificial Intelligence?
Data-driven methods to perform specific targeted tasks
Common AI techniques
Fuzzy logic, neural networks, genetic algorithms, decision
trees
Choosing between AI techniques
Understand the role of your data
Do experts understand the system? (have a model)
Do experts expect to understand the system? (readability)
Pre and post processing
Image processing techniques on spatial grids
lakshman@ou.edu 34