12/9/2016 ArtificialIntelligenceNeuralNetworks

ArtificialIntelligenceNeuralNetworks
https://www.tutorialspoint.com/artificial_intelligence/artificial_intelligence_neural_networks.htm
Copyrighttutorialspoint.com

YetanotherresearchareainAI,neuralnetworks,isinspiredfromthenaturalneuralnetworkofhumannervous
system.

WhatareArtificialNeuralNetworksAN N s?
Theinventorofthefirstneurocomputer,Dr.RobertHechtNielsen,definesaneuralnetworkas

"...acomputingsystemmadeupofanumberofsimple,highlyinterconnectedprocessingelements,
whichprocessinformationbytheirdynamicstateresponsetoexternalinputs.

BasicStructureofANNs
TheideaofANNsisbasedonthebeliefthatworkingofhumanbrainbymakingtherightconnections,canbe
imitatedusingsiliconandwiresaslivingneuronsanddendrites.

Thehumanbrainiscomposedof100billionnervecellscalledneurons.Theyareconnectedtootherthousand
cellsbyAxons.Stimulifromexternalenvironmentorinputsfromsensoryorgansareacceptedbydendrites.These
inputscreateelectricimpulses,whichquicklytravelthroughtheneuralnetwork.Aneuroncanthensendthe
messagetootherneurontohandletheissueordoesnotsenditforward.

https://www.tutorialspoint.com/cgibin/printpage.cgi 1/8
12/9/2016 ArtificialIntelligenceNeuralNetworks

ANNsarecomposedofmultiplenodes,whichimitatebiologicalneuronsofhumanbrain.Theneuronsare
connectedbylinksandtheyinteractwitheachother.Thenodescantakeinputdataandperformsimpleoperations
onthedata.Theresultoftheseoperationsispassedtootherneurons.Theoutputateachnodeiscalledits
activationornodevalue.

Eachlinkisassociatedwithweight.ANNsarecapableoflearning,whichtakesplacebyalteringweightvalues.
ThefollowingillustrationshowsasimpleANN

TypesofArtificialNeuralNetworks
TherearetwoArtificialNeuralNetworktopologiesFreeForwardandFeedback.

FeedForwardANN
Theinformationflowisunidirectional.Aunitsendsinformationtootherunitfromwhichitdoesnotreceiveany
information.Therearenofeedbackloops.Theyareusedinpatterngeneration/recognition/classification.They
havefixedinputsandoutputs.

https://www.tutorialspoint.com/cgibin/printpage.cgi 2/8
12/9/2016 ArtificialIntelligenceNeuralNetworks

FeedBackANN
Here,feedbackloopsareallowed.Theyareusedincontentaddressablememories.

https://www.tutorialspoint.com/cgibin/printpage.cgi 3/8
12/9/2016 ArtificialIntelligenceNeuralNetworks

WorkingofANNs
Inthetopologydiagramsshown,eacharrowrepresentsaconnectionbetweentwoneuronsandindicatesthe
pathwayfortheflowofinformation.Eachconnectionhasaweight,anintegernumberthatcontrolsthesignal
betweenthetwoneurons.

Ifthenetworkgeneratesagoodordesiredoutput,thereisnoneedtoadjusttheweights.However,ifthe
networkgeneratesapoororundesiredoutputoranerror,thenthesystemalterstheweightsinordertoimprove
subsequentresults.

MachineLearninginANNs
ANNsarecapableoflearningandtheyneedtobetrained.Thereareseverallearningstrategies

SupervisedLearningItinvolvesateacherthatisscholarthantheANNitself.Forexample,theteacher
feedssomeexampledataaboutwhichtheteacheralreadyknowstheanswers.

Forexample,patternrecognizing.TheANNcomesupwithguesseswhilerecognizing.Thentheteacher
providestheANNwiththeanswers.Thenetworkthencomparesitguesseswiththeteacherscorrect
answersandmakesadjustmentsaccordingtoerrors.

UnsupervisedLearningItisrequiredwhenthereisnoexampledatasetwithknownanswers.For
example,searchingforahiddenpattern.Inthiscase,clusteringi.e.dividingasetofelementsintogroups
accordingtosomeunknownpatterniscarriedoutbasedontheexistingdatasetspresent.

ReinforcementLearningThisstrategybuiltonobservation.TheANNmakesadecisionbyobservingits
environment.Iftheobservationisnegative,thenetworkadjustsitsweightstobeabletomakeadifferent
requireddecisionthenexttime.

BackPropagationAlgorithm

Itisthetrainingorlearningalgorithm.Itlearnsbyexample.Ifyousubmittothealgorithmtheexampleofwhat
youwantthenetworktodo,itchangesthenetworksweightssothatitcanproducedesiredoutputforaparticular
inputonfinishingthetraining.

BackPropagationnetworksareidealforsimplePatternRecognitionandMappingTasks.

BayesianNetworksBN
Thesearethegraphicalstructuresusedtorepresenttheprobabilisticrelationshipamongasetofrandomvariables.
BayesiannetworksarealsocalledBeliefNetworksorBayesNets.BNsreasonaboutuncertaindomain.

Inthesenetworks,eachnoderepresentsarandomvariablewithspecificpropositions.Forexample,inamedical
diagnosisdomain,thenodeCancerrepresentsthepropositionthatapatienthascancer.

Theedgesconnectingthenodesrepresentprobabilisticdependenciesamongthoserandomvariables.Ifoutoftwo
nodes,oneisaffectingtheotherthentheymustbedirectlyconnectedinthedirectionsoftheeffect.Thestrength
oftherelationshipbetweenvariablesisquantifiedbytheprobabilityassociatedwitheachnode.

ThereisanonlyconstraintonthearcsinaBNthatyoucannotreturntoanodesimplybyfollowingdirectedarcs.
HencetheBNsarecalledDirectedAcyclicGraphsDAGs .

BNsarecapableofhandlingmultivaluedvariablessimultaneously.TheBNvariablesarecomposedoftwo
dimensions

Rangeofprepositions
Probabilityassignedtoeachoftheprepositions.

https://www.tutorialspoint.com/cgibin/printpage.cgi 4/8
12/9/2016 ArtificialIntelligenceNeuralNetworks

ConsiderafinitesetX={X1,X2,,Xn}ofdiscreterandomvariables,whereeachvariableXimaytakevalues
fromafiniteset,denotedbyValXi .IfthereisadirectedlinkfromvariableXitovariable,Xj,thenvariableXiwill
beaparentofvariableXjshowingdirectdependenciesbetweenthevariables.

ThestructureofBNisidealforcombiningpriorknowledgeandobserveddata.BNcanbeusedtolearnthecausal
relationshipsandunderstandvariousproblemdomainsandtopredictfutureevents,evenincaseofmissingdata.

BuildingaBayesianNetwork
AknowledgeengineercanbuildaBayesiannetwork.Thereareanumberofstepstheknowledgeengineerneeds
totakewhilebuildingit.

ExampleproblemLungcancer.Apatienthasbeensufferingfrombreathlessness.Hevisitsthedoctor,
suspectinghehaslungcancer.Thedoctorknowsthatbarringlungcancer,therearevariousotherpossiblediseases
thepatientmighthavesuchastuberculosisandbronchitis.

GatherRelevantInformationofProblem

Isthepatientasmoker?Ifyes,thenhighchancesofcancerandbronchitis.
Isthepatientexposedtoairpollution?Ifyes,whatsortofairpollution?
TakeanXRaypositiveXraywouldindicateeitherTBorlungcancer.

IdentifyInterestingVariables

Theknowledgeengineertriestoanswerthequestions

Whichnodestorepresent?
Whatvaluescantheytake?Inwhichstatecantheybe?

Fornowletusconsidernodes,withonlydiscretevalues.Thevariablemusttakeonexactlyoneofthesevaluesat
atime.

Commontypesofdiscretenodesare

BooleannodesTheyrepresentpropositions,takingbinaryvaluesTRUET andFALSEF .

OrderedvaluesAnodePollutionmightrepresentandtakevaluesfrom{low,medium,high}describing
degreeofapatientsexposuretopollution.

IntegralvaluesAnodecalledAgemightrepresentpatientsagewithpossiblevaluesfrom1to120.Even
atthisearlystage,modelingchoicesarebeingmade.

Possiblenodesandvaluesforthelungcancerexample

Node
Type Value NodesCreation
Name

{LOW,
Polution Binary HIGH,
MEDIUM}

{TRUE,
Smoker Boolean
FASLE}

Lung {TRUE,
Boolean
Cancer FASLE}

{Positive,
XRay Binary
Negative}

https://www.tutorialspoint.com/cgibin/printpage.cgi 5/8
12/9/2016 ArtificialIntelligenceNeuralNetworks

CreateArcsbetweenNodes

Topologyofthenetworkshouldcapturequalitativerelationshipsbetweenvariables.

Forexample,whatcausesapatienttohavelungcancer?Pollutionandsmoking.Thenaddarcsfromnode
PollutionandnodeSmokertonodeLungCancer.

Similarlyifpatienthaslungcancer,thenXrayresultwillbepositive.ThenaddarcsfromnodeLungCancerto
nodeXRay.

SpecifyTopology

Conventionally,BNsarelaidoutsothatthearcspointfromtoptobottom.ThesetofparentnodesofanodeXis
givenbyParentsX .

TheLungCancernodehastwoparentsreasonsorcauses :PollutionandSmoker,whilenodeSmokerisan
ancestorofnodeXRay.Similarly,XRayisachildconsequenceoref f ectsofnodeLungCancerandsuccessor
ofnodesSmokerandPollution.

ConditionalProbabilities

Nowquantifytherelationshipsbetweenconnectednodes:thisisdonebyspecifyingaconditionalprobability
distributionforeachnode.Asonlydiscretevariablesareconsideredhere,thistakestheformofaConditional
ProbabilityTableC P T .

First,foreachnodeweneedtolookatallthepossiblecombinationsofvaluesofthoseparentnodes.Eachsuch
combinationiscalledaninstantiationoftheparentset.Foreachdistinctinstantiationofparentnodevalues,we
needtospecifytheprobabilitythatthechildwilltake.

Forexample,theLungCancernodesparentsarePollutionandSmoking.Theytakethepossiblevalues={H , T ,
H , F ,L, T ,L, F }.TheCPTspecifiestheprobabilityofcancerforeachofthesecasesas<0.05,0.02,0.03,

0.001>respectively.

Eachnodewillhaveconditionalprobabilityassociatedasfollows

https://www.tutorialspoint.com/cgibin/printpage.cgi 6/8
12/9/2016 ArtificialIntelligenceNeuralNetworks

ApplicationsofNeuralNetworks
Theycanperformtasksthatareeasyforahumanbutdifficultforamachine

AerospaceAutopilotaircrafts,aircraftfaultdetection.

AutomotiveAutomobileguidancesystems.

MilitaryWeaponorientationandsteering,targettracking,objectdiscrimination,facialrecognition,
signal/imageidentification.

ElectronicsCodesequenceprediction,ICchiplayout,chipfailureanalysis,machinevision,voice
synthesis.

FinancialRealestateappraisal,loanadvisor,mortgagescreening,corporatebondrating,portfoliotrading
program,corporatefinancialanalysis,currencyvalueprediction,documentreaders,creditapplication
evaluators.

IndustrialManufacturingprocesscontrol,productdesignandanalysis,qualityinspectionsystems,
weldingqualityanalysis,paperqualityprediction,chemicalproductdesignanalysis,dynamicmodelingof
chemicalprocesssystems,machinemaintenanceanalysis,projectbidding,planning,andmanagement.

MedicalCancercellanalysis,EEGandECGanalysis,prostheticdesign,transplanttimeoptimizer.

SpeechSpeechrecognition,speechclassification,texttospeechconversion.

TelecommunicationsImageanddatacompression,automatedinformationservices,realtimespoken
languagetranslation.

TransportationTruckBrakesystemdiagnosis,vehiclescheduling,routingsystems.

SoftwarePatternRecognitioninfacialrecognition,opticalcharacterrecognition,etc.
https://www.tutorialspoint.com/cgibin/printpage.cgi 7/8
12/9/2016 ArtificialIntelligenceNeuralNetworks

TimeSeriesPredictionANNsareusedtomakepredictionsonstocksandnaturalcalamities.

SignalProcessingNeuralnetworkscanbetrainedtoprocessanaudiosignalandfilteritappropriatelyin
thehearingaids.

ControlANNsareoftenusedtomakesteeringdecisionsofphysicalvehicles.

AnomalyDetectionAsANNsareexpertatrecognizingpatterns,theycanalsobetrainedtogeneratean
outputwhensomethingunusualoccursthatmisfitsthepattern.

https://www.tutorialspoint.com/cgibin/printpage.cgi 8/8