NaturalLanguageProcessing

whatitdoes whatisinvolved whyisitdifficult y briefhistory

sentencef structuredrepnsofmeaning
"how old is my help3 doc file?" howoldismyhelp3.docfile? Lisp:(query(filedetail'date"C:/help3.doc")) "thelargecatchasedtherat" Logic:(E Logic: (E1s1 large(s1) L feline(s1)) )L L (E1s2 rodent(s2))L chased(s1,s2 ) "theyoungboyateabadapple" CDGraph CD Graph ...seenextpage... see next page

CDgraph "theyoungboyateabadapple"

whatisinvolved
symboliccomputation symbolic computation ie:symbolsmanipulatedbysymbolprocessors search&inference knowledgerepresentationtechniques

whyisitdifficult
prejudice,politics,etc ambiguity... y syntactic semantic pragmatic

examplesentences
theoldmantheboats mycardrinkspetrol IsawtheEiffelTowerflyingtoParis p y heopenedthedoorwiththekey heopenedthedoorwiththesqueakinghinge y theboykickedtheballunderthetree theboykickedthewallunderthetree putthebottlesintheboxontheshelfbythedoor

(brief)historyoflanguageprocessing
1950s 1956 1960s 1970s 1980s 1990s 2000+ Russianf Englishtranslation Chomsky Patternmatching g p Parsing&someKnRep Kn&inference bigdreamssmallresults big dreams small results quietlypromising

matching: g Sir

matching: g Student

matching: g Elisa

amodern approach

input sentence

morphological processing lexicon syntax analysis (parsing) grammar semantic rules semantic analysis

contextual information

pragmatic analysis

target g representation

step1 morphologicalprocessing
objective:stripwordsintoroots&modifiers issues inflection derivation compounding

( p (catpl f cats) ) (happyadj f happinessnoun) (toothpaste)

morphologicalprocessing notes
all(?)spokenlngsexhibitmorphology easiertohandleinwrittenlngsifnoticonic somemorphologydescribesinfmbeyondsyntax g p y ( , , ) eg: proximity (Tamil,Setswana,etc) case speaker/listenerpeerrelationship speaker / listener peer relationship

morphologyexamples morphology examples

Noun + Suffix Syntactic case y Meaning g Chennai-ukku dative: destination To Madras Chennai-ukku-irundu dative: source From Madras Chennai-le containment In Madras Chennai-ai object (formal) Madras Fig. 2.2. Suffix Attachment for Noun Cases (Tamil author's spelling)

Proximity Time Things (inanimate) Near N i-ppa (thi ti i (this time: now) ) i-ndtha (thi thi i dth (this thing: thi ) this) Far a-ppa (that time: then) a-ndtha (that thing: that) Question e-ppa (what time: when) e-ndtha (what thing: which) Fig. 2.3. Proximity Information as Prefix Tags (Tamil)

Proximity Cow Student Near Speaker kgomo e (this cow) mo-ithuti yo (this student) Near Listener kgomo e-o (that cow) mo-ithuti yo-o (that student) g ( ) y ( ) Far kgomo e-le (that cow) mo-ithuti yo-le (that student) Fig. 2.4. Proximity by Demostrative Pronoun Inflection (Setswana)

step2 syntaxanalysis
objectives: 1 checkforcorrectness 2 producephrasestructure uses parser grammar lexicon

arulebasedsearchengine context freeproductionrules contextfree production rules dictionaryofwords&theircategories

syntaxrules
partsofspeech rulesofcombination rules of combination consider id thecatchasesthemouse alllargeblackdogschasecats

example1 usingLkit
(build-lexicon '((a determiner) (cat noun) (dog noun) (the determiner) (chased verb))) (build-grammar '((s1 (sentence -> noun-phrase verb-phrase)) 1 (np (noun-phrase -> determiner noun)) (vp (verb-phrase -> verb noun-phrase)) ))

example1 output
(parse 'sentence '(the dog chased a cat)) complete-edge 0 5 s1 sentence (the dog ...) nil s1 sentence -> (noun-phrase verb-phrase) > (noun phrase verb phrase) Syntax (sentence (noun-phrase (determiner the) (noun dog)) (verb-phrase (verb chased) (noun-phrase (determiner a) (noun cat)))) Semantics (sentence)

sowhat? wewantmeaning i

Remember:"theyoungboyateabadapple"
howcansemanticsbeencodedassymbols? the boy / an apple? boy/an young/old,happy/sad,good/bad? howcansemanticsbegenerated? h i b d? whatcanbeinferredfromsemantics?

Reminder:"theyoungboyateabadapple"

symbolicrepresentationofsemantics
(actor (root boy) (id boy#732) (tags animate human male) (qual (age (val 5) (approx 3))) (quant specific)) (action (primitve INGEST)) (object (root apple) (id nil) (tags physob veg fruit food) (qual (phy-state -4)) (quant non-specific))

semanticsinlexicon
asimpleexample
(build-lexicon '((a det (cat noun (chased verb (dog noun (the det ))

any ) feline ) hunts ) canine ) specific)

semanticsingrammarrules
(s1 (sentence -> noun-phrase verb-phrase) (actor . noun-phrase) ( (action . verb-phrase.action) p ) (object . verb-phrase.object) ) (np (noun phrase -> det noun) (noun-phrase > (det . noun) ) (vp (verb-phrase -> verb noun-phrase) (action . verb) (object . noun-phrase) j p )

semantics results
(parse 'sentence '(the dog chased a cat)) complete-edge 0 5 s1 sentence (the dog...) nil s1 sentence -> (noun-phrase verb-phrase) ( p p ) Syntax (sentence (noun-phrase (det the) (noun dog)) (verb-phrase (verb phrase (verb chased) (noun-phrase (det a) (noun cat)))) Semantics (sentence (actor (specific canine)) (action hunts) (object (any feline))) j y

semanticsinlexicon checks1
(a (all (cat (cats (chase (chases (dog (dogs (the det det noun noun verb verb noun noun det (sems (sems (sems (sems (sems (sems (sems (sems (sems . . . . . . . . . any)) every)) feline) (num feline) (num hunts) (num hunts) (num canine) (num canine) (num specific))

. . . . . .

sing)) plur)) plur)) sing)) sing)) plur))

semanticsingrammar checks1
(s1 (sentence -> noun-phrase verb-phrase) (actor . noun-phrase.sems) ( (action . verb-phrase.action) p ) (object . verb-phrase.object) ; check number of noun-phrase & verb-phrase (if (noun phrase number (noun-phrase.number = verb-phrase.number) numeric-agreement-ok numeric-agreement-bad ) )

semantics results
(parse 'sentence '(the dog chases a cat)) complete-edge 0 5 s1 sentence (the dog...) nil s1 sentence -> (noun-phrase verb-phrase) ( p p ) Syntax (sentence (noun-phrase (det the) (noun dog)) (verb-phrase (verb phrase (verb chases) (noun-phrase (det a) (noun cat)))) Semantics (sentence (actor specific canine) (action . hunts) (object any feline) j y numeric-agreement-ok)

semantics results
(parse 'sentence '(the dogs chases a cat)) complete-edge 0 5 s1 sentence (the dog...) nil s1 sentence -> (noun-phrase verb-phrase) ( p p ) Syntax (sentence (noun-phrase (det the) (noun dog)) (verb-phrase (verb phrase (verb chases) (noun-phrase (det a) (noun cat)))) Semantics (sentence (actor specific canine) (action . hunts) (object any feline) j y numeric-agreement-bad)

semanticsingrammar checks2
(s1 (sentence -> noun-phrase verb-phrase) (fail if noun-phrase.number /= verb-phrase.number) p ) (actor . noun-phrase.sems) (action . verb-phrase.action) (object . verb phrase object) verb-phrase.object) )

semantics results
(parse 'sentence '(the dog chases a cat)) Semantics ( (sentence ( (actor specific canine) p ) (action . hunts) (object any feline))

(parse 'sentence '(the dogs chases a cat)) .... failed ....

semanticsingrammar checks3
(s1 (sentence -> noun-phrase verb-phrase) (glitch numeric-agreement if not noun-phrase.number p = verb-phrase.number) (actor . noun-phrase.sems) (action . verb phrase action) verb-phrase.action) (object . verb-phrase.object) )

semantics results
(parse 'sentence '(the dogs chases a cat)) complete-edge 0 5 s1 sentence (the dogs...) nil Glitches: (numeric-agreement) ( g ) s1 sentence -> (noun-phrase verb-phrase) Syntax (sentence (noun phrase (det the) (noun dogs)) (noun-phrase (verb-phrase (verb chases) (noun-phrase (det a) (noun cat)))) Semantics (sentence (actor specific canine) (action . hunts) (object any feline))

example2 lexicon
(a (cat (chase (dog (the (black (large (small det noun verb noun det adj adj adj any ) feline ) hunts ) canine ) specific) (color black)) (size 7/10)) (size 3/10))

example2 grammar
(build-grammar '((np ( (( p (noun-phrase -> ?det *adj noun) p j ) (if det (quantification . det) (quantification undefined)) (qualifiers . *.adj) (object . noun) )) ))

example2 results
(parse 'noun-phrase '(small black dog)) complete-edge 0 3 np noun-phrase (small...) nil np p noun-phrase -> (?det *adj noun) p ( j ) Syntax (noun-phrase (adj small) (adj black) (noun dog)) Semantics (noun-phrase (quantification undefined) (qualifiers ((size . 3/10)) ((color . black))) (object canine))

example2 results
small dogs chase the small cats and large dogs chase the large cats (sentence conjunction ((actor (quant undefined) (qual (size . 3/10)) (object . canine)) (action . hunts) (object (quant . specific) ( ( bj t ( t ifi ) (qual ( i l (size . 3/10)) (object . feline))) ((actor (quant undefined) (qual (size . 7/10)) (object . canine)) (action . hunts) (object (quant . specific) (qual (size . 7/10)) (object . feline))))

semanticprocessing(oneapproach)
semanticrulesingrammarf 1ststagecase frame verbformf primitiveactioncaseframe disambiguate&filladditionalcaseframeslots checkreferenceswithworldand/ordialog dostatementlevelinference integratewithdialog q g doeventsequencedialog

step1:producerawcaseframe
verbcases
thecatchasedtheratinthekitchen thecatchasedtheratintothekitchen

commoncases
source destination location starttime endtime duration instrument beneficiary

theambiguityproblem
eg:theboykickedtheballunderthetree grammarrules
Sf SPP S f NP VP NP f ?det *adj noun NPf NPPP NP f NP PP

exampleframe#1
actor (quant specific) (tags animate male human) (qual (age (range 3 13))) (root boy) action (root kick) object (root ball) (tags manip) (posn-relative (locator beneath) (object (root tree) ...etc... ) etc

exampleframe#2
actor (quant specific) (tags animate male human) (qual (age (range 3 13))) (root boy) action (root kick) object (root ball) (tags manip) dest (posn-relative (locator beneath) (object (root tree) ...etc... ) etc

exampleverbform#1
primitive prohibited slots legal strike object (tags manip) instrument (part-of $actor foot) start-time, end-time, duration instrument, beneficiary, location source, dest

illegal

exampleverbform#2
primitive required slots legal push object (tags manip) instrument (part-of $actor foot) source, dest, start-time, end-time, instr, beneficiary, locatn, duration

semanticprocessing(oneapproach)
semanticrulesingrammarf 1ststagecase frame verbformf primitiveactioncaseframe disambiguate&filladditionalcaseframeslots checkreferenceswithworldand/ordialog dostatementlevelinference integratewithdialog q g doeventsequencedialog

integrationwithdialog
dialogshave... players(actors) props(objects) ( ) locations(fromcaseframes) themes(derived) eventsequences(fromthemes) event sequences (from themes) plans(fromthemesand/orderived)

eventsequence
setof... players(actors) props(objects) seriesof... semanticallyencodedactivities(matched) semantically encoded activities (matched) escapes,exceptions&alternatives

reading grammars, etc g g ,

A good source of links & references... Computational Analysis of Prepositions http://knol.google.com/k/abdul-baqisharaf/computational-analysis-of-prepositions/3hc3uny2z7r41/4# if you only plan to read one article... Baldwin, T. Kordoni, V and Villavicencio, A. 2009. Prepositions in Applications: A Survey and Introduction to the Special Issue ". Computational Linguistics 35 (2): 119149. also... Litkowski, Kenneth C. and Orin Hargraves. 2007. SemEval-2007 task 06: Wordsense disambiguation of prepositions. In Proceedings of the 4th International Workshop on Semantic Evaluations, pages 2429, Prague. Disambiguation of Preposition Sense Using Linguistically Motivated Features, Stephen Tratz and Dirk Hovy. Proceedings of the NAACL HLT Student Research Workshop and Doctoral Consortium, pages 96100, Boulder, C l B ld Colorado, J d June 2009. c 2009 Association for Computational Linguistics 2009 A i ti f C t ti l Li i ti

reading grammars etc grammars,

the NLP dictionary: www.cse.unsw.edu.au/~billw/nlpdict.html for practical help with building grammars check the following (it is about 10 years old but then so is the English language :o) A Grammar Writers Cookbook. Miriam Butt, Tracy Holloway King, Marma-Eugenia g Nio and Fridirique Segond also (for writing larger grammars) it is useful to find a book on grammar for tutors and/or students of English as a second language. for a broad (if a little formal) take on semantics try dipping into... Semantics-Oriented Natural Language Processing Mathematical Models and Algorithms. Vladimir Fomichov A. 2010

reading kn rep for NLP

logic and knowledge representation a guide http://dspace.dsto.defence.gov.au/dspace/bitstream/1947/9996/1/DSTO TR http://dspace.dsto.defence.gov.au/dspace/bitstream/1947/9996/1/DSTO-TR2324%20PR.pdf representing events for NLP http://www.google.co.uk/url?sa t&rct j&q knowledge%20representation%20%22 http://www.google.co.uk/url?sa=t&rct=j&q=knowledge%20representation%20%22 representing%20events%22&source=web&cd=6&sqi=2&ved=0CEgQFjAF&url=h ttp%3A%2F%2Fwww.aaai.org%2Focs%2Findex.php%2FFSS%2FFSS10%2Fpap er%2Fdownload%2F2183%2F2819&ei=f6oWT_e7DeKC4gTMpaijBA&usg=AFQjC NFYmurwJR9oqfCRBimVprWRK45kew&cad=rja semantic networks & frames (2005) http://www.cs.bham.ac.uk/~jxb/IAI/w6.pdf VERL: An Ontology Framework for Video Events (2005) http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1524892