2014 Fourth International Conference on Advances in Computing and Communications
Multilingual Machine Translation with Semantic and
Disambiguation
Meera M
College of Engineering, Cherthala, Managed by IHRD
Established by the Government of Kerala
Email: mmeera18@gmail.com
AbstractMachine Translation is the process of translating
one natural language into another using automated and com-
puterized means.The proposed system is a rule based multilin-
gual unidirectional translation system which translates English
sentences into corresponding Malayalam and Hindi sentences.
During translation it incorporates morphological and syntactic
information present in the working pair of languages. And it also
performs word sense disambiguation, semantic analysis and mor-
phological processing of the generated target sentence.Machine
learning based word sense disambiguation and First Order
Predicate Logic(FOPL) based semantic checking are the two main
features of this paper.The word arrangement in dictionary with
POS tags and word sense is an other important factor that can
contribute to the efciency of the translation system.Hence the
system can achieve 74% accuracy for both translation which
includes all the tenses and different type of sentences in English.
Keywords : Machine Translation,Morphology Genera-
tion,Word Sense Disambiguation
I. I NTRODUCTION
Language is an effective medium of communication. It
effectively represents the ideas and feelings of the human mind.
All around the world more than 5000 languages are used. It
shows the existence of linguistic diversity. It is difcult for
an individual to know and understand all the languages of
the world. Hence, the methodology of translation is adopted
to communicate the messages from one language to another.
Machine translation is one of the research areas under com-
putational linguistics. In the machine translation systems, a
computerized system is used for the translation of one natural
language to another.
Machine translation systems [1] can be designed for most
of the languages. If it is specically designed for two particular
languages then it is called bilingual system, or if for more than
a single pair of languages then it is called multilingual systems.
The language inputted for the translation is called the source
language(SL) and language outputted by the translation system
is called the target language(TL). Based on the direction of
translation the machine translation systems can categorize into
two as, unidirectional and bidirectional. In a unidirectional
system source language is converted to target language format
during the translation. But in the case of bidirectional systems
translation in either direction is possible that means from SL
to TL and TL to SL.
978-1-4799-4363-0/14 $31.00 2014 IEEE
DOI 10.1109/ICACC.2014.60
Sony P
College of Engineering, Cherthala, Managed by IHRD
Established by the Government of Kerala
Email: spsony@gmail.com
II. R ELATED W ORKS
The history of machine translation dates back to the sev-
enteenth century. From there on wards so many researches
were done in the area of machine translation. But most
of them are based on statistical and inter lingual approach.
AnglaBharti, a multilingual translation system [1], is one
among them which uses a rule based transfer approach for
translation. The system analyses English sentences and cre-
ates an intermediate structure called PLIL(Pseudo Lingua for
Indian Languages).For translation it uses a pattern directed
approach using context free grammar like structures and the
strategy used for translation lies in between the transfer and
the inter lingual approach. IIT, kanpur developed an English-
Hindi rule based translation system named AnglaHindi [1],
which combines example based and AnglaBharti approach. By
adding an additional layer over AnglaBharti, In 2005 a new
statistical translation system for English-Hindi is developed.
In 2009, Amrita Institute of Technology developed an English
to Malayalam translation system which incorporates morpho-
logical and syntactic information. Combining rule based and
statistical approach IISC Bangalore created a translation sys-
tem named Shakti [1] which translate English to Marathi and
Telugu using transfer based strategy.The English to Hindi MT
system Mantra, developed by Applied Artifcial Intelligence
(AAI) group of CDAC,Bangalore, in 1999 uses transfer based
approach. The system translates domain specic documents
in the eld of personal administration; specically gazette
notications, ofce orders, ofce memorandums and circulars.
III. CLASSIFICATION OF MACHINE TRANSLATION
APPROACHES
The Machine Translation process can be broadly classied
into the following approaches Direct Machine Translation,
Rule Based Machine Translation and Corpus Based Machine
Translation [1]. The classication hierarchy is given in Fig. 1.
Fig. 1: Machine Translation Approaches Classication
In Direct Translation [1], with the help of a bilingual
dictionary a direct word by word translation of the input is
223
carried out and after which some syntactical rearrangements
are made. The Rule Based Translation System [1]considers
semantic, morphological and syntactic information and based
on these rules the source language is transformed to the target
language through an intermediate representation. The Inter
lingual approach converts words into an intermediate language
IL, which is a universal language created by the system, to use
it as an intermediate for translation into more than one target
language.
The Transfer based approach [1] uses translation rules to
translate the input language to the output language which is
done in three phases. Firstly, the source language is converted
into an intermediate representation which is subsequently
converted into target language representation in the second
phase. The third phase involves generation of the nal target
language.
In corpus based approach [1], the translation system is
trained with a bilingual text corpus to get the desired output.
In Statistical Machine Translation, a bilingual corpus is trained
and statistical parameters are derived in order to reach the
most likely translation. While the Example Based Machine
Translation System usually uses previous translated examples
to translate from source to target language.
IV. P ROBLEM D EFINITION
Analysis on the language features show that, for every
English sentence there exist at least one sentence in Malayalam
and Hindi Languages. So our problem is to nd that correct
sentence in those languages for the given English sentence with
the help of a machine. The block diagram for the proposed
system is given in Fig.2. In the diagram black box view
of system F, with input X and output Y is shown. For the
transformation of X into Y, the system F, depends on semantic
checker S, morphology generator M and disambiguator D.
So the problem of machine translation can be modeled in
mathematical formula as, X, Y, Z : F(X) Y, Z where
F(X) : D S M
Fig. 2: Block Diagram
V. P ROPOSED S YSTEM
The proposed system presents a rule based direct machine
translation system which performs a unidirectional machine
translation from English to Malayalam and Hindi with the help
of a bilingual dictionary. For this system source language is
English and target languages are Malayalam and Hindi. Since
it works for more than two languages it is a multilingual
translation system. Apart from the previous related works in
the area of machine translation, the proposed system focus
on word sense disambiguation [11] and rst order predicate
logic(FOPL) based semantic checking.
The proposed system introduces verb classication based
FOPL rule for semantic checking, parse tree node numbering
224
method for removing ambiguity in CFG, Unicode combina-
tion rules for morphology generation and machine learning
approach for word sense disambiguation. Rule based method
in the proposed system helps to make translation without the
cost of corpus creation. The unexpected result generated by the
statistical approach due to the misalignment of words in the
corpus is overcame in this proposed system. so the generated
results are not deceiving.
The verb classication and FOPL based semantic checking
used in this system can easily separate the semantically wrong
sentence from the correct one with very simple steps. In
case of structure rearrangement phase the newly introduced,
parse tree node numbering method can avoid the ambiguity in
CFG rules while generating the target sentences. The unicode
combination rules can effectively blend with the rules used for
translation and can generate new morpheme with morpholog-
ical variations suited with the target language syntax. In the
case of word sense disambiguation machine learning method
can correctly identify the sense of words based on the context
used in the sentences, so the system can disambiguate the
word meaning with the help of these identied senses. Since
the proposed system is a combination of all these method, it
can take the advantage over most of the existing systems and
produce highly efcient translations.
The architecture of the translation system is shown in Fig.
3. In this system the translation process is divided into three
phases as i)sentence analysis phase ii) structure rearrangement
phase iii) target sentence generation phase. Detailed ow chart
is shown in Fig. 4
Fig. 3: Architecture
A. Sentence Analysis Phase
English sentence has twelve tenses. The data inputted to the
system may be sentence, Wh question or yes/no question and
belongs to one of the twelve tense forms. Accurate translation
can be generated by analyzing the nature of the input. For that
in this phase English data undergoes tokenization and parts of
speech tagging [7]. Using the obtained tags, category of the
sentence is analyzed for further processing.

Fig. 4: Flow Diagram
B. Structure Rearrangement Phase
It is the heart of this translation system, which performs
semantic checking and structure rearrangement. Input is parsed
using Stanford Parser [9]developed by the Stanford University.
The output from the parser consist of parse tree and depen-
dency tree. The obtained results are converted into context free
grammar(CFG) and predicates for subsequent processing.
1) Parsing: Inputted sentence is parsed using the Stanford
English parser. The parse tree generated by the parser is con-
verted into a numbered format to avoid ambiguity during target
sentence generation. The generated parse tree and numbered
parse tree for the sentence He went to school is shown in
Figure. 5.
Fig. 5: Generated Parse Tree and Numbered Parse Tree
2) Semantic Checking: For the checking of semantic all
English verbs are classied based on nature of subjects [5]
who can do these verbs as shown in Fig. 6.
The classication shows that class H is a subset of class O
which is a subset of class E. That means human subject can
be combined with all verbs, living subjects can be combined
with all verbs except with class H verbs, non living subject
can combined with the verbs other than in class H and O. Any
mismatch in the combination leads to semantic error.
Fig. 6: Verb Classication
The dependency tree obtained from the parser consist of
Stanford dependencies, from these dependencies extract the
subject and verb of the sentence. Subject verb combination
based rst order predicate logic(FOPL) are used for checking
the semantic correctness. The dependency obtained from the
225
parser is converted into FOPL format as: x, y, subject(x)
verb(y) cando(x, y). The predicate evaluates to true only if
x, y belongs to valid combination of subject, verb classes. The
true value of the predicate indicates the semantic correctness
of the sentence.
For example if the inputted sentence are i)Monkey ate a
banana and ii) Banana ate a Monkey, the dependency tree
obtained are shown in Fig. 7.
i)Monkey ate a Banana ii)Banana ate a Monkey
Fig. 7: Stanford Dependency Tree
Stanford dependencies for the sentences are given above.
From the above dependencies, the subject of rst the sentence
is Monkey and verb is ate and that of the second sentence is
Banana and ate. In verb classication ate belongs to class
O so it cant be combined with nonliving subject Banana but
it can be with living subject Monkey. So the predicate for the
rst one evaluates to true but for the other is false. That shows
the semantic correctness of example 1 and semantic error of
example 2.
3) CFG Creation and Rearrangement: The created parse
tree is processed to get the Context free Grammar(CFG)
corresponds to the sentence. Context free grammar is generated
for each branch of the parse tree. Then grammar rules are
modied according to target language syntax. The grammar
generated for the parse tree in Fig. 5 and the modied grammar
is given below.
1S 2NP 4VP 1S 2NP 4VP
2NP 3PRP 2NP 3PRP
3PRP He 3PRP He
4VP 5VBD 6PP 4VP 6PP 5VBD
5VBD went 5VBD went
6PP 7TO 8NP 6PP 8NP 7TO
7TO to 7TO to
8NP school 8NP school
4) Sentence Creation: Structure of inputted sentence is
altered according to target language format using newly formed
grammar and it gives a new sentence having different word
order from the actual one. The rearranged sentence generated
with the modied grammar is shown in Fig. 8.
Fig. 8: Rearranged Sentence Parse Tree
C. Target Sentence Generation Phase
All words in the rearranged sentence undergo stemming.
After that word by word translation is performed using the
dictionary. The system uses a Unicode based morphology
processing method [4]. In the morphology generation mor-
phemes are combined to form the new word based on the
previously identied sentence structure and tenses. Since Hindi
language follows the gender matching rule, for English to
Hindi translation an extra gender processing step is used which
add post position based on the gender of the next word.
1) Disambiguation: Disambiguation [11] is an open prob-
lem in natural language processing, which governs the process
of identifying which sense of a word or preposition which is
used in a sentence, when the word/preposition has multiple
meanings.
In this work Weka tool is used for the disambiguation of
prepositions and words [11]. The sentence used for training
purpose is converted into a feature vector containing four
elds. The four elds include previous tag, word, next tag,
sense. The NNge(Non-Nested Generalized Exampler) classier
in Weka is used to classify these vectors based on sense eld.
The result from the classier is converted into rule format for
further processing in the subsequent steps.
2) Stemming : In this step each word is converted into its
root form by deleting the afxes such as ed, s, es, ing, s.
Spelling rules in English are used in reverse form to perform
stemming on each words This step helps for morphology
generation and dictionary lookup. The split afxes are stored
separately and used at the time of morpheme generation.
3) Dictionary Lookup: This step is used to nd the correct
translation of a single word in the source sentence into its
corresponding target word.To decrease the dictionary search
time the contents are organized in 26 les based on their
starting letter.
4) Morphology generation: Analyze the tense forms and
prepositions in the sentence to make the morphological vari-
ations. Morphology generation is performed using Unicode
processing [4]. Adjacent words are compared to nd the
Unicode at the beginning and the end of the morphemes.
By using this Unicode combination rule new morphemes are
generated.
VI. O BSERVATION AND R ESULT
Since it is a rule based translation approach accuracy of
translation mainly depends on the correctness of dened rules.
For disambiguation Weka tool is used for creating a rule.
Disambiguation can be done more accurately with sufcient
amount of training sentences. Sometimes the structure rear-
rangement phase creates erroneous result because of the free
word order nature of Malayalam and Hindi languages. Apart
from statistical approach this rule based method can give a
correct translation for most of the inputted sentences. The
dictionary content is an other factor that affects the correct-
ness of translations. With more words in dictionary number
of translated sentences can be increased.For the evaluation
precision and recall value are used. The formula used for
calculating the precision and recall are given below.where
the candidate solution is generated by the proposed machine
translation system and reference solution is generated with
human translation.The obtained results for both translations
with different corpus are modeled as graph shown in Fig.9.
Precision =Number of word in the candidate solution
correctly aligned with reference solution / Number of words
in candidate solution
Recall=Number of word in the candidate solution correctly
aligned with reference / Number of words in reference solution
Fig. 9: Observation Result
VII. C ONCLUSION
This work introduces an effective methodology for English
to Malayalam and Hindi translation based on the rule based ap-
proach. The proposed translation system can successfully work
for almost all simple sentences in their twelve tense forms,
their negatives and question forms. Apart from other translation
systems it considers the semantic and disambiguation, and is
a success for these. As the result of combining the newly
introduced method with machine translation the evaluation
result shows an accuracy of 74% with harmonic mean(Fmean)
of .74. The languages like Malayalam and Hindi are very much
morphologically rich and agglutinative, the performance can be
further improved by adding more morphological inections to
the system.
R EFERENCES
[1] Antony P. J, Machine Translation Approaches and Survey for Indian
Languages , Computational Linguistics and Chinese Language Process-
ing,Vol. 18, No. 1, March 2013, pp. 47-78
[2] Mary Priya Sebastian,K. Sheena Kurian,G. Santhosh Kumar , Align-
ment Model and Training Technique in SMT from English to
Malayalam,Springer-Verlag Berlin Heidelberg 2010, IC3 2010,Part
I,CCIS 94,p.305-315
[3] Mallamma V. Reddy, M. Hanumanthappa , Indic Language Machine
Translation Tool: English to Kannada/Telugu,Proceedings of Multimedia
Processing, Communication and Computing Applications,Springer India
2013,p.200-213
[4] Karin Kipper, Anna Korhonen, Morphological Analyzer for Malayalam
Using Machine Learning, Language Resources and Evaluation ,Volume
42, Issue 1 , p.21-40
[5] Karin Kipper, Anna Korhonen, A large-scale classication of English
verbs,Language Resources and Evaluation, Issue 1, Volume 42, p.21-40
[6] Nisheeth Joshi, Hemant Darbari, Human and Automatic Evaluation
of English to Hindi Machine Translation Systems, Proceedings of
the Second International Conference on Computer Science Engineer-
ing and Applications,Springer-Verlag Berlin Heidelberg 2010, Volume
166,2012,AISC 106,p.423-432
[7] Remya Rajan, Remya Sivan, Remya Ravindran, K.P Soman. Rule
based machine translation from english to malayalam. In Conference
Proceedings on International Conference on Advances in Computing,
Control, and Telecommunication Technologies, pages 439441, 2009
[8] Mary Priya Sebastian, G Santhosh Kumar,English to malayalam trans-
lation: a statistical approach. In Proceedings of the 1st Amrita ACM-W
Celebration on Women in Computing in India, page 64. ACM, 2010.
[9] Nishtha Jaiswal, Renu Balyan, and Anuradha Sharma. A step towards
human-machine unication using translation memory and machine trans-
lation system. In International Conference on Languages, Literature and
Linguistics, pages 6468.2011.
[10] Raghavendra Udupa U, Tanveer A. Faruquie , An English-Hindi Sta-
tistical Machine Translation System, First International Joint Conference
2004, Hainan Island, China,p.315-325
[11] Roberto Navigli, Word Sense Disambiguation: A Survey, ACM Com-
puting Surveys (CSUR) 2009,Volume 41, Issue 2, Article No. 10
[12] Jignashu Parikh, Pushpak Bhattacharyya , Interlingua-based English-
Hindi Machine Translation and Language Divergence, Machine Trans-
lation, Volume 16, Issue 4 , p.251-304.
226