Beruflich Dokumente
Kultur Dokumente
535
project that concentrates on electronic documentation of cuneiform script is on the left; the transliteration is in the
ancient cuneiform, consisting of cuneiform texts, images, middle and the modern English translation is on the right.
transliterations and glossaries. It is managed at UCLA, Sumerian written in cuneiform script does not have the
USA and the Max Planck Institution for the History of concept of upper- or lowercase, and as a result, in monolin-
Science, Berlin. BDTNS is a database that manages more gual contexts, the typical step of case normalization is not
than 95,300 administrative Sumerian cuneiform tablets necessary as all text is rendered in lowercase in the translit-
during the Neo-Sumerian period. eration format used by CDLI. However, signs rendered in
uppercase occur frequently and denote a number of special
Previous Work on Sumerian Texts circumstances, most commonly, that the phonetic equiva-
lent of the sign is unknown. With respect to our system,
Research on the application of machine learning and NLP the presence of uppercase signs is rarely an issue as long
techniques to ancient languages is not abundant. By and as the spelling and casing is consistently attested in the cor-
large, the application of computer technology has been lim- pus. This consistency is provided by the lemmatization re-
ited to electronic publishing and string searching. Such ef- quired of Sumerologists in order to submit initial transliter-
forts applied to Sumerian are even more sparse. (Tablan et ations (Foxvog 2014).
al. 2006) described the creation of a tool for Sumerian lin- Royal epithets notwithstanding, Sumerian personal names
guistic analysis and corpus search applied to Sumerian lit- are exclusively comprised of a single word, almost always
erature. This work focused on the application of a morpho- consisting of at least two signs. In cases where disambigua-
logical model for noun and verb recognition, developed by tion is required, a patronymic may added (for example, szu-
Sumerologists. However, it did not extend beyond simple esz4-tar2 dumu zu-zu, “Su-Estar, son of Zuzu”). This disam-
word and morpheme recognition. (Jaworski 2008) devel- biguation is frequent in practice due to the relatively shallow
oped an approach to extraction of information from the same pool of personal names used (Limet 1960).
economic documents of concern to us by beginning with an
ontology of the world presupposed by this corpus and, in Lemmata
conjunction with syntax and semantic analysis. The claim
of this work is that it supports research into the Sumerian 62.1% of the tablets (53,146 tablets) in the CDLI corpus are
language, officials participating in the activities the tablets accompanied by annotations, dubbed “lemmata” which pro-
record, and into document classification. Our previous work vide, stemification, translation and categorization of linguis-
includes (Brewer et al. 2014) and (Luo et al. 2015). Both tic elements within a line. Thus, the example given in Fig-
are addressed to the recognition of proper names in Sume- ure 2 gives a line of text followed by its lematization, indi-
rian text. (Brewer et al. 2014) uses a naive edit-distance and cated with a line-initial “#.”
pattern-matching algorithm to attempt to identify personal In this example, the lemmatization indicates that the
names. (Luo et al. 2015) applies a DL-Cotrain based unsu- word “GAN2” derives from the Sumerian stem “iku” and
pervised learning method with 3 seed rules. This unsuper- is understood to mean “unit.” Similarly, “ur-{gesz}-gigir”
vised method achieves a high recall (92.5%) and a low pre- is a personal name; “nu-banda3” with the Sumerian stem
cision (56.0%). Some comparisons among these work are “nubanda” means “overseer”, a common profession in
described in the later section. Sumerian society; the word “gu4” with the Sumerian stem
“gud” means “ox.” Sometimes, when the function of a work
is uncertain, the lemmatization offers more than one cate-
Data of CDLI gory. For example, the lemma “GN|FN” indicates that the
In this project, we still use the CDLI repository as we did corresponding can be either a geographical name or the
in our previous work for its restriction to the ASCII charac- name of a field (analogous the “Potter’s Field.”).
ter set is more convenient than the UTF-8 encoding used by
BDTNS, as well as for comparison purpose. Of this reposi- Damaged Tablets
tory we use the 53,146 tablets having lemmata. In CDLI corpus, some transliterations have “[” and “]” at-
In the following, we will give more detailed introduction taching to a sign. It indicates the sign is damaged. More
of the CDLI data. specifically, “[” indicates that the following sign is damaged
on the left edge, whereas “]” indicates the following sign
CDLI: Words and Signs is damaged on the right edge (Sahala 2012). “[x]” indicates
Although the texts we are investigating were originally that there is a sign that cannot be read due to the damage on
written in cuneiform script, scholars have traditionally both edges and “[...]” indicates that there are several unread-
worked with transliterations using the English alphabet. Ho- able signs.
mophony, which is very common in cuneiform, is handled
by numerical subscripts, i.e., “gu,” “gu2 ” and “gu3 ” refer Supervised Proper Name Recognizer
to distinct signs with the same sound value. Sumeriologists An important task in supervised learning is to propose a
have developed a system of in-text annotations important to set of useful features. Because there is no previous work as
the application of NLP techniques. such, we proposed a set of features that capture either the
Figure 1 shows the tablet with an id of P105955 context or the spelling of the word of interest. We reported
from CDLI repository ( http://cdli.ucla.edu/). The original the system performance of a variety of supervised learning
536
Figure 1: Tablet with ID of P105955 from CDLI.
537
word index in line. 0-indexed. given the weighted sum of the input features, gives results
left context. None if this is the first word in the line. close to 1 when the true class is “positive” or “success”
right context. None if this is the last word in the line. (In the case of this work, a personal name) and close to 0
line context. All words in line, space-delimited. otherwise.
if word is alone on line.
if left context is dumu (“child (of)”). 4. Naive Bayes: Makes use of Bayes Theorem to train a sim-
%This may suggest a personal name in a patronymic. ple classification model. Starts with the observed data (in-
if right context is dumu (“child (of)”). cluding the distribution of names/non-names in the train-
if line context is kiword. ing set), and calculates the most probable class for each
%This may suggest a seller in a transaction. word. Despite the assumption that the features are all con-
if line context is igi word. ditionally independent given the true class, Naive Bayes
%This may suggest a witness to a transaction. has enjoyed some success as a classification tool.
if line context is igi word-sze3. 5. Stochastic Gradient Descent (SGD): Does Stochastic Gra-
%This may suggest a witness to a transaction. dient Descent (i.e. updates the gradient of the loss with
if line context is the Personnenkeil 1(disz) word. each instance examined, rather than averaging it across
%This may suggest a list of named individuals. all instances in the training set) on a logistic regression
if line context is kiszib3 word. model.
%This may suggest the individual responsible for sealing
%a tablet. 6. Linear Support Vector Machine: Support Vector Ma-
if line context is giri3 word. chines try to discover a hyperplane across the set of fea-
if first sign in word is repeated. tures that divides the data points in the “personal name”
%Sumerian names tend to favor repeated syllables. class from the others or, if it cannot, to find a hyperplane
last sign in word is repeated. that misclassifies as few points as possible (called a soft
if any sign in word is repeated. margin).
if word is a common profession. 7. Random Forest Classifier: A collection of 500 decision
if word contains a profession. trees, trained on different subsets of the data and com-
if left context is a profession. bined to form a stronger classifier. Each tree contributes
if left context contains a profession. its estimation of the probabilities of the different output
if right context is a profession. classes to the final result.
if right context contains a profession.
if word starts with ur. Results
if word starts with lu2-.
if word ends with -mu. For the most part, the results obtained from the different al-
if word contains {d}. gorithms were quite similar, though there were some inter-
%short for “dingir”(“deity”), the divine determinative. esting patterns.
if word contains {ki} “place”.
if word contains any determinative. Method Precision Recall Fscore
if word’s transliteration contains the letter q. Decision Tree 0.898 0.475 0.621
if word contains lugal (“king; large”). Gradient Boosting 0.872 0.660 0.752
if word contains a number. Logistic Regression 0.858 0.662 0.748
if word followed by sag. Naive Bayes 0.831 0.611 0.704
if word followed by zarin. Gradient Descent (SGD) 0.864 0.653 0.744
if word followed by a numeric classifier. SVM 0.856 0.655 0.742
if line context begins with iti (“month”). Random Forest 0.932 0.458 0.614
if line context begins with mu (“year”).
Table 2: Test Set 1 - data with no damaged words
Table 1: 36 features applied in training data and test data
538
Each classifier produced very similar results on both data Conclusion and Future Work
sets. This indicates that the damaged signs that can be an-
notated in the lemmata are also easy for the machine to rec- In this paper we reported our ongoing work on a supervised
ognize through the context or the spelling components. It proper name recognizer of the Sumerian corpus of CDLI
would be interesting to see how the system works on the database.
damaged signs that are not initially marked in the lemmata. As the first work of applying supervised method on Sume-
However the evaluation on such data set would be more chal- rian proper name recognition, our major contribution is two-
lenging given that it would involve more human expertise. fold. One is in feature selection. With the result from our
The Decision Tree classifier and the Random Forest clas- previous work on unsupervised learning on this task, and
sifier (which was constructed from multiple decision trees) with the help from a domain expert in our home univer-
both traded low recall for a higher precision to a greater ex- sity, we proposed a set of 36 features for both training and
tent than the others, in particular the Random Forest, which testing. The experiments have shown that different machine
had both the highest precision and lowest recall of all the learning methods, when exploiting these features, can per-
algorithms. Of the others, the Gradient Boosting classifier, form with high precision, which indicates the features are
Logistic Regression, Stochastic Gradient Descent, and the accurate enough to predict true names. Secondly, our work
Support Vector Machine all produced very similar results. shows that the decision tree based method is not the best op-
The Naive Bayes classifier, on the other hand, does not seem tion in this setting, which indicates that the combinations of
to be the best model for this task, at least as used here. It had features are more predictive of the class label in this appli-
the lowest precision, at 83%, and the lowest recall after the cation. However the decision tree do not provide such infor-
models involving decision trees, at 61%. mation. The relative poor performance of the Naive Bayes
method may be due to the fact that some of the features here
The similarity between the results of different methods
are more correlated with each other. Other machine learning
raised suspicions that they were learning very similar mod-
methods gave very similar result and performance. In com-
els, and that many words labeled correctly as personal names
parison with the previous work on unsupervised Sumerian
were identified by all or most of the algorithms. To test this,
proper name recognition (Luo et al. 2015), it suggested that
a system was set up in which each algorithm’s label for a
the current feature set is accurate but not large enough. How
word was interpreted as a ‘vote’ that that class be assigned
to obtain a richer feature set, compare to the adoption of a
to that word, and each word was classified according to the
machine learning strategy, is more important for this task.
label that received the most votes across all methods. This
conjecture seems to be confirmed, although more investiga- For the future work, we would like to do a more in-depth
tion is still under way; the resulting system achieved a recall comparison between the features we proposed here and the
of 65% and precision of 86% on both test sets. rules resulting from the unsupervised method. We believe
that it would help to further incorporate good features. It
would also be interesting to do analysis of the relevance of
Comparison with the Unsupervised Method the features. More feature engineering and analysis would
(Luo et al. 2015) reported a DL-CoTrain based unsuper- tell us what types of features are more salient and worth ex-
vised Proper Name Recognizer for CDLI data set. With 3 ploring. Since our work has drawn more attention from the
seeds rules and 150 iterations, this system generated over Assyriologists across the world, we have been provided a
2,000 decision rules, and achieves a high recall (92.5%) and richer set of features on this task. In addition, our experi-
a low precision (56.0%), which means it produces a lot of mental results showed that some methods tend to favor pre-
false positives. This indicates that a subset of these over cision , while others recalls. We need to do more exploration
2,000 rules are accurate enough that it predicts most of the on how the training process and parameter optimization af-
true names. However it also generates a lot of rules which fected the results, and see if the process has been sufficiently
are not accurate enough that 44% of the names it predicts optimized.
are not labelled as names by the annotators.
In the supervised setting, the system exhibits opposite be- Acknowledgments
haviors. It achieves a low recall (around 65%) but a high
precision (around 86%), meaning that the supervised system The authors wish to thank Steven Garfinkle from Depart-
can predict a name with a high accuracy but is not competent ment of History of Western Washington University, and Mr.
enough to identify much more than a simple majority of the Clinton Burkhart for their assistance and valuable input as a
names. This may be due to the fact that the features used are language expert in this work.
very good ones when they apply but their number and vari-
ety are insufficient to provide full coverage. Thus, we need References
a larger feature set.
Some of the 36 features in the current supervised set- Brewer, F.; Burkhart, C.; Houng, J.; Luo, L.; Riley, D.;
ting are selected from the high confident contextual rules Toner, Brandon. Liu, Y.; and Hearne, J. 2014. A preliminary
and spelling rules generated by the DL-Cotrain method. One study into named entity recognition in cuneiform tablets.
possible way to improve the current supervised learning re- In The third Pacific Northwest Regional Natural Language
sult is to include more features from the decision list of DL- Processing Workshop, 1–3.
Cotrain. Foxvog, D. 2014. An introduction to sumerian grammar.
539
Jaworski, W. 2008. Contents modeling of neo-sumerian ur
iii economic text corpus. In Proceedings of the 22nd Interna-
tional Conference on Computational Linguistics, 369–376.
Limet, H. 1960. L’Anthroponymie sumerienne dans les doc-
uments de la 3e dynastie d’Ur. Paris: Socit d’dition Les
Belles Lettres.
Luo, L.; Liu, Y.; Hearne, J.; and Burkhart, C. 2015. Unsuper-
vised sumerian personal name recognition. In Proceedings
of the Twenty-Eighth International Florida Artificial Intel-
ligence Research Society Conference, FLAIRS 2015, Holly-
wood, Florida. May 18-20, 2015., 193–198.
Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.;
Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss,
R.; Dubourg, V.; Vanderplas, J.; Passos, A.; Cournapeau, D.;
Brucher, M.; Perrot, M.; and Duchesnay, E. 2011. Scikit-
learn: Machine learning in Python. Journal of Machine
Learning Research 12:2825–2830.
Sahala, A. 2012. Notation in sumerian transliteration. Tech-
nical report, University of Helsinki.
Tablan, V.; Peters, W.; Maynard, D.; and Cunningham, H.
2006. Creating tools for morphological analysis of sume-
rian. In Proceedings of the Fifth International Conference
on Language Resources and Evaluation, 1762–1765.
Weibull, N. 2004. A historical survey of number systems.
Technical report, Chalmers University of Technology.
Widell, M. 2008. The ur iii metal loans from ur. In Garfinkle,
S., and Cale Johnson, J., eds., The Growth of an Early State
in Mesopotamia: Studies in Ur III Administration. Madrid:
Consejo Superior de Investigationes Cientficas. 207–223.
540