Transforming Semi-Structured Indigenous Dictionary into Machine-Readable Dictionary
Keywords:
Machine-Readable Dictionary, SemiStructured Data, Indigenous Dictionary, Regular Expressions, Python,Abstract
Creating a machine-readable dictionary for an indigenous language is not an easy process and thus, transforming an existing indigenous dictionary into a machinereadable dictionary is one approach to speed up the process. This paper presents the sequential transformation of two bilingual Sarawak indigenous dictionaries, Melanau-MukahMalay and Iban-Malay dictionaries, from their initial semistructured form into their structured representation. The transformation makes use of an OCR to convert the original PDF format of the dictionaries into HTML files, which is then analysed by the Python HTMLParser to extract only the content of the dictionaries. The extracted content is saved in plain text file. To understand the original structure of each dictionary, the textual units in the plain text file are converted into generic symbols. The observation of the collocations of the generic symbols yields to the writing of regular expressions that can delimit each dictionary element. The result is a machinereadable dictionary stored in comma-separated values format. The inspection of each column in the comma-separated values file indicates that the written regular expressions offer a good coverage of the different dictionary elements present in the studied dictionaries. Therefore, the proposed sequential transformation is efficient in accomplishing the conversion of a semi-structured indigenous dictionary into a structured machine-readable dictionary.References
Bakliwal, V. V. Devadath, and C. V. Jawahar, "Align Me: A framework to generate parallel porpus psing OCRs & bilingual dictionaries," Proceedings of the 6th Workshop on South and Southeast Asian Natural Language Processing, Osaka, Japan, pp. 183-187, 2016.
E. H. Klapicová, "Composition of the entry in a bilingual dictionary," SKASE Journal of Theoretical Linguistics, vol. 2, no. 3, pp. 57-74, 2005.
T. Ejarvec, and N. Ide, "Markup enhancement: Converting CEE dictionaries into TEI, and beyond," In F. Kiefer, G. Kiss, & J. Pajzs (Ed.), Papers in Computational Lexicography COMPLEX'99 Linguistics Institute, Budapest: Hungarian Academy of Sciences, pp. 211-217, 1999.
J. Mayfield and P. McNamee, "Converting On-Line Bilingual Dictionaries from Human-Readable to Machine-Readable Form," SIGIR’02, Tampere, Finland: ACM, 2002.
E. Balabanova, and K. Ivanova, "Creating a machine-readable version of Bulgarian valence dictionary: (A case study of CLaRK system application)," First Workshop on Treebanks and Linguistic Theories (TLT2002), Sozopol, Bulgaria, pp. 1-12, 2002.
J. Gracia, M. Villegas, A. Gómez-Pérez, and N. Bel, "The apertium bilingual dictionaries on the web of data," Semantic Web - Interoperability, Usability, Applicability, pp. 1-10, 2017.
M. Maxwell and A. Bills, "Endangered data for endangered languages: Digitizing print dictionaries," Proceedings of the 2nd Workshop on the Use of Computational Methods in the Study of Endangered Languages, Honolulu, Hawai‘i, USA: Association for Computational Linguistics, pp. 85-91, 2017.
E.-L. Ng, A. W. Yeo, and B. Ranaivo-Malançon, "Identification of closely related indigenous languages: An orthographic approach," Proceedings of the International Conference on Asian Language Processing (IALP), Singapore: IEEE, pp. 230-235, 2009.
B. Karagol-Ayan, D. Doermann, and A. Weinbe, "Adaptive transformation-based learning for improving dictionary tagging," 11th Conference of the European Chapter of the Association for Computer Linguistics, Trento, Italy: EACL, The Association for Computer Linguistics, pp. 257-264, 2006.
A. A. Krizhanovsky, "Transformation of Wiktionary entry structure into tables and relations in a relational database schema," Computing Research Repository (CoRR), 2010. Retrieved June 2017, from https://arxiv.org/abs/1011.1368
M. Padró, N. Bel, and S. Necsulescu, "Towards the fully automatic merging of lexical resources: A step forward," LREC 2012 Workshop on Language Resource, Istanbul, Turkey: European Language Resources Association, pp. 8-14, 2012.
Downloads
Published
How to Cite
Issue
Section
License
TRANSFER OF COPYRIGHT AGREEMENT
The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):
- All proprietary right other than copyright, such as patent rights
- The right to make further copies of all or part of the published article for my use in classroom teaching
- The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
- The right to make copies of the published work for internal distribution within the institution that employs me
I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.
