Extension of Language Model to Solve Inconsistency, Incompleteness, and Short Query in the Collection of Cultural Heritage

Authors

  • K. L. Tan Department of Computer Science, Faculty of Art, Computing and Creative Industry,Sultan Idris Education University, Tanjong Malim, Perak Darul Ridzuan 35900, Malaysia.
  • C. K Lim Department of Computer Science, Faculty of Art, Computing and Creative Industry,Sultan Idris Education University, Tanjong Malim, Perak Darul Ridzuan 35900, Malaysia.

Keywords:

Cultural heritage, Information retrieval, Language Model,

Abstract

With the explosive growth of online information such as email messages, news articles, and scientific literature, many institutions and museums are converting their cultural collections from physical data to digital format. However, this conversion results in the issues of inconsistency and incompleteness. Besides, the usage of inaccurate keywords also results in short query problem. Most of the time, the inconsistency and incompleteness are caused by the aggregation fault in annotating a document itself while the short query problem is caused by naive user who has prior knowledge and experience in cultural heritage domain. In this paper, we presented an approach to solve the problem of inconsistency, incompleteness and short query by incorporating the Term Similarity Matrix into the Language Model. Our approach is tested on the Cultural Heritage in CLEF (CHiC) collection, which consists of short queries and documents. The results show that the proposed approach is effective and has improved the accuracy in retrieval time.

References

G. Amati and C. J. van Rijsbergen. Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Transaction on Information Systems, 20(4):357–389, 2002.

M. Bendersky and W. B. Croft. Discovering key concepts in verbose queries. International ACM SIGIR Conference, pp. 491–498, 2008.

A. Berger and J. Lafferty. Information retrieval as statistical translation. International ACM SIGIR Conference, pp. 222–229, 1999.

D. Carmel, N. Zwerdling, and S. Yogev. Entity oriented search and exploration for cultural heritage collections: the EU-Cultura project. International conference companion on World Wide Web, pp. 227–230, 2012.

J.-P. Chevallet. X-iota: An open xml framework for ir experimentation. Lecture Notes in Computer Science, vol. 8, pp. 263–280, 2005.

P. Clough, N. Ford, and M. Stevenson. Personalizing access to cultural heritage collections using pathways. International Workshop on Personalized Access to Cultural Heritage, 2011.

F. Crestani. Exploiting the similarity of non-matching terms at retrieval time. Journal of Information Retrieval, 2:25–45, 2000.

S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6):391–407, 1990.

Y. Jing and W. B. Croft. An association thesaurus for information retrieval. In RIAO Conference Proceedings, pages 146–160, 1994.

M. Karimzadehgan and C. Zhai. Estimation of statistical translation models based on mutual information for ad hoc information retrieval. International ACM SIGIR conference, pp. 323–330, 2010.

R. Krovetz. Viewing morphology as an inference process, pp. 191–202, 1993.

V. Lavrenko and W. B. Croft. Relevance based language models. International ACM SIGIR, pp. 120–127, 2001.

C. D. Manning, P. Raghavan, and H. Schutze. Introduction to Information Retrieval. Cambridge University Press, New York, 2008.

B. Markines, C. Cattuto, F. Menczer, D. Benz, A. Hotho, and G. Stumme. Evaluating similarity measures for emergent semantics of social tagging. International Conference on World Wide Web, pp. 641–650, 2009.

H. J. Peat and P. Willett. The limitations of term co-occurrence data for query expansion in document retrieval systems. Journal of the American Society for Information Science, 42:378–383, 1991.

F. Peng, N. Ahmed, X. Li, and Y. Lu. Context sensitive stemming for web search. International ACM SIGIR conference, pp. 639–646, 2007.

J. M. Ponte and W. B. Croft. A language modeling approach to information retrieval. International ACM SIGIR conference, pp. 275–281, 1998.

M. F. Porter. Readings in information retrieval: An algorithm for suffix stripping, pp. 313–316. Morgan Kaufmann Publishers Inc., 1997.

C. J. V. Rijsbergen. Information Retrieval. Butterworth-Heinemann, Newton, MA, USA, 2nd edition, 1979.

S. E. Robertson. Overview of the okapi projects. Journal of Documentation, 53(1):3–7, 1997.

G. Salton, editor. The SMART Retrieval System Experiments in Automatic Document Processing. Prentice Hall, 1971.

G. Salton. The smart project in automatic document retrieval. International ACM SIGIR conference, pp. 356-358, 1991.

G. Salton and C. Buckley. Term-weighting approaches in automatic text retrieval. In Information Processing And Management, pages 513–523, 1988.

G. Srinivas, N. Tandon, and V. Varma. A weighted tag similarity measure based on a collaborative weight model. International workshop on Search and mining user-generated contents, pp. 79–86, 2010.

J. Xu and W. B. Croft. Query expansion using local and global document analysis. International ACM SIGIR Conference, pp. 4-11, 1996.

L. Zhao and J. Callan. Automatic term mismatch diagnosis for selective query expansion. International ACM SIGIR conference, pp. 515-524, 2012

S. Yogev, H. Roitman, D. Carmel, N. Zwerdling. Towards expressive exploratory search over entity-relationship data. International conference companion on World Wide Web, pp. 83-92, 2012.

C. Zhai, J. Lafferty. A study of smoothing methods for language models applied to information retrieval. ACM Transaction Information System, pp. 179-214, 2004.

Downloads

Published

2017-09-15

How to Cite

Tan, K. L., & Lim, C. K. (2017). Extension of Language Model to Solve Inconsistency, Incompleteness, and Short Query in the Collection of Cultural Heritage. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 9(2-12), 119–123. Retrieved from https://jtec.utem.edu.my/jtec/article/view/2780

Issue

Vol. 9 No. 2-12: The Challenge of Computer and Engineering in Building Sustainable Service Systems II

Section

Articles

License

TRANSFER OF COPYRIGHT AGREEMENT

The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):

  1. All proprietary right other than copyright, such as patent rights
  2. The right to make further copies of all or part of the published article for my use in classroom teaching
  3. The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
  4. The right to make copies of the published work for internal distribution within the institution that employs me

I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.