Recall, Precision, and F-Measure for Evaluating Information Retrieval System in Electronic Document Management Systems (EDMS)

Annisa Fajriyati Sadeli; Ike Iswary  Lawanda

doi:10.24252/kah.v11i2a8

Annisa Fajriyati Sadeli Departemen Ilmu Perpustakaan dan Infromasi, Fakultas Ilmu Pengetahuan Budaya, Universitas Indonesia
(ID)
Ike Iswary Lawanda Departemen Ilmu Perpustakaan dan Informasi, Fakultas Ilmu Pengetahuan Budaya, Universitas Indonesia
(ID)

DOI: https://doi.org/10.24252/kah.v11i2a8

Keywords: Information retrieval, recall, precision, F-measure

Abstract

The Legal Division of PT. ABC has implemented EDMS using the Integrated Legal Document Management Application, Centralized Legal Administration System (CLAS), to make the management of documents more accessible. The urgency is the system's accuracy in retrieving data or document input. The study employed a quantitative method with an experimental approach. Data collection techniques were observation, application search tests, documentation, and literature. Data analysis regarding the recall, precision, and F-measure calculation formulas will be compared with the standard information retrieval system effectiveness. The calculation results show that the average recall value for the search equals 1 or 100%. For the precision value that uses quotation marks, it is equal to 0.9713, while for searching without quotation marks, it is equal to 0.3920. From these results, the precision value is lower than the recall value. The precision value of 0.9713 is in the range of 0.68–1.00, which is placed as high precision, meaning that information retrieval is effective. The F-measure value is 0.98, close to 1, so the tracing system is sound. CLAS is quite effective in retrieving information according to user needs.

Downloads

Download data is not yet available.

References

Alshibly, H., Chiong, R., & Bao, Y. (2016). Investigating the Critical Success Factors for Implementing Electronic Document Management Systems in Governments: Evidence From Jordan. Information Systems Management, 33(4), 287–301. https://doi.org/10.1080/10580530.2016.1220213

Bama, Ss., & Ahmed, Msi. (2015). A Survey on Performance Evaluation Measures for Information Retrieval System. International Research Journal of Engineering and Technology. www.irjet.net

Bramer, W. M., Giustini, D., & Kramer, B. M. R. (2016). Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: A prospective study. Systematic Reviews, 5(1). https://doi.org/10.1186/s13643-016-0215-7

Ceri, S., Bozzon, A., Brambilla, M., Della Valle, E., Fraternali, P., & Quarteroni, S. (2013). An Introduction to Information Retrieval. In Web Information Retrieval (pp. 3–11). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-39314-3_1

Faila Saufa, A., & Wahyu, J. (2017). Evaluasi Sistem Temu Kembali Informasi Koha di Perpustakaan Universitas Muhammadiyah Surakarta (UMS). Khizanah Al-Hikmah : Jurnal Ilmu Perpustakaan, Informasi, Dan Kearsipan, 5(2), 140–151. https://doi.org/10.24252/kah.v5i2a1

Goker, A., & Davies, J. (2009). Information Retrieval: Searching in the 21st century. Laserwords Private Ltd.

Grossman, D. A., & Frieder, O. (2004). Information Retrieval. Springer Netherlands. https://doi.org/10.1007/978-1-4020-3005-5

Koehrsen, W. (2023, March 8). Precision and Recall: How to Evaluate Your Classification Model. Builtin.Com.

Lancaster, F. W. (1979). Information Retrieval Systems: Characteristics, Testing, and Evaluation (2nd ed.). John Willey.

Manning, C. D., Raghavan, Prabhakar., & Schütze, Hinrich. (2008). Introduction to information retrieval. Cambridge University Press.

Martin, :, & Nilawati, L. (2019). Recall dan Precision Pada Sistem Temu Kembali Informasi Online Public Access Catalogue (OPAC) di Perpustakaan. Paradigma : Jurnal Komputer Dan Informatika Universitas Bina Sarana Informatika, 21(1), 77–84. https://doi.org/10.31294/p.v20i2

Miao, J., & Zhu, W. (2022). Precision–recall curve (PRC) classification trees. Evolutionary Intelligence, 15(3), 1545–1569. https://doi.org/10.1007/s12065-021-00565-2

Musvida, E. (2017). Recall Dan Precision pada Sistem Temu Kembali Informasi Bidang Ilmu Fikih di Perpustakaan UIN Ar-Raniry Banda Aceh.

Song, J. J. S., Lee, W., & Afshar, J. (2019). An effective High Recall Retrieval method. Data and Knowledge Engineering, 123. https://doi.org/10.1016/j.datak.2017.07.006

Sukma, A., Zaman, B., & Purwanti, E. (2015). Klasifikafi Dokumen Temu Kembali Informasi dengan K-Nearest Neghbour Information Retrieval Document Classified with K-Nearest Neighbor. In RECORD AND LIBRARY JOURNAL (Vol. 1, Issue 2).

Zhang, E., & Zhang, Y. (2018). F-Measure. In: Liu,L., Ozsu,M.T. In L. Liu & M. T. Ozsu (Eds.), Encyclopedia of Database System. Springer. https://doi.org/https://remote-lib.ui.ac.id:2075/10.1007/978-1-4614-8625-9_483