Towards Citation Justice: Ensuring an Inclusive Search and Literature Retrieval: A Western Librarian's Perspective

Floor Agnes Andrea Ruiter

doi:10.18357/kula.311

Authors

Floor Agnes Andrea Ruiter Maastricht University Library, Maastricht University https://orcid.org/0000-0002-2192-2181

DOI:

https://doi.org/10.18357/kula.311

Keywords:

inclusive literature retrieval, search strategy bias, citation inequity, systematic bias

Abstract

The main sources of citation injustice are the collective biases in the scientific community, including literature retrieval bias, which has direct effects on citation inequality. In this commentary I focus on index bias in literature databases, inherent/unconscious bias during search strategy development, and systematic bias of controlled vocabularies. Exemplar literature search strategies and/or retrieval analyses comparing Web of Science and OpenAlex are used to demonstrate these biases. Moreover, this commentary offers steps to consider during literature search preparation, search query building, and the search itself, which can lead to a more inclusive representation of the literature in the topic of interest and reduce citation inequity.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

Alonso-Álvarez, Patricia. 2025. “A Small Step Towards the Epistemic Decentralization of Science: A Dataset of Journals and Publications Indexed in African Journals Online.” Journal of Data and Information Science. https://doi.org/10.2478/jdis-2025-0034.

Amar-Zifkin, Alexandre, Taline Ekmekjian, Virginie Paquet, and Tara Landry. 2025. “Algorithmic Indexing in MEDLINE Frequently Overlooks Important Concepts and May Compromise Literature Search Results.” Journal of the Medical Library Association 113 (1): 39–48. https://doi.org/10.5195/jmla.2025.1936.

Beukeboom, Camiel J. 2014. “Mechanisms of Linguistic Bias: How Words Reflect and Maintain Stereotypic Expectancies.” In Social Cognition and Communication, edited by Joseph P. Forgas, Orsolya Vincze, and János László. Psychology Press. https://doi.org/10.4324/9780203744628.

Bullard, Julia, Brian Watson, and Caitlin Purdome. 2022. “Misrepresentation in the Surrogate: Author Critiques of ‘Indians of North America’ Subject Headings.” Cataloging & Classification Quarterly 60 (6–7): 599–619. https://doi.org/10.1080/01639374.2022.2090039.

Céspedes, Lucía, Diego Kozlowski, Carolina Pradier, Maxime Holmberg Sainte-Marie, Natsumi Solange Shokida, Pierre Benz, Constance Poitras, Anton Boudreau Ninkov, Saeideh Ebrahimy, Philips Ayeni, Sarra Filali, Bing Li, and Vincent Larivière. 2025. “Evaluating the Linguistic Coverage of OpenAlex: An Assessment of Metadata Accuracy and Completeness.” Journal of the Association for Information Science and Technology 76 (6): 884–95. https://doi.org/10.1002/asi.24979.

Chatterjee, Paula, and Rachel M. Werner. 2021. “Gender Disparity in Citations in High-Impact Journal Articles.” JAMA Network Open 4 (7): e2114509. https://doi.org/10.1001/jamanetworkopen. 2021.14509.

Chilisa, Bagele. 2020. Indigenous Research Methodologies. 2nd ed. Sage. https://uk.sagepub.com/en-gb/eur/indigenous-research-methodologies/book241776#description.

Didier, Nicolas. 2025. “Decolonizing Public Administration in Latin America: A Systematic Literature Review of Trending Discussions in the Region.” Public Administration and Development: 1–18. https://doi.org/10.1002/pad.2092.

Drabinski, Emily. 2013. “Queering the Catalog: Queer Theory and the Politics of Correction.” The Library Quarterly: Information, Community, Policy 83 (2): 94–111. https://doi.org/10.1086/669547.

Epstein, Robert, and Ronald E. Robertson. 2015. “The Search Engine Manipulation Effect (SEME) and Its Possible Impact on the Outcomes of Elections.” Proceedings of the National Academy of Sciences of the United States of America 112 (33): E4512–E4521. https://doi.org/10.1073/pnas.1419828112.

Freelon, Deen, Meredith L. Pruden, Kirsten A. Eddy, and Rachel Kuo. 2023. “Inequities of Race, Place, and Gender Among the Communication Citation Elite, 2000–2019.” Journal of Communication 73 (4): 356–67. https://doi.org/10.1093/joc/jqad002.

Glickman, Moshe, and Tali Sharot. 2025. “How Human–AI Feedback Loops Alter Human Perceptual, Emotional and Social Judgements.” Nature Human Behaviour 9 (2): 345–59. https://doi.org/10.1038/s41562-024-02077-2.

Grames, Eliza M., Andrew N. Stillman, Morgan W. Tingley, and Chris S. Elphick. 2019. “An Automated Approach to Identifying Search Terms for Systematic Reviews Using Keyword Co-Occurrence Networks.” Methods in Ecology and Evolution 10 (10): 1645–54. https://doi.org/10.1111/2041-210X.13268.

Gültzow, Thomas, Efrat Neter, and Hanne M. L. Zimmermann. 2023. “Making Research Look Like the World Looks: Introducing the ‘Inclusivity & Diversity Add-On for Preregistration Forms’ Developed During an EHPS2022 Pre-Conference Workshop.” The European Health Psychologist 22 (6), 1063–69. https://ehps.net/ehp/index.php/contents/article/view/3436/.

Hadfield, Ruth M. 2020. “Delay and Bias in PubMed Medical Subject Heading (MeSH®) Indexing of Respiratory Journals.” Preprint, medRxiv, October 4. https://doi.org/10.1101/2020.10.01.20205476.

Harding, Louise, Caterina J. Marra, and Judy Illes. 2021. “Establishing a Comprehensive Search Strategy for Indigenous Health Literature Reviews.” Systematic Reviews 10 (1): 115. https://doi.org/10.1186/s13643-021-01664-y.

Holstrom, Chris. 2021. “The Sears List of Subject Headings: Social and Cultural Dimensions in Historical, Theoretical, and Design Contexts.” Proceedings from North American Symposium on Knowledge Organization 8: 1–11. https://doi.org/10.7152/nasko.v8i1.15865.

Holvey, Kate. 2025. “How Maastricht Is Advocating for Inclusive Research and Decolonisation.” Overton (blog). https://www.overton.io/blog/how-maastricht-is-advocating-for-inclusive-research-and-decolonisation. Archived at: https://perma.cc/G58V-ZAJ8.

Hovy, Dirk, and Shrimai Prabhumoye. 2021. “Five Sources of Bias in Natural Language Processing.” Language and Linguistics Compass 15 (8): e12432. https://doi.org/10.1111/lnc3.12432.

Howard, Sara A., and Steven A. Knowlton. 2018. “Browsing Through Bias: The Library of Congress Classification and Subject Headings for African American Studies and LGBTQIA Studies.” Library Trends 67 (1): 74–88. https://doi.org/10.1353/lib.2018.0026.

Jaeger-McEnroe, Emily. 2025. “Rethinking Authority and Bias: Modifying the CRAAP Test to Promote Critical Thinking about Marginalized Information.” College & Research Libraries News 86 (1): 12–17. https://doi.org/10.5860/crln.86.1.12.

Kacperski, Celina, Mona Bielig, Mykola Makhortykh, Maryna Sydorova, and Roberto Ulloa. 2024. “Examining Bias Perpetuation in Academic Search Engines: An Algorithm Audit of Google and Semantic Scholar.” First Monday 29 (11). https://doi.org/10.5210/fm.v29i11.13730.

Kugley, Shannon, Anne Wade, James Thomas, Quenby Mahood, Anne-Marie Klint Jørgensen, Karianne Hammerstrøm, and Nila Sathe. 2017. “Searching for Studies: A Guide to Information Retrieval for Campbell Systematic Reviews.” Campbell Systematic Reviews 13 (1): 1–73. https://doi.org/10.4073/cmg.2016.1.

Lin, Cong, Yuxin Gao, Na Ta, Kaiyu Li, and Hongyao Fu. 2023. “Trapped in the Search Box: An Examination of Algorithmic Bias in Search Engine Autocomplete Predictions.” Telematics and Informatics 85: 102068. https://doi.org/10.1016/j.tele.2023.102068.

Maddi, Abdelghani, Marion Maisonobe, and Chérifa Boukacem-Zeghmouri. 2025. “Geographical and Disciplinary Coverage of Open Access Journals: OpenAlex, Scopus, and WoS.” PLOS ONE 20 (4): e0320347. https://doi.org/10.1371/journal.pone.0320347.

Martin-Booran Mirraboopa, Karen. 2003. “Ways of Knowing, Being and Doing: A Theoretical Framework and Methods for Indigenous and Indigenist Re-search.” Journal of Australian Studies 27 (76): 203–14. https://doi.org/10.1080/14443050309387838.

McGowan, Jessie, Margaret Sampson, Douglas M. Salzwedel, Elise Cogo, Vicki Foerster, and Carol Lefebvre. 2016. “PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement.” Journal of Clinical Epidemiology 75: 40–46. https://doi.org/10.1016/j.jclinepi.2016.01.021.

Medical Library Association. 2022. “Open Letter to NLM Regarding MeSH Term Changes.” Medical Library Association. https://www.mlanet.org/article/open-letter-to-nlm-regarding-mesh-term-changes/. Archived at: https://perma.cc/9K6F-CY8A.

Mehta, Noor, Rhiannon Callahan, Rachel Greenberg, Kaitlin Kerr, Miranda Melson, K. J. Rawson, Cailin Roles, Eamon Schlotterback, and Cecilia Wolfe. n.d. “Global Terms.” Digital Transgender Archive. https://www.digitaltransgenderarchive.net/learn/terms. Archived at: https://perma.cc/2FB3-MN5R.

Moura, Maria Aparecida. 2024. “Information and Code Biases: Social Differentiation, Intersectionality and Decoloniality in Knowledge Organization Systems.” Knowledge Organization 51 (7): 514–20. https://doi.org/10.5771/0943-7444-2024-7-514.

Narragunnawali: Reconciliation in Education. n.d. “Terminology Guide: A Guide to Using Respectful and Inclusive Language and Terminology.” https://www.narragunnawali.org.au/about/terminology-guide.

O’Keefe, Hannah, Judith Rankin, Sheila A. Wallace, and Fiona Beyer. 2023. “Investigation of Text-Mining Methodologies to Aid the Construction of Search Strategies in Systematic Reviews of Diagnostic Test Accuracy—A Case Study.” Research Synthesis Methods 14 (1): 79–98. https://doi.org/10.1002/jrsm.1593.

Overton. 2025. “Guide: How to Use Overton to Decolonise Your Research and Teaching.” Overton.io. https://www.overton.io/how-to-use-overton-to-decolonise-your-research-and-teaching. Archived at: https://perma.cc/C6LX-4P6G.

Paez, Arsenio. 2017. “Gray Literature: An Important Resource in Systematic Reviews.” Journal of Evidence-Based Medicine 10 (3): 233–40. https://doi.org/10.1111/jebm.12266.

Peterson‐Salahuddin, Chelsea. 2024. “From Information Access to Production: New Perspectives on Addressing Information Inequity in Our Digital Information Ecosystem.” Journal of the Association for Information Science & Technology 75 (10): 1134–51. https://doi.org/10.1002/asi.24879.

Pieper, Dawid, and Livia Puljak. 2021. “Language Restrictions in Systematic Reviews Should Not Be Imposed in the Search Strategy but in the Eligibility Criteria if Necessary.” Journal of Clinical Epidemiology 132:146–47. https://doi.org/10.1016/j.jclinepi.2020.12.027.

Pranckutė, Raminta. 2021. “Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World.” Publications 9 (1): 12. https://doi.org/10.3390/publications9010012.

Priem, Jason, Heather Piwowar, and Richard Orr. 2022. “OpenAlex: A Fully-Open Index of Scholarly Works, Authors, Venues, Institutions, and Concepts.” Preprint, v2, arXiv, June 17. https://doi.org/10.48550/arXiv.2205.01833.

Scopus 2025. “Content Policy and Selection” Elsevier. https://www.elsevier.com/products/scopus/content/content-policy-and-selection.

Simard, Marc-André, Isabel Basson, Madelaine Hare, Vincent Larivière, and Philippe Mongeon. 2025. “Examining the Geographic and Linguistic Coverage of Gold and Diamond Open Access Journals in OpenAlex, Scopus, and Web of Science.” Quantitative Science Studies 6: 1–29. https://doi.org/10.1162/QSS.a.1.

Stern, Cindy, and Jos Kleijnen. 2020. “Language Bias in Systematic Reviews: You Only Get Out What You Put in.” JBI Evidence Synthesis 18 (9): 1818–19. https://doi.org/10.11124/JBIES-20-00361.

Tennant, Jonathan P. 2020. “Web of Science and Scopus Are Not Global Databases of Knowledge.” European Science Editing 46: e51987. https://doi.org/10.3897/ese.2020.e51987.

Tynan, Lauren, and Michelle Bishop. 2022. “Decolonizing the Literature Review: A Relational Approach.” Qualitative Inquiry 29 (3–4): 498–508. https://doi.org/10.1177/10778004221101594.

Udoh, Emmanuel Sebastian, Xiaojun Yuan, and Abebe Rorissa. 2024. “A Framework for Defining Algorithmic Fairness in the Context of Information Access.” Proceedings of the Association for Information Science and Technology 61 (1): 667–72. https://doi.org/10.1002/pra2.1077.

Vong, Silvia. 2021. “From Dispositions to Positionality: Addressing Dispositions of the Student Researcher in the ACRL Framework.” The Journal of Academic Librarianship 47 (6): 102457. https://doi.org/10.1016/j.acalib.2021.102457.

Weeks, Thomas C., and Melissa E. Johnson. 2024. “Using Positionality to Address Student Bias in Information Seeking.” Proceedings of the Association for Information Science and Technology 61 (1): 1126–28. https://doi.org/10.1002/pra2.1206.

Wijnhoven, Fons, and Jeanna van Haren. 2021. “Search Engine Gender Bias.” Frontiers in Big Data 4: 622106. https://doi.org/10.3389/fdata.2021.622106.

WoS 2025. “Journal Evaluation Process and Selection Criteria.” Clarivate. https://clarivate.com/academia-government/scientific-and-academic-research/research-discovery-and-referencing/web-of-science/web-of-science-core-collection/editorial-selection-process/journal-evaluation-process-selection-criteria/.

Yunkaporta, Tyson, and Donna Moodie. 2021. “Thought Ritual: An Indigenous Data Analysis Method for Research.” In Indigenous Knowledges: Privileging Our Voices, edited by Tarquam McKenna, Donna Moodie, and Pat Onesta. Brill. https://doi.org/10.1163/9789004461642_006.