Automated gender identification vs. manual review: Gender assignment of editorial board members in Information and Library Science Journals
Main Article Content
Abstract
In recent years, computer programmes that automatically assign gender based on a person’s name and country of affiliation have been increasingly used in gender studies of authors of scientific publications. This study aims to compare the results generated by the automated genderize.io programme with those obtained through manual gender identification. To assess the accuracy of genderize.io, the gender of Editorial Board Members (EBMs) from 84 journals in the field of Information and Library Science was analysed. The comparison revealed discrepancies: genderize.io incorrectly classified 80 out of 1,419 men as women, and 124 out of 2,580 women were misidentified as men. Additionally, genderize.io classified the gender of 123 EBMs as unknown. While the manual method achieved a 99.15 percent accuracy rate, genderize.io had a slightly lower accuracy of 91.51 percent. There was, however, strong agreement between the two methodologies (Cohen's Kappa = 0.829, p < 0.001). Genderize.io exhibited a 7.71 percent inaccuracy rate, particularly underestimating the number of women. The study concludes that while automated software like genderize.io is effective for large-scale analyses and useful for library and information professionals, manual review is recommended for smaller studies to ensure higher accuracy.
Downloads
Article Details
It is a condition of publication that manuscripts submitted to the journal have not been published, accepted for publication, nor simultaneously submitted for publication elsewhere. By submitting a manuscript, the author(s) agree that copyright for the article is transferred to the publisher, if and when the manuscript is accepted for publication.
References
Batumalai, V., Kumar, B., & Sundaresan, P. (2023). Trends in gender of first and senior authors of articles published in JMIRO. Journal of Medical Imaging and Radiation Oncology, 67(2), 179-184. https://doi.org/10.1111/1754-9485.13492.
Gadek, L., Dammann, C., Savich, R., Mmuo-Oji, C., Barrera, L., Gallagher, P. G., & Machut, K. (2023). Gender analysis of Journal of Perinatology authorship during COVID-19. Journal of Perinatology, 43, 518-522. https://doi.org/10.1038/s41372-022-01551-x.
Gottlieb, M., Krzyzaniak, S. M, Mannix, A., Parsons, M., Mody, S., Kalantari, A., Ashraf, H., & Chan, T. M. (2021). Sex distribution of editorial board members among emergency medicine journals. Annals of Emergency Medicine, 77(1), 117-123. https://doi.org/10.1016/j.annemergmed.2020.03.027.
Hartzler, A. L., Leroy, G., Daurelle, B., Ochoa, M., Williamson, J., Cohen, D., & Stipelman, C. (2021). Comparison of women and men in biomedical informatics scientific dissemination: Retrospective observational case study of the AMIA Annual Symposium: 2017–2020. Journal of the American Medical Informatics Association (JAMIA), 28(9), 1928-1935. https://doi.org/10.1093/jamia/ocab097.
Heiberger, R. H. (2022). Applying machine learning in sociology: How to predict gender and reveal research preferences. KZfSS - Cologne Journal of Sociology and Social Psychology, 74(1), 383-406. https://doi.org/10.1007/s11577-022-00839-2.
Jadidi, M., Karimi, F., Lietz, H., & Wagner, C. (2017). Gender disparities in science? Dropout, productivity, collaborations and success of male and female computer scientists. Advances in Complex Systems, 21(03n04), 1750011. https://doi.org/10.1142/S0219525917500114.
Karimi, F., Wagner, C., Lemmerich, F., Jadidi, M., & Strohmaier, M. (2016). Inferring gender from names on the Web: A comparative evaluation of gender detection methods. Proceedings of the 25th International Conference Companion on World Wide Web (pp. 53-54). https://doi.org/10.1145/2872518.2889385.
Lund, B. D., & Shamsi, A. (2023). Women authorship in library and information science journals from 1981 to 2020: Is equitable representation being attained? Journal of Information Science, 49(5), 1335-1343. https://doi.org/10.1177/01655515211050026.
Narasimhan, V. L., Bhargavi, G. V., & Sultana, A. (2016). A subjective examination of implicit root stereotypes of STEM disciplines. In M. E. Auer, & K. S. Kim (Eds). Engineering Education for a Smart Society. World Engineering Education Forum & Global Engineering Deans Council 2016 (pp. 225-236). Springer Cham.
Nguyen, E., Robinson, R., & Hoover, R. M. (2021). Women as first authors in key pharmacy journals: Analysis by publication type. Journal of the American Pharmacists Association, 61(1), e26-e29. https://doi.org/10.1016/j.japh.2020.08.037.
Santamaría, L., & Mihaljević, H. (2018). Comparison and benchmark of name-to-gender inference services. PeerJ Computer Science, 4, e156. https://doi.org/10.7717/peerj-cs.156.
Sebo, P. (2021a). Performance of gender detection tools: A comparative study of name-to-gender inference services. Journal of the Medical Library Association (JMLA), 109(3), 414-421. https://doi.org/10.5195/jmla.2021.1185.
Sebo, P. (2021b). Using genderize.io to infer the gender of first names: How to improve the accuracy of the inference. Journal of the Medical Library Association (JMLA), 109(4), 609-612. https://doi.org/10.5195/jmla.2021.1252.
Sebo, P. (2022). How accurate are gender detection tools in predicting the gender for Chinese names? A study with 20,000 given names in Pinyin format. Journal of the Medical Library Association (JMLA), 110(2), 205-211. https://doi.org/10.5195/jmla.2022.1289.
Schlein, S. M., Pollock, N. W., Polukoff, N. E., Brown, A. B., Byrne, A., & Keyes, L. E. (2022). Gender equity in membership, leadership, and award recognition in the Wilderness Medical Society. Wilderness & Environmental Medicine, 33(3), 275-283. https://doi.org/10.1016/j.wem.2022.04.006.
Sixto-Costoya, A., Alonso-Arroyo, A., Castelló-Cogollo, L., Aleixandre-Benavent, R., & Valderrama-Zurián. J. C. (2022). Gender presence on the editorial boards of journals in the Women’s Studies subject category. Women’s Studies International Forum, 93, 102617. https://doi.org/10.1016/j.wsif.2022.102617.
Wais, K. (2016). Gender prediction methods based on first names with genderizeR. The R Journal, 8(1), 17-37. https://doi.org/10.32614/RJ-2016-002.
Waldhorn, I., Dekel, A., Morozov, A., Alon, E. S., Stave, D., Tsrooya, N. B., Schlosser, S., Markel, G., Bomze, D., & Meirson, T. (2022). Trends in women’s leadership of oncology clinical trials. Frontiers in Oncology, 12, 885275. https://doi.org/10.3389/fonc.2022.885275.
Wang, A., Dunlop, R., Allavena, R., & Palmieri, C. (2022). Gender representation on journal editorial boards in the field of veterinary sciences. Research in Veterinary Science, 148, 21-26. https://doi.org/10.1016/j.rvsc.2022.05.001.