Innovation & Research Practice

Paper: Research Abstract

Variation and Outliers in Search Results among MEDLINE-Based Platforms: A Longitudinal Study

Tuesday, May 7
2:35 PM - 2:50 PM
Room: Columbus AB (East Tower, Ballroom/Gold Level)

Objectives : This study is part of a larger project to understand the search result variations across MEDLINE-based platforms. With this current research, we seek to examine longitudinal changes in how five MEDLINE-based platforms respond to basic searches.

Methods : We examined five MEDLINE-based platforms by creating sets of queries for each platform and comparing search count results. The five platforms include: EBSCOhost/MEDLINE, Ovid/MEDLINE, ProQuest/MEDLINE, PubMed/MEDLINE, and Web of Science/MEDLINE. Our queries were organized into 29 sets of five queries each (one query per platform). The five queries in each set were designed to be equivalent and were modified only to match the syntax required by the each platform. Searches are being run monthly for 8 months. To answer our research question, we used a repeated measures approach based on the median absolute deviations (MAD) and the modified z-scores (m_i) to examine the search result counts in each query set. We centered our scores on the PubMed count in each set of queries in order to compare how result counts deviate from PubMed/MEDLINE.

Results : Initial results from one month of search results indicate substantial variance across platforms. Web of Science/MEDLINE results were most likely to deviate from PubMed/MEDLINE followed by ProQuest/MEDLINE. Both EBSCO/MEDLINE and Ovid/MEDLINE exhibited deviations, though Ovid/MEDLINE returned results most consistent with PubMed/MEDLINE. Preliminary longitudinal results seem to reflect initial findings, with continued and substantial variances across platforms.

Conclusions : It appears proprietary, platorm-specific interventions (possibly including data ingest workflows, term indexing and retrieval algorithms, and interface features) are affecting the retrieval of structurally similar searches across MEDLINE-based platforms. These variances have the potential to impact clinical decision-making, reproducibility studies, systematic reviews, or any research which depends on the consistent and predictable performance of searches.

Send Email for Tyler Nix

Robert M. Shapiro, II

Visiting Assistant Professor
University of Kentucky
University of Kentucky
Lexington, Kentucky


Send Email for Robert Shapiro

C. Sean Burns

Assistant Professor
School of Information Science
Lexington, Kentucky

Sean Burns is an assistant professor at the University of Kentucky's School of Information Science. He teaches in the library science and the information, communication, technology (ICT) programs.


Send Email for C. Sean Burns

Jeffrey T. Huber

Director and Professor
University of Kentucky
University of Kentucky
Lexington, KY

Jeffrey T. Huber is director and professor in the School of Library and Information Science at the University of Kentucky. He has more than twenty years of experience as a library and information science faculty members. Huber is the author of numerous publications and presentations related to health sciences librarianship.


Send Email for Jeffrey Huber


Variation and Outliers in Search Results among MEDLINE-Based Platforms: A Longitudinal Study

Audio Slides Video

Attendees who have favorited this

Please enter your access key

The asset you are trying to access is locked. Please enter your access key to unlock.

Send Email for Variation and Outliers in Search Results among MEDLINE-Based Platforms: A Longitudinal Study