Using Free Text Mining Software to Analyze Chat Reference Transcripts: A Pilot Study

Objectives: Several recent publications have demonstrated the value of mining chat reference transcripts using complex and costly text mining software and human-labor-intensive methods to conduct quantitative and qualitative analysis. Voyant is a free online suite of text mining tools. Can Voyant be used in place of other text mining software and methods to identify common themes discussed by users and librarians in chat services?

Methods: Transcripts from the chat reference service from an academic medical library covering July 2015- February 2019 were gathered, cleaned, and uploaded into Voyant. Based on prior qualitative research examples, authors developed themes expected to be found and lists of common phrases, keywords, and concepts correlating within themes. Various tools within Voyant were used to analyze data for quantitative and thematic properties. The outputs from each tool were compared and evaluated on time and effort spent to conduct the analysis; value/relevance of data for quality assurance, evaluation, and training development for reference services; and comparative value to other Voyant tools. We anticipate that different tools within Voyant will be more valuable when librarians have different kinds of questions and expected uses for chat reference transcripts, covering timeliness, audience, depth/sophistication (research question versus ready reference), focus of question, and training needs.

Results: Using the tools within Voyant is relatively simple, but requires data preparation and planning to be successful. The trends, terms, and collocates tools within Voyant are useful for quantitative data such as timeliness, length, and volume. The collocates, contexts, and phrases tools, along with additional analysis, are most useful for qualitative or thematic data such as differentiation between ready reference and research questions, further analysis of research needs, focus of questions, and training needs. No tools were able to successfully identify types of patrons.

Conclusions: Preparing data for Voyant tools and thinking about how to analyze the data is tedious, but using the tools is not. Output from Voyant tools can be used along with other strategies to evaluate the success of chat reference services, drive changes in services, and provide content for training for reference staff. The information could also be used to support decisions relating to public services in outreach, communications, and instruction.