Developing Archival AI chatbots: risks, benefits, and future directions

AI chatbots are increasingly used for navigating and analysing the contents of major archives and collections. Applying Retrieval Augmented Generation to existing large language models, these tools draw on indexes of the relevant collections to answer, in natural language, users’ questions. This presentation brings together three professionals from history, digital humanities, the GLAM sector, and computer science to discuss the risks, benefits, and future directions of these tools. We explore how AI chatbots could be used to optimise the accessibility of collections in ways that maintain, or even enhance, research integrity.
The discussion draws on hands-on experiences with developing and using archival AI chatbots. Caio Mello reflects on the challenges and questions faced by his team in developing Barista, a chatbot designed as part of the ‘Impresso – Media Monitoring of the Past II’ project to transform prompts into search queries on the Impresso Web App. Yves Maurer brings insights from the design and launch of chat.eluxemburgensia.lu, a pioneering AI chatbot released in 2023 to help users search heritage documents digitized by the National Library of Luxembourg (BNL). Finola Finn complements these inputs with methodological and epistemological considerations, examining the implications of using chatbots for historical research and public engagement. Together, the speakers offer suggestions for how providers could carefully design, frame, and describe the intended use of archival chatbots. Throughout, the discussion centers around two key themes:
--Thoughtful UX design and communication
Given the current influx of very similar tools available for public use (from ChatGPT to customer service bots), a key issue in the development of archival AI chatbots is expectation management. Each chatbot works in a different way and was built for a different purpose, but these distinctions and their downstream effects are often not immediately clear to users. So, how can we best inform users of chatbots’ capabilities and limitations? Have users developed certain knowledge and habits through engagement with other chatbots that might need curbing or reorienting? What skills and information do we need to make available for users to be able to use chatbots effectively and responsibly?
--Maintaining rigorous research practices
In addition to being highly powerful finding aids, allowing users to locate and access relevant documents more easily, some archival AI chatbots are also presented as being capable of providing useful, automated answers to questions about the past. What are the epistemological differences between these two uses of archival chatbots (i.e. navigation vs analysis), and why is it important to demarcate them? What are the implications and possibilities for integrating these tools into the research practices of humanities scholars and their efforts to interrogate archives and collections?