Introduction to the IMLS-funded Project ORAL-AI

ORAL-AI: High Efficacy Content Representation and User Engagement of Oral Digital Collections with Multimodal LLMs", 2026–2029.

Project Overview

The University of North Texas, in collaboration with the University of Illinois Urbana-Champaign, seeks $490,397 for a 3-year Applied Research grant project that aims to expand meaningful public access to large-scale oral history collections by developing and evaluating a responsible, privacy-aware multimodal AI framework that supports cross-modal semantic retrieval, evidence-grounded question answering, and contextual narrative summarization.

By strengthening libraries’capacity to responsibly integrate emerging AI technologies into digital collections, the project advances NLGL Goal 2.1 (Promote broad public engagement) and Goal 3.2 (Promote access to museum and library collections). The project will pilot and evaluate the framework across three nationally significant collections—the Densho Digital Repository (Densho collection), the Library of Congress Civil Rights History Project (Civil Rights collection), and the Veterans History

Project (Veterans History collection)—representing diverse historical contexts, multimedia formats, and narrative complexity.

Building on the team’s prior empirical studies of oral history collections and large language model evaluation, the project will investigate three research questions:

• RQ1: What are the capabilities, limitations, and risks of current multimodal large language models in oral digital collections?

• RQ2: How can multimodal retrieval and generation systems be designed to mitigate identified limitations while preserving interpretive integrity and professional stewardship?

• RQ3: How does AI-assisted multimodal access affect user engagement, comprehension, and professional practice compared to traditional discovery systems?

  

Project Team

Faculty Member

PI Dr. Haihua Chen is Assistant Professor of Data Science and Director of the Intelligent Data Engineering and Analytics Lab at UNT. His research focuses on artificial intelligence, multimodal large language models, information retrieval, and data-centric AI for digital libraries and cultural heritage collections. He has published over 60 peer-reviewed articles in leading venues such as EMNLP, WWW, ACM MM, ICDM, IP&M, and JCDL. Dr. Chen has conducted empirical studies applying NLP and multimodal learning to oral history archives. He will lead overall project coordination and technical direction, including benchmark design, multimodal LLM evaluation, cross-modal alignment development, evidence-grounded generation strategies, and system implementation.

Co PI Dr. Jiangping Chen is Professor of Information Sciences at UIUC. Her expertise spans digital libraries,

multilingual and cross-language information retrieval, information services, and large language model applications. She has led multiple IMLS National Leadership Grants focused on digital collection access and multilingual information services. In ORAL-AI, she will lead user-centered evaluation design, development of professional guidelines and training materials, and alignment with archival standards. She will also co-lead national dissemination efforts.

Student Researcher

Mehri Sattari is a Ph.D. student in Information Science with a concentration in Data Science at the University of North Texas. She holds an M.Sc. in Industrial Engineering and has experience applying machine learning and data-driven methods to complex real-world problems. Her research interests include artificial intelligence, machine learning, multimodal AI, data quality, data-centric AI, and the reliability and evaluation of AI systems. Her prior research has included work on data quality and AI reliability in autonomous driving applications.