Utilizing AI/ML to Enhance Public Engagement with Large-scale and Multi-modal GLAM Collections

GLAM institutions serve as vital cultural and educational repositories, safeguarding humanity’s artistic, historical, and intellectual heritage. The collections held by GLAM institutions are not only vast but also diverse in format (spanning texts, images, audio, video, and other media) and complexity (usually semi-structure and unstructured). A national museum, for example, may have an archive that includes historical documents, high-resolution images of artworks, audio interviews, and even immersive virtual tours of past exhibitions. These large scale and multi-modal collections provide rich opportunities for research and public engagement, but they also pose significant organizational challenges. The ability of GLAM institutions to effectively manage and make accessible their rapidly growing digital collections has not kept pace with the scale of the collections themselves. However, existing studies either focus on a single data type or reply on manual processes or isolated tools that cannot seamlessly and efficiently organize, analyze, and interpret these large-scale and multi-modal resources. Therefore, there is a critical need within the GLAM sector: the growing expectation that institutions integrate AI/ML into their workflows.

Figure 1. Overall research framework

This project will create a unified AI/ML platform based on deep learning, multi-modal LLMs, and generative AI techniques that automates key processes such as metadata generation, content-based search, and cross-modal analysis. For example, the platform will leverage image recognition technologies to identify objects in historical photos, pair them with relevant textual descriptions from archival records, and even match them to spoken-word data from interviews. This type of cross-modal linking will enable users to navigate collections more dynamically, breaking down the silos between different media types and uncovering new connections that might otherwise remain hidden. In addition, by automating these labor-intensive tasks, the platform will allow GLAM professionals to focus more of their attention on curatorial and interpretive work. This combination of advanced ML techniques and practical applications will help GLAM institutions maximize the impact of their digital collections on both scholarly research and public engagement.

This project will serve diverse communities through developing an innovative and sustainable AI/ML-powered platform for streamlining the organization and analysis of large-scale and multi-modal GLAM collections. It will address the following research questions (RQs): RQ1: How can AI/ML tools be effectively integrated into GLAM institutions to automate metadata generation and enable cross-modal search? RQ2: What impact do AI/ML-driven interfaces have on the public's engagement with multi-modal digital collections? RQ3: How can AI/ML enhance accessibility for underrepresented communities and smaller institutions with limited resources? RQ4: What are the best practices for using AI/ML tools in GLAM institutions to balance efficiency, quality, and inclusivity?

Related papers:

Li, Y., Mandaloju, T., & Chen, H. (2025). Exploring Public Perceptions of Generative AI in Libraries: A Social Media Analysis of X Discussions. arXiv preprint arXiv:2507.07047.

Chen, H., Kim, J., Chen, J., & Sakata, A. (2024). Demystifying oral history with natural language processing and data analytics: a case study of the Densho digital collection. The Electronic Library, 42(4), 643-663.