Evaluating Generative AI for Domain-specific Applications

With the advancement of AI-powered language models, such as ChatGPT, generative AI (AIGC, a.k.a AI-generated content) are being discussed and used in every aspect of human society. Generative AI has strong ability to analyze and create text, images, code, and beyond. In addition to the fundamental techniques behind AIGC and the popular tasks of AIGC, it is also being widely applied to different fields and domains, including question answering, marketing, healthcare, gaming, music, drug discovery, etc. Researchers and scientists have also started discussing the general design principles for generative AI applications. However, it is undeniable that the generative AI revolution is still in the early stage, and it can change and improve with the potential for even greater future capability. This project aims to investigate: (1) Frameworks and methodologies of evaluating the capacity and limitations of Generative AI in specific domains. (2) The applications of Generative AI in high stake domains, such as legal, medical, cyber security, and others. (3) Generative AI for information extraction, information retrieval, questions answering, text summarization, and other similar tasks. (4) Data argumentation with Generative AI. (5) Quality assurance for Generative AI. (6) Methodologies of quality evaluation of AI generated content. (7) Generative AI for future teaching and learning.

Related papers

Ding, J., Chen, H., Kolapudi, S., Pobbathi, L., & Nguyen, H. (2023, October). Quality Evaluation of Summarization Models for Patent Documents. In 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS) (pp. 250-259). IEEE.

Nguyen, H., Chen, H., Maganti, R., Hossain, K. T., & Ding, J. (2023, July). Measurement and Identification of Informative Reviews for Automated Summarization. In 2023 IEEE International Conference on Artificial Intelligence Testing (AITest) (pp. 146-151). IEEE.

Zhao, H., Chen, H., & Yoon, H. J. (2023, July). Enhancing Text Classification Models with Generative AI-aided Data Augmentation. In 2023 IEEE International Conference on Artificial Intelligence Testing (AITest) (pp. 138-145). IEEE.