IndoBART optimization for question answer generation system with longformer attention

Abstract

The Incorporation of Question Answering system holds immense potential for addressing Indonesia’s educational disparities between the abundance of high school students and the limited number of teachers in Indonesia. These studies aim to enhance the Question Answering System model tailored for the Indonesian language dataset through enhancements to the Indonesian IndoBART model. Improvement was done by incorporating Longformer’s sliding windows attention mechanism into the IndoBART model, it would increase model proficiency in managing extended sequence tasks such as question answering. The dataset used in this research was TyDiQA multilingual dataset and translated the SQuADv2 dataset. The evaluation indicates that the Longformer-IndoBART model outperforms its predecessor on the TyDiQA dataset, showcasing an average 26% enhancement across F1, Exact Match, BLEU, and ROUGE metrics. Nevertheless, it experienced a minor setback on the SQuAD v2 dataset, leading to an average decrease of 0.6% across all metrics.