Extraction of association rules in a diabetic dataset using parallel FP-growth algorithm under apache spark

Abstract

This research paper focuses on enhancing the frequent pattern growth (FP-growth) algorithm, an advanced version of the Apriori algorithm, by employing a parallelization approach using the Apache Spark framework. Association rule mining, particularly in healthcare data for predicting and diagnosing diabetes, necessitates the handling of large datasets which traditional methods may not process efficiently. Our method improves the FP-growth algorithm’s scalability and processing efficiency by leveraging the distributed computing capabilities of apache spark. We conducted a comprehensive analysis of diabetes data, focusing on extracting frequent itemsets and association rules to predict diabetes onset. The results demonstrate that our parallelized FP-growth (PFP-growth) algorithm significantly enhances prediction accuracy and processing speed, offering substantial improvements over traditional methods. These findings provide valuable insights into disease progression and management, suggesting a scalable solution for large-scale data environments in healthcare analytics.