A multi-expert approach to content-based image retrieval using feature fusion and late re-ranking

Abstract

As digital data rapidly grows, content-based image retrieval (CBIR) has become important for optimizing collections of visual data. This work proposes a retrieval framework which operates in two stages and improves accuracy by using systematic fusion of features. In the first stage, first-stage wide-scope descriptors called bag-of-visual-words (BoVW), scattering wavelet transform (SWT), discrete cosine transform (DCT), and principal component analysis (PCA) retrieve initial candidate images. The second stage undertakes detailed re-ordering of candidate images by implementing the local binary pattern (LBP), histogram of oriented gradients (HOG), and singular value decomposition (SVD) descriptors to re-evaluate similarity scores. Each individual descriptor returned results for mean average precision for the top 10 retrieved images (mAP, top-10) of between 0.63 and 0.79 and the fused framework achieved 0.88, which is evidence of the viability of complementary feature integration. These findings support the hypothesis that while multiple descriptors performed well and delivered high retrieval accuracy, hierarchical fusion of multiple handcrafted descriptors does not involve the computational costs associated with deep learning methods.