MIVPG and Instance Correlation: Enhanced Multi-Instance Learning

Table of Links

Abstract and 1 Introduction

2. Related Work

   2.1. Multimodal Learning

   2.2. Multiple Instance Learning

3. Methodology

   3.1. Preliminaries and Notations

   3.2. Relations between Attention-based VPG and MIL

   3.3. MIVPG for Multiple Visual Inputs

   3.4. Unveiling Instance Correlation in MIVPG for Enhanced Multi-instance Scenarios

4. Experiments and 4.1. General Setup

   4.2. Scenario 1: Samples with Single Image

   4.3. Scenario 2: Samples with Multiple Images, with Each Image as a General Embedding

   4.4. Scenario 3: Samples with Multiple Images, with Each Image Having Multiple Patches to be Considered and 4.5. Case Study

5. Conclusion and References

Supplementary Material

A. Detailed Architecture of QFormer

B. Proof of Proposition

C. More Experiments

3.4. Unveiling Instance Correlation in MIVPG for Enhanced Multi-instance Scenarios

Subsequently, the aggregated low-rank matrix can be reintegrated with the original embeddings, as shown in Equation 9. This low-rank projection effectively reduces the time complexity to O(MM′).

Proposition 2. MIVPG, when equipped with the CSA (Correlated Self-Attention) module, continues to fulfill the essential properties of MIL

We prove the proposition 2 in the supplementary B.

In summary, as depicted in Figure 2a, we establish that QFormer falls under the MIL category and is a specialized instance of our proposed MIVPG. The latter extends to visual inputs with multiple dimensions, accounting for instance correlation.