This story on HackerNoon has a decentralized backup on Sia.
Transaction ID: wdPm5HqxwGR9Qv85jcno-CZhHOhodmVLBsy43E0YfJY
Cover

Ideal Contribution Estimation: Taylor Expansion and Gradient Cache

Written by @instancing | Published on 2025/12/4

TL;DR
The ideal contribution of generated data is approximated using a first-order Taylor expansion based on the gradient dot product.

Abstract and 1 Introduction

  1. Related work

    2.1. Generative Data Augmentation

    2.2. Active Learning and Data Analysis

  2. Preliminary

  3. Our method

    4.1. Estimation of Contribution in the Ideal Scenario

    4.2. Batched Streaming Generative Active Learning

  4. Experiments and 5.1. Offline Setting

    5.2. Online Setting

  5. Conclusion, Broader Impact, and References

A. Implementation Details

B. More ablations

C. Discussion

D. Visualization

4. Our method

4.1. Estimation of Contribution in the Ideal Scenario

Moreover, we can employ the classic first-order Taylor expansion to approximate ϕ(g, θ), which has also been widely used in previous work (Pruthi et al., 2020; He et al., 2023). Note that, it is possible to use more sophisticated methods here, e.g., Koh and Liang (2017).

Lemma 4.1. The loss of a network f on a dataset U can be approximated by a first-order approximation:

So the contribution of g to f can be approximated by:

Authors:

(1) Muzhi Zhu, with equal contribution from Zhejiang University, China;

(2) Chengxiang Fan, with equal contribution from Zhejiang University, China;

(3) Hao Chen, Zhejiang University, China (haochen.cad@zju.edu.cn);

(4) Yang Liu, Zhejiang University, China;

(5) Weian Mao, Zhejiang University, China and The University of Adelaide, Australia;

(6) Xiaogang Xu, Zhejiang University, China;

(7) Chunhua Shen, Zhejiang University, China (chunhuashen@zju.edu.cn).


This paper is available on arxiv under CC BY-NC-ND 4.0 Deed (Attribution-Noncommercial-Noderivs 4.0 International) license.

[story continues]


Written by
@instancing
Pioneering instance management, driving innovative solutions for efficient resource utilization, and enabling a more sus

Topics and
tags
deep-learning|computational-efficiency|data-contribution|first-order-taylor-expansion|lvis-test-set-sampling|gradient-cache|taylor-expansion|what-is-taylor-expansion
This story on HackerNoon has a decentralized backup on Sia.
Transaction ID: wdPm5HqxwGR9Qv85jcno-CZhHOhodmVLBsy43E0YfJY