GistNet: a Geometric Structure Transfer Network for Long-Tailed Recognition

Haoxiang Li2

Hao Kang2

Gang Hua2

UC San Diego1, Wormpex AI Research2


The problem of long-tailed recognition, where the number of examples per class is highly unbalanced, is considered. It is hypothesized that the well known tendency of standard classifier training to overfit to popular classes can be exploited for effective transfer learning. Rather than eliminating this overfitting, e.g. by adopting popular class-balanced sampling methods, the learning algorithm should instead leverage this overfitting to transfer geometric information from popular to low-shot classes. A new classifier architecture, GistNet, is proposed to support this goal, using constellations of classifier parameters to encode the class geometry. A new learning algorithm is then proposed for GeometrIc Structure Transfer (GIST), with resort to a combination of loss functions that combine class-balanced and random sampling to guarantee that, while overfitting to the popular classes is restricted to geometric parameters, it is leveraged to transfer class geometry from popular to few-shot classes. This enables better generalization for few-shot classes without the need for the manual specification of class weights, or even the explicit grouping of classes into different types. Experiments on two popular long-tailed recognition datasets show that GistNet outperforms existing solutions to this problem.







Architecture: Schematic representation of GeometrIc Structure Transfer networks.


  1. Linear Classifier implies similar feature space geometry among classes.
  2. Geometric Transfer Classifier.
  3. paper
  4. GIST Training: combining random and class-balanced sampling.
    1. Class-specifc parameters are trained with class-balanced sampling.
    2. Structure parameters are trained with random sampling.


Training, evaluation and deployment code available on GitHub.


Bo Liu

UC San Diego

Haoxiang Li

Wormpex AI Research

Hao Kang

Wormpex AI Research

Gang Hua

Wormpex AI Research


Gang Hua was supported partly by National Key R&D Program of China Grant 2018AAA0101400 and NSFC Grant 61629301. Bo Liu and Nuno Vasconcelos were partially supported by NSF awards IIS-1637941, IIS-1924937, and NVIDIA GPU donations.