Home People Research Publications Demos
News Jobs Prospective
About Internal

Variable Margin Canonical Gradient Boost

The problem of controlling the margin of a classifier is studied. A detailed analytical study is presented on how properties of the classification risk, such as its optimal link and minimum risk functions, are related to the shape of the loss, and its margin enforcing properties. It is shown that for a class of risks, denoted canonical risks, asymptotic Bayes consistency is compatible with simple analytical relationships between these functions. These enable a precise characterization of the loss for a popular class of link functions. It is shown that, when the risk is in canonical form and the link is inverse sigmoidal, the margin properties of the loss are determined by a single parameter. Novel families of Bayes consistent loss functions, of variable margin, are derived. These families are then used to design boosting style algorithms with explicit control of the classification margin. The new algorithms generalize well established approaches, such as LogitBoost. Experimental results show that the proposed variable margin losses outperform the fixed margin counterparts used by existing algorithms. Finally, it is shown that best performance can be achieved by cross-validating the margin parameter.

Experimental Results:

A number of easily reproducible experiments were conducted to study the effect of variable margin losses on the accuracy of the resulting classifiers. Canonical logistic and canonical boosting outperform both LogitBoost and AdaBoost in 7 and 5 of the ten datasets, respectively.


Publications: Variable margin losses for classifier design.
Hamed Masnadi-Shirazi and Nuno Vasconcelos.
Neural Information Processing Systems (NIPS), Vancouver, Canada, Dec 2010.
(acceptance rate 25%)

Contact: Nuno Vasconcelos, Hamed Masnadi-Shirazi


Copyright @ 2007 www.svcl.ucsd.edu