Iteratively reweighted least squares classifier and its ℓ₂- and ℓ₁-regularized Kernel versions

This paper introduces a new classifier design method based on regularized iteratively reweighted least squares criterion function. The proposed method uses various approximations of misclassification error, including: linear, sigmoidal, Huber and logarithmic. Using the represented theorem a kernel version of classifier design method is introduced. The conjugate gradient algorithm is used to minimize the proposed criterion function. Furthermore, ℓ1-regularized kernel version of the classifier is introduced. In this case, the gradient projection is used to optimize the criterion function. Finally, an extensive experimental analysis on 14 benchmark datasets is given to demonstrate the validity of the introduced methods.