Low complexity VLSI architecture for improved primal–dual support vector machine learning core

Support vector machine (SVM) is an efficient and popular machine learning tool used for supervised classification. It is a statistical tool for classification with good generalization capacity, higher accuracy, and reliability [1]. Cortes and Vapnik invented SVM in 1993 [2], as an extension of the perceptron learning technique that guarantees a hyperplane separating the data classes for learning. SVM has applications in the fields of medical image analysis [3], handwriting recognition [4], financial engineering [5], face recognition [6], speech recognition [7], bioinformatics [8], text and hypertext categorization [9], prediction [10], [11] and various remote sensing applications including detection and classification of multispectral, hyperspectral, radar data, etc. [12], [13], [14]. SVM classification comprises mainly two stages, training the classifier and testing the incoming data based on the learned model. In the training phase, support vectors are extracted to build a model, classifying an incoming test sample [15]. The memory requirement and computational complexity of SVM are related to the number of training samples/ dataset size, dimensionality, and the number of support vectors. In large-scale classification problems with real-time data, the classification process takes considerable time due to the need for large data throughput [16]. There are several techniques to train the SVM; the most important are decomposition, active set, and interior point methods. The original problem is divided into several sub-problems in decomposition and active set methods, making them easier to solve. The decomposition method optimizes a fixed number of variables at a time. Advantages of this method include using limited memory resources and efficiently dealing with a large dataset compared to traditional SVM training. Sequential minimal optimization (SMO) is the most commonly used and efficient decomposition training method [17]. Library for Support Vector Machines (LIBSVM) [18] and $SV{M}^{light}$ [19] are two other commonly used decomposition methods for SVM training. As only a few variables are updated in each iteration, decomposition methods have slow convergence. The active set method is a sequential training technique, primarily targeted for training small and medium-sized problems [20], [21]. Support vector machine-quadratic problem (SVM-QP), support vector machine-revised simplex quadratic problem (SVM-RSQP), and simplex methods [22] are primarily used active set methods. If training time is limited or the data size is large, the active set method may not be efficient. The interior-point method (IPM) efficiently classifies large-scale datasets as it scales efficiently, reducing its computational complexity [23]. It also provides excellent stability, robustness, and good early approximation. The interior point method is suitable for parallel and multicore environments. Classification performance of interior point method for large, noisy and dense data is consistent and accurate. The number of iterations required grow very slowly with the problem dimension [24].

Primal–Dual Support Vector Machine (PDSVM) [25] is a type of interior point methods other than the barrier point method. PDSVM is an SVM training method based on Primal–Dual linear programming applications [26]. PDSVM is preferred over the barrier method due to its accuracy and efficiency [27]. PDSVM is robust in terms of accurate solutions, especially in optimal global solutions. In this method, the number of iterations required grows slowly with an increase in the number of constraints. In PDSVM, both primal and dual information has been utilized to solve quadratic problems with bounded variables [28]. Based on the nature of the constraints, it is solved in its original space itself without the need for transforming them to some other space. This results in saving memory required for computation. PDSVM method is relatively accurate and faster in classifying problems of medium, large size as well as noisy data [27]. The most commonly used SVM training technique is SMO owing to its advantages. PDSVM has better scalability and faster computation than SMO. PDSVM scales better with increased data size as compared to SMO, making it a better SVM training method for medium and larger dataset problems [29].

Real-time classification problems, including medical image analysis, handwriting recognition, face recognition, etc., are primarily of medium or large set data size in nature. Also, in practical applications, larger possible data set is used for learning as the classification accuracy increases with data size. PDSVM is a highly suitable SVM classifier for real-time classification applications.

Field-programmable gate array (FPGA) is an appropriate device for hardware implementation of SVM, owing to its high performance, programming flexibility, and efficiency in achieving a high accuracy rate [30]. FPGA has a similar classification and a better processing time accuracy compared to general purpose central processing unit (CPU). Real-time classification applications commonly require fast hardware support. Computationally complex algorithms like the PDSVM can be easily implemented in FPGA by exploiting the parallelism available in FPGA with reasonable accuracy [31]. Accordingly, various works presenting hardware implementation of SVM in FPGA have been found in the literature [3], [32], [33]. Anguita et al. have done an efficient implementation of kernel based perceptron on fixed-point digital hardware [34]. Very large scale integration (VLSI) architecture of SMO has been discussed in [35]. A multi-class SVM-based recognition system in FPGA has been proposed by Manikandan et al. [36]. However, there is a scarcity of literature discussing dedicated hardware for primal–dual SVM in all hardware platforms, including FPGA. A dedicated classification system based on a primal–dual method can be utilized as a standalone device for machine learning applications. It can also be easily integrated into embedded systems intended for various image analysis applications.