Berikut sebagian isi makalahnya :
Support
Vector Machines (SVM) for solving pattern recognition and nonlinear function
estimation problems have been introduced in [7]. The idea of SVM is mapping the
training data nonlinearly into a higher-dimensional feature space, then
construct a separating hyperplane with maximum margin there. This yields a
nonlinear decision boundary in input space. By the use of a kernel function,
either polynomial, splines, radial basis function (RBF) or multilayer
perceptron, it is possible to compute the separating hyperplane without
explicitly carrying out the map into the feature space. While classical Neural
Networks techniques suffer from the existence of many local minima, SVM
solutions are obtained from quadratic programming problems possessing a global
solution.
Recently,
least squares (LS) versions of SVM have been investigated for classification
[5] and function estimation [6]. In these LS-SVM formulations one computes the
solution by solving a linear system instead of quadratic programming. This is
due to the use of equality instead of inequality constraints in the problem
formulation. In [1, 4] such linear systems have been called Karush-Kuhn-Tucker
(KKT) systems and their numerical stability has been investigated. This linear
system can be efficiently solved by iterative methods such as conjugate
gradient [2], and enables solving large scale classification problems. As an
example we show the excellent performance on a multi two-spiral benchmark
problem, which is known to be a difficult test case for neural network
classifiers [3].
LEAST
SQUARES SUPPORT VECTOR MACHINES
Given
a training set of N data points 
, where 
is the k-th input
pattern and 
is the k-th output
pattern, the classifier can be constructed using the support vector method in
the form
, where is the k-th input
pattern and is the k-th output
pattern, the classifier can be constructed using the support vector method in
the form
where
are called support
values and b is a constant. The 
is the kernel, which
can be either 
(linear SVM); 
(polynomial SVM of
degree d); 
(multilayer perceptron
SVM), or 
(RBF SVM), where 
, and
are constants.
are called support
values and b is a constant. The is the kernel, which
can be either (linear SVM); (polynomial SVM of
degree d); (multilayer perceptron
SVM), or (RBF SVM), where , and are constants.
For
instance, the problem of classifying two classes is defined as
This
can also be written as
where 
is a nonlinear function mapping of the input space to a
higher dimensional space. LS-SVM classifiers\
is a nonlinear function mapping of the input space to a
higher dimensional space. LS-SVM classifiers\
subjects to the equality constraints
The Lagrangian is defined as
with Lagrange multipliers 
(called support
values).
(called support
values).
The conditions for optimality are given by
for 
. After elimination of 
and 
one obtains the solution
. After elimination of and one obtains the solution
with
and 
. Mercer’s condition
is applied to the matrix 
with
and . Mercer’s condition
is applied to the matrix with
The kernel parameters, i.e. s for RBF kernel, can be
optimally chosen by optimizing an upper bound on the VC dimension. The support
values ak
are proportional to the errors at the data points in the LS-SVM case, while
in the standard SVM case many support values are typically equal to zero. When
solving large linear systems, it becomes needed to apply iterative methods [2].
