O.L. Mangasarian. Multisurface method of pattern separation. IEEE Transactions on Information Theory IT-14 (6), 1968: 801--807.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....lists is fairly natural, if one is to take an iterative approach to data classification. There are other methods which similarly make use of such an iterative approach, by classifying some points of the data set, removing these from consideration, and proceeding. Magasarian s multisurface method [18] also has this character. At each stage, it finds two parallel hyperplanes (as close together as possible) such that the points not enclosed between the two planes all have the same classification. It then removes these points and repeats. We can see that the MSM method may be regarded as ....

O.L. Mangasarian. Multisurface method of pattern separation. IEEE Transactions on Information Theory IT-14 (6), 1968: 801--807.

....and e#ciency in evaluation are extremely important. In this paper, we combine two independent but related research directions developed for solving the two class linear discrimination problem. The first is the linear programming (LP) methods stemming from the Multisurface Method of Mangasarian [12, 13]. This method and it s later extension the Robust Linear Programming (RLP) approach [6] have been used in a highly successfully breast cancer diagnosis system [26] The second direction is the quadratic programming (QP) methods based on Vapnik s Statistical Learning Theory [24, 25] Statistical ....

....polynomial time nonlinear methods based on LP use a multi step approaches. The methods of Roy et al. [20, 19, 18] use clustering in conjuction with LP to generate neural networks in polynomial time. Another approach is to recursively construct piecewise linear discriminants using a series of LP s [13, 2, 15]. These approaches could also be used with SVM but we limit discussion to nonlinear discriminants constructed using the SVM kernel type approaches. After the introduction to the existing multiclass methods, M RLP and k SVM, we will show how same idea used in the M RLP, can be adapted to ....

[Article contains additional citation context not shown here]

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968. 29

....work at Wisconsin The application of machine learning techniques to problems in breast cancer diagnosis at the University of Wisconsin began in 1989 with the collaborative work of W. H. Wolberg (Surgery and Human Oncology) and O. L. Mangasarian (Computer Sciences) The Multisurface Method (MSM) [53, 54] of pattern separation was first applied to a collection of cases represented by nine subjectively evaluated cytological features [57, 58, 92] The MSM procedure uses a linear programming model to place successive pairs of separating planes in the feature space of the input examples, building a ....

....separated by benign and malignant cases, appears in Appendix A. 2.4 Application to breast cancer diagnosis 2.4. 1 Classification method: Multisurface Method Tree (MSM T) The classification procedure used to separate benign from malignant samples is a variant on the Multisurface Method (MSM) [53, 54] known as MSM Tree (MSMT) 7, 10] This method uses a linear programming [22] model to iteratively place a series of separating planes in the feature space of the examples. If the two sets of points are linearly separable, the first plane will be placed between them. If the sets are not linearly ....

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....function g has the following form: 2 g(x) 8 : 1 if x 2 A 0 if x 2 B: 1) Many different algorithms exist for constructing the approximation g of g and the approximation may have many different functional forms. Examples include a separating plane based function [96, 97], the backpropagation algorithm for artificial neural networks (ANNs) 77, 111] decision tree construction algorithms utilizing various node decision criteria [29, 134, 5] spline methods for classification [165, 162] and probabilistic graphical dependency models [76, 32] Evaluating an ....

.... z) 2 e 0 s fi fi fi fi fi fi fi fi fi fi fi fi fi fi GammaAw efl e y; Bw Gamma efl e z; y 0; z 0; Gammas w s: 9 = 2 (0; 1) 35) We note that an optimization formulation was proposed and implemented in [97] for computing a separating plane by forcing the bounding planes to be as far apart as possible. 48 Usually the support vector machine problem is formulated using the 2 norm in the objective [161, 6] Since the 2 norm is dual to itself, it follows that the margin is also measured in the 2 norm ....

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....aspirates from a patient s breast [15] The measurements were graded 1 to 10 at the time of sample collection, with 1 being the closest to benign and 10 the most anaplastic. A linear programming based method for pattern separation called the Multisurface Method has been proposed by Mangasarian [16]. A computer program that implements this method for the WBCD has been in use at the University of Wisconsin Hospital since 1990 [17] For our experiment, 315 samples were randomly selected for training, 35 samples were selected for cross validation, and 349 for testing. 2. United States ....

O.L. Mangasarian, "Multisurface method of pattern separation," IEEE Transactions on Information Theory, vol. 14, no. 6, pp. 801-807, 1968.

....that exactly this technique of problem solving in feed forward networks has been implemented explicitly by hand by a number of researchers. Such methods use so called constructive or growth algorithms, either through explicit use of nodeobjects [12, 6, 5] or pure mathematical abstractions [11]. They tend to be more efficient than back propagation in performing the necessary mappings, but they do 12 Frank J. Smieja not have the other favorable non linear properties found in a net trained using backpropagation, namely the history information contained in the hyperplane masses and ....

O.L. Mangasarian. Multisurface method of pattern separation. IEEE Transactions on information theory, IT-14(6):801--807, 1968.

....constructive algorithms It is also possible to approach the network growth issue from a geometrical standpoint. Two illustrations of such an approach are the construction of a piecewise linear classification surface in the input vector space using the Multisurface Method (MSM) of Mangasarian [15], and a more opportunistic slicing up of the input vector space, using the Minimal Resources method of Ruj an and Marchand. Both approaches basically decide on the separation hyperplanes to be used for this training set, and then transfer the results simply to a network architecture. 3.1 The ....

.... surface that successfully separates two sets of vectors with intersecting convex hulls (i.e. that are not linearly separable) First a method is described for finding the optimal hyperplane for separating two linearly separable sets of vectors (patterns) using the following algorithm [15]: maximize ff; fi; w 8 : ff Gamma fi j Aw eff; Bw efi; Gammae w e 9 = 6) where the two vector sets to be separated, A and B, are represented by the matrices A and B respectively, e is a vector filled with ones, w is the weight vector defining the orientation of the hyperplane and ....

[Article contains additional citation context not shown here]

O.L. Mangasarian. Multisurface method of pattern separation. IEEE Transactions on information theory, IT-14(6):801--807, 1968.

....= 1; n as: max w;ff;fi ff Gamma fi s:t: Aw Gamma ffe 0 GammaBw fie 0 Gammae w e w j = Gamma1 (11) A solution of one the 2n problems with the greatest value of ff Gamma fi is the optimal answer. 1 This approach, called the Multisurface Method of Pattern Recognition (MSM) [15], was used in the initial implementation of the automated breast cancer diagnosis system described in the introduction [19, 34] The second general method is to fix ff Gamma fi 0. If we set ff Gamma fi = 2 by defining ff = fl 1 and fi = fl Gamma 1, then Problem (7) becomes min w;fl 1 2 ....

....u;v 1 2 kA 0 u Gamma B 0 vk 2 s:t: e 0 u = 1 e 0 v = 1 u 0 v 0 (20) Proof. The dual problem maximizes the Lagrangian function of (19) L(w; ff; fi; u; v) subject to the constraints that the partial derivatives of the Lagrangian with respect to the primal variables are equal to zero [15]. Specifically, the dual of (20) is: max w;ff;fi;u;v L(w; ff; fi; u; v) 1 2 kwk 2 Gamma (ff Gamma fi) Gamma u 0 (Aw Gamma eff) Gamma v 0 ( GammaBw efi) s:t: L w = w Gamma A 0 u B 0 v = 0 L ff = Gamma1 e 0 u = 0 L fi = 1 Gamma e 0 v = 0 u 0; v 0 (21) ....

[Article contains additional citation context not shown here]

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....to this norm is the 1 norm and accordingly kwk 0 = kwk 1 in (1.24) which leads to the following linear programming formu 16 lation: min w; y;s e 0 y e 0 s s.t. D(Aw e ) y e s w s y 0: 1. 26) We note that the rst paper on the multisurface method on pattern separation [33] also proposed and implemented the idea of separating the bounding planes, just as the SVM approach does. Usually the SVM problem is formulated using the 2 norm in the objective [57, 2] Since the 2 norm is dual to itself, it follows that the distance between the parallel planes de ning the ....

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801-807, 1968.

....medical examinations. These 569 vectors, along with the known outcomes, represent a training set with which a classifier can be constructed to diagnose future examples. These examples were used to train a linear programming based diagnostic system by a variant of the multisurface method (MSM) [14, 15] called MSMTree (MSM T) 1, 2] which we briefly describe now. Let m malignant n dimensional vectors be stored in the m Theta n matrix A, and k benign n dimensional vectors be stored in the k Theta n matrix B. The points in A and B are strictly separable by a plane in the n dimensional real ....

.... y and z of the inequalities (2) This intuitively plausible linear program has significant theoretical and computational consequences [2] such as naturally eliminating the null point w = 0 from being a solution, a difficulty that other linear programming formulations exclude in an ad hoc manner [9, 10, 14]. Once the plane x T w = fl has been obtained, the same procedure can be applied recursively to one or both of the newly created halfspaces x T w fl and x T w fl, if warranted by the presence of an unacceptable mixture of benign and malignant points in the halfspace. Figure 2 shows an ....

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....that represented by an LTU was needed to correctly map these simple four points into the set f0; 1g. Curiously enough, however, it should be noted that even before Minsky Papert proposed their classical XOR counterexample, a linear programming based piecewise linear separator was proposed in 1968 [28] that could easily and correctly handle this problem, and which in fact can be represented as a neural network [7] See Figure 14 and discussion following Algorithm 2.2 below. We shall now use this example to motivate a general multisurface method (MSM) for separating the sets A and B of the ....

....of LTU s. In fact, it can be shown [24, 29] that such a neural network can separate any two disjoint sets in R n given a sufficient number of hidden units. See Theorem 2.2 below. This is equivalent to the multisurface method separating such disjoint sets given a sufficient number of planes [28]. Hopefully, it should be clear now that the mapping that we are after from R n into f0; 1g can be rather complex depending on the sets A and B: The two representations that have been described, neural networks and multisurface separation, are both valid representations of this mapping. ....

[Article contains additional citation context not shown here]

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....objective function of the mathematical program (17) is convex but its feasible region, which is the unit sphere in the dual norm k Delta k 0 , is not convex. It is precisely this essential nonconvex condition that has been either ignored in most previous work [12, 9, 8, 2] or used heuristically [13, 19] to enforce nonzeroness of w but not as a distance normalization constraint. Thus in these papers, the sum of the distances of misclassified points to the separating plane has not been the real objective function that has been minimized. In fact the nonconvexity of the program (17) leads to ....

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....function h : R n Gamma R m on R n . The feasible region of (17) which is the unit sphere in the dual norm k Delta k 0 , is however not convex. It is precisely this essential nonconvex condition that has been either ignored in most previous work [12, 9, 8, 2] or used heuristically [13, 20] to enforce nonzeroness of w but not as a distance normalization constraint. Thus in these papers, the sum of the distances of misclassified points to the separating plane has not been the real objective function that has been minimized. In [5] a 2 norm error term is used, instead of the 1 norm ....

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....olvi cs.wisc.edu. This material is based on research supported by Air Force Office of Scientific Research Grant F49620 94 1 0036 and National Science Foundation Grant CCR 9322479. plane (1) that minimizes a weighted average of the sum of the distances of the misclassified points to the plane [7, 2] as follows: min w;fl;y;z ( e T y m e T z k j Aw y efl e; Bw Gamma z efl Gamma e; y 0; z 0 ) 3) Here the rows of the matrices A 2 R m Thetan and B 2 R k Thetan represent the m points in A and the k points in B respectively, while e is a vector of ones of appropriate ....

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....two point sets in the n dimensional real space R n , and show that its complexity ranges from polynomial time to NP complete, depending on the measure of error employed. When the traditional distance of a misclassified point to a separating plane is used as an error, a single linear program [6, 15, 16, 4] usually solves the problem. Recently [10, 18, 2, 7] a more complex, and for certain applications more realistic, error measure has been considered, namely the number of misclassified points by a separating plane. This problem, even though shown to be NP complete [7] can be effectively solved by ....

....not intersect. Thus, one resorts in the general case to optimizing some error criterion in the satisfaction of (2) The simplest such criterion is to use linear programming in order to construct a plane (1) that maximizes a weighted sum of the distances of correctly classified points to the plane [16, 4] as follows: max w; y;z Phi e T y e T z j Aw e y; Bw e Gamma z; y e; z e Psi (3) Here the rows of the matrices A 2 R m Thetan and B 2 R k Thetan represent the m points in A and the k points in B respectively, while e is a vector of ones of appropriate dimension. The ....

[Article contains additional citation context not shown here]

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....is not linearly separable, but is bilinearly separable. It can be solved by a neural network with 2 layers of linear threshold units [25, 18] see Figure 3) Other methods of separation by more than one plane, for example multisurface methods (MSM) of pattern separation, have also been proposed [14, 5, 6] and extensively used for medical diagnosis [29, 17, 5] MSM which has been shown to be equivalent to a feed forward neural network with a single hidden layer [5] can be trained by a greedy algorithm using linear programming [14, 5, 6] Thus bilinear and MSM separation can be thought of as ....

.... methods (MSM) of pattern separation, have also been proposed [14, 5, 6] and extensively used for medical diagnosis [29, 17, 5] MSM which has been shown to be equivalent to a feed forward neural network with a single hidden layer [5] can be trained by a greedy algorithm using linear programming [14, 5, 6]. Thus bilinear and MSM separation can be thought of as alternative linear programming based methods for solving problems that are usually solved by neural networks. It is interesting to note that the bilinear separation depicted in Figure 3(a) which corresponds to the topology of the bilinear ....

O.L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

....the sets are not piecewise linear separable is evidenced by the relatively poor training set accuracy shown in Table 1. A single linear program is insufficient for solving such problems. However, the results of several multicategory linear programs can be combined by using multisurface methods [12, 3, 2]. To demonstrate this, we used the multicategory linear program to create the multivariate splits in the multisurface method tree algorithm (MSMT) 2, 18] MSMT multicategory works by applying the linear program (8) to a k class classification problem. The resulting piecewise linear surface ....

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

No context found.

O. L. Mangasarian, Multisurface Method of Pattern Separation, IEEE Transactions on Information Theory IT-14(6), 1968, pp. 801-807.

....into decision regions and associate a prediction with each. For example linear discriminant analysis determines linear separators and neural networks compute non linear decision surfaces [64] Decision tree or rule based classifiers make a piecewise constant approximation of the decision surface [26, 81, 6]. The third class of methods is by far the most commonly used and studied. It is usually more practical because it sidesteps the harder problem of determining the density and just concentrates on separating various regions of the sapce. 3.1.2 Mathematical Programming Formulations We address the ....

....models possess this property. For example the robust linear separator of [9] has a very simple geometric representation as a plane separating most or all the elements of two sets, which has been used to advantage in medical applications [92, 91] Similarly the multisurface separation method [81, 88] can be geometrically depicted as placing the sets to be separated into distinct polyhedral compartments. 8. Local versus global methods. Because of size considerations we often do not want to model the whole data set. We therefore need to pre partition the data and model it locally. One way to ....

[Article contains additional citation context not shown here]

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Transactions on Information Theory, IT-14:801--807, 1968.

No context found.

O. L. Mangasarian. Multi-surface method of pattern separation. IEEE Trans on Information Theory, IT-14:801--807, 1968.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC