F. Girosi and G. Anzellotti. Convergence rates of approximation by translates. Technical Report 1288, MIT Art. Intell. Lab., 1992.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....bound on the finite dimensional subspace spanfOE( t i )g. We note that (1.4) is sharp in the sense that for any ffi one can find a perturbation of noise level ffi which yields equality in (1.4) Although there exists a large variety of results about the approximation by neural networks (c.f. [3, 7, 13, 21, 26, 29] and the references quoted there) the stability aspect has been largely neglected so far, only few authors give a rigourous treatment of regularization methods for the approximation problem (cf. 14, 15, 16, 33, 40] In Section 2 we will show that network approximation in Sobolev spaces is ....

....of gradients, which is necessary for the numerical solution of the minimization problem. We will give an analysis for arbitrary choice of the parameter t, obviously all approximation results hold for the optimal choice of t, too (which is the case so far mainly studied in the literature, cf. [3, 7, 13, 26, 33]) The stability results deduced in Section 4 cannot be applied to the case of optimized t in a simple way, since the dependence of t upon the data f ffi has to be examined additionally. An extension of the stability results to this nonlinear problem will be one of our main future projects. So ....

[Article contains additional citation context not shown here]

F.Girosi, G.Anzellotti, Convergence rates of approximation by translates, AI Memo 1288 (AI Laboratory, MIT, Cambridge, Massachusetts, 1995).

....Theorem 4.1, 4.3) holds and fi plays the role of . 5. Applications In this section we apply the above results to some typical constructions for neural networks. The two classes we consider are perceptrons with one hidden layer and translation networks, whose use for approximation problems (cf. [10]) and deconvolution (cf. 6] has been investigated recently. Perceptrons are a classical construction for neural networks. They consist of an input layer of ridge type and an activation function oe as in (1.3) The activation function oe is usually chosen as a Heaviside function or a smoothed ....

F. Girosi and G. Anzellotti, Convergence rates of approximation by translates, AI Memo 1288 (AI Laboratory, MIT, Cambridge, Massachusetts), 1995.

....functions with expansion coefficient vectors lying in the kernel of D, however, will not be dampened at all. Example 4 (Sobolev Regularization) Smoothness properties of functions f can be enforced effectively by minimizing the Sobolev norm of a given order. Our exposition in this point follows [15]: The Sobolev Space H s;p (V ) s 2 N, 1 p 1) is defined as the space of those L p functions on V whose derivatives up to the order s are L p functions. It is a Banach space with the norm jjf jj H s;p (V ) X jfljs jj D fl f jj Lp (24) where fl is a multi index and D fl is the ....

F. Girosi and G. Anzellotti. Convergence rates of approximation by translates. Technical Report AIM-1288, Artificial Intelligence Laboratory, Massachusetts Institute of Technology (MIT), Cambridge, Massachusetts, 1992.

....the proofs are constructive, an algorithm to achieve the theoretical bounds is provided as well. Other nonlinear approximation techniques have also benefited from the solution to this problem: approximation by hinged hyperplanes [8] projection pursuit regression [28] and radial basis functions [17]. In all these related approximation problems the solution can always be constrained to fall in the closure of the convex hull of a subset of functions (e.g. hinged hyperplanes, ridge functions or radial basis functions in the examples mentioned above) 3 Constructive Solutions For the sake of ....

F. Girosi and G. Anzellotti. Convergence rates of approximation by translates. Technical Report 1288, MIT Art. Intell. Lab., 1992.

....in Neural Network Models Two significant bounds on the generalisation error for sigmoid and radial basis networks have been developed by Barron (in [7] and Niyogi [8] respectively. These bounds are based on a lemma by Jones on the convergence rate of particular approximation schemes [9 11, 8, 12] and on the Vapnik Chervonenkis Dimension [13] Barron s and Niyogi s (with probability 1 Gamma ff, where ff is ideally a small number) generalisation bounds are given by E Barron O( 1 n ) O( nd ln ) 1) E Niyogi O( 1 n ) O( nd ln(n ) Gamma ln ff ] 1=2 ) 2) where O(h) ....

F Girosi and G Anzellotti, "Convergence rates of approximation by translates," Tech. Rep. AIM-1288, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, April 1995.

....functions with expansion coefficient vectors lying in the kernel of D, however, will not be dampened at all. Example 2 (Sobolev Regularization) Smoothness properties of functions f can be enforced effectively by minimizing the Sobolev norm of a given order. Our exposition in this point follows [15]: The Sobolev Space H s;p (V ) s 2 N, 1 p 1) is defined as the space of those L p functions on V whose derivatives up to the order s are L p functions. It is a Banach space with the norm kfk H s;p (V ) X jfljs k D fl fkLp (19) where fl is a multi index and D fl is the ....

F. Girosi and G. Anzellotti. Convergence rates of approximation by translates. Technical Report AIM-1288, Artificial Intelligence Laboratory, Massachusetts Institute of Technology (MIT), Cambridge, Massachusetts, 1992.

....4 ) nonlinear approximants also suffer the curse of dimensionality, the scaling may be less drastic. Moreover there are interesting classes of functions which avoid the curse of dimensionality: however this is because the classes become increasingly constrained as the dimensionality increases [9]. It has thus been suggested that work in this area should concentrate on the nonlinear approximants [17] We disagree with [17] and contend that before a nonlinear approximant based theory can be developed, the linear theory must be in place first; besides which, it is reasonable to expect the ....

F. Girosi and G. Anzellotti. Convergence rates of approximation by translates. Technical Report 1288, MIT AI Laboratory, April 1995.

No context found.

F. Girosi and G. Anzellotti. Convergence rates of approximation by translates. Technical Report 1288, MIT Art. Intell. Lab., 1992.

No context found.

F. Girosi and G. Anzellotti. Convergence rates of approximation by translates. Technical Report AIM1288, Artificial Intelligence Laboratory, Massachusetts Institute of Technology (MIT), Cambridge, MA, 1992.

No context found.

Girosi, F. and Anzellotti, G. (1992). Convergence rates of approximation by translates (Tech. Rep. A.I. memo 1288). Artificial Intelligence Laboratory, Massachusetts Institute of Technology.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC