W. Duch and N. Jankowski. Survey of neural transfer functions. Neural Computing Surveys, 2:163--212, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... further mathematical questions concerning the capacity of possibly restricted architectures can be investigated, such as the capacity of FNNs with restricted weights [42] the uniqueness of parameters [44] or the design of alternative transfer functions for FNNs respectively kernels for SVMs [43, 72]. A problem adapted number of neurons can be achieved with alternative techniques which increase the number of neurons during training, such as proposed for RBF networks by Li, Luo, and Qi in this volume. 2.2 Complexity of training Usually, network training aims at finding weights such that the ....

W. Duch and N. Jankowski. Survey of neural transfer functions. Neural Computing Surveys, 2:163 212, 1999.

....this definition of variables, the distance variable does 38 make sense. The basic requirement is that if the distance measure between two values is small, both should belong to similar classifications [48] As of the output function, the S shaped sigrnoidal functions may be the most popular [49,50]. The sigrnoidal function yields a value between 0 and 1 in terms of the activation function value A and a slope (scale) parameter s. The logistic function [49] o (s) 1 1 e (100) illustrates the typical form of the sigrnoidal functions. This section will focus on developing of the kemel ....

....should belong to similar classifications [48] As of the output function, the S shaped sigrnoidal functions may be the most popular [49,50] The sigrnoidal function yields a value between 0 and 1 in terms of the activation function value A and a slope (scale) parameter s. The logistic function [49] o (s) 1 1 e (100) illustrates the typical form of the sigrnoidal functions. This section will focus on developing of the kemel sigrnoidal functions with the help of the kernel distance functions. It is stressed that although the kernel distance function has the origin of PDEs, any ....

[Article contains additional citation context not shown here]

W. Duch and N. Jankowski, Survey of neural transfer functions, Neural Computing Surveys, 2, 163-212, 1999.

....there are no restriction on the type of transfer functions in the FSM model but so far only localized functions were used. The simplest functions with suitable properties for probability density modeling are of the Gaussian type or an approximate Gaussian type, for example the bicentral functions [6]: N i b i s b i s i e D x e e D x e Bi 1 ) 1 ) s b D x where x e x = 1 1 ) and x is the input vector. Shape adaptation of the density ) s b D x Bi is possible by shifting centers D, changing spreads b and ....

....localized functions like: triangular, trapezoidal, rectangular. Rectangular functions: 0 1 ) i i x D if G D x are very useful for extraction of crisp logical rules in the FSM network. Further details about this network are available in [5] MLP2LN MLP2LN [6] is a smooth transformation from an MLP network into a network performing logical operations (Logical Network, LN) This transformation is achieved during network training by: gradually increasing the slope of sigmoidal functions to obtain crisp decision regions simplifying the network structure ....

W. Duch N. Jankowski (1999), Survey of neural transfer functions. Neural Computing Surveys 2: 163-213

....networks use many di#erent architectures and many di#erent transfer functions. The problems considered in this paper will deal rather with the local learning than global. Especially the local and semilocal transfer functions will be described. For extensive review of other transfer function see [ Duch and Jankowski, 1999 ] The best known local learning models are the radial basis function networks [ Powell, 1987; Broomhead and Lowe, 1988; Dyn, 1989; Poggio and Girosi, 1990 ] adaptive kernel methods and local risk minimization [ Bottou and Vapnik, 1992; Vapnik, 1995; Girosi, 1998 ] The Radial Basis Function ....

....(for example, Euler s angles) and to calculate the derivatives necessary for backpropagation training procedure. Rotated densities in all dimensions may be obtained in two ways using transfer functions with just N 1 additional parameters per neuron. In the first approach (for the second see [ Duch and Jankowski, 1999; 1997 ] product form of the combination of sigmoids is used (see Fig. 3) CP (x; t, t # , R) N Y i #(R i x t i ) #(R i x t # i ) 10) SCP (x; t, t # , p, r, R) N Y i p i #(R i x t i ) r i #(R i x t # i ) 11) where R i is the i th row of the ....

W. Duch and N. Jankowski. Survey of neural transfer functions. Neural Computing Surveys, (2):163--212, 1999.

No context found.

W. Duch and N. Jankowski. Survey of neural transfer functions. Neural Computing Surveys, 2:163--212, 1999.

No context found.

W. Duch, N. Jankowski, Survey of neural transfer functions. Neural Computing Surveys 2 (1999) 163-213

No context found.

W. Duch and N. Jankowski, "A Survey of Neural Transfer Functions," Neural Computing Surveys, vol. 2, pp. 163--213, 1999.

No context found.

W. Duch and N. Jankowski, "A survey of neural transfer functions," Neural Comput. Surv., vol. 2, pp. 163--213, 1999.

No context found.

Duch W., Jankowski, N.: Survey of neural transfer functions. Neural Computing Surveys 2 (1999) 163-213

No context found.

Duch W., Jankowski N. (1999) Survey of neural transfer functions. Neural Computing Surveys, vol. 2, pp. 163-213.

No context found.

W. Duch and N. Jankowski, "A survey of neural transfer functions," Neural Comput. Surv., vol. 2, pp. 163--213, 1999.

No context found.

W. Duch and N. Jankowski, "Survey of neural transfer functions". Neural Computing Surveys, Vol. 2, pp. 163-213, 1999

....is the hypothyroid dataset, for which the best optimized MLPs still give about 1.5 of error [24] while logical rules reduce it to 0. 64 [9, 5] Some real world examples showing the differences between RBF and MLP networks that are mainly due to the transfer functions used were presented in [12]. Artificial data for which networks using sigmoidal functions (hyperplanar, delocalized decision borders) need O(N) parameters while networks with localized transfer functions (for example Gaussians) need O(N 2 ) parameters, and artificial data in which the situation is reversed, are presented ....

.... Artificial data for which networks using sigmoidal functions (hyperplanar, delocalized decision borders) need O(N) parameters while networks with localized transfer functions (for example Gaussians) need O(N 2 ) parameters, and artificial data in which the situation is reversed, are presented in [12]. Although all these networks are universal approximators in real applications their speed of learning and final network complexity may differ significantly. In principle neural networks may learn any mapping, but the inner ability to learn quickly in a given case may require flexible brain ....

[Article contains additional citation context not shown here]

W. Duch and N. Jankowski. Survey of neural transfer functions. Neural Computing Surveys, 2:163--212, 1999.

....at the feature level and in the simplest case d(x i , y i ) x i y i . Mahalanobis distance, more general quadratic distance function and many correlation factors are also suitable for metric functions, for example Camberra, Chebychev, Kendall s Rank Correlation or Chi square distance [5]. All these function may replace the Euclidean distance used in the definition of many transfer functions. For symbolic features the Modified Value Difference Metric (MVDM) is useful [6] A value difference for feature j of two N dimensional vectors x,y with discrete (symbolic) elements, in a C ....

....for any class. d q V (x j , y j ) is used as d( in the Minkovsky distance function for symbolic attributes, combined with other distance functions for numerical attributes. Many variants of MVDM have been discussed in the literature and may be used in neural network activation functions [5, 6]. To avoid problems with calculation of gradients instead of using VDM type metrics directly one may use numerical input vectors obtained by replacing symbolic and discrete attributes with p(C i x j ) probabilities. Resulting numerical vectors have the number of components equal to the number ....

[Article contains additional citation context not shown here]

W. Duch, N. Jankowski, "Survey of neural transfer functions", Neural Computing Surveys 2: 163--213, 1999.

....N EURAL networks of the most popular multi layer perceptron (MLP) type perform discrimination of the input feature space using hyperplanes. Many other transfer function have been proposed to increase the flexibility of contours used for estimation of decision borders [1] for a recent review see [2]) Perhaps the best known alternative to sigmoidal functions are localized Gaussian functions and other radial basis functions. Viewing the problem of learning from geometrical point of view functions performed by neural nodes should enable tessellation of the input space in the most flexible way ....

....tests) the best model. Networks with activation given by Eq. 3) or (4) have not yet been implemented but such models seem to be quite promising. The change of the shapes of decision borders has been accomplished before by adding new type of units to neural networks (see the forthcoming review [2]) For example, Ridella et al. 19] used circular units in their Circular Backpropagation Networks. Different type of circular units have been used by Kirby and Miranda [20] in their implementation two sigmoidal units are coupled together and their output is restricted to lie on a unit circle. ....

W. Duch, N. Jankowski, Survey of Neural Transfer Functions.Neural Computing Surveys (submitted, 1999)

....at the feature level and in the simplest case d(x i , y i ) x i y i . Mahalanobis distance, more general quadratic distance function and many correlation factors are also suitable for metric functions, for example Camberra, Chebychev, Kendall s Rank Correlation or Chi square distance [5]. All these function may replace the Euclidean distance used in the definition of many transfer functions. For symbolic features the Modified Value Difference Metric (MVDM) is useful [6] A value difference for feature j of two N dimensional vectors x,y with discrete (symbolic) elements, in a C ....

....for any class. d q V (x j , y j ) is used as d( in the Minkovsky distance function for symbolic attributes, combined with other distance functions for numerical attributes. Many variants of MVDM have been discussed in the literature and may be used in neural network activation functions [5, 6]. To avoid problems with calculation of gradients instead of using VDM type metrics directly one may use numerical input vectors obtained by replacing symbolic and discrete attributes with p(C i x j ) probabilities. Resulting numerical vectors have the number of components equal to the number ....

[Article contains additional citation context not shown here]

W. Duch, N. Jankowski, "Survey of neural transfer functions", Neural Computing Surveys 2: 163--213, 1999.

....2) CP (x; t, t # , R) N # i # #(R i x t i ) #(R i x t # i ) # (9) where R i is the i th row of the rotation matrix R with the following structure: R = # # # # # s 1 # 1 0 . s N 1 #N 1 0s N # # # # # (10) For other bicentral transfer function extensions see (Jankowski, 1999; Duch and Jankowski, 1999). 2 10 0 10 10 0 10 0 0.05 0.1 0.15 0.2 Biases 1 1 Slopes .5 .5 10 0 10 10 0 10 0 0.5 1 Biases 5 5 Slopes 1 1 10 0 10 10 0 10 0 0.5 1 Biases 5 5 Slopes 3 3 10 0 10 10 0 10 0 0.2 0.4 0.6 0.8 Biases 2 5 Slopes 1 2 10 0 10 10 0 10 0 0.5 1 Biases 3.5 5 Slopes 3 3 10 0 10 10 0 10 0 0.2 ....

Duch, W. and Jankowski, N. (1999). Survey of neural transfer functions. Neural Computing Surveys. (submitted).

....t, t # , R) N Y i #(R i x t i ) #(R i x t # i ) 9) where R i is the i th row of the rotation matrix R with the following structure: R = 2 6 6 6 4 s 1 # 1 0 . s N 1 #N 1 0 s N 3 7 7 7 5 (10) For other bicentral transfer function extensions see (Jankowski, 1999; Duch and Jankowski, 1999). 10 0 10 10 0 10 0 0.05 0.1 0.15 0.2 Biases 1 1 Slopes .5 .5 10 0 10 10 0 10 0 0.5 1 Biases 5 5 Slopes 1 1 10 0 10 10 0 10 0 0.5 1 Biases 5 5 Slopes 3 3 10 0 10 10 0 10 0 0.2 0.4 0.6 0.8 Biases 2 5 Slopes 1 2 10 0 10 10 0 10 0 0.5 1 Biases 3.5 5 Slopes 3 3 10 0 10 10 0 10 0 0.2 0.4 ....

Duch, W. and Jankowski, N. (1999). Survey of neural transfer functions. Neural Computing Surveys, (2):163-- 212.

....(Eq. 9) CP (x; t, t # , R) N Y i #(R i x t i ) #(R i x t # i ) 9) where R i is the i th row of the rotation matrix R with the following structure: R = 2 6 6 6 4 s 1 # 1 0 . s N 1 #N 1 0 s N 3 7 7 7 5 (10) For other biradial transfer function extensions see [6, 4]. Classification using IncNet. Independed IncNet networks are constructed for each class for a given problem. Each of them receives input vector x and 1 if index of i th sub network is equal to desired class number, otherwise 0. The output of i th network defines how much a given case belongs ....

W. Duch and N. Jankowski. Survey of neural transfer functions. Neural Computing Surveys, 7, 1999. (submitted).

No context found.

W. Duch and N. Jankowski. Survey of neural transfer functions. Neural Computing Surveys, 2:163--212, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC