DMCA
A Data Mining Approach in the Analysis of the Weight Space of Multilayer Perceptron Solving Complex Real World Tasks
BibTeX
@MISC{Adam_adata,
author = {S. Adam and D. A. Karras and M. N Vrahatis and Hpeirou Arta},
title = {A Data Mining Approach in the Analysis of the Weight Space of Multilayer Perceptron Solving Complex Real World Tasks},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract: One of the main reasons for the slow convergence and the suboptimal generalization results of MLP (Multilayer Perceptrons) based on gradient descent training is the lack of a proper initialization of the weights to be adjusted. Even sophisticated learning procedures are not able to compensate for bad initial values of weights, while good initial guess leads to fast convergence and or better generalization capability even with simple gradientbased error minimization techniques. Although initial weight space in MLPs seems so critical there is no study so far of its properties with regards to which regions lead to solutions or failures concerning generalization and convergence in real world problems. There exist only some preliminary studies for toy problems, like XOR. A data mining approach, based on Self Organizing Feature Maps (SOM), is involved in this paper to demonstrate that a complete analysis of the MLP weight space is possible even in the case of complex real world problems. This is the main novelty of this paper. The conclusions drawn from this novel application of SOM algorithm in MLP analysis extend significantly previous preliminary results in the literature. MLP initialization procedures are overviewed along with all conclusions so far drawn in the literature and an extensive experimental study on more representative tasks, using our data mining approach, reveals important initial weight space properties of MLPs, extending previous knowledge and literature results.
Keyphrases
data mining approach weight space mlp analysis generalization capability good initial guess main novelty bad initial value mlp weight space preliminary study novel application mlp initialization procedure complete analysis som algorithm suboptimal generalization result previous preliminary result simple gradientbased error minimization technique multilayer perceptrons literature result proper initialization slow convergence toy problem previous knowledge complex real world problem representative task real world problem extensive experimental study main reason sophisticated learning procedure gradient descent training self organizing feature map initial weight space
