Abstract

This study empirically investigates variations of hill climbing algorithms for training artificial neural networks on the 5-bit parity classification task. The experiments compare the algorithms when they use different combinations of random number distributions, variations in the step size and changes of the neural networks ' initial weight distribution. A hill climbing algorithm which uses inline search is proposed. In most experiments on the 5-bit parity task it performed better than simulated annealing and standard hill climbing. 1 Introduction Hill climbing is an optimisation algorithm which historically was first used for maximisation tasks. Like gradient descent algorithms in continuous spaces it approaches a local extremum but instead of using the gradient, hill climbing uses random local search to determine the direction and size of each new step. The terminology in the literature is not uniform. In this study hill climbing (HC) is the algorithm presented in Table 1 and hill ...

Citations