In this thesis, we present new methods for solving nonlinear optimization problems. These problems are difficult to solve because the nonlinear constraints form feasible regions that are difficult to find, and the nonlinear objectives contain local minima that trap descent-type search methods. In order to find good solutions in nonlinear optimization, we focus on the following two key issues: how to handle nonlinear constraints and how to escape from local minima. We use a Lagrange-multiplier-based formulation to handle nonlinear constraints, and develop Lagrangian methods with dynamic control to provide faster and more robust convergence. We extend the traditional Lagrangian theory for the continuous space to the discrete space and develop efficient discrete Lagrangian methods. To overcome local minima, we design a new trace-based global-search method that relies on an external traveling trace to pull a search trajectory out of a local optimum in a continuous fashion without having to restart the search from a new starting point. Good starting points identified in the global search are used in the local search to identify true local optima. By combining these new methods, we develop a prototype, called Novel (Nonlinear Optimization Via External Lead), that solves nonlinear constrained and unconstrained problems in a unified framework.
|
4828
|
Genetic Algorithms
– Goldberg
- 1989
|
|
2172
|
Optimization by simulated annealing
– Kirkpatrick, Jr, et al.
- 1983
|
|
1316
|
Genetic Algorithms + Data Structures = Evolution Programs
– Michalewicz
- 1994
|
|
834
|
Reducibility among Combinatorial Problems
– Karp
- 1972
|
|
778
|
A computing procedure for quantification theory
– Davis, Putnam
- 1960
|
|
738
|
A machine-oriented logic based on the resolution principle
– Robinson
- 1965
|
|
686
|
and Nonlinear Programming
– Luenberger, Linear
- 1984
|
|
658
|
Practical methods of optimization
– Fletcher
- 1987
|
|
629
|
Foundations of Constraint Satisfaction
– Tsang
- 1993
|
|
607
|
A simplex method for function minimization
– Nelder, Mead
- 1965
|
|
599
|
Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall
– DENNIS, SCHNABEL
- 1983
|
|
574
|
Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming
– Goemans, Williamson
- 1995
|
|
556
|
A new method for solving hard satisfiability problems
– Selman, Levesque, et al.
- 1992
|
|
551
|
The cascade-correlation learning architecture
– Fahlman, Lebiere
- 1991
|
|
496
|
The complexity of theorem-proving procedures
– Cook
- 1971
|
|
474
|
Linear Programming and Extensions
– Dantzig
- 1963
|
|
418
|
A logical calculus of the ideas immanent in nervous activity
– McCulloch, Pitts
- 1943
|
|
401
|
Parallel networks that learn to pronounce english text
– Sejnowski, Rosenberg
- 1987
|
|
354
|
Tabu search
– Glover, Laguna
- 1997
|
|
347
|
Global Optimization Using Interval Analysis
– HANSEN, WALSTER
- 2004
|
|
332
|
Minimizing conflicts: A heuristic repair method for constraint satisfaction and scheduling problems
– Minton, Johnston, et al.
- 1992
|
|
322
|
A learning algorithm for continually running fully recurrent neural networks
– Williams, Zipser
- 1990
|
|
285
|
Noise strategies for improving local search
– Selman, Kautz, et al.
- 1994
|
|
283
|
Greedy Randomized Adaptive Search Procedure
– Feo, Resende
- 1995
|
|
273
|
Parallel distributed processing: Explorations in the microstructure of cognition: Vol. 1. Foundations
– Rumelhart, McClelland
- 1986
|
|
257
|
Iterative solution of nonlinear equations in several variables
– Ortega, Rheinboldt
- 1970
|
|
249
|
and easy distributions of SAT problems
– Hard
- 1992
|
|
245
|
Increased Rates of Convergence Through Learning Rate Adaptation. Neural Networks
– Jacobs
- 1988
|
|
223
|
Faster-Learning Variations on Back-Propagation: An Empirical Study
– Fahlman
- 1988
|
|
207
|
A thermodinamical approach to the traveling salesman problem: an efficient simulated annealing algorithm
– Cerny
- 1985
|
|
196
|
Domain independent extensions to GSAT: Solving large structured satis - ability problems
– Selman, Kautz
- 1993
|
|
188
|
Local Search Strategies for Satisfiability Testing
– Selman, Kautz, et al.
- 1995
|
|
172
|
Multiple Criteria Optimization: Theory, Computation, and Application
– Steuer
- 1986
|
|
169
|
The breakout method for escaping from local minima
– Morris
- 1993
|
|
164
|
The reactive tabu search
– Battiti, Tecchiolli
- 1994
|
|
159
|
A scaled conjugate gradient algorithm for fast supervised learning
– Møller
- 1993
|
|
149
|
Analysis of hidden units in a layered network trained to classify sonar targets
– Gorman, Sejnowski
- 1988
|
|
144
|
Global optimization: deterministic approaches, 2nd ed
– Horst, Tuy
- 1993
|
|
137
|
Zilinskas. Global optimization
– Törn
- 1989
|
|
129
|
An Introduction to Simulated Evolutionary Optimization
– Fogel
- 1994
|
|
126
|
The traveling salesman problem and minimum spanning trees
– Held, Karp
- 1970
|
|
116
|
Towards an understanding of hill-climbing procedures for SAT
– Gent, Walsh
- 1993
|
|
112
|
Linear-space best-first search
– Korf
- 1993
|
|
109
|
Gradient Calculations for Dynamic Recurrent Neural Networks: A Survey
– Pearlmutter
- 1995
|
|
106
|
Generalization of back-propagation to recurrent neural networks
– Pineda
- 1987
|
|
105
|
Minimising multimodal functions of continuous variables with the “simulated annealing” algorithm
– Corana, Marchesi, et al.
- 1987
|
|
101
|
Genetic algorithms and neural networks: optimizing connections and connectivity
– Whitley, Starkweather, et al.
- 1990
|
|
101
|
An e cient gradient-based algorithm for on-line training of recurrent network trajectories
– Williams, Peng
- 1990
|
|
100
|
Global optimization
– Horst, Tuy
- 1990
|
|
99
|
Genet: A connectionist architecture for solving constraint satisfaction problems by iterative improvement
– Davenport, Tsang, et al.
- 1994
|
