Nonlinear Optimization for Electrical Engineers

Instructor: Mohamed Bakr

Date	Lecture	Description
	0	Course Outline
Sept. 18th	1	Introduction: Historical Background, statement of optimization problem
	2	Introduction: Classifications of Optimization problems, Mathematical background
Sept. 25^th	3	Classical Optimization Methods: single variable optimization, unconstrained multivariate optimization
	4	Equality Constraints: Solution by Direct substitution, Method of constrained variation
	5	Equality Constriants: Method of Lagrange multipliers
Oct. 2^nd	6	Inequality constraints: Kuhn-Tucker Conditions, Constraint qualification
	7	One Dimensional Search: why one dimensional search?, Search with Fixed Step Size, Search with Accelerated Step size
	8	One Dimensional Search: Interval halving Method, Fibonacci Method, Golden Section Search
Oct. 16^th	9	One Dimensional Search: Interpolation Methods, Newton Method
	10	One Dimensional Search: Quasi-Newton Method, Secant Method, Practical Consideration
Oct. 23^rd	11	Unconstrained Nonlinear Optimization: Introduction and basic concepts
	12	Direct Search Methods: Random Walks, Grid Search, Univariate Method
	13	Conjugate Gradient Methods: Hooke and Jeeves Method, Powell’s Method, Simplex Method
Oct 30^th	14	Indirect Methods: Steepest Descent, Conjugate Directions, Conjugate Gradients
	15	2^nd Order Methods: Newton Method, Marquardt Method
	16	2^nd Order Methods (Cont’d): Quasi Newton Methods, The DFP formula, the BFGS formula, summary
	17	2^nd Order Methods (Cont’d): Linear Least Squares, Nonlinear Least Squares, Newton-Gauss method, applications
	20	Constrained Nonlinear Optimization: Introduction, Random Methods, Complex Method
	19	Some Constrained Optimization Methods: Zoutendijk’s method of feasible directions
	20	Constrained Optimization (Cont’d): Rosen’s Gradient projection Method
	21	Constrained Optimization (Cont’d): Quadratic Programming
	22	Constrained Optimization (Cont’d): Sequential Quadratic Programming
	23	Constrained Optimization (Cont’d): Penalty and Barrier Methods
	24	Global Optimization Techniques: Genetic Algorithms
	25	Global Optimization Techniques (Cont’d): Simulated annealing
	26	Global Optimization Techniques(Cont’d): Particle Swarm Optimization
	27	Linear Programming: The Simplex Method
	28	Linear Programming: Interior Point Methods
	29	Space Mapping Optimization and Modeling: Basic Concepts, classical Space Mapping, Aggressive Space Mapping
	30	Space Mapping (Cont’d): surrogate-based optimization, Output Space Mapping
	31	Adjoint Variable Methods: The Dynamic Case