Nonlinear Optimization for Electrical Engineers

Instructor: Mohamed Bakr

Date	Lecture	Description
Jan. 15^th	0	Course Outline
Jan 18^th	1	Introduction: Historical Background, statement of optimization problem
	2	Introduction: Classifications of Optimization problems, Mathematical background
Jan. 25^th	3	Classical Optimization Methods: single variable optimization, unconstrained multivariate optimization
	4	Equality Constraints: Solution by Direct substitution, Method of constrained variation
	5	Equality Constriants: Method of Lagrange multipliers
Feb 1^st	6	Inequality constraints: Kuhn-Tucker Conditions, Constraint qualification
	7	One Dimensional Search: why one dimensional search?, Search with Fixed Step Size, Search with Accelerated Step size
	8	One Dimensional Search: Interval halving Method, Fibonacci Method, Golden Section Search
Feb. 8^th	9	One Dimensional Search: Interpolation Methods, Newton Method
	10	One Dimensional Search: Quasi-Newton Method, Secant Method, Practical Consideration
Feb. 15^th		Reading Week (No Class)
Feb. 22^nd	11	Unconstrained Nonlinear Optimization: Introduction and basic concepts
	12	Direct Search Methods: Random Walks, Grid Search, Univariate Method
	13	Conjugate Gradient Methods: Hooke and Jeeves Method, Powell’s Method, Simplex Method
March 1^st	14	Indirect Methods: Steepest Descent, Conjugate Directions, Conjugate Gradients
	15	2^nd Order Methods: Newton Method, Marquardt Method
March 8^th	16	2^nd Order Methods (Cont’d): Quasi Newton Methods, The DFP formula, the BFGS formula, summary
	17	2^nd Order Methods (Cont’d): Linear Least Squares, Nonlinear Least Squares, Newton-Gauss method, applications
March 15^th	18	Linear Programming: The Simplex Method
	19	Linear Programming: Interior Point Methods
March 22^nd	20	Constrained Nonlinear Optimization: Introduction, Random Methods, Complex Method
	21	Some Constrained Optimization Methods: Zoutendijk’s method of feasible directions
	22	Constrained Optimization (Cont’d): Rosen’s Gradient projection Method
March 29^th	23	Constrained Optimization (Cont’d): Quadratic Programming
	24	Constrained Optimization (Cont’d): Sequential Quadratic Programming
	25	Constrained Optimization (Cont’d): Penalty and Barrier Methods
April 12^th	26	Global Optimization Techniques: Genetic Algorithms
	27	Global Optimization Techniques (Cont’d): Simulated annealing
	28	Global Optimization Techniques(Cont’d): Particle Swarm Optimization
April 19^th	29	Space Mapping Optimization and Modeling: Basic Concepts, classical Space Mapping, Aggressive Space Mapping
	30	Space Mapping (Cont’d): surrogate-based optimization, Output Space Mapping
May 10^th	31	Adjoint Variable Methods: The Dynamic Case