This first-year graduate course offers a rigorous yet accessible introduction to the algorithmic foundations of optimization, emphasizing both theory and applications in computer science and engineering. Topics include convex sets and functions, necessary and sufficient conditions for optimality, unconstrained and constrained optimization (gradient descent, Newton’s method, projection methods, Lagrange multipliers, KKT conditions), duality theory, and augmented Lagrangian methods. If time permits, we will also cover stochastic, subgradient, and proximal techniques for large-scale or nondifferentiable problems. The course integrates lectures with problem-solving sessions and programming assignments, enabling students to analyze convergence behavior, understand algorithmic trade-offs, and implement methods for real-world applications in machine learning, resource allocation, and network design.
Prerequisites: Students wishing to enroll should have completed, or have an equivalent background in:
- Multivariable Calculus, including directional derivatives and Taylor expansions;
- Linear Algebra, covering vector spaces, inner products, eigenvalues, and positive‑definite matrices.
Once the quarter starts, instructor approval is required to maintain enrollment in the course, including if students do not have the listed pre-requisite courses completed.