Nick Korbit

Hi, I'm Nick! I am a PhD student at IMT Lucca, where I work in the DYSCO lab under the supervision of Mario Zanon and Alberto Bemporad. And DYSCO is what I do - studying DYnamical Systems, Control and Optimization, focusing on scalable second-order optimization algorithms for modern-day neural networks.

Prior to starting my PhD, I was working on autonomous delivery robots at Starship Technologies and modeling risk at CompatibL.

Email  /  Google Scholar  /  GitHub  /  LinkedIn

I want to make neural network training fast, efficient and scalable. To achieve this, I primarily focus on second-order optimization methods, with a specific interest in the Gauss-Newton approach as well as diagonal Hessian approximation.

We propose a stochastic second-order optimization algorithm - EGN - that efficiently computes the descent direction by using a low-rank Gauss-Newton Hessian approximation.

We propose IGND - a scale-invariant, easy-to-tune, fast-converging stochastic optimization algorithm based on approximate second-order information with nearly the same per-iteration complexity as Stochastic Gradient Descent.

I find JAX to be a great tool for experimenting with new optimization algorithms. During my PhD I've been inspired by specialty JAX repositories like Patrick Kidger's diffrax and equinox, as well as Robert Lange's gymnax. So, as part of my thesis I've been working on somax - a second-order complement to optax (which is a library of mostly first-order solvers).

Somax is a library of stochastic second-order methods for machine learning optimization written in JAX. Somax is based on the JAXopt StochasticSolver API, and can be used as a drop-in replacement for JAXopt as well as Optax solvers.

How to Design Machine Learning Experiments [In Preparation]
Course Co-creator and Instructor

Kudos to Jon Barron for this template!