Adam

When you’re training a deep learning model, it can take days for an optimization algorithm to minimize the loss function. A new approach could save time.

Everyone has a favorite optimization method, but it’s not always clear which one works best in a given situation. New research aims to establish a set of benchmarks. Researchers evaluated 14 popular optimizers using the Deep Optimization Benchmark Suite some of them introduced last year.