Last updated 3 years ago
To get out of saddle points or local minima
When model could be sensitive to initial learning rate
scheduler = torch.optim.lr_scheduler.CyclicLR(optimizer, 1e-4, 1e-3, cycle_momentum=False)
Link to reading
