The 1cycle Policy
Picking the right learning rate at different iterations helps model to converge quickly. The 1cycle policy gives very fast results when training complex models. It follows the Cyclical Learning Rate (CLR) to obtain faster training time with regularization effect but with a slight modification. Specifically, it uses one cycle that is smaller than the total... » read more