2021
07-09
07-09
pytorch中Schedule与warmup_steps的用法说明
1.lr_scheduler相关lr_scheduler=WarmupLinearSchedule(optimizer,warmup_steps=args.warmup_steps,t_total=num_train_optimization_steps)其中args.warmup_steps可以认为是耐心系数num_train_optimization_steps为模型参数的总更新次数一般来说:num_train_optimization_steps=int(total_train_examples/args.train_batch_size/args.gradient_accumulation_steps)Schedule用来调节学习率,拿线性变换调整来说,下面代...
继续阅读 >