A Warmup Scheduler for Pytorch
Project description
Warmup Scheduler Pytorch
Description
A Warmup Scheduler in Pytorch to make the learning rate change at the beginning of training for warmup.
Install
Notice: need to install pytorch>=1.1.0 manually.
The official website is PyTorch
Then install as follows:
pip install warmup_scheduler_pytorch
Usage
Detail to see GitHub example.py file.
import torch
from torch.optim import SGD # example
from torch.optim.lr_scheduler import CosineAnnealingLR # example
from warmup_scheduler_pytorch import WarmUpScheduler
model = Model()
optimizer = SGD(model.parameters(), lr=0.1)
lr_scheduler = CosineAnnealingLR(optimizer, T_max=100, eta_min=0.01)
data_loader = torch.utils.data.DataLoader(...)
warmup_scheduler = WarmUpScheduler(optimizer, lr_scheduler,
len_loader=len(data_loader),
warmup_steps=100,
warmup_start_lr=0.01,
warmup_mode='linear')
epochs = 100
for epoch in range(epochs):
for batch_data in data_loader:
output = model(...)
# loss = loss_fn(output, ...)
# loss.backward()
optimizer.step()
optimizer.zero_grad()
warmup_scheduler.step()
# lr_scheduler.step() is no longer needed
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for warmup_scheduler_pytorch-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b3e67db7db952b3cb38801528a4c76107fb57143b29cf0c8e1aeef2561b1b31f |
|
MD5 | 18ec324cce14fc2d74116c5f9b9788d3 |
|
BLAKE2b-256 | 58fe514711c5f50d8bb7324f6177eb24fb785a071abd7d986b1cd97c46930b8f |
Close
Hashes for warmup_scheduler_pytorch-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bdfb411fcd46d1da69cbc1634ac55c9292b13230600cb8ca8b7dfe1377647553 |
|
MD5 | 90a96dd0069277baf26a91cbd6645935 |
|
BLAKE2b-256 | af0a640dfb2fbddeaae89d8d7dd84307b75be7fe4dd668844c4f01a518e6cc4c |