ml.lr_schedulers.cosine

Defines a cosine learning rate scheduler.

class ml.lr_schedulers.cosine.CosineLRSchedulerConfig(name: str = '???', total_steps: int = '${task.max_steps}', num_resets: int = 0, phase: int = '???', ramp_up_percent: float = 0.05, ramp_up_steps: int = '???', eta_min: float = 0.01, eta_max: float = 1.0)[source]

Bases: BaseLRSchedulerConfig

total_steps: int = '${task.max_steps}'

num_resets: int = 0

phase: int = '???'

ramp_up_percent: float = 0.05

ramp_up_steps: int = '???'

eta_min: float = 0.01

eta_max: float = 1.0

classmethod resolve(config: CosineLRSchedulerConfig) → None[source]

Runs post-construction config resolution.

Parameters:: config – The config to resolve

class ml.lr_schedulers.cosine.CosineLRScheduler(config: BaseConfigT)[source]

Bases: BaseLRScheduler[CosineLRSchedulerConfig]

get_lr_scale(state: State) → float[source]

Given a state, returns the current learning rate.

Parameters:: state – The current trainer state
Returns:: The computed learning rate to use