eThK z S r SSKrSSKrSSKrSSKJr SSKJrJr SSKr SSK
J r SSK J
r
JrJrJr SSKJr SS KJr \R* " \5 r\ " S
S 5 5 r " S S
5 r\ " S S\5 5 r " S S5 r " S S\5 r " S S\5 r " S S\5 r " S S\5 r " S S\\5 r g) zJ
Callbacks to use with the Trainer class and customize the training loop.
N) dataclass)OptionalUnion)tqdm )HPSearchBackendIntervalStrategy SaveStrategy
has_length)TrainingArguments)loggingc \ rS rSr% SrSr\\ \S' Sr \
\S' Sr \
\S' Sr \
\S ' Sr
\
\S
' Sr\
\S ' Sr\\
\S ' Sr\
\S
' Sr\
\S' Sr\\S' Sr\\\\4 \S' Sr\\ \S' Sr\\
\S' Sr\\ \S' Sr\\S' Sr \\S' Sr \\S' Sr \\ \S' Sr \\\ \\\
\4 4 \S' Sr!\S \S ' S r"S \4S jr#\$S \4S j5 r%S! r&S" r'S#r(g)$ TrainerState# a
A class containing the [`Trainer`] inner state that will be saved along the model and optimizer when checkpointing
and passed to the [`TrainerCallback`].
In all this class, one step is to be understood as one update step. When using gradient accumulation, one update
step may require several forward and backward passes: if you use `gradient_accumulation_steps=n`, then one update
step requires going through *n* batches.
Args:
epoch (`float`, *optional*):
Only set during training, will represent the epoch the training is at (the decimal part being the
percentage of the current epoch completed).
global_step (`int`, *optional*, defaults to 0):
During training, represents the number of update steps completed.
max_steps (`int`, *optional*, defaults to 0):
The number of update steps to do during the current training.
logging_steps (`int`, *optional*, defaults to 500):
Log every X updates steps
eval_steps (`int`, *optional*):
Run an evaluation every X steps.
save_steps (`int`, *optional*, defaults to 500):
Save checkpoint every X updates steps.
train_batch_size (`int`, *optional*):
The batch size for the training dataloader. Only needed when
`auto_find_batch_size` has been used.
num_input_tokens_seen (`int`, *optional*, defaults to 0):
When tracking the inputs tokens, the number of tokens seen during training (number of input tokens, not the
number of prediction tokens).
total_flos (`float`, *optional*, defaults to 0):
The total number of floating operations done by the model since the beginning of training (stored as floats
to avoid overflow).
log_history (`List[Dict[str, float]]`, *optional*):
The list of logs done since the beginning of training.
best_metric (`float`, *optional*):
When tracking the best model, the value of the best metric encountered so far.
best_global_step (`int`, *optional*):
When tracking the best model, the step at which the best metric was encountered.
Used for setting `best_model_checkpoint`.
best_model_checkpoint (`str`, *optional*):
When tracking the best model, the value of the name of the checkpoint for the best model encountered so
far.
is_local_process_zero (`bool`, *optional*, defaults to `True`):
Whether or not this process is the local (e.g., on one machine if training in a distributed fashion on
several machines) main process.
is_world_process_zero (`bool`, *optional*, defaults to `True`):
Whether or not this process is the global main process (when training in a distributed fashion on several
machines, this is only going to be `True` for one process).
is_hyper_param_search (`bool`, *optional*, defaults to `False`):
Whether we are in the process of a hyper parameter search using Trainer.hyperparameter_search. This will
impact the way data will be logged in TensorBoard.
stateful_callbacks (`List[StatefulTrainerCallback]`, *optional*):
Callbacks attached to the `Trainer` that should have their states be saved or restored.
Relevant callbacks should implement a `state` and `from_state` function.
Nepochr global_step max_stepsi
logging_steps
eval_steps
save_stepstrain_batch_sizenum_train_epochsnum_input_tokens_seen
total_flos log_history best_metricbest_global_stepbest_model_checkpointTis_local_process_zerois_world_process_zeroFis_hyper_param_search
trial_name trial_paramsTrainerCallbackstateful_callbacksc U R c / U l U R c 0 U l g [ U R [ 5 ( a g 0 nU R H n[ U[ 5 ( d [ S[
U5 35 eUR R nX1; aA [ X [ 5 ( d X /X' X R UR 5 5 M UR 5 X' M Xl g )NzNAll callbacks passed to be saved must inherit `ExportableState`, but received ) r r%
isinstancedictExportableState TypeErrortype __class____name__listappendstate)selfr% callbacknames U/var/www/auris/envauris/lib/python3.13/site-packages/transformers/trainer_callback.py
__post_init__TrainerState.__post_init__u s # !D " " *&(D #
//
6
6 "$ 3 3!(_>>#himnviwhxy ))22- &&8&>EE4F4L3M*0&,33HNN4DE/7~~/?&, 4 '9 # json_pathc [ R " [ R " U 5 SSS9S- n[ USSS9 nUR U5 SSS5 g! , ( d f g= f) zDSave the content of this instance in JSON format inside `json_path`. T)indent sort_keys
wutf-8encodingN)jsondumps dataclassesasdictopenwrite)r1 r8 json_stringfs r4 save_to_jsonTrainerState.save_to_json sK jj!3!3D!9!tTW[[
)S7
3q
GGK 4
3
3s A
A"c [ USS9 nUR 5 nSSS5 U " S0 [ R " W5 D6$ ! , ( d f N*= f)z3Create an instance from the content of `json_path`.r? r@ N )rF readrB loads)clsr8 rI texts r4 load_from_jsonTrainerState.load_from_json s@ )g
.!668D /&TZZ%&& /
.s A
Ac S HC n[ X S35 nUc M US: a [ R " X$- 5 n[ X S3U5 ME g)zt
Calculates and stores the absolute value for logging,
eval, and save steps based on if it was a proportion
or not.
)r
evalsave_stepsNr )getattrmathceilsetattr)r1 argsr step_kind num_stepss r4
compute_stepsTrainerState.compute_steps sN 5I 6&:;I$ q= $ )*? @I 6 2I> 5r7 c v UR b- UR b UR UR 5 U l SU l Ub? SSKJn UR [ R :X a UR OUnU" U5 U l X l
X0l UR 5 U l UR 5 U l
g)z9
Stores the initial training references needed in `self`
Nr ) hp_params)hp_name_trialr" r# transformers.integrationsrb hp_search_backendr SIGOPT assignmentsr r r r )r1 trainerr r trialrb rh s r4 init_training_references%TrainerState.init_training_references s ?? &7>>+E &oognn =DO ;/6/H/HOLbLb/b%++hmK )+ 6D " 0 %,%B%B%D"%,%B%B%D"r7 )r r r r r r% r" r# ))r-
__module__ __qualname____firstlineno____doc__r r float__annotations__r intr r r r r r r r r r. r( strr r r r boolr r! r" r# r r% r5 rJ classmethodrR r_ rk __static_attributes__rM r7 r4 r r # sV 9v "E8E?!KIs M3 JJ&*hsm* c !"3"J*.Kd3:&'.#'K% '&*hsm*+/8C=/"&4&"&4&"'4' $J
$<@L$sE#uc4"7 889@26 ./696!c ! 's ' ' ?Er7 r c 6 \ rS rSrSrS\4S jr\S 5 rSr g)r) a
A class for objects that include the ability to have its state
be saved during `Trainer._save_checkpoint` and loaded back in during
`Trainer._load_from_checkpoint`.
These must implement a `state` function that gets called during the respective
Trainer function call. It should only include parameters and attributes needed to
recreate the state at a particular time, to avoid utilizing pickle/maintain standard
file IO writing.
Example:
```python
class EarlyStoppingCallback(TrainerCallback, ExportableState):
def __init__(self, early_stopping_patience: int = 1, early_stopping_threshold: Optional[float] = 0.0):
self.early_stopping_patience = early_stopping_patience
self.early_stopping_threshold = early_stopping_threshold
# early_stopping_patience_counter denotes the number of times validation metrics failed to improve.
self.early_stopping_patience_counter = 0
def state(self) -> dict:
return {
"args": {
"early_stopping_patience": self.early_stopping_patience,
"early_stopping_threshold": self.early_stopping_threshold,
},
"attributes": {
"early_stopping_patience_counter": self.early_stopping_patience_counter,
}
}
```returnc [ S5 e)NzInternal class that just calls the list of callbacks in order.c / U l U H nU R U5 M X l X0l X@l XPl S U l S U l [ S U R 5 5 ( d# [ R SU R - 5 g g )Nc 3 B # U H n[ U[ 5 v M g 7fN)r' DefaultFlowCallback.0cbs r4 +CallbackHandler.__init__.. s P 2:b"566 s zThe Trainer will not work properly if you don't have a `DefaultFlowCallback` in its callbacks. You
should add one before training with `trainer.add_callback(DefaultFlowCallback). The current list ofcallbacks is
:) callbacks add_callbackmodelprocessing_class optimizer lr_schedulertrain_dataloadereval_dataloaderanyloggerwarning
callback_list)r1 r r r r r r s r4 __init__CallbackHandler.__init__ s B b !
0 " ( $ # P PPP NN$ $$%
Qr7 c j [ U[ 5 ( a U" 5 OUn[ U[ 5 ( a UO UR nX0R Vs/ s H oDR PM sn; a) [ R SU S3S- U R - 5 U R R U5 g s snf )NzYou are adding a zH to the callbacks of this Trainer, but there is already one. The currentzlist of callbacks is
:)r' r+ r, r r r r r/ )r1 r2 r cb_classcs r4 r CallbackHandler.add_callback s %h55XZ8 )(D 9 98x?Q?Q ^^<^ ^< < NN#H:-uv+,$$%
b!
=s B0c " [ U[ 5 ( aC U R H2 n[ X!5 ( d M U R R U5 Us $ g U R H' nX!:X d M
U R R U5 Us $ g r r' r+ r remover1 r2 r s r4 pop_callback CallbackHandler.pop_callback sk h % %nn b++NN))"- I %
nn>NN))"- I %r7 c [ U[ 5 ( aA U R H0 n[ X!5 ( d M U R R U5 g g U R R U5 g r r r s r4 remove_callback CallbackHandler.remove_callback sQ h % %nn b++NN))"- %
NN ! !( +r7 c F SR S U R 5 5 $ )Nr= c 3 L # U H oR R v M g 7fr )r, r- r s r4 r 0CallbackHandler.callback_list.. s H 2 .. s "$)joinr r} s r4 r CallbackHandler.callback_list s yyH HHHr7 r\ r0 r c ( U R SXU5 $ )Nr
call_eventr1 r\ r0 r s r4 r CallbackHandler.on_init_end }d7CCr7 c 6 SUl U R SXU5 $ )NFr )r r r s r4 r CallbackHandler.on_train_begin s ',$ /gFFr7 c ( U R SXU5 $ )Nr r r s r4 r CallbackHandler.on_train_end ~tGDDr7 c 6 SUl U R SXU5 $ )NFr )r r r s r4 r CallbackHandler.on_epoch_begin s $)! /gFFr7 c ( U R SXU5 $ )Nr r r s r4 r CallbackHandler.on_epoch_end r r7 c R SUl SUl SUl U R SXU5 $ )NFr )r r r r r s r4 r CallbackHandler.on_step_begin s- ""' #WEEr7 c ( U R SXU5 $ )Nr r r s r4 r %CallbackHandler.on_pre_optimizer_step s 6WMMr7 c ( U R SXU5 $ )Nr r r s r4 r !CallbackHandler.on_optimizer_step s 2DIIr7 c ( U R SXU5 $ )Nr r r s r4 r CallbackHandler.on_substep_end s /gFFr7 c ( U R SXU5 $ )Nr r r s r4 r CallbackHandler.on_step_end r r7 c 2 SUl U R SXX4S9$ )NFr r )r r r1 r\ r0 r r s r4 r CallbackHandler.on_evaluate s "' }d7TTr7 c $ U R SXX4S9$ )Nr r r r s r4 r CallbackHandler.on_predict s |T'SSr7 c 6 SUl U R SXU5 $ )NFr )r r r s r4 r CallbackHandler.on_save s #y$w??r7 c 2 SUl U R SXX4S9$ )NFr )logs)r r )r1 r\ r0 r r s r4 r CallbackHandler.on_log# s "xgIIr7 c ( U R SXU5 $ )Nr r r s r4 r "CallbackHandler.on_prediction_step' s 3T'JJr7 c U R Ha n[ Xa5 " UUU4U R U R U R U R
U R U R S.UD6nUc M_ UnMc U$ )N)r r r r r r )r rX r r r r r r )r1 eventr\ r0 r r r2 results r4 r CallbackHandler.call_event* s H X- jj!%!6!6 .. ! . .!%!6!6 $ 4 4 F ! ' r7 )r r r r r r r N) r- rm rn ro rp r r r r propertyr r r r r r r r r r r r r r r r r r r r rw rM r7 r4 r r s H& "
, I ID 1 D, DQ_ DG#4 G\ GTb GE!2 E< ER` EG#4 G\ GTb GE!2 E< ER` EF"3 FL FSa F N*; NL N[i NJ&7 J JWe JG#4 G\ GTb GD 1 D, DQ_ DU 1 U, UQ_ UT 0 T TP^ T@ - @l @^ @J, J\ JN JK'8 K KXf Kr7 r c D \ rS rSrSrS\S\S\4S jrS\S\S\4S jr Sr
g )
r i> zp
A [`TrainerCallback`] that handles the default flow of the training loop for logs, evaluation and checkpoints.
r\ r0 r c UR S:X a UR ( a SUl UR [ R
:X a$ UR UR - S:X a SUl UR [ R
:X a> UR UR - S:X a! UR UR :: a SUl
UR [ R
:X a4 UR S: a$ UR UR - S:X a SUl UR UR : a, SUl UR [ R
:X a SUl U$ )Nr Tr )r logging_first_stepr logging_strategyr STEPSr
eval_strategyr
eval_delayr
save_strategyr
r r r r r s r4 r DefaultFlowCallback.on_step_endC s# !d&=&=!%G $4$:$: :u?P?PSXSfSf?fjk?k!%G
"2"8"8 8!!E$4$4495#4#44&*G #
,"4"4 4 1$!!E$4$449"&G /+/G (!!\%7%77&*#r7 c UR [ R :X a SUl UR [ R :X a! UR
UR :: a SUl UR [ R :X a SUl
U$ )NT) r) r EPOCHr r+ r, r r r- r
r r s r4 r DefaultFlowCallback.on_epoch_endc sp $4$:$: :!%G !1!7!7 7DOOu{{ s@ 1 , Q_ @
!2
<
R`
r7 r c \ \ rS rSrSrSS\4S jjrS rS rSS jr S r
S
r SS jr S r
S
rg)ProgressCallbackis z
A [`TrainerCallback`] that displays the progress of training or evaluation.
You can modify `max_str_len` to control how long strings are truncated when logging.
max_str_lenc , SU l SU l Xl g)z
Initialize the callback with optional max_str_len parameter to control string truncation length.
Args:
max_str_len (`int`):
Maximum length of strings to display in logs.
Longer strings will be truncated with a message.
N) training_barprediction_barr4 )r1 r4 s r4 r ProgressCallback.__init__y s ! "&r7 c f UR ( a [ UR SS9U l SU l g )NT)total
dynamic_ncolsr )r r r r6 current_stepr s r4 r ProgressCallback.on_train_begin s& & & $5??$ OD r7 c UR ( aD U R R UR U R -
5 UR U l g g r )r r6 updater r< r s r4 r ProgressCallback.on_step_end sC & & $ $U%6%69J9J%J K % 1 1D 'r7 Nc UR ( a_ [ U5 ( aN U R c% [ [ U5 U R
S L SS9U l U R R
S5 g g g )NT)r: leaver; r )r r r7 r lenr6 r? )r1 r\ r0 r r r s r4 r #ProgressCallback.on_prediction_step se & &:o+F+F""*&* o.d6G6G46O_c'#
& &q ) ,G &r7 c UR ( a/ U R b U R R 5 S U l g g r r r7 closer s r4 r ProgressCallback.on_evaluate 6 & &"".##))+"&D 'r7 c UR ( a/ U R b U R R 5 S U l g g r rF r s r4 r ProgressCallback.on_predict rI r7 c UR ( a U R b 0 nUR 5 HW u px[ U[ 5 ( a9 [ U5 U R : a S[ U5 SU R S3Xg' MS XU' MY UR SS 5 n SU; a [ US S5 US' U R R [ U5 5 g g g )Nz%[String too long to display, length: z > z/. Consider increasing `max_str_len` if needed.]r r r: )
r r6 r r' rt rC r4 poproundrG )
r1 r\ r0 r r r shallow_logsr r _s
r4 r ProgressCallback.on_log s & &4+<+<+H L
a%%#a&43C3C*C?Axs4K[K[J\ ]H H !O
'(O % t4A,&(-l7.CQ(G W% # #C $5 6! ,I &r7 c j UR ( a" U R R 5 S U l g g r )r r6 rG r s r4 r ProgressCallback.on_train_end s* & & # # % $D 'r7 )r< r4 r7 r6 )d r )r- rm rn ro rp rs r r r r r r r r rw rM r7 r4 r3 r3 s s6
'C '
2
*' ' 7&%r7 r3 c " \ rS rSrSrSS jrSrg)PrinterCallbacki z7
A bare [`TrainerCallback`] that just prints the logs.
Nc b UR SS 5 nUR ( a [ U5 g g )Nr )rM r print)r1 r\ r0 r r r rP s r4 r PrinterCallback.on_log s' HH\4 ( & & $K 'r7 rM r )r- rm rn ro rp r rw rM r7 r4 rV rV s r7 rV c T \ rS rSrSrS
S\S\\ 4S jjrS r S r
S r S \ 4S
jr
S rg )EarlyStoppingCallbacki a
A [`TrainerCallback`] that handles early stopping.
Args:
early_stopping_patience (`int`):
Use with `metric_for_best_model` to stop training when the specified metric worsens for
`early_stopping_patience` evaluation calls.
early_stopping_threshold(`float`, *optional*):
Use with TrainingArguments `metric_for_best_model` and `early_stopping_patience` to denote how much the
specified metric must improve to satisfy early stopping conditions. `
This callback depends on [`TrainingArguments`] argument *load_best_model_at_end* functionality to set best_metric
in [`TrainerState`]. Note that if the [`TrainingArguments`] argument *save_steps* differs from *eval_steps*, the
early stopping will not occur until the next save step.
early_stopping_patienceearly_stopping_thresholdc * Xl X l SU l g )Nr r\ r] early_stopping_patience_counter)r1 r\ r] s r4 r EarlyStoppingCallback.__init__ s '>$(@%/0,r7 c 2 UR ( a [ R O[ R nUR b< U" XBR 5 ( a- [ XBR -
5 U R : a SU l g U =R S-
sl g )Nr r )greater_is_betternpgreaterlessr absr] r` )r1 r\ r0 r metric_valueoperators r4 check_metric_value(EarlyStoppingCallback.check_metric_value sk !%!7!72::RWW $ \#4#4 5 5L#4#4458U8UU34D 0 0 0A 5 0r7 c UR ( d [ R S5 UR c S5 eUR [
R :w d S5 eg )NzUsing EarlyStoppingCallback without load_best_model_at_end=True. Once training is finished, the best model will not be loaded automatically.zBEarlyStoppingCallback requires metric_for_best_model to be definedzAEarlyStoppingCallback requires IntervalStrategy of steps or epoch)load_best_model_at_endr r metric_for_best_modelr+ r NOr s r4 r $EarlyStoppingCallback.on_train_begin sb ** NN^
))5
P
5 !!%5%8%88
O
8r7 c UR nUR S5 ( d SU 3nUR U5 nUc [ R SU S35 g U R XX75 U R U R : a SUl g g )Neval_z@early stopping required metric_for_best_model, but did not find z so early stopping is disabledT) rn
startswithgetr r rj r` r\ r )r1 r\ r0 r r r metric_to_checkrh s r4 r !EarlyStoppingCallback.on_evaluate s 44 ))'22 %o%6 7O {{?3 NNRSbRc d
WC / /43O3O O+/G ( Pr7 rz c R U R U R S.SU R 0S.$ )N)r\ r] r` r r_ r} s r4 r0 EarlyStoppingCallback.state s7 ,0+G+G,0,I,I
243W3W
r7 )r\ r` r] N)r g )r- rm rn ro rp rs r rq r rj r r r( r0 rw rM r7 r4 r[ r[ s< 1 1S[\aSb 1 6
0"
t
r7 r[ )!rp rD rB rY r typingr r numpyrd tqdm.autor
trainer_utilsr r r
r
training_argsr utilsr
get_loggerr- r r r) r r$ r r r3 rV r[ rM r7 r4 r s ! " V V ,
H % XE XE XEv) )X :
_ :
:
zO
O
d@o @F2/ 2jG% G%To I
O_ I
r7