JThC SSKJrJrJr SSKrSSKJr SSKJrJrJ r J r J r J r J r JrJrJrJrJrJrJr SS/r"SS\5rS S \S \ S \S \ S \S 3 -\lS\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\S\4SjrS\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\S\4Sjr\ "\S9S!S\\S\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\S\4S jj5rg)")castOptionalUnionN)Tensor)_capturable_doc_default_to_fused_or_foreach_differentiable_doc_disable_dynamo_if_unsupported _foreach_doc!_get_capturable_supported_devices_get_scalar_dtype _get_value _maximize_doc _params_doc_use_grad_for_differentiable _view_as_real OptimizerParamsTAdamaxadamaxc^\rSrSrSSSSS.S\S\\\4S\\\4S\S\S \ \ S \ S \ S \ 4U4S jjjjr U4Sjr Sr \SSj5rSrU=r$)rF)maximizedifferentiable capturableparamslrbetaseps weight_decayforeachrrrc >[U[5(aUR5S:wa [S5eSU::d[SU35eSU::d[SU35eSUSs=::aS:dO [SUS35eSUSs=::aS:dO [S US35eSU::d[S U35e[ UUUUUUUU S 9n [ T U]X5 g) NrzTensor lr must be 1-elementzInvalid learning rate: zInvalid epsilon value: r?z#Invalid beta parameter at index 0: z#Invalid beta parameter at index 1: zInvalid weight_decay value: )rrr r!r"rrr) isinstancernumel ValueErrordictsuper__init__) selfrrrr r!r"rrrdefaults __class__s J/var/www/auris/envauris/lib/python3.13/site-packages/torch/optim/adamax.pyr+Adamax.__init__s b& ! !bhhjAo:; ;by6rd;< <cz6se<= =eAh$$B58*MN NeAh$$B58*MN Nl";L>JK K%)!   *cP>[TU]U5 URGHnURSS5 URSS5 URSS5 URSS5 USHnURR U/5n[ U5S:wdM0[R"US5(aMP[US5nUS(a([R"U[5URS 9O[R"U[5S 9US'M GM g) Nr"rFrrrrstepdtypedevicer5) r* __setstate__ param_groups setdefaultstategetlentorch is_tensorfloattensorrr6)r,r;grouppp_statestep_valr.s r/r8Adamax.__setstate__Cs U#&&E   Y -   Z /   -u 5   \5 18_**..B/w<1$U__WV_-M-M$WV_5H !. $,=,?#\\(:K:MN FO % 'r1c SnUSGHvnURcMU[R"U5-nURU5 URR(a [ S5eURUR5 UR Un [U 5S:XaUS(a([R"S[5URS9O[R"S[5S 9U S '[R"U[RS 9U S '[R"U[RS 9U S 'URU S 5 URU S 5 URU S 5 GMy U$)NFrz(Adamax does not support sparse gradientsrrr4r$r7r3) memory_formatexp_avgexp_inf)gradr> is_complexappend is_sparse RuntimeErrorr;r=zerosrr6rA zeros_likepreserve_format) r,rBparams_with_gradgradsexp_avgsexp_infs state_steps has_complexrCr;s r/ _init_groupAdamax._init_groupVsJ xAvv~ 5++A. .K  # #A &vv"#MNN LL JJqME5zQ\*KK*;*=ahhOc1B1DEf $)#3#3U%:%:$i $)#3#3U%:%:$i  OOE), - OOE), -   uV} -7!:r1c~UR5 SnUb%[R"5 U"5nSSS5 URHan/n/n/n/n/nUSupUSn USn USn USnUSnUSnUS nUR X4XVXx5n[ UUUUUU U U U U UUUUUS 9 Mc U$!,(df  N=f) zPerforms a single optimization step. Args: closure (Callable, optional): A closure that reevaluates the model and returns the loss. Nrr rr!r"rrr) r beta1beta2rr!r"rrrrY) _cuda_graph_capture_health_checkr> enable_gradr9rZr)r,closurelossrBrTrUrVrWrXr]r^r rr!r"rrrrYs r/r3 Adamax.stepys --/  ""$y%&&E-/ "$E%'H%'H(*K >LE,CtB 0LI&GZ(H"#34N|,J**(K  )!-%' )'L S%$s B.. B<rH)gMb`?)g?g+?g:0yE>rNN)__name__ __module__ __qualname____firstlineno__rrr@rtuplerboolr+r8rZrr3__static_attributes__ __classcell__)r.s@r/rrs$(%1"&$+$ $+$+ %- $+UE\" $+  $+  $+$$+$+$+$+$+L&!F"4"4r1aImplements Adamax algorithm (a variant of Adam based on infinity norm). .. math:: \begin{aligned} &\rule{110mm}{0.4pt} \\ &\textbf{input} : \gamma \text{ (lr)}, \beta_1, \beta_2 \text{ (betas)},\theta_0 \text{ (params)},f(\theta) \text{ (objective)}, \: \lambda \text{ (weight decay)}, \\ &\hspace{13mm} \epsilon \text{ (epsilon)} \\ &\textbf{initialize} : m_0 \leftarrow 0 \text{ ( first moment)}, u_0 \leftarrow 0 \text{ ( infinity norm)} \\[-1.ex] &\rule{110mm}{0.4pt} \\ &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \: \textbf{do} \\ &\hspace{5mm}g_t \leftarrow \nabla_{\theta} f_t (\theta_{t-1}) \\ &\hspace{5mm}if \: \lambda \neq 0 \\ &\hspace{10mm} g_t \leftarrow g_t + \lambda \theta_{t-1} \\ &\hspace{5mm}m_t \leftarrow \beta_1 m_{t-1} + (1 - \beta_1) g_t \\ &\hspace{5mm}u_t \leftarrow \mathrm{max}(\beta_2 u_{t-1}, |g_{t}|+\epsilon) \\ &\hspace{5mm}\theta_t \leftarrow \theta_{t-1} - \frac{\gamma m_t}{(1-\beta^t_1) u_t} \\ &\rule{110mm}{0.4pt} \\[-1.ex] &\bf{return} \: \theta_t \\[-1.ex] &\rule{110mm}{0.4pt} \\[-1.ex] \end{aligned} For further details regarding the algorithm we refer to `Adam: A Method for Stochastic Optimization`_. z Args: a lr (float, Tensor, optional): learning rate (default: 2e-3) betas (Tuple[float, float], optional): coefficients used for computing running averages of gradient and its square eps (float, optional): term added to the denominator to improve numerical stability (default: 1e-8) weight_decay (float, optional): weight decay (L2 penalty) (default: 0) z zd .. _Adam\: A Method for Stochastic Optimization: https://arxiv.org/abs/1412.6980 rrUrVrWrXr r]r^rr!rrrrYc "[U5GHupXnU (dUOU*nX.nX>nXNn[RR5(ddU (a][ 5nUR R UR R :XaUR R U;d SUS35eUS- nU S:waURXS9n[R"U5(aX[R"U5n[R"U5n[R"U5n[R"U5nURUSU- 5 U (dC[R"URU5UR5RU5US9 O[R"URU5R!S5UR5RU5R#S5/S5nUR%[R&"USSS95 U (a3UU-S- nUR)U5 UU-nUR+UU5 GMWSU[-U5-- nUU- nUR+UUU*S 9 GM g) NIIf capturable=True, params and state_steps must be on supported devices: .rralpha)outF)keepdim)value) enumerater>compiler is_compilingr r6typeaddrM view_as_reallerp_maximummul_absadd_cat unsqueeze unsqueeze_copy_amaxdiv_addcdiv_r)rrUrVrWrXr r]r^rr!rrrrYiparamrLrJrKstep_tcapturable_supported_devicesnorm_bufneg_bias_correctiondenombias_correctionclrs r/_single_tensor_adamaxrs:"f%x#t$++~~**,,+L+N ( !!V]]%7%77LL%%)EE {[[wZxxyz {F !  1 88E86D   E " "&&u-E%%d+D((1G((1G  dAI& MM U# $  yye$..q1488:??33G3R3RST3UVH MM%**Xq%@ A #(-!"3   $ $R (11E NN7E *%:f+="==O&C NN7GC4N 8m&r1c "^U (aS5e[U5S:Xag[RR5(d>U (a7[ SS9m[ U4Sj[ X555(d STS35e[R"XX#U/5nUR5GHuunnnnnn[[[U5n[[[U5n[[[U5n[[[U5n[[[U5nU (a[UUUU5 U (a[R"U5n[RR5(d>USR(a*[R "U[R""SS S 9SS 9 O[R "US 5 U S:wa4U (a[R "UUU S 9 O[R$"UUU S 9n[R&"UUS U- 5 [R("UU5 U (dU S:Xa[R*"U5nO[R,"U5 [R "UU5 [R."UU5 U (aw[R0"UU5n[R2"US 5 [R4"UU5 [R6"UU5n[R8"UUU5 GM|UVs/sHnS U[;U5-- PM nnUVs/sHn[;U5U- S -PM nn[R8"UUUU5 GM gs snfs snf)Nz#_foreach ops don't support autogradrF) supports_xlac3># UHTupURRURR:H=(a URRT;v MV g7frd)r6rx).0rCr3rs r/ '_multi_tensor_adamax..FsN 4 HHMMT[[-- - > !== >3sAArnror%cpu)r6rpr)r=r>rvrwr allzipr"_group_tensors_by_device_and_dtypevaluesrlistrr _foreach_negis_cpu _foreach_add_rA _foreach_add_foreach_lerp_ _foreach_mul_ _foreach_abs _foreach_abs__foreach_maximum_ _foreach_pow _foreach_sub_ _foreach_div_ _foreach_mul_foreach_addcdiv_r) rrUrVrWrXr r]r^rr!rrrrYgrouped_tensorsgrouped_params_grouped_grads_grouped_exp_avgs_grouped_exp_infs_grouped_state_steps__grouped_params grouped_gradsgrouped_exp_avgsgrouped_exp_infsgrouped_state_stepsbias_correctionsrr3bc step_sizers @r/_multi_tensor_adamaxr+s9"DDD  6{a >> & & ( (Z'H( $ v3    wWWsVttu v  w  BB K8O  " " $   d6lO<T&\>: V .?@V .?@"4<1EF   /?AQ  !..}=M ~~**,,1DQ1G1N1N   #U\\#e%DC     3Q 7 1 ##M>V % 2 2!>! -}a%iH ,e4LA-!..}=M    . M3/  0-@ $11%9LM     0! 4    0" 5&&'79IJE  # #N4De L;N :M$EZ---:M  ?OO>N*R.2-3>NIO  # # 02BI C %z Ps *N N )single_tensor_fnr"c [RR5(d"[SU55(d [ S5eUc [ XSS9upU(a.[R R5(a [ S5eU(a*[R R5(d[nO[nU"UUUUUU U U U UUUU US9 g)zjFunctional API that performs adamax algorithm computation. See :class:`~torch.optim.Adamax` for details. c3V# UHn[U[R5v M! g7frd)r&r>r)rts r/radamax..s!5-8 1ell##[s')zPAPI has changed, `state_steps` argument must contain a list of singleton tensorsNF) use_fusedz6torch.jit.script not supported with foreach optimizers) r r]r^rr!rrrYr) r>rvrwrrPr jit is_scriptingrr)rrUrVrWrXr"rrrrYr r]r^rr!rfuncs r/rrs4 >> & & ( (5-8522 ^  1 e 599))++STTuyy--//#$   !%r1)NFFFF)typingrrrr>r optimizerrr r r r r rrrrrrrr__all__r__doc__rr@rjrrrrHr1r/rs(( $ X RYRl4       5+`G9 LG9 <G96lG96l G9 f G9 G9 G9 G9 G9G9G9G9G9G9Tm Lm <m6lm6l m f m m m m mmmmmm` 1FG# < L< <<6l<6l < f <d^<<<<< < < !<" #<$%<H<r1