JTh@SSKJrJrJrJr SSKrSSKJr SSKJrJ r J r J r J r J r JrJrJrJrJrJrJr SS/r"SS\5rS S \S \ S \S \S \ S 3 -\lS\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\4SjrS\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\4Sjr\ "\S9S S\\S\\S\\S\\S\\S\S\\S\S\S\S\S\S\S\4Sjj5rg)!)AnycastOptionalUnionN)Tensor) _capturable_doc_default_to_fused_or_foreach_differentiable_doc_disable_dynamo_if_unsupported _foreach_doc!_get_capturable_supported_devices_get_scalar_dtype _maximize_doc _params_doc_use_grad_for_differentiable _view_as_real OptimizerParamsTAdadeltaadadeltac^\rSrSrSSSSS.S\S\\\4S\S\S\S \\ S \ S \ S \ 4U4S jjjjr U4Sjr S\ \ \4S\\S\\S\\S\\S\\4 Sjr\SSj5rSrU=r$)rF) capturablemaximizedifferentiableparamslrrhoeps weight_decayforeachrrrc ^>[U[5(aUR5S:wa [S5eSU::d[SU35eSUs=::aS::dO [SU35eSU::d[SU35eSU::d[SU35e[ UUUUUUUU S 9n [ T U]X5 g) NrzTensor lr must be 1-elementgzInvalid learning rate: ?zInvalid rho value: zInvalid epsilon value: zInvalid weight_decay value: )rrr r!rrr"r) isinstancernumel ValueErrordictsuper__init__) selfrrrr r!r"rrrdefaults __class__s L/var/www/auris/envauris/lib/python3.13/site-packages/torch/optim/adadelta.pyr*Adadelta.__init__s b& ! !bhhjAo:; ;by6rd;< <c S 23%89 9cz6se<= =l";L>JK K%!)   *cP>[TU]U5 URGHnURSS5 URSS5 URSS5 URSS5 USHnURR U/5n[ U5S:wdM0[R"US5(aMP[US5nUS(a([R"U[5URS 9O[R"U[5S 9US'M GM g) Nr"rFrrrrstepdtypedevicer4) r) __setstate__ param_groups setdefaultstategetlentorch is_tensorfloattensorrr5)r+r:grouppp_statestep_valr-s r.r7Adadelta.__setstate__@s U#&&E   Y -   Z /   -u 5   \5 18_**..B/w<1$U__WV_-M-M$WV_5H !. $,=,?#\\(:K:MN FO % 'r0rAparams_with_gradgrads square_avgs acc_deltas state_stepsc SnUSGHvnURcMU[R"U5-nURU5 URR(a [ S5eURUR5 UR Un [U 5S:XaUS(a([R"S[5URS9O[R"S[5S9U S '[R"U[RS 9U S '[R"U[RS 9U S 'URU S 5 URU S 5 URU S 5 GMy U$) NFrz*Adadelta does not support sparse gradientsrrr3r6r2) memory_format square_avg acc_delta) gradr= is_complexappend is_sparse RuntimeErrorr:r<zerosrr5 zeros_likepreserve_format) r+rArFrGrHrIrJ has_complexrBr:s r. _init_groupAdadelta._init_groupSsN xAvv~ 5++A. .K  # #A &vv"#OPP LL JJqME5zQ\*KK*;*=ahhOR/@/BCf ',&6&6U%:%:'l#&+%5%5U%:%:&k"   u\2 3   eK0 1   uV} -9!<r0c~UR5 SnUb%[R"5 U"5nSSS5 URHan/n/n/n/n/nUSUSUSUSUSUSUSUS 4un n n n n nnnUR X4XVXx5n[ UUUUUU U U U U UUUUS 9 Mc U$!,(df  N=f) zPerform a single optimization step. Args: closure (Callable, optional): A closure that reevaluates the model and returns the loss. Nrrr r!r"rrr) rrr r!r"rrrrX) _cuda_graph_capture_health_checkr= enable_gradr8rYr)r+closurelossrArFrGrHrIrJrrr r!r"rrrrXs r.r2 Adadelta.step~s  --/  ""$y%&&E-/ "$E(*K')J(*Kd e e n%i j!&'l#  **ZK  )!-%' ='^ e%$s B.. B<rL)r$g?gư>rNN)__name__ __module__ __qualname____firstlineno__rrr?rrboolr*r7r(strrlistrYrr2__static_attributes__ __classcell__)r-s@r.rrs $'"&"+!$"+"+ %- "+ "+  "+  "+$"+"+"+"+"+H&)CH~)v,)F| ) &\ ) L )&\)V"="=r0aImplements Adadelta algorithm. .. math:: \begin{aligned} &\rule{110mm}{0.4pt} \\ &\textbf{input} : \gamma \text{ (lr)}, \: \theta_0 \text{ (params)}, \: f(\theta) \text{ (objective)}, \: \rho \text{ (decay)}, \: \lambda \text{ (weight decay)} \\ &\textbf{initialize} : v_0 \leftarrow 0 \: \text{ (square avg)}, \: u_0 \leftarrow 0 \: \text{ (accumulate variables)} \\[-1.ex] &\rule{110mm}{0.4pt} \\ &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \: \textbf{do} \\ &\hspace{5mm}g_t \leftarrow \nabla_{\theta} f_t (\theta_{t-1}) \\ &\hspace{5mm}if \: \lambda \neq 0 \\ &\hspace{10mm} g_t \leftarrow g_t + \lambda \theta_{t-1} \\ &\hspace{5mm} v_t \leftarrow v_{t-1} \rho + g^2_t (1 - \rho) \\ &\hspace{5mm}\Delta x_t \leftarrow \frac{\sqrt{u_{t-1} + \epsilon }}{ \sqrt{v_t + \epsilon} }g_t \hspace{21mm} \\ &\hspace{5mm} u_t \leftarrow u_{t-1} \rho + \Delta x^2_t (1 - \rho) \\ &\hspace{5mm}\theta_t \leftarrow \theta_{t-1} - \gamma \Delta x_t \\ &\rule{110mm}{0.4pt} \\[-1.ex] &\bf{return} \: \theta_t \\[-1.ex] &\rule{110mm}{0.4pt} \\[-1.ex] \end{aligned} For further details regarding the algorithm we refer to `ADADELTA: An Adaptive Learning Rate Method`_. z Args: ar lr (float, Tensor, optional): coefficient that scale delta before it is applied to the parameters (default: 1.0) rho (float, optional): coefficient used for computing a running average of squared gradients (default: 0.9). A higher value of `rho` will result in a slower average, which can be helpful for preventing oscillations in the learning process. eps (float, optional): term added to the denominator to improve numerical stability (default: 1e-6). weight_decay (float, optional): weight decay (L2 penalty) (default: 0) z zd .. _ADADELTA\: An Adaptive Learning Rate Method: https://arxiv.org/abs/1212.5701 rrGrHrIrJrrr r!rrrrXc^[RR5(d>U (a7[SS9m[ U4Sj[ X555(d STS35e[ XX#U5GHupnnnUS- nU (dUOU*nUS:waUR XS9n[R"U 5(aB[R"U5n[R"U5n[R"U5nURU5RXSU- S 9 UR U5R5nUR U5R5nU (aUR5nURU5RU5 URU5RUUSU- S 9 [R"U 5(a[R"U5nU RUU*S9 GM g) NF supports_xlac3># UHTupURRURR:H=(a URRT;v MV g7frar5type.0rBr2capturable_supported_devicess r. *_single_tensor_adadelta.. N 4 HHMMT[[-- - > !== >3AAIIf capturable=True, params and state_steps must be on supported devices: .rralphavalue)r=compiler is_compilingrallzipaddrQ view_as_realmul_addcmul_sqrt_clonediv_view_as_complexadd_)rrGrHrIrJrrr r!rrrrXparamrPrNrOr2stddeltarss @r._single_tensor_adadeltars" >> & & ( (Z'H( $ v3    wWWsVttu v  w 58{ 50ZD  #t$ 1 88E86D   E " "++J7J**95I%%d+D%%dC%@nnS!'') c"((* KKME 3T"s$$UES$A   E " "))%0E 5 $15r0c ^U (aS5e[RR5(d>U (a7[SS9m[ U4Sj[ X555(d STS35e[ U5S:Xag[R"XX#U/5n U R5GHuunnnnnn[[[U5n[[[U5n[[[U5n[[[U5n[[[U5nU (a[UUUU5 [RR5(d>USR(a*[R"U[R "SS S 9SS 9 O[R"US 5 U (a[R""U5nUS:wa4U (a[R"UUUS 9 O[R$"UUUS 9n[R&"UU5 [R("UUUS U- S 9 [R$"UU5n[R*"U5 [R$"UU5n[R*"U5 [R,"UU5 [R&"UU5 [R&"UU5 [R("UUUS U- S 9 U (aQ[/U[R5(a2[R&"UU*5 [R"UU5 GM[R"UUU*S 9 GM g)Nz#_foreach ops don't support autogradFrlc3># UHTupURRURR:H=(a URRT;v MV g7frarorqs r.rt)_multi_tensor_adadelta..Arvrwrxryrr$cpu)r5rzrr|)r=r~rrrrr<r"_group_tensors_by_device_and_dtypevaluesrrhrris_cpu _foreach_add_r@ _foreach_neg _foreach_add _foreach_mul__foreach_addcmul__foreach_sqrt_ _foreach_div_r%)rrGrHrIrJrrr r!rrrrXgrouped_tensorsdevice_params_ device_grads_device_square_avgs_device_acc_deltas_device_state_steps__ device_params device_gradsdevice_square_avgsdevice_acc_deltasdevice_state_stepsrdeltasrss @r._multi_tensor_adadeltar*s DDD  >> & & ( (Z'H( $ v3    wWWsVttu v  w  6{aBB  =O  " " $   T&\>: DL-8 !$v,0CD f/AB!$v,0CD  |-?AR ~~**,,1CA1F1M1M   "ELLU$C3     2A 6  --l;L 1 ##L-|T$11 -|   .4   l!c'   !3S9 S!##$5s; V$ FC( FL1 -s3  166SQ *R66    ,    v 6    vbS Aq %r0)single_tensor_fnr"c [RR5(d"[SU55(d [ S5eUc [ XSS9upU(a.[R R5(a [ S5eU(a*[R R5(d[nO[nU"UUUUUU U U U U UUUS9 g)znFunctional API that performs Adadelta algorithm computation. See :class:`~torch.optim.Adadelta` for details. c3V# UHn[U[R5v M! g7fra)r%r=r)rrts r.rtadadelta..s!5-8 1ell##[s')zPAPI has changed, `state_steps` argument must contain a list of singleton tensorsNF) use_fusedz6torch.jit.script not supported with foreach optimizers)rrr r!rrrrX) r=r~rrrTr jit is_scriptingrr)rrGrHrIrJrr"rrXrrr r!rrfuncs r.rrs6 >> & & ( (5-8522 ^  1 e 599))++STTuyy--//%&   !%r0)FNFF)typingrrrrr=r optimizerr r r r r rrrrrrrr__all__r__doc__rhr?rfrrrrLr0r.rs-- " z "ayaJ8       90 j3% L3% <3%f3%V 3% f 3% 3% 3% 3%3%3%3%3%3%laB LaB <aBfaBV aB f aB aB aB aBaBaBaBaBaBH 1HI" = L= <=f=V = f ==d^=== = = = !="#=J=r0