o ‡ZŽh2ã@sjUddlmZddlZddlmmZddlmZddlm Z gZ e e e d<ejjGdd„dƒƒZdS)é)ÚOptionalN)ÚTensor)Ú2_scripted_functional_optimizer_deprecation_warningÚ__all__c@s‚eZdZ         ddeededededed ed ed ed ed efdd„Zdedeefdd„Z deeefdd„Z dS)Ú_FunctionalSGDç{®Gáz„?çFÚparamsÚlrÚmomentumÚ dampeningÚ weight_decayÚnesterovÚmaximizeÚforeachÚfusedÚ_allow_empty_param_listc Cs|tdd||||dœ|_||_||_||_| |_tj t tj t t tj ffi¡|_ t |ƒdkr7| s7tdƒ‚d|i|_dS)Né)Ú stacklevel)r r r r rz%optimizer got an empty parameter listr )rÚdefaultsrrrrÚtorchÚjitZannotateÚdictrÚstrÚstateÚlenÚ ValueErrorÚ param_group) Úselfr r r r r rrrrr©rúU/var/www/auris/lib/python3.10/site-packages/torch/distributed/optim/functional_sgd.pyÚ__init__s ü$z_FunctionalSGD.__init__ÚparamÚgradc Cs|jd}|jd}|jd}|jd}|g}g}g} d} |durK|  |¡|jr+d} ||jvr5i|j|<|j|} d| vrD| d¡n| | d¡t ¡tj|| ||||||j|j | |j |j ddd Wdƒn1sqwY|j|} |d } | dur‰| | d<dSdS) z[Similar to self.step, but operates on a single parameter and its gradient. r r r r FNTÚmomentum_buffer© r r r r rrÚhas_sparse_gradrrZ grad_scaleZ found_infr) rÚappendÚ is_sparserrÚno_gradÚFÚsgdrrrr) rr"r#r r r r r Úmomentum_buffer_listÚgradsr&rr$rrr Ú step_param;sR          òÿ  ÿz_FunctionalSGD.step_paramÚ gradientscCs€|jd}g}g}g}|jd}|jd}|jd}|jd} t|ƒt|ƒkr:tddt|ƒ›dd t|ƒ›ƒ‚d } t||ƒD]7\} } | durx| | ¡| | ¡| jrXd } | |jvrbi|j| <|j| } d | vrq| d¡qA| | d ¡qAt  ¡t j ||||||| |j |j | |j|jddd Wdƒn1sŸwYt|ƒD]\}}|j|} ||}|dur½|| d <q¨dS)Nr r r r r zEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: FTr$r%)rrrrÚzipr'r(rrr)r*r+rrrrÚ enumerate)rr/r Zparams_with_gradr-r,r r r r r&r"ZgradientrÚiÚpr$rrr Ústepmsn     ÿ þÿ      € òÿ €üz_FunctionalSGD.stepN) rrrrFFFFF) Ú__name__Ú __module__Ú __qualname__ÚlistrÚfloatÚboolr!rr.r4rrrr rsBõþýüûúùø ÷ ö õ!2r)ÚtypingrrZtorch.optim._functionalZoptimZ _functionalr*rZ,torch.distributed.optim._deprecation_warningrrr8rÚ__annotations__rÚscriptrrrrr Ús