a h(@sTddlZddlmZddlmZddlmZmZmZm Z ddl m Z m Z m Z ddlZddlmZddlmZddlmZe dd d Ze d Zdd dZeZeZGdddZe ddd dZe dejeejZGddde eeeefZefe eeee eefefgee eefeeffdddZ!ejee"e#efdddZ$dS)N) OrderedDict)wraps)CallableGenericOptionalProtocol) Concatenate ParamSpecTypeVar)_State)_get_root_modules_TT) covariant_P__composable_api_state_keycCs|dttS)N_)struuiduuid4)stringrT/var/www/auris/lib/python3.9/site-packages/torch/distributed/_composable/contract.pygenerate_state_keysrc@s eZdZdS) RegistryItemN)__name__ __module__ __qualname__rrrrrsr_TStater )boundr_Mc@s4eZdZejejedddZej e dddZ dS) _ContractFn)argskwargsreturncOsdSNr)selfr!r"rrr__call__'z_ContractFn.__call__moduler#cCsdSr$r)r%r)rrrstate)r'z_ContractFn.stateN) rrrrr!r"r r&nnModulerr*rrrrr &sr ) state_clsr#csBtttttftfttttfttfdfdd }|S)a% Decorate a function as a composable distributed API, where the first argument of the function must be an :class:`nn.Module` instance or sequence of :class:`nn.Module` instances. The decorator verifies that the decorated function does not modify fully-qualified names (FQNs) for parameters, buffers, or modules. The decorated function can return different module instances than the input modules; the FQN invariant will be enforced following the input order. When a function ``func`` is decorated by ``@contract()``, a ``.state(module: nn.Module)`` method will be installed to the decorated function. Then you can retrieve and modify the state on a module by calling ``func.state(module)``. Example:: >>> # xdoctest: +SKIP >>> import torch.nn as nn >>> >>> class MyModel(nn.Module): >>> def __init__(self) -> None: >>> super().__init__() >>> self.l1 = nn.Linear(10, 10) >>> self.l2 = nn.Linear(10, 10) >>> >>> def forward(self, x): >>> return self.l2(self.l1(x)) >>> >>> @contract() >>> def my_feature(module: nn.Module) -> nn.Module: >>> my_feature.state(module).some_state = "any value" >>> return module >>> >>> model = MyModel() >>> my_feature(model.l1) >>> assert my_feature.state(model.l1).some_state == "any value" >>> my_feature(model.l2) >>> model(torch.randn(2, 10)).sum().backward() )funcr#csFtttjtjtdfdd }tjtdfdd }||_|S)N)r)r!r"r#cs|}t|tjr|g}n tt|}}t}g}g}g} |D]}t} t} |jt | } t| t svt d| |jt | } t| t st d| | vsj | vrt dj d|| || j ||t||t|| t|q@|g|Ri|}|dur:|}t|tjrP|g}n tt|}g}g}g}|D]<}|t||t||t|qlt| }t|}||krt j d|d|dtttttdd d }t||D]&\}}|t|t|d qt||D]&\}}|t|t|d q6t| |D]&\}}|t|t|d qh|S)Nz-Distributed composable API states corrupted: z/Distributed composable API registry corrupted: zOEach distinct composable distributed API can only be applied to a module once. z3 has already been applied to the following module: zB should return the same number of modules as input modulesInputs: z modules Outputs: z modules) orig_fqnsnew_fqns check_keycSst||kr dSt|t|}}||}||}t|s>t|rXt|d|d|nt|d|d|dS)NzVComposable distributed API implementations cannot modify FQNs. FQNs only in original: z FQNs only in new: z[Composable distributed API implementations cannot modify the order of FQNs. Original FQNs: z New FQNs: )setlen RuntimeError)r/r0r1Z orig_fqn_setZ new_fqn_setZ orig_onlyZnew_onlyrrr check_fqns(z;contract..inner..wrapper..check_fqnzChecking parameters: zChecking buffers: zChecking modules: ) isinstancer+r,r listrr__dict__ setdefault STATE_KEYdictAssertionError REGISTRY_KEYrappendZnamed_parametersZ named_buffersZ named_modulesr3rzipkeys)r)r!r"Z inp_modulemodulesr*Z registry_itemZall_orig_named_paramsZall_orig_named_buffersZall_orig_named_modulesZdefault_all_stateZdefault_registryZ all_stateregistryupdatedZupdated_modulesZall_new_named_paramsZall_new_named_buffersZall_new_named_modulesZnum_orig_modulesZnum_new_modulesr5Zorig_named_paramsZnew_named_paramsZorig_named_buffersZnew_named_buffersZorig_named_modulesZnew_named_modules)r.r-rrwrapper_s                  z(contract..inner..wrapperr(cs|jtiSr$)r8r9r:getr)r.rr get_states z*contract..inner..get_state) rrrr!r"r+r,r r*)r.rDrHr-rGrinner[szcontract..inner)rrrrrr r)r-rJrrIrcontract,s /rKr(cCs t|tdS)z Get an ``OrderedDict`` of composable APIs that have been applied to the ``module``, indexed by the API name. If no API has been applied, then this returns ``None``. N)getattrr=rFrrr _get_registrysrM)r)%r collectionsr functoolsrtypingrrrrZtyping_extensionsrr r ZtorchZtorch.nnr+Z#torch.distributed._composable_stater Ztorch.distributed.utilsr r rrr:r=rrr,r7rr typerKr;rrMrrrrs8        G