[Th "SSKrSSKrSSKrSSKrSSKJr SSKJrJr SSK J r SSK r SSK J r SSKJr SSKJs Jr SSK Jr SSKJr /SQr\R."5r"SS 5r\r"S S 5r"S S \ R8\\5r"SS\5r"SS\ R8\\\S9r "SS\ 5r!Sr"Sr#"SS\!5r$S-Sjr%Sr&S.Sjr'Sr(\'"SSS9r)\'"S S!\&S"9r*\'"S#S$S%S&9r+\'"S'S(S9r,\%"S)S*S!S9r-"S+S,\!5r.g)/N) OrderedDict)AnyOptional) deprecated) _functions)custom_function_call) FunctionCtxBackwardCFunction FunctionMetaFunctiononce_differentiableInplaceFunctionNestedIOFunctionc\rSrSrS\R 4SjrS\R 4SjrS\R 4Sjr\ "S\ S9S 5r S\R 4S jr S \ 4S jrS rg)r "tensorscXlg)a Save given tensors for a future call to :func:`~Function.backward`. ``save_for_backward`` should be called at most once, in either the :func:`setup_context` or :func:`forward` methods, and only with tensors. All tensors intended to be used in the backward pass should be saved with ``save_for_backward`` (as opposed to directly on ``ctx``) to prevent incorrect gradients and memory leaks, and enable the application of saved tensor hooks. See :class:`torch.autograd.graph.saved_tensors_hooks`. Note that if intermediary tensors, tensors that are neither inputs nor outputs of :func:`forward`, are saved for backward, your custom Function may not support double backward. Custom Functions that do not support double backward should decorate their :func:`backward` method with ``@once_differentiable`` so that performing double backward raises an error. If you'd like to support double backward, you can either recompute intermediaries based on the inputs during backward or return the intermediaries as the outputs of the custom Function. See the `double backward tutorial `_ for more details. In :func:`backward`, saved tensors can be accessed through the :attr:`saved_tensors` attribute. Before returning them to the user, a check is made to ensure they weren't used in any in-place operation that modified their content. Arguments can also be ``None``. This is a no-op. See :ref:`extending-autograd` for more details on how to use this method. Example:: >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_AUTOGRAD) >>> class Func(Function): >>> @staticmethod >>> def forward(ctx, x: torch.Tensor, y: torch.Tensor, z: int): >>> w = x * z >>> out = x * y + y * z + w * y >>> ctx.save_for_backward(x, y, w, out) >>> ctx.z = z # z is not a tensor >>> return out >>> >>> @staticmethod >>> @once_differentiable >>> def backward(ctx, grad_out): >>> x, y, w, out = ctx.saved_tensors >>> z = ctx.z >>> gx = grad_out * (y + y * z) >>> gy = grad_out * (x + z + w) >>> gz = None >>> return gx, gy, gz >>> >>> a = torch.tensor(1., requires_grad=True, dtype=torch.double) >>> b = torch.tensor(2., requires_grad=True, dtype=torch.double) >>> c = 4 >>> d = Func.apply(a, b, c) N)to_save)selfrs O/var/www/auris/envauris/lib/python3.13/site-packages/torch/autograd/function.pysave_for_backwardFunctionCtx.save_for_backward#s r cxUH.n[U[R5(aM$UcM)S5e Xlg)aSave given tensors for a future call to :func:`~Function.jvp`. ``save_for_forward`` should be called at most once, in either the :func:`setup_context` or :func:`forward` methods, and all arguments should be tensors. In :func:`jvp`, saved objects can be accessed through the :attr:`saved_tensors` attribute. Arguments can also be ``None``. This is a no-op. See :ref:`extending-autograd` for more details on how to use this method. Example:: >>> # xdoctest: +SKIP >>> class Func(torch.autograd.Function): >>> @staticmethod >>> def forward(ctx, x: torch.Tensor, y: torch.Tensor, z: int): >>> ctx.save_for_backward(x, y) >>> ctx.save_for_forward(x, y) >>> ctx.z = z >>> return x * y * z >>> >>> @staticmethod >>> def jvp(ctx, x_t, y_t, _): >>> x, y = ctx.saved_tensors >>> z = ctx.z >>> return z * (y * x_t + x * y_t) >>> >>> @staticmethod >>> def vjp(ctx, grad_out): >>> x, y = ctx.saved_tensors >>> z = ctx.z >>> return z * grad_out * y, z * grad_out * x, None >>> >>> a = torch.tensor(1., requires_grad=True, dtype=torch.double) >>> t = torch.tensor(1., dtype=torch.double) >>> b = torch.tensor(2., requires_grad=True, dtype=torch.double) >>> c = 4 >>> >>> with fwAD.dual_level(): >>> a_dual = fwAD.make_dual(a, t) >>> d = Func.apply(a_dual, b, c) Nzgsave_for_forward expects all arguments to be tensors; you should save non-tensors as attributes on ctx.) isinstancetorchTensorsaved_for_forward)rrtensors rsave_for_forwardFunctionCtx.save_for_forward^s>\Ffell33v~ 9 E ")rargscXlg)aMark given tensors as modified in an in-place operation. This should be called at most once, in either the :func:`setup_context` or :func:`forward` methods, and all arguments should be inputs. Every tensor that's been modified in-place in a call to :func:`forward` should be given to this function, to ensure correctness of our checks. It doesn't matter whether the function is called before or after modification. Examples:: >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_AUTOGRAD) >>> class Inplace(Function): >>> @staticmethod >>> def forward(ctx, x): >>> x_npy = x.numpy() # x_npy shares storage with x >>> x_npy += 1 >>> ctx.mark_dirty(x) >>> return x >>> >>> @staticmethod >>> @once_differentiable >>> def backward(ctx, grad_output): >>> return grad_output >>> >>> a = torch.tensor(1., requires_grad=True, dtype=torch.double).clone() >>> b = a * a >>> Inplace.apply(a) # This would lead to wrong gradients! >>> # but the engine would not know unless we mark_dirty >>> # xdoctest: +SKIP >>> b.backward() # RuntimeError: one of the variables needed for gradient >>> # computation has been modified by an inplace operation N) dirty_tensorsrr"s r mark_dirtyFunctionCtx.mark_dirtys F"rz`mark_shared_storage` is deprecated. Tensors with shared storages are automatically tracked. Note that calls to `set_()` are not tracked)categorycgN)rpairss rmark_shared_storageFunctionCtx.mark_shared_storages rcXlg)aPMark outputs as non-differentiable. This should be called at most once, in either the :func:`setup_context` or :func:`forward` methods, and all arguments should be tensor outputs. This will mark outputs as not requiring gradients, increasing the efficiency of backward computation. You still need to accept a gradient for each output in :meth:`~Function.backward`, but it's always going to be a zero tensor with the same shape as the shape of a corresponding output. This is used e.g. for indices returned from a sort. See example:: >>> class Func(Function): >>> @staticmethod >>> def forward(ctx, x): >>> sorted, idx = x.sort() >>> ctx.mark_non_differentiable(idx) >>> ctx.save_for_backward(x, idx) >>> return sorted, idx >>> >>> @staticmethod >>> @once_differentiable >>> def backward(ctx, g1, g2): # still need to accept g2 >>> x, idx = ctx.saved_tensors >>> grad_input = torch.zeros_like(x) >>> grad_input.index_add_(0, idx, g1) >>> return grad_input N)non_differentiabler%s rmark_non_differentiable#FunctionCtx.mark_non_differentiables <#'rvaluecXlg)aSet whether to materialize grad tensors. Default is ``True``. This should be called only from either the :func:`setup_context` or :func:`forward` methods. If ``True``, undefined grad tensors will be expanded to tensors full of zeros prior to calling the :func:`backward` and :func:`jvp` methods. Example:: >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_AUTOGRAD) >>> class SimpleFunc(Function): >>> @staticmethod >>> def forward(ctx, x): >>> return x.clone(), x.clone() >>> >>> @staticmethod >>> @once_differentiable >>> def backward(ctx, g1, g2): >>> return g1 + g2 # No check for None necessary >>> >>> # We modify SimpleFunc to handle non-materialized grad outputs >>> class Func(Function): >>> @staticmethod >>> def forward(ctx, x): >>> ctx.set_materialize_grads(False) >>> ctx.save_for_backward(x) >>> return x.clone(), x.clone() >>> >>> @staticmethod >>> @once_differentiable >>> def backward(ctx, g1, g2): >>> x, = ctx.saved_tensors >>> grad_input = torch.zeros_like(x) >>> if g1 is not None: # We must check for None now >>> grad_input += g1 >>> if g2 is not None: >>> grad_input += g2 >>> return grad_input >>> >>> a = torch.tensor(1., requires_grad=True) >>> b, _ = Func.apply(a) # induces g2 to be undefined N)materialize_grads)rr3s rset_materialize_grads!FunctionCtx.set_materialize_gradss X"'r)r$r5r0rrN)__name__ __module__ __qualname____firstlineno__rrrr r&r FutureWarningr-r1boolr6__static_attributes__r+rrr r "su9%,,9v4)4)l#" #"J 6     'U\\'@,'4,'rr c$\rSrSr\S5rSrg) _HookMixinicjUc [5n[R"U5nXUR'X4$r*)rhooksRemovableHandleid)backward_hookshookhandles r_register_hook_HookMixin._register_hooks4  !(]N&&~6$(vyy!%%rr+N)r8r9r:r; staticmethodrHr>r+rrr@r@s&&rr@c*\rSrSrSrSrSrSrSrg)r iz< This class is used for internal autograd work. Do not use. cURRnURRnU[RLaU[RLa [ S5eU[RLaUOUnU"U/UQ76$)z@ Apply method used when executing this Node during the backward zsImplementing both 'backward' and 'vjp' for a custom Function is not allowed. You should only implement one of them.) _forward_clsbackwardvjpr RuntimeError)rr" backward_fnvjp_fnuser_fns rapplyBackwardCFunction.apply$su ''00 ""&& h// /F(,,4N  #(,,6&Kt#d##rc<URR"U/UQ76$)zE Apply method used when executing forward mode AD during the forward )rMjvpr%s r apply_jvpBackwardCFunction.apply_jvp5s   $$T1D11rc8URRU5$r*)rM_compiled_autograd_key)rs rr[(BackwardCFunction._compiled_autograd_key<s  77==rr+N) r8r9r:r;__doc__rTrXr[r>r+rrr r s$"2>rr c,^\rSrSrSrU4SjrSrU=r$)r i@zFunction metaclass. This metaclass sets up the following properties: _backward_cls: The Function class corresponding to the differentiated version of this function (which is generated on the fly by this metaclass). c>[US-[4SU05n[[5UlSUl[ USS5(aURRUlX@l [TU]-XU5 g)NBackwardrM_lazy_backward_info) typer nextAUTOGRAD_FUNCTION_COUNTER_autograd_function_id _bw_modulegetattrra bw_module _backward_clssuper__init__)clsnamebasesattrsrQ __class__s rrkFunctionMeta.__init__Iss :  13nc5J -11J,K )!%  3-t 4 4%(%<%<%F%FK "' e,rr+r8r9r:r;r]rkr> __classcell__rps@rr r @s - -rr c \rSrSr\S\S\S\4Sj5r\S\S\\S4S \S\4S j5r\S\S \S\4S j5r \ r \S\S \S\4Sj5r Sr g)_SingleLevelFunctioniVr"kwargsreturnc[S5e)aDefine the forward of the custom autograd Function. This function is to be overridden by all subclasses. There are two ways to define forward: Usage 1 (Combined forward and ctx):: @staticmethod def forward(ctx: Any, *args: Any, **kwargs: Any) -> Any: pass - It must accept a context ctx as the first argument, followed by any number of arguments (tensors or other types). - See :ref:`combining-forward-context` for more details Usage 2 (Separate forward and ctx):: @staticmethod def forward(*args: Any, **kwargs: Any) -> Any: pass @staticmethod def setup_context(ctx: Any, inputs: Tuple[Any, ...], output: Any) -> None: pass - The forward no longer accepts a ctx argument. - Instead, you must also override the :meth:`torch.autograd.Function.setup_context` staticmethod to handle setting up the ``ctx`` object. ``output`` is the output of the forward, ``inputs`` are a Tuple of inputs to the forward. - See :ref:`extending-autograd` for more details The context can be used to store arbitrary data that can be then retrieved during the backward pass. Tensors should not be stored directly on `ctx` (though this is not currently enforced for backward compatibility). Instead, tensors should be saved either with :func:`ctx.save_for_backward` if they are intended to be used in ``backward`` (equivalently, ``vjp``) or :func:`ctx.save_for_forward` if they are intended to be used for in ``jvp``. zEYou must implement the forward function for custom autograd.Function.NotImplementedError)r"rws rforward_SingleLevelFunction.forwardYsT" S  rctxinputs.outputc[S5e)a5There are two ways to define the forward pass of an autograd.Function. Either: 1. Override forward with the signature ``forward(ctx, *args, **kwargs)``. ``setup_context`` is not overridden. Setting up the ctx for backward happens inside the ``forward``. 2. Override forward with the signature ``forward(*args, **kwargs)`` and override ``setup_context``. Setting up the ctx for backward happens inside ``setup_context`` (as opposed to inside the ``forward``) See :meth:`torch.autograd.Function.forward` and :ref:`extending-autograd` for more details. z!setup_context is not implemented.rz)r~rrs r setup_context"_SingleLevelFunction.setup_contexts""EFFr grad_outputsc[S5e)aIDefine a formula for differentiating the operation with backward mode automatic differentiation. This function is to be overridden by all subclasses. (Defining this function is equivalent to defining the ``vjp`` function.) It must accept a context :attr:`ctx` as the first argument, followed by as many outputs as the :func:`forward` returned (None will be passed in for non tensor outputs of the forward function), and it should return as many tensors, as there were inputs to :func:`forward`. Each argument is the gradient w.r.t the given output, and each returned value should be the gradient w.r.t. the corresponding input. If an input is not a Tensor or is a Tensor not requiring grads, you can just pass None as a gradient for that input. The context can be used to retrieve tensors saved during the forward pass. It also has an attribute :attr:`ctx.needs_input_grad` as a tuple of booleans representing whether each input needs gradient. E.g., :func:`backward` will have ``ctx.needs_input_grad[0] = True`` if the first input to :func:`forward` needs gradient computed w.r.t. the output. zwYou must implement either the backward or vjp method for your custom autograd.Function to use it with backward mode AD.rz)r~rs rrN_SingleLevelFunction.backwards."   r grad_inputsc[S5e)aDefine a formula for differentiating the operation with forward mode automatic differentiation. This function is to be overridden by all subclasses. It must accept a context :attr:`ctx` as the first argument, followed by as many inputs as the :func:`forward` got (None will be passed in for non tensor inputs of the forward function), and it should return as many tensors as there were outputs to :func:`forward`. Each argument is the gradient w.r.t the given input, and each returned value should be the gradient w.r.t. the corresponding output. If an output is not a Tensor or the function is not differentiable with respect to that output, you can just pass None as a gradient for that input. You can use the :attr:`ctx` object to pass any value from the forward to this functions. z`You must implement the jvp function for custom autograd.Function to use it with forward mode AD.rz)r~rs rrW_SingleLevelFunction.jvps$" @  rr+N) r8r9r:r;rJrr|tuplerrNrOrWr>r+rrrvrvVs+ s+ c+ c+ + ZG3Gc3hGGGG  c # #  : C  C C  rrv) metaclassch^\rSrSrSrSrSrSr\S5r \ U4Sj5r \S5r S r U=r$) r iaOBase class to create custom `autograd.Function`. To create a custom `autograd.Function`, subclass this class and implement the :meth:`forward` and :meth:`backward` static methods. Then, to use your custom op in the forward pass, call the class method ``apply``. Do not call :meth:`forward` directly. To ensure correctness and best performance, make sure you are calling the correct methods on ``ctx`` and validating your backward function using :func:`torch.autograd.gradcheck`. See :ref:`extending-autograd` for more details on how to use this class. Examples:: >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_AUTOGRAD) >>> class Exp(Function): >>> @staticmethod >>> def forward(ctx, i): >>> result = i.exp() >>> ctx.save_for_backward(result) >>> return result >>> >>> @staticmethod >>> def backward(ctx, grad_output): >>> result, = ctx.saved_tensors >>> return grad_output * result >>> >>> # Use it by calling the apply method: >>> # xdoctest: +SKIP >>> output = Exp.apply(input) cR[R"URS3[SS9 g)Nz should not be instantiated. Methods on autograd functions are all static, so you should invoke them on the class itself. Instantiating an autograd function will raise an error in a future version of PyTorch.) stacklevel)warningswarnrpDeprecationWarningrr"rws rrkFunction.__init__s* ~~4 4   rc[S5e)NzLegacy autograd function with non-static forward method is deprecated. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function))rPrs r__call__Function.__call__s _  rFc[S5e)aDefine the behavior for this autograd.Function underneath :func:`torch.vmap`. For a :func:`torch.autograd.Function` to support :func:`torch.vmap`, you must either override this static method, or set ``generate_vmap_rule`` to ``True`` (you may not do both). If you choose to override this staticmethod: it must accept - an ``info`` object as the first argument. ``info.batch_size`` specifies the size of the dimension being vmapped over, while ``info.randomness`` is the randomness option passed to :func:`torch.vmap`. - an ``in_dims`` tuple as the second argument. For each arg in ``args``, ``in_dims`` has a corresponding ``Optional[int]``. It is ``None`` if the arg is not a Tensor or if the arg is not being vmapped over, otherwise, it is an integer specifying what dimension of the Tensor is being vmapped over. - ``*args``, which is the same as the args to :meth:`~Function.forward`. The return of the vmap staticmethod is a tuple of ``(output, out_dims)``. Similar to ``in_dims``, ``out_dims`` should be of the same structure as ``output`` and contain one ``out_dim`` per output that specifies if the output has the vmapped dimension and what index it is in. Please see :ref:`func-autograd-function` for more details. zrTo use autograd.Function with vmap, you must either override the vmap staticmethod or set generate_vmap_rule=True.rz)infoin_dimsr"s rvmap Function.vmaps8" @  rcT>Sn[UR5nU(aU"UR/UQ70UD6n[RR 5(d/[ RRU5n[TU]("U0UD6$U(d [S5e[U/UQ70UD6$)Nc[R"U5nUR"U0UD6nUR5 UR$r*)inspect signaturebindapply_defaultsr")funcr"rwr bound_argss rbind_default_args)Function.apply..bind_default_args1s;))$/I"88J  % % '?? "rzIn order to use an autograd.Function with functorch transforms (vmap, grad, jvp, jacrev, ...), it must override the setup_context staticmethod. For more details, please see https://pytorch.org/docs/main/notes/extending.func.html) _is_setup_context_definedrr|r_C _are_functorch_transforms_active _functorchutilsunwrap_dead_wrappersrjrTrPr)rlr"rwris_setup_ctx_definedrps rrTFunction.apply/s # 99J9JK $S[[B4B6BDxx88::##88>D7=$1&1 1#J $C9$9&99rcUR4$r*)re)r~s rr[Function._compiled_autograd_keyKs))++rr+)r8r9r:r;r]rkrgenerate_vmap_rulerJr classmethodrTr[r>rsrts@rr r s[B    @::6,,rr c(U[R:g$r*)rvr)fns rrrPs %33 33rcF^[R"T5U4Sj5nU$)Nc>[R"5 T"U/UQ76nSSS5 [R"5(dW$[SU55nU(dW$[ W[ 5(dU4n[ R"S[U55nSnU"UVs/sH oe"U5PM sn6$!,(df  N=fs snf)Nc3|# UH2n[U[R5=(a URv M4 g7fr*)rrr requires_grad).0args r 7once_differentiable..wrapper..fs, KOCJsELL ) ?c.?.? ?4s:<sRtrying to differentiate twice a function that was marked with @once_differentiablec:UbUR5nSUlU$)NT)detachr)vars rfake_requires_grad@once_differentiable..wrapper..fake_requires_gradxsjjl$(!Jr) rno_gradis_grad_enabledanyrrr DelayedErrorlen)r~r"outputsrerr_fnrvrs rwrapper$once_differentiable..wrapperUs ]]_ntnG$$&&N KO  N'5))jG(( ) L   w?w!*1-w?@@O_N@s B6!C6 C) functoolswraps)rrs` rr r Ts(__R(A(AT Nrc0^\rSrSrSrSU4SjjrSrU=r$)riy This class is here only for backward compatibility reasons. Use :class:`Function` instead of this for any new use case. c.>[TU]5 Xlgr*)rjrkinplace)rrrps rrkInplaceFunction.__init__s  r)r)Frrrts@rrrs rrc ^^^^UUUU4SjmT$)Nc>T"U5(aT"U5$Ucg[U[[45(a<U4SjU5n[US5(a[ U5"U6$[ U5"U5$[U[ 5(aUVs0sH o"T"X5_M sn$[ S[R"U5-T(aST-S--5eS-5es snf)Nc34># UH nT"U5v M g7fr*r+)rx_maps rr,_nested_map.._map..s+s!d1ggss_fieldsAAuto nesting doesn't know how to process an input object of type . Accepted types: , or lists/tuples of them) rlistrhasattrrbdict ValueErrorrtypename)objmappedrr condition condition_msgrs rr_nested_map.._maps S>>c7N [ dE] + ++s+FsI&&Cy&))9V$ $ T " "-01StCF|OS1 1+..%& %)=8;VV      2s Cr+)rrrrs```@r _nested_maprs2 KrcH[US5(aUR5$U$)N _jit_unwrap)rr)rs r_jit_unwrap_structuredrs!sM""  Jrc$^^^^^UUUUU4SjmT$)Nc3># TbT"U5nT"U5(aUv gUcg[U[[45(aUHnT"U5ShvN M g[U[5(a(UR 5HnT"U5ShvN M gT(aUv g[ S[ R"U5-T(aST-S--5eS-5eNNQ7f)Nrrrr)rrrrvaluesrrr)ro_iter allow_unknownrr conversions rr_iter_filter.._iters  !S/C S>>I [  dE] + + 8## T " "ZZ\ 8##" I+..%& %)=8;VV      $$s%A C!C/n[US5(aURU5$[U[[45(d USUSS4$UH4nUcUR U5 MT"X5up@UR U5 M6 [ U5"U5U4$)N _jit_wrapr)rrrrrappendrb)inputprotoreseres_eunflatten_helpers rr$_unflatten..unflatten_helpers,. 5+ & &??5) )%$//8U12Y& &Ay 1 /9  5!  E{3&&rrr+)rrrs @r _unflattenrs ' E )! ,,rc`USL=(d$ [U[RR5$r*)rrrValuers rrsa4i8:a88rzjit's Values or None)rc6[U[R5$r*rrrrs rrrjELL)rTensors)rrc6[U[R5$r*rrs rrrr rTzTensors (permissive))rrcLUSL=(d [U[R5$r*rrs rrrsa4i6:a66rzTensors or Nonec6[U[R5$r*rrs rrrr rcUR$r*)datars rrrsQVVrc^\rSrSrSrU4SjrU4SjrS\S\4Sjr\r S\S\4S jr S\SS 4S jr \ U4S j5r S\S \SS 4SjrS\S \SS 4SjrS\SS 4SjrS\SS 4SjrSrU=r$)rirc>Xl[[U55n[TU]"U6n[ X0R 5nU$r*) _nested_inputr _iter_tensorsrj _do_forwardr_nested_output)rr flat_input flat_outputnested_tensorsrps rrNestedIOFunction._do_forwards="=/0 g):6 #K1D1DErcH>X l[TU] X5nU(dU?U?U$r*)retain_variablesrj _do_backwardr_to_save_nested)r gradientsrresultrps rrNestedIOFunction._do_backward s, 0%iB#$ rrrxcr[XR5nUR"U6n[[ U55$)z Shared backward utility. )rrbackward_extendedr_iter_None_tensors)rrnested_gradientsrs rrNNestedIOFunction.backwards7&i1D1DE'')9:'/00rr"c[UR5nUR"U6nU?X0l[ [ U55$)z Shared forward utility. )_map_tensor_datarforward_extendedrrr)rr"rrs rr|NestedIOFunction.forwards@*$*<*<=&&7  $]6*++rNcB[[U55UlXlg)z) See :meth:`Function.save_for_backward`. N)rrrrr%s rr"NestedIOFunction.save_for_backward&s]401 #rcB>[TU]n[XR5$)z% See :meth:`Function.saved_tensors`. )rj saved_tensorsrr)r flat_tensorsrps rr-NestedIOFunction.saved_tensors-s w, ,(<(<==rrwc8[[X455Ulg)z" See :meth:`Function.mark_dirty`. N)rrr$rs rr&NestedIOFunction.mark_dirty5s#=$#@Arc8[[X455Ulg)z/ See :meth:`Function.mark_non_differentiable`. N)rrr0rs rr1(NestedIOFunction.mark_non_differentiable;s#( tn(E"Frrc[e)z User defined forward. rz)rrs rr(!NestedIOFunction.forward_extendedA "!r grad_outputc[e)z User defined backward. rz)rr7s rr""NestedIOFunction.backward_extendedGr6r)rrrr$r0rr)r8r9r:r;r]rrrrNrr|rpropertyr-r&r1r(r"r>rsrts@rrrs13131H,S,S,$s$t$>>BBsBtB GSGCGDG "s"t" "c"d""rrr*)FNN)/rr itertoolsr collectionsrtypingrrtyping_extensionsrrtorch._Crtorch._functorchrtorch.utils.hooksrrBr"torch._functorch.autograd_functionr__all__countrdr _ContextMethodMixinr@ _FunctionBaser rbr rvr rr rrrrr_iter_jit_valuesr_iter_tensors_permissiver#r'rr+rrrIsS# ( %!!C &OO-l'l'`"&&>((+z>B-4-,w k:w t},#},@4,^h: @-( 8()% ()( "6FW)+;9 P"xP"r