JThJSrSSKrSSKJrJr SSKJr SSKr"SS\5r"SS\5r "S S \5r "S S \5r "S S\5r "SS\5r "SS\5r"SS\5rSrSrS$SjrSrS$SjrS$SjrSrSrSrSrSrS rS!rS"rS#rg)%zPruning methods.N)ABCabstractmethod)Iterablech\rSrSr%Sr\\S'Sr\S5r Sr \ SS.S j5r S S jr S rS rg)BasePruningMethod zAbstract base class for creation of new pruning techniques. Provides a skeleton for customization requiring the overriding of methods such as :meth:`compute_mask` and :meth:`apply`. _tensor_namecN[XRURU55 g)a]Multiply the mask into original tensor and store the result. Multiplies the mask (stored in ``module[name + '_mask']``) into the original tensor (stored in ``module[name + '_orig']``) and stores the result into ``module[name]`` by using :meth:`apply_mask`. Args: module (nn.Module): module containing the tensor to prune inputs: not used. N)setattrr apply_mask)selfmoduleinputss L/var/www/auris/envauris/lib/python3.13/site-packages/torch/nn/utils/prune.py__call__BasePruningMethod.__call__s ))4??6+BCcg)aoCompute and returns a mask for the input tensor ``t``. Starting from a base ``default_mask`` (which should be a mask of ones if the tensor has not been pruned yet), generate a random mask to apply on top of the ``default_mask`` according to the specific pruning method recipe. Args: t (torch.Tensor): tensor representing the importance scores of the parameter to prune. default_mask (torch.Tensor): Base mask from previous pruning iterations, that need to be respected after the new mask is applied. Same dims as ``t``. Returns: mask (torch.Tensor): mask to apply to ``t``, of same dims as ``t`` N)r t default_masks r compute_maskBasePruningMethod.compute_mask srcURc SUS35e[XRS-5n[XRS-5nURURS9U-nU$)aYSimply handles the multiplication between the parameter being pruned and the generated mask. Fetches the mask and the original tensor from the module and returns the pruned version of the tensor. Args: module (nn.Module): module containing the tensor to prune Returns: pruned_tensor (torch.Tensor): pruned version of the input tensor Module z has to be pruned_mask_origdtype)r getattrtor)r rmaskorig pruned_tensors rr BasePruningMethod.apply_mask4ss    ) / VH- . / )v007:;v007:;djj1D8 rNimportance_scorescSnU"XU/UQ70UD6n[X5nUb(URUR:Xd SUSU35eOUn[U[5(d9UR US-U5 UR U [ R"U5n O7[XS-5R5R[ RS9n URX9S9n URUS-U 5 [XURU55 URU5 U$![ aKn [U[5(d/[XS-5nUR X(5 UR US- U eSn A ff=f) arAdd pruning on the fly and reparametrization of a tensor. Adds the forward pre-hook that enables pruning on the fly and the reparametrization of a tensor in terms of the original tensor and the pruning mask. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. args: arguments passed on to a subclass of :class:`BasePruningMethod` importance_scores (torch.Tensor): tensor of importance scores (of same shape as module parameter) used to compute mask for pruning. The values in this tensor indicate the importance of the corresponding elements in the parameter being pruned. If unspecified or None, the parameter will be used in its place. kwargs: keyword arguments passed on to a subclass of a :class:`BasePruningMethod` c SnSn/nURR5HFup[U [5(dMU RU:XdM.U nUR U5 US- nMH US::dSUSUS35eUHnURU M U"U0UD6n X*lUb][U[ 5(aURU 5 Un U $[U[5(a[ U5n U RU 5 U n U $)NrzFAvoid adding multiple pruning hooks to the same tensor z of module z. Use a PruningContainer.)_forward_pre_hooksitems isinstancerr appendPruningContaineradd_pruning_method) clsrnameargskwargs old_methodfoundhooks_to_removekhookmethod containers r_get_composite_method6BasePruningMethod.apply.._get_composite_methodas2JE!O!44::<d$5664;L;LPT;T!%J#**1-QJE =  P!F+fX5NP P%--a0% $)&)F"& %j*:;;11&9'FM  ,=>> 0 4 8 F&"3"3F"; <  , ,V 4  f&677vg~6))$5&&tg~6G  sAD!! E6+AE11E6cUb"URUR:XdS5eOUnUbUO[R"U5nXRX2S9-$)aSCompute and returns a pruned version of input tensor ``t``. According to the pruning rule specified in :meth:`compute_mask`. Args: t (torch.Tensor): tensor to prune (of same dimensions as ``default_mask``). importance_scores (torch.Tensor): tensor of importance scores (of same shape as ``t``) used to compute mask for pruning ``t``. The values in this tensor indicate the importance of the corresponding elements in the ``t`` that is being pruned. If unspecified or None, the tensor ``t`` will be used in its place. default_mask (torch.Tensor, optional): mask from previous pruning iteration, if any. To be considered when determining what portion of the tensor that pruning should act on. If None, default to a mask of ones. Returns: pruned version of tensor ``t``. z8importance_scores should have the same shape as tensor tr@)rArDrEr)r rrr's rpruneBasePruningMethod.prunesd*  (!''1772 JI J2!" '3'?|U__UVEW $$%6$RRRrcURc SUS35eURU5n[XR5(a[XR5 URURS-nUR UlURURS- UR URS- [XRU5 g)aBRemove the pruning reparameterization from a module. The pruned parameter named ``name`` remains permanently pruned, and the parameter named ``name+'_orig'`` is removed from the parameter list. Similarly, the buffer named ``name+'_mask'`` is removed from the buffers. Note: Pruning itself is NOT undone or reversed! Nrz: has to be pruned before pruning can be removedrr)r r hasattrdelattrrCdata_buffersr )r rweightr#s rremoveBasePruningMethod.removes    ) X VHV W X )( 6,, - - F-- .!!$"3"3g"=>KK   t007: ; OOD--7 8))40rr)NN)__name__ __module__ __qualname____firstlineno____doc__str__annotations__rrrr classmethodrMrPrX__static_attributes__rrrrr sS  D  &,:>~~@S<1rrc<\rSrSrSrSrSrSrSrSr Sr S r g ) r/iaContainer holding a sequence of pruning methods for iterative pruning. Keeps track of the order in which pruning methods are applied and handles combining successive pruning calls. Accepts as argument an instance of a BasePruningMethod or an iterable of them. c&SUl[U[5(d#URUlUR U5 g[ U5S:Xa)USRUlUR US5 gUHnUR U5 M g)Nrr*r)_pruning_methodsr-rr r0len)r r3r:s r__init__PruningContainer.__init__sz?A$)) $ 1 1D   # #D ) Y!^ $Q 4 4D   # #DG ,''/rc"[U[5(dUb[[U5S35eUbGURUR:wa-[ SURSUS3SURS3-5eU=R U4- slg)zAdd a child pruning ``method`` to the container. Args: method (subclass of BasePruningMethod): child pruning method to be added to the container. Nz$ is not a BasePruningMethod subclassz._combine_masksSsqH{{{1H""n4ai$$ 4vu--(Qjj7 ,C7$LVHU $xxf ,R 1cQ ,RxSWXX T{mf,'C$$0QWWT{mf,!#=f>Q>Q=R!STT"..qvDI.NL(OO(..OAHMO!-Ss 0 E=Erz)r rrrr:r"s rrPruningContainer.compute_mask7s+8= ~&&r*f6 r)rer N) rZr[r\r]r^rgr0rsrwr|rrbrrrr/r/s& 0+&*+*]rr/c@^\rSrSrSrSrSr\U4Sj5rSr U=r $)IdentityizsUtility pruning method that does not prune any units but generates the pruning parametrization with a mask of ones.rc UnU$rqrr rrr"s rrIdentity.compute_masks  rc">[TU]X5$)jAdd pruning on the fly and reparametrization of a tensor. Adds the forward pre-hook that enables pruning on the fly and the reparametrization of a tensor in terms of the original tensor and the pruning mask. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. superrM)r1rr2 __class__s rrMIdentity.applysw}V**rr) rZr[r\r]r^rrrarMrb __classcell__rs@rrrs$~!L + +rrcF^\rSrSrSrSrSrSr\U4Sj5r Sr U=r $)RandomUnstructurediaPrune (currently unpruned) units in a tensor at random. Args: name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. rc&[U5 Xlgrq_validate_pruning_amount_initamountr rs rrgRandomUnstructured.__init__%f- rc^UR5n[URU5n[XC5 UR [ R S9nUS:waX[ R"U5n[ R"URS5US9nSURS5UR'U$)Nr>rrr8) nelement_compute_nparams_topruner_validate_pruning_amountrGrDrH rand_liketopkviewindices)r rr tensor_sizenparams_topruner"probrs rrRandomUnstructured.compute_masksjjl 34;; L !>!!0G0G!H a ??1%D::diim?D*+DIIbM$,, ' rc >[TU]XUS9$)acAdd pruning on the fly and reparametrization of a tensor. Adds the forward pre-hook that enables pruning on the fly and the reparametrization of a tensor in terms of the original tensor and the pruning mask. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. rr)r1rr2rrs rrMRandomUnstructured.applys"w}V&}99rr rZr[r\r]r^rrgrrarMrbrrs@rrrs- "L (::rrcJ^\rSrSrSrSrSrSr\SU4Sjj5r Sr U=r $) L1Unstructuredia^Prune (currently unpruned) units in a tensor by zeroing out the ones with the lowest L1-norm. Args: amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. rc&[U5 Xlgrqrrs rrgL1Unstructured.__init__rrc\UR5n[URU5n[XC5 UR [ R S9nUS:waW[ R"[ R"U5RS5USS9nSURS5UR'U$)Nr>rrFr8largest) rrrrrGrDrHrabsrr)r rrrrr"rs rrL1Unstructured.compute_masksjjl 34;; L !>!!0G0G!H a ::eiil//3PUVD*+DIIbM$,, ' rc >[TU]XX4S9$)aAdd pruning on the fly and reparametrization of a tensor. Adds the forward pre-hook that enables pruning on the fly and the reparametrization of a tensor in terms of the original tensor and the pruning mask. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. importance_scores (torch.Tensor): tensor of importance scores (of same shape as module parameter) used to compute mask for pruning. The values in this tensor indicate the importance of the corresponding elements in the parameter being pruned. If unspecified or None, the module parameter will be used in its place. rr'r)r1rr2rr'rs rrML1Unstructured.applys ,w}   rrrqrrs@rrrs-"L ,  rrcN^\rSrSrSrSrSSjrSr\SU4Sjj5r Sr U=r $) RandomStructuredi-aPrune entire (currently unpruned) channels in a tensor at random. Args: amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. dim (int, optional): index of the dim along which we define channels to prune. Default: -1. rc2[U5 XlX lgrq)rrr)r rrs rrgRandomStructured.__init__;s%f- rc0[U5 [XR5 URURn[ UR U5n[ XC5 SnUS:XaUnU$U"XRX45nXbRURS9-nU$)aCompute and returns a mask for the input tensor ``t``. Starting from a base ``default_mask`` (which should be a mask of ones if the tensor has not been pruned yet), generate a random mask to apply on top of the ``default_mask`` by randomly zeroing out channels along the specified dim of the tensor. Args: t (torch.Tensor): tensor representing the parameter to prune default_mask (torch.Tensor): Base mask from previous pruning iterations, that need to be respected after the new mask is applied. Same dims as ``t``. Returns: mask (torch.Tensor): mask to apply to ``t``, of same dims as ``t`` Raises: IndexError: if ``self.dim >= len(t.shape)`` c[R"U5n[R"XCS9RnXE:n[R"U5n[ S5/[ UR5-nXhU'SXx'U$)Nrr*)rDrandkthvaluevalues zeros_likerrfrA) rr nchannelsnchannels_topruner threshold channel_maskr"rs r make_mask0RandomStructured.compute_mask..make_maskisi::i(DtAHHI+L##A&D;-#agg,.C#HDIKrrr) _validate_structured_pruning_validate_pruning_dimrrArrrr!r)r rrrrrr"s rrRandomStructured.compute_maskAs, %Q' a*ggdhh' 34;; L !>  a D  Q+GD OO$**O5 5D rc >[TU]XX4S9$)aAdd pruning on the fly and reparametrization of a tensor. Adds the forward pre-hook that enables pruning on the fly and the reparametrization of a tensor in terms of the original tensor and the pruning mask. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. dim (int, optional): index of the dim along which we define channels to prune. Default: -1. rrr)r1rr2rrrs rrMRandomStructured.applys&w}V&}BBrrrrrs@rrr-s0  L =~CCrrcN^\rSrSrSrSrSSjrSr\S U4Sjj5r Sr U=r $) LnStructuredia=Prune entire (currently unpruned) channels in a tensor based on their L\ ``n``-norm. Args: amount (int or float): quantity of channels to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. n (int, float, inf, -inf, 'fro', 'nuc'): See documentation of valid entries for argument ``p`` in :func:`torch.norm`. dim (int, optional): index of the dim along which we define channels to prune. Default: -1. rc>[U5 XlX lX0lgrq)rrnr)r rrrs rrgLnStructured.__init__s%f- rc[U5 [XR5 URURn[ UR U5nX4- n[ XC5 [XRUR5n[R"XeSS9nSnUS:XaUn U $U"XRUR5n XRU RS9-n U $)aCompute and returns a mask for the input tensor ``t``. Starting from a base ``default_mask`` (which should be a mask of ones if the tensor has not been pruned yet), generate a mask to apply on top of the ``default_mask`` by zeroing out the channels along the specified dim with the lowest L\ ``n``-norm. Args: t (torch.Tensor): tensor representing the parameter to prune default_mask (torch.Tensor): Base mask from previous pruning iterations, that need to be respected after the new mask is applied. Same dims as ``t``. Returns: mask (torch.Tensor): mask to apply to ``t``, of same dims as ``t`` Raises: IndexError: if ``self.dim >= len(t.shape)`` Trc[R"U5n[S5/[UR5-nX$U'SX4'U$)Nr*)rDrrrfrA)rrrr"rs rr,LnStructured.compute_mask..make_masksA##A&D;-#agg,.CHDIKrrr)rrrrArrr _compute_normrrDrrr!r) r rrrrnparams_tokeepnormrrr"s rrLnStructured.compute_masks, %Q'a*ggdhh' 34;; L$6 !> Q1zz$$?  a D  Q$,,7D OO$**O5 5D rc (>[TU]UUUUUUS9$)aAdd pruning on the fly and reparametrization of a tensor. Adds the forward pre-hook that enables pruning on the fly and the reparametrization of a tensor in terms of the original tensor and the pruning mask. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. n (int, float, inf, -inf, 'fro', 'nuc'): See documentation of valid entries for argument ``p`` in :func:`torch.norm`. dim (int): index of the dim along which we define channels to prune. importance_scores (torch.Tensor): tensor of importance scores (of same shape as module parameter) used to compute mask for pruning. The values in this tensor indicate the importance of the corresponding elements in the parameter being pruned. If unspecified or None, the module parameter will be used in its place. )rrrr'r)r1rr2rrrr'rs rrMLnStructured.applys-4w}  /   r)rrrrrqrrs@rrrs/  LDL    rrcB^\rSrSrSrSrSr\U4Sj5rSr U=r $)CustomFromMaskircXlgrqr")r r"s rrgCustomFromMask.__init__s rcURURR:XdeX RRURS9-nU$)Nr)rAr"r!rrs rrCustomFromMask.compute_masks@!!TYY__444iill1C1ClDD rc >[TU]XUS9$)rrr)r1rr2r"rs rrMCustomFromMask.apply"sw}V}55rr) rZr[r\r]rrgrrarMrbrrs@rrrs&L  6 6rrc0[RX5 U$)aApply pruning reparametrization without pruning any units. Applies pruning reparametrization to the tensor corresponding to the parameter called ``name`` in ``module`` without actually pruning any units. Modifies module in place (and also return the modified module) by: 1) adding a named buffer called ``name+'_mask'`` corresponding to the binary mask applied to the parameter ``name`` by the pruning method. 2) replacing the parameter ``name`` by its pruned version, while the original (unpruned) parameter is stored in a new parameter named ``name+'_orig'``. Note: The mask is a tensor of ones. Args: module (nn.Module): module containing the tensor to prune. name (str): parameter name within ``module`` on which pruning will act. Returns: module (nn.Module): modified (i.e. pruned) version of the input module Examples: >>> # xdoctest: +SKIP >>> m = prune.identity(nn.Linear(2, 3), 'bias') >>> print(m.bias_mask) tensor([1., 1., 1.]) )rrM)rr2s ridentityr2s> NN6 Mrc2[RXU5 U$)aPrune tensor by removing random (currently unpruned) units. Prunes tensor corresponding to parameter called ``name`` in ``module`` by removing the specified ``amount`` of (currently unpruned) units selected at random. Modifies module in place (and also return the modified module) by: 1) adding a named buffer called ``name+'_mask'`` corresponding to the binary mask applied to the parameter ``name`` by the pruning method. 2) replacing the parameter ``name`` by its pruned version, while the original (unpruned) parameter is stored in a new parameter named ``name+'_orig'``. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. Returns: module (nn.Module): modified (i.e. pruned) version of the input module Examples: >>> # xdoctest: +SKIP >>> m = prune.random_unstructured(nn.Linear(2, 3), 'weight', amount=1) >>> torch.sum(m.weight_mask == 0) tensor(1) )rrM)rr2rs rrandom_unstructuredrUsBV62 Mrc.[RXX#S9 U$)aWPrune tensor by removing units with the lowest L1-norm. Prunes tensor corresponding to parameter called ``name`` in ``module`` by removing the specified `amount` of (currently unpruned) units with the lowest L1-norm. Modifies module in place (and also return the modified module) by: 1) adding a named buffer called ``name+'_mask'`` corresponding to the binary mask applied to the parameter ``name`` by the pruning method. 2) replacing the parameter ``name`` by its pruned version, while the original (unpruned) parameter is stored in a new parameter named ``name+'_orig'``. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. importance_scores (torch.Tensor): tensor of importance scores (of same shape as module parameter) used to compute mask for pruning. The values in this tensor indicate the importance of the corresponding elements in the parameter being pruned. If unspecified or None, the module parameter will be used in its place. Returns: module (nn.Module): modified (i.e. pruned) version of the input module Examples: >>> # xdoctest: +SKIP >>> m = prune.l1_unstructured(nn.Linear(2, 3), 'weight', amount=0.2) >>> m.state_dict().keys() odict_keys(['bias', 'weight_orig', 'weight_mask']) r)rrM)rr2rr's rl1_unstructuredrzs$LV Mrc2[RXX#5 U$)aPrune tensor by removing random channels along the specified dimension. Prunes tensor corresponding to parameter called ``name`` in ``module`` by removing the specified ``amount`` of (currently unpruned) channels along the specified ``dim`` selected at random. Modifies module in place (and also return the modified module) by: 1) adding a named buffer called ``name+'_mask'`` corresponding to the binary mask applied to the parameter ``name`` by the pruning method. 2) replacing the parameter ``name`` by its pruned version, while the original (unpruned) parameter is stored in a new parameter named ``name+'_orig'``. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. dim (int): index of the dim along which we define channels to prune. Returns: module (nn.Module): modified (i.e. pruned) version of the input module Examples: >>> # xdoctest: +SKIP >>> m = prune.random_structured( ... nn.Linear(5, 3), 'weight', amount=3, dim=1 ... ) >>> columns_pruned = int(sum(torch.sum(m.weight, dim=0) == 0)) >>> print(columns_pruned) 3 )rrM)rr2rrs rrandom_structuredrsJ65 Mrc 0[RXX#XES9 U$)aYPrune tensor by removing channels with the lowest L\ ``n``-norm along the specified dimension. Prunes tensor corresponding to parameter called ``name`` in ``module`` by removing the specified ``amount`` of (currently unpruned) channels along the specified ``dim`` with the lowest L\ ``n``-norm. Modifies module in place (and also return the modified module) by: 1) adding a named buffer called ``name+'_mask'`` corresponding to the binary mask applied to the parameter ``name`` by the pruning method. 2) replacing the parameter ``name`` by its pruned version, while the original (unpruned) parameter is stored in a new parameter named ``name+'_orig'``. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. amount (int or float): quantity of parameters to prune. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. n (int, float, inf, -inf, 'fro', 'nuc'): See documentation of valid entries for argument ``p`` in :func:`torch.norm`. dim (int): index of the dim along which we define channels to prune. importance_scores (torch.Tensor): tensor of importance scores (of same shape as module parameter) used to compute mask for pruning. The values in this tensor indicate the importance of the corresponding elements in the parameter being pruned. If unspecified or None, the module parameter will be used in its place. Returns: module (nn.Module): modified (i.e. pruned) version of the input module Examples: >>> from torch.nn.utils import prune >>> m = prune.ln_structured( ... nn.Conv2d(5, 3, 2), 'weight', amount=0.3, dim=1, n=float('-inf') ... ) r&)rrM)rr2rrrr's r ln_structuredrs&Rf Mrc [U[5(d [S5eUbUO0n[U[5(d [S5e[R R RUVVs/sH!upEURXE4[XE55PM# snn5n[R R RUVVs/sH1upE[XES-[R"[XE555PM3 snn5n[5nSUl U"S 0UD6n SU l U RS:wa[SUSU R35eURU 5 URXg5n S n UHCupE[XE5n U R!5n XX-R#U 5n[%XEUS 9 X- n ME gs snnfs snnf) a Globally prunes tensors corresponding to all parameters in ``parameters`` by applying the specified ``pruning_method``. Modifies modules in place by: 1) adding a named buffer called ``name+'_mask'`` corresponding to the binary mask applied to the parameter ``name`` by the pruning method. 2) replacing the parameter ``name`` by its pruned version, while the original (unpruned) parameter is stored in a new parameter named ``name+'_orig'``. Args: parameters (Iterable of (module, name) tuples): parameters of the model to prune in a global fashion, i.e. by aggregating all weights prior to deciding which ones to prune. module must be of type :class:`nn.Module`, and name must be a string. pruning_method (function): a valid pruning function from this module, or a custom one implemented by the user that satisfies the implementation guidelines and has ``PRUNING_TYPE='unstructured'``. importance_scores (dict): a dictionary mapping (module, name) tuples to the corresponding parameter's importance scores tensor. The tensor should be the same shape as the parameter, and is used for computing mask for pruning. If unspecified or None, the parameter will be used in place of its importance scores. kwargs: other keyword arguments such as: amount (int or float): quantity of parameters to prune across the specified parameters. If ``float``, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If ``int``, it represents the absolute number of parameters to prune. Raises: TypeError: if ``PRUNING_TYPE != 'unstructured'`` Note: Since global structured pruning doesn't make much sense unless the norm is normalized by the size of the parameter, we now limit the scope of global pruning to unstructured methods. Examples: >>> from torch.nn.utils import prune >>> from collections import OrderedDict >>> net = nn.Sequential(OrderedDict([ ... ('first', nn.Linear(10, 4)), ... ('second', nn.Linear(4, 1)), ... ])) >>> parameters_to_prune = ( ... (net.first, 'weight'), ... (net.second, 'weight'), ... ) >>> prune.global_unstructured( ... parameters_to_prune, ... pruning_method=prune.L1Unstructured, ... amount=10, ... ) >>> print(sum(torch.nn.utils.parameters_to_vector(net.buffers()) == 0)) tensor(10) z4global_unstructured(): parameters is not an IterableNz=global_unstructured(): importance_scores must be of type dictrtemprzROnly "unstructured" PRUNING_TYPE supported for the `pruning_method`. Found method z of type rrr)r-rrldictrDnnutilsparameters_to_vectorgetr rEr/r rr0rnumelview_ascustom_from_mask) parameterspruning_methodr'r4rr2relevant_importance_scoresrr;r: final_maskpointerparam num_param param_masks rglobal_unstructuredrs| j( + +NOO->-J)PR ' . .WXX"'!D!D#- ",  ! !6.'&2G H", "88>>66#- ", F7NEOOGFFFuM  J7 #E  s 0(G 8G c2[RXU5 U$)aPrune tensor corresponding to parameter called ``name`` in ``module`` by applying the pre-computed mask in ``mask``. Modifies module in place (and also return the modified module) by: 1) adding a named buffer called ``name+'_mask'`` corresponding to the binary mask applied to the parameter ``name`` by the pruning method. 2) replacing the parameter ``name`` by its pruned version, while the original (unpruned) parameter is stored in a new parameter named ``name+'_orig'``. Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. mask (Tensor): binary mask to be applied to the parameter. Returns: module (nn.Module): modified (i.e. pruned) version of the input module Examples: >>> from torch.nn.utils import prune >>> m = prune.custom_from_mask( ... nn.Linear(5, 3), name='bias', mask=torch.tensor([0, 1, 0]) ... ) >>> print(m.bias_mask) tensor([0., 1., 0.]) )rrM)rr2r"s rrrus:t, MrcURR5HNup#[U[5(dMURU:XdM.UR U5 URU Us $ [ SUSUS35e)aRemove the pruning reparameterization from a module and the pruning method from the forward hook. The pruned parameter named ``name`` remains permanently pruned, and the parameter named ``name+'_orig'`` is removed from the parameter list. Similarly, the buffer named ``name+'_mask'`` is removed from the buffers. Note: Pruning itself is NOT undone or reversed! Args: module (nn.Module): module containing the tensor to prune name (str): parameter name within ``module`` on which pruning will act. Examples: >>> m = random_unstructured(nn.Linear(5, 7), name='weight', amount=0.2) >>> m = remove(m, name='weight') z Parameter 'z ' of module z/ has to be pruned before pruning can be removed)r+r,r-rr rXrn)rr2r8r9s rrXrXs{&,,224 d- . .43D3D3L KK ))!,M 5  dV<x/^_ rcUR5H>upURR5Hn[U[5(dM g M@ g)a4Check if a module is pruned by looking for pruning pre-hooks. Check whether ``module`` is pruned by looking for ``forward_pre_hooks`` in its modules that inherit from the :class:`BasePruningMethod`. Args: module (nn.Module): object that is either pruned or unpruned Returns: binary answer to whether ``module`` is pruned. Examples: >>> from torch.nn.utils import prune >>> m = nn.Linear(5, 7) >>> print(prune.is_pruned(m)) False >>> prune.random_unstructured(m, name='weight', amount=0.2) >>> print(prune.is_pruned(m)) True TF) named_modulesr+rr-r)r_ submoduler9s r is_prunedrsH,,,. 00779D$ 122:/ rcD[U[R5(d[SUS35e[U[R5(aUS:d=[U[R5(d.[ U5S:d[ U5S:a[ SUS35egg) aPValidate helper to check the range of amount at init. Args: amount (int or float): quantity of parameters to prune. If float, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If int, it represents the absolute number of parameters to prune. Raises: ValueError: if amount is a float not in [0, 1], or if it's a negative integer. TypeError: if amount is neither a float nor an integer. Note: This does not take into account the number of parameters in the tensor to be pruned, which is known only at prune. zInvalid type for amount: z. Must be int or float.rg?gamount=zG should either be a float in the range [0, 1] or a non-negative integerN)r-numbersRealrlIntegralfloatrnrs rrrs$ fgll + +3F8;RSTT67++,,! vw// 0 0 6]S E&MC$7fXd e  %8 1rcp[U[R5(aX:a[SUSU35egg)aValidate that the pruning amount is meaningful wrt to the size of the data. Validation helper to check that the amount of parameters to prune is meaningful wrt to the size of the data (`tensor_size`). Args: amount (int or float): quantity of parameters to prune. If float, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If int, it represents the absolute number of parameters to prune. tensor_size (int): absolute number of parameters in the tensor to prune. rz: should be smaller than the number of parameters to prune=N)r-rrrnrrs rrrsE$&'**++0DfXWXcWd e  1E+rcpURn[U5S::a[SUS[U5S35eg)aMValidate that the tensor to be pruned is at least 2-Dimensional. Validation helper to check that the tensor to be pruned is multi- dimensional, such that the concept of "channels" is well-defined. Args: t (torch.Tensor): tensor representing the parameter to prune Raises: ValueError: if the tensor `t` is not at least 2D. r*zZStructured pruning can only be applied to multidimensional tensors. Found tensor of shape z with z dimsN)rArfrn)rrAs rrrsE GGE 5zQ ?gVCJ#.B177)LMMrc[[UR555nUS:aX2nURU5 [R "XUS9nU$)a"Compute the L_n-norm of a tensor along all dimensions except for the specified dimension. The L_n-norm will be computed across all entries in tensor `t` along all dimension except for the one identified by dim. Example: if `t` is of shape, say, 3x2x4 and dim=2 (the last dim), then norm will have Size [4], and each entry will represent the `L_n`-norm computed using the 3x2=6 entries for each of the 4 channels. Args: t (torch.Tensor): tensor representing the parameter to prune n (int, float, inf, -inf, 'fro', 'nuc'): See documentation of valid entries for argument p in torch.norm dim (int): dim identifying the channels to prune Returns: norm (torch.Tensor): L_n norm computed across all dimensions except for `dim`. By construction, `norm.shape = t.shape[-1]`. r)pr)listrrrXrDr)rrrdimsrs rrrAsG( aeeg D QwiKK ::a$ 'D Krrq)r^rabcrrcollections.abcrrDrr/rrrrrrrrrrrrrrXrrrrrrrrrrr+s#$ x1x1vO(Od+ +28:*8:v? &? DfC(fCR~ $~ B6&66 F"J)X&R,^tnB<: < 0 *+4Nr