eTh_::SSKJr SSKrSSKJrJr SSKJr SSKrSSK J r J r SSKJ r SSK JrJrJrJr \ R$/r\R("\5r\"S S S 9r\"S S S 9r\"S S S 9r\"SS S 9r\"SS S 9r\"SS S 9r\"SS S 9r\"SS S 9r\R>RA5r!\"S5(a\!(aSr"S!S"Sjjr#"SS\ RH5r%S#S$Sjjr&S%S&Sjjr'S'Sjr(S(Sjr)SS.S)Sjjr*S*Sjr+S+Sjr,S r-g),) annotationsN) lru_cachewraps)Callable) storage_ptr storage_size)nn)is_torch_greater_or_equalis_torch_xla_availableis_torchdynamo_compilingloggingz2.6T) accept_devz2.4z2.3z2.2z2.1z2.0z1.13z1.12z2.5cJSSKJn U"XURUR5$)z A function that calls the internal `_softmax_backward_data` PyTorch method and that adjusts the arguments according to the torch version detected. r)_softmax_backward_data)torchrdimdtype)parent grad_outputoutputrselfrs R/var/www/auris/envauris/lib/python3.13/site-packages/transformers/pytorch_utils.pysoftmax_backward_datar1s - !+vzz4:: NNcURURR5nURRX!5R 5R 5nUR bZUS:Xa)UR R 5R 5nO+UR UR 5R 5n[URR55n[U5XR'[R"USUSUR SLS9RURR5nSURl URRUR55 SURl UR bKSUR l UR RWR55 SUR l U$)a| Prune a linear layer to keep only entries in index. Used to remove heads. Args: layer (`torch.nn.Linear`): The layer to prune. index (`torch.LongTensor`): The indices to keep in the layer. dim (`int`, *optional*, defaults to 0): The dimension on which to keep the indices. Returns: `torch.nn.Linear`: The pruned layer as a new layer with `requires_grad=True`. Nr r)biasFT)toweightdevice index_selectdetachclonerlistsizelenr Linear requires_gradcopy_ contiguouslayerindexrWbnew_size new_layers rprune_linear_layerr2<sa HHU\\(( )E !!#-446<<>A zz !8 !!#))+A 5!((*002AELL%%'(HJHM (1+x{49OPSSTYT`T`TgTghI%*I" 1<<>*%)I" zz', $Q\\^,'+ $ rc<^\rSrSrSrU4SjrSSjrSrSrU=r $)Conv1D^a 1D-convolutional layer as defined by Radford et al. for OpenAI GPT (and also used in GPT-2). Basically works like a linear layer but the weights are transposed. Args: nf (`int`): The number of output features. nx (`int`): The number of input features. cF>[TU]5 XlX l[R "[ R"X!55Ul[R "[ R"U55Ul [RRURSS9 g)Ng{Gz?)std) super__init__nfnxr Parameterremptyrzerosrinitnormal_)rr:r; __class__s rr9Conv1D.__init__isa ll5;;r#67 LLR1   .rc:SR"S0URD6$)NzConv1D(nf={nf}, nx={nx}))format__dict__)rs r__repr__Conv1D.__repr__qs)00A4==AArc UR5SSUR4-n[R"URUR SURS55UR 5nUR U5nU$)N)r%r:raddmmrviewr)rxsize_outs rforwardConv1D.forwardts^668CR=DGG:- KK 166"affRj#94;; G FF8 r)rr:r;r)returnstr) __name__ __module__ __qualname____firstlineno____doc__r9rGrO__static_attributes__ __classcell__)rAs@rr4r4^s/Brr4chURURR5nURRX!5R 5R 5nUS:Xa)UR R 5R 5nO+UR UR 5R 5n[URR55n[U5XR'[USUS5RURR5nSURl URRUR55 SURl SUR l UR RUR55 SUR l U$)a Prune a Conv1D layer to keep only entries in index. A Conv1D work as a Linear layer (see e.g. BERT) but the weights are transposed. Used to remove heads. Args: layer ([`~pytorch_utils.Conv1D`]): The layer to prune. index (`torch.LongTensor`): The indices to keep in the layer. dim (`int`, *optional*, defaults to 1): The dimension on which to keep the indices. Returns: [`~pytorch_utils.Conv1D`]: The pruned layer as a new layer with `requires_grad=True`. rr FT)rrr r!r"r#rr$r%r&r4r(r)r*r+s rprune_conv1d_layerr[{s; HHU\\(( )E !!#-446<<>A ax JJ    % % ' JJu  $ $ & , , .ELL%%'(HJHMx{HQK033ELL4G4GHI%*I" 1<<>*%)I"#(INN  NN(#'INN rc[U[R5(a[XUcSS9$US9$[U[5(a[ XUcSS9$US9$[ SUR35e)a Prune a Conv1D or linear layer to keep only entries in index. Used to remove heads. Args: layer (`Union[torch.nn.Linear, Conv1D]`): The layer to prune. index (`torch.LongTensor`): The indices to keep in the layer. dim (`int`, *optional*): The dimension on which to keep the indices. Returns: `torch.nn.Linear` or [`~pytorch_utils.Conv1D`]: The pruned layer as a new layer with `requires_grad=True`. rrr zCan't prune layer of class ) isinstancer r'r2r4r[ ValueErrorrA)r,r-rs r prune_layerr`sl%##!%ANN#NN E6 " "!%ANN#NN6u6GHIIrc^^^ [U5S:d US35e[[R"T5R5nU[U5:wa[ SUS[U5S35eUS:aUSR TnUH4nUR TU:wdM[ SUSUR T35e USR TU-S:wa![ SUSR TS U35eUSR TU-m [ UU 4S jU55n[ U4S j[U655n[R"UTS 9$T"U6$) a This function chunks the `input_tensors` into smaller input tensor parts of size `chunk_size` over the dimension `chunk_dim`. It then applies a layer `forward_fn` to each chunk independently to save memory. If the `forward_fn` is independent across the `chunk_dim` this function will yield the same result as directly applying `forward_fn` to `input_tensors`. Args: forward_fn (`Callable[..., torch.Tensor]`): The forward function of the model. chunk_size (`int`): The chunk size of a chunked tensor: `num_chunks = len(input_tensors[0]) / chunk_size`. chunk_dim (`int`): The dimension over which the `input_tensors` should be chunked. input_tensors (`Tuple[torch.Tensor]`): The input tensors of `forward_fn` which will be chunked Returns: `torch.Tensor`: A tensor with the same shape as the `forward_fn` would have given if applied`. Examples: ```python # rename the usual forward() fn to forward_chunk() def forward_chunk(self, hidden_states): hidden_states = self.decoder(hidden_states) return hidden_states # implement a chunked forward function def forward(self, hidden_states): return apply_chunking_to_forward(self.forward_chunk, self.chunk_size_lm_head, self.seq_len_dim, hidden_states) ```rz" has to be a tuple/list of tensorszforward_chunk_fn expects z arguments, but only z input tensors are givenz/All input tenors have to be of the same shape: z, found shape zThe dimension to be chunked z( has to be a multiple of the chunk size c3B># UHoRTTS9v M g7f)r]N)chunk).0 input_tensor chunk_dim num_chunkss r ,apply_chunking_to_forward..s"$ugtWc%7%7 %7%Rgtsc3.># UH nT"U6v M g7fNrD)rdinput_tensors_chunk forward_fns rrhrisuZtCVj*=>Ztsr]) r&inspect signature parametersr_shapetupleziprcat) rm chunk_sizerf input_tensorsnum_args_in_forward_chunk_fn tensor_shapereinput_tensors_chunks output_chunksrgs ` ` @rapply_chunking_to_forwardr{sR }  !Wm_4V#WW !$'w'8'8'D'O'O#P #s='99'(D'EEZ[^_l[mZno    A~$Q'--i8 )L!!), < El^T##/#5#5i#@"AC*   ! !) ,z 9Q >.}Q/?/E/Ei/P.QR"|%  #1%++I6*D  %$ugt$uuuZ]_sZtuu yyI66 } %%rcZ^[R"X5n[U5U- nUH mT[U4SjU55- mSUT'M" UR S5R 5R S5n[R"[U55UR5nX4$)a  Finds the heads and their indices taking `already_pruned_heads` into account. Args: heads (`List[int]`): List of the indices of heads to prune. n_heads (`int`): The number of heads in the model. head_size (`int`): The size of each head. already_pruned_heads (`Set[int]`): A set of already pruned heads. Returns: `Tuple[Set[int], torch.LongTensor]`: A tuple with the indices of heads to prune taking `already_pruned_heads` into account and the indices of rows/columns to keep in the layer weight. c36># UHoT:aSOSv M g7f)r rNrD)rdhheads rrh3find_pruneable_heads_and_indices..sM8L14x!Q.8LsrrJr ) ronessetsumrLr*eqaranger&long)headsn_heads head_sizealready_pruned_headsmaskr-rs @r find_pruneable_heads_and_indicesrs ::g )D J- -EcM8LMMMT  99R= # # % ( ( +D#ll3t95d;@@BE <r)indexingc.[R"USU06$)z Wrapper around torch.meshgrid to avoid warning messages about the introduced `indexing` argument. Reference: https://pytorch.org/docs/1.13/generated/torch.meshgrid.html r)rmeshgrid)rtensorss rrrs >>7 6X 66rc[(a[SSKJn [X5(aEUR 5nUR UR 5R5UR4$UR RS:Xa/[5(a SSK nURRU5nO [U5nUR U[U54$)a Unique identifier to a tensor storage. Multiple different tensors can share the same underlying storage. For example, "meta" tensors all share the same storage, and thus their identifier will all be equal. This identifier is guaranteed to be unique and constant for this tensor's storage during its lifetime. Two tensor storages with non-overlapping lifetimes may have the same id. r)DTensorxlaN)"is_torch_greater_or_equal_than_2_0torch.distributed.tensorrr^to_localr storagedata_ptrnbytestyper torch_xla_XLAC_xla_get_tensor_idrr)tensorr local_tensorr unique_ids rid_tensor_storager$s*)4 f & &!??,L==,"6"6"8"A"A"CV]]R R }}U"'='?'? OO66v> ' ==)\&%9 99rcURRS:Xa[(d[R"U5nUR S:XaUR S5nURURSS5RUR S55RSS9R5R5$[R"X5$)a Same as `torch.isin` without flags, but MPS-friendly. We can remove this function when we stop supporting torch <= 2.3. See https://github.com/pytorch/pytorch/issues/77764#issuecomment-2067838075 Args: elements (`torch.Tensor`): Input elements test_elements (`torch.Tensor` or `int`): The elements to check against. Returns: `torch.Tensor`: A boolean tensor of the same shape as `elements` that is True for `elements` in `test_elements` and False otherwise mpsrr r])r r"is_torch_greater_or_equal_than_2_4rrndim unsqueezetilerqrrboolsqueezeisin)elements test_elementss risin_mps_friendlyr@su$-O-O ]3    ")33A6M}}]003Q7::=;R;RST;UVZZ_`Zaffhpprrzz(22rc^^UU4SjnU$)z LRU cache decorator from standard functools library, but with a workaround to disable caching when torchdynamo is compiling. Expected to work with class methods. c6>^[T5UUU4Sj5nU$)Nc8>[5(d[USTR35(d=URSTR3[ T0TD6"TR U555 UR STR35"U0UD6$T"U/UQ70UD6$)N_cached_)r hasattrrS __setattr__r__get____getattribute__)rargskwargsfunclru_args lru_kwargss rwrapperGcompile_compatible_method_lru_cache..decorator..wrapper_s+--tx %?@@$$"4==/2Ix4V:4VW[WcWcdhWi4j,,x -GH$YRXYYD242622r)r)rrrrs` r decorator6compile_compatible_method_lru_cache..decorator^s t 3  3rrD)rrrs`` r#compile_compatible_method_lru_cacherXs " r)r)r, nn.Linearr-torch.LongTensorrintrQr)r )r,r4r-rrrrQr4rk)r,nn.Linear | Conv1Dr-rrz int | NonerQr)rmzCallable[..., torch.Tensor]rurrfrrQ torch.Tensor) rz list[int]rrrrrzset[int]rQz!tuple[set[int], torch.LongTensor])rz!torch.Tensor | list[torch.Tensor]rz str | NonerQztuple[torch.Tensor, ...])rrrQztuple[torch.device, int, int])rrrztorch.Tensor | intrQr). __future__rrn functoolsrrtypingrrsafetensors.torchrrr utilsr r r r LayerNormALL_LAYERNORM_LAYERS get_loggerrSlogger"is_torch_greater_or_equal_than_2_6r"is_torch_greater_or_equal_than_2_3"is_torch_greater_or_equal_than_2_2"is_torch_greater_or_equal_than_2_1r#is_torch_greater_or_equal_than_1_13#is_torch_greater_or_equal_than_1_12 distributed is_available_torch_distributed_availablerr2Moduler4r[r`r{rrrrrrDrrrsy#& 7gg ~   H %%>uQU%V"%>uQU%V"%>uQU%V"%>uQU%V"&?uQU%V"%>uQU%V"&?SW&X#&?SW&X# %00==?U##(DODRYY:BJ,K&+K&K&K&  K&\ "/2JR&6RV7:830r