fThS5SrSSKrSSKrSSKJrJrJrJr SSKr SSK r SSK r SSK J r SSK Jr SSKJr SSKJrJr SS KJr SS KJrJr SS KJrJrJrJrJr SS KJ r SS K!J"r"J#r#J$r$J%r% SSK&J'r' \#"5(a SSK(J)r) SSK*J+r+ \%RX"\-5r.S\ R^S\0S\04Sjr1"SS\ Rd5r3"SS\ Rh5r5"SS\ Rh5r6S\50r7"SS\ Rh5r8\""SS \ 55r9"S!S"\95r:"S#S$\95r;\""S%S&\955r<\""S'S(9"S)S*\9\55r="S+S,\95r>"S-S.\9\5r?/S/Qr@g)0z=PyTorch MarianMTModel model, ported from the Marian C++ repo.N)ListOptionalTupleUnion)nn)CrossEntropyLoss)ACT2FN)CacheEncoderDecoderCache)GenerationMixin)AttentionMaskConverter_prepare_4d_attention_mask)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentions!CausalLMOutputWithCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutput)PreTrainedModel)auto_docstringis_torch_flex_attn_availableis_torchdynamo_compilinglogging) MarianConfig) BlockMask)make_flex_block_causal_mask input_ids pad_token_iddecoder_start_token_idcURUR5nUSS2SS24R5USS2SS24'X#SS2S4'Uc [S5eUR US:HU5 U$)z) Shift input ids one token to the right. Nrrz1self.model.config.pad_token_id has to be defined.i) new_zerosshapeclone ValueError masked_fill_)rrr shifted_input_idss b/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/marian/modeling_marian.pyshift_tokens_rightr*=sz"++IOO<(CRC0668ae4adLMM""#4#[TU]X5 gN)super__init__)selfr/r0r1 __class__s r)r6,MarianSinusoidalPositionalEmbedding.__init__Ps 6r+cURRup[R"[ U5VVs/sH?n[ U5Vs/sH%oC[R "SSUS--U- 5- PM' snPMA snn5n[ R"XURRSS9nUS-S:XaUS-OUS-S-n[ R"[R"USS2SSS2455USS2SU24'[ R"[R"USS2SSS2455USS2US24'[R"USS9Ulgs snfs snnf) z Identical to the XLM create_sinusoidal_embeddings except features are not interleaved. The cos features are in the 2nd half of the vector. [dim // 2:] i'F)dtype requires_gradrrN)r=)weightr$nparrayrangepowertorchemptyr< FloatTensorsincosr Parameter)r7n_posdimposj position_encoutsentinels r) _init_weight0MarianSinusoidalPositionalEmbedding._init_weightSs* [[&& xxX]^cXd eXdQTs LABHHUAaL3$677 LXd e kk%DKK,=,=US"Qw!|3!8#(a"..rvvl1add76K/LMAqzM!--bff\!QTT'5J.KLAxyLll3e< M esE ,E6E E input_ids_shapepast_key_values_length position_idsc>UcBUSSupE[R"X"U-[RURRS9n[ TU]U5$)z3`input_ids_shape` is expected to be [bsz x seqlen].Nr;)r<device)rCarangelongr>rVr5forward)r7rRrSrTbszseq_lenr8s r)rY+MarianSinusoidalPositionalEmbedding.forwardbsX  *2A.LC <<&(HPUPZPZcgcncncucuLw|,,r+)r>r4rN)__name__ __module__ __qualname____firstlineno____doc__intrr6rPrCno_gradSizeTensorrY__static_attributes__ __classcell__r8s@r)r-r-MsN7c7#7HUXM7ei77 = ]]_sw -$zz -CF -ZbchcocoZp -  - -r+r-c^\rSrSrSrSS\S\S\S\S\S\S \\ S \\4U4S jjjr SS \ RS \\ RS\\ S\\ RS\\ RS\S\\ RS\\ R\\ R\\\ R44SjjrSrU=r$)MarianAttentionpz=Multi-headed attention from 'Attention Is All You Need' paper embed_dim num_headsdropout is_decoderbias is_causalconfig layer_idxc t>[T U]5 XlX lX0lX-UlXplUR U-UR:wa[SURSUS35eUR S-UlX@l X`l Xl Uc>UR(a-[RSURRS35 [ R""XUS9Ul[ R""XUS9Ul[ R""XUS9Ul[ R""XUS9Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).gzInstantiating a decoder z without passing `layer_idx` is not recommended and will lead to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` when creating this class.rq)r5r6rmrnrohead_dimrsr&scalingrprrrtlogger warning_oncer8r^rLineark_projv_projq_projout_proj) r7rmrnrorprqrrrsrtr8s r)r6MarianAttention.__init__ss  "" !.  MMI %$.. 8MdnnM]$YKr3 }}d* $""     *4>>+B+B*CD,,  ii 4@ ii 4@ ii 4@  )TB r+ hidden_stateskey_value_statespast_key_valueattention_masklayer_head_maskoutput_attentionscache_positionr2c USLnUR5upn URU5RU SURUR5R SS5n XR -n Ub][U[5(aFURRUR5n U(a URnOURnOUnU(aUOUnU(a=Ub:W (a3WRURnURURnOUR!U5nUR#U5nURU SURUR5R SS5nURU SURUR5R SS5nUbNU(dUOSnWR%UUURSU05unnU(aSURUR'XR-SUR4nU R&"U6n UR&"U6nUR&"U6nURS5n[(R*"U UR SS55nUR5XR-U U4:wa.[-SXR-U U4SUR535eUb]USS2SS2SS2SUR.S 24nURXRU U5U-nURXR-U U5n[0R2R5USS 9nUbUR5UR4:wa*[-S UR4SUR535eURSSSS5URXRU U5-nURXR-U U5nU(a=URXRU U5nURXR-U U5nOSn[0R2R7UUR6UR8S 9n[(R*"UU5nUR5XR-XR4:wa7[-S XR-XR4SUR535eURXRXR5nUR SS5nUR'XUR:5nUR=U5nUUU4$)z#Input shape: Batch x Time x ChannelNr"rr;rTz$Attention weights should be of size z , but is rJz/Head mask for a single layer should be of size ptrainingz `attn_output` should be of size )sizer~viewrnrw transposerx isinstancer is_updatedgetrtcross_attention_cacheself_attention_cache key_cache value_cacher|r}updatereshaperCbmmr&r$r functionalsoftmaxrorrmr)r7rrrrrrris_cross_attentionrZtgt_len_ query_statesrcurr_past_key_valuecurrent_states key_states value_states proj_shapesrc_len attn_weightsattn_weights_reshaped attn_probs attn_outputs r)rYMarianAttention.forwards.T9',,.a{{=166sBPTP]P]^hhijlmn #ll2  %.*=>>+66::4>>J %*8*N*N'*8*M*M'&4#-?)] ."<,66t~~FJ.::4>>JL^4J;;~6L#b$..$--PZZ[\^_`J',,S"dnndmmT^^_`bcdL)7It+>+E+E dnn?OQ_>`,( L&@DN--dnn=NN*B > #++Z8 ''4 #++Z8 //!$yyz/C/CAq/IJ    3#7'"J J6nn8LgW^7_6`a %%'(*   %+Aq!5Kz7G7G7K5K,KLN',,S..'7SVddL',,S>>-A7GTL}},,\r,B  &##%$..):: Et~~FWEXY',,./1+//2q!<|?P?PQTVdVdfmov?wwL',,S>>-A7GTL  %1$5$5c>>7T[$\ !055cNN6JGU\]L$( !]]**<4<AAr+) rsrormrwrrrpr|rtrnrr~rxr})FTFNN)NNNNFN)r^r_r`rarbrcfloatboolrrr6rCrfr rrYrgrhris@r)rkrkpsYG  )-#'%C%C%C %C  %C  %C%C&%CC=%C%CT48*.1526"'15pB||pB#5<<0pB! pB !. pB "%,,/ pB pB!.pB u||Xell3XeELL>Q5RR SpBpBr+rkc^\rSrSrS S\S\\4U4SjjjrS S\RS\RS\RS\\ S \ \R\\R44 S jjr S r U=r$)MarianEncoderLayerirsrtc>[TU]5 URUl[UR "URUR URUUS9Ul[R"UR5Ul URUl [URUlUR Ul[R""URUR$5Ul[R""UR$UR5Ul[R"UR5Ulg)N)rmrnrorsrt)r5r6d_modelrmMARIAN_ATTENTION_CLASSES_attn_implementationencoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normror activation_function activation_fnactivation_dropoutr{encoder_ffn_dimfc1fc2final_layer_normr7rsrtr8s r)r6MarianEncoderLayer.__init__s 1&2M2MNnn44,,  %'LL$@!~~ #F$>$>?"(";";99T^^V-C-CD99V33T^^D " T^^ <r+rrrrr2cUnURUUUUS9upn[RRXRURS9nXQ-nUR U5nUnUR URU55n[RRXRURS9nURU5n[RRXRURS9nXQ-nURU5nUR[R:Xa[R"U5R5(d)[R "U5R5(aC[R""UR5R$S- n[R&"X*US9nU4n U(aX4- n U $)aW Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )rrrrri)minmax)rrrrorrrrrrrr<rCfloat16isinfanyisnanfinforclamp) r7rrrrresidualrr clamp_valueoutputss r)rYMarianEncoderLayer.forward"s$!)-')+/ *8* & Q  --m||VZVcVc-d  0 11-@  **488M+BC  --m?V?Vaeanan-o /  --m||VZVcVc-d  0 --m<   %-- / KK & * * , , M0J0N0N0P0P++m&9&9:>>EK!KK <[YM "   &Gr+) rrrormrrrrrr4F)r^r_r`rarrrcr6rCrErrrYrgrhris@r)rrs=|= ==0-2 0((0))0** 0 $D> 0 u  (5+<+<"== > 00r+reagerc^\rSrSrSS\S\\4U4SjjjrSS\RS\\RS\\RS\\RS \\RS \\RS \\ S \\ S \\ S\\RS\ \R\\ \R\R444SjjrSrU=r$)MarianDecoderLayeriYrsrtc T>[TU]5 URUl[UR "URUR URSSUUS9UlURUl [URUl URUl [R"UR5Ul[UR "URUR URSUUS9Ul[R"UR5Ul[R&"URUR(5Ul[R&"UR(UR5Ul[R"UR5Ulg)NT)rmrnrorprrrsrt)rorprsrt)r5r6rrmrrdecoder_attention_headsrrror rrrrrr encoder_attnencoder_attn_layer_normr{decoder_ffn_dimrrrrs r)r6MarianDecoderLayer.__init__Zs6 1&2M2MNnn44,, ~~ #F$>$>?"(";";$&LL$@!4V5P5PQ NN  * *,,  (*||DNN'C$99T^^V-C-CD99V33T^^D " T^^ <r+rrencoder_hidden_statesencoder_attention_maskrcross_attn_layer_head_maskrr use_cacherr2c  Un URUUUUUU S9upn[RRXRURS9nX-nUR U5nSn Ub`Un UR UUUUUUS9upn[RRXRURS9nX-nURU5nUn URURU55n[RRXRURS9nURU5n[RRXRURS9nX-nURU5nU4nU(aXU 4- nU (aX4- nU$)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. encoder_hidden_states (`torch.FloatTensor`): cross attention input to the layer of shape `(batch, seq_len, embed_dim)` encoder_attention_mask (`torch.FloatTensor`): encoder attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. cross_attn_layer_head_mask (`torch.FloatTensor`): mask for cross-attention heads in a given layer of size `(decoder_attention_heads,)`. past_key_value (`Tuple(torch.FloatTensor)`): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. )rrrrrrrN)rrrrrr) rrrrorrrrrrrrr)r7rrrrrrrrrrrself_attn_weightscross_attn_weightsrs r)rYMarianDecoderLayer.forwardysD!<@>>'))+/) >G  ( (Gr+) rrrormrrrrrrrr4) NNNNNNFTN)r^r_r`rarrrcr6rCrfr rrrErYrgrhris@r)rrYs1=|= ==D268<9=26=A*.,1$(15S||S!.S ( 5 S !) 6 S "%,,/ S%-U\\$:S!S$D>SD>S!.S u  (51B1BEDUDU1U+V"WW XSSr+rc L\rSrSr\rSrSrSrSr S\ \ R\ R\44SjrSS\ \R"S4S\R"S \R"S \S \4 S jjr\S\R"S \S\S\R.S \R"S\4 Sj5r\S5rSrg)MarianPreTrainedModelimodelTmodulec@URRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[5(aUR5 g[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[R5(aJUR R RS5 URR R5 gg)Nr)meanstd?)rsinit_stdrrr{r>datanormal_rqzero_r-rP Embeddingr1rfill_)r7rrs r) _init_weights#MarianPreTrainedModel._init_weightss"kk"" fbii ( ( MM   & &CS & 9{{&   &&('  C D D    !  - - MM   & &CS & 9!!- ""6#5#56<<>.  - - MM   $ $S ) KK   " " $.r+rr input_tensorrpast_key_valuesrc URRS:XaUbUS:HR5(aU$gURRS:Xa,[U[R 5(a [ U5nU$UbUR5OSnUb UROSnURRS:Xa5U(d.U(d'[R"UUUURS9(agURnURSn U(aUR5n O5[U[R 5(aURS OXi-S-n URUU U UUURSS 9n URRS:XaZUbWUR R"S ;a=U(d6[R$"U5R&n [R("X5n U $) Nflash_attention_2rflex_attentionrFsdpa) inputs_embedsrS is_trainingrr")sequence_length target_lengthr<r batch_size)cudaxpunpu)rsrrrrCrfrget_seq_lengthis_compileabler_ignore_causal_mask_sdparr<r$get_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionrVtyperr_unmask_unattended) r7rrrrrpast_seen_tokensusing_compilable_cacher<rr causal_mask min_dtypes r)_update_causal_mask)MarianPreTrainedModel._update_causal_masks ;; + +/B B)~/D.I.I.K.K%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`aCRC^!?!?di ;; + +v 5>T]n%>>*'7 MM  ""&,,Q/ !+??AMnell;;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCK[Kr+rrr<rc UbUR5S:XaUnU$[R"U5Rn[R"X4XUR S9nUS:wa[R "USS9nU[R"X$R S9URSS5:-nUSSSS2SS24RUSSS5nUbUR5nURSn USS2SS2SS2SU 24USS2SSSS24RUR 5-n U S:Hn USS2SS2SS2SU 24RX5USS2SS2SS2SU 24'U$) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuer<rVr)diagonalrVr"r)rJrCrrfullrVtriurWrexpandr%r$to masked_fill) rrrr<rrkwargsr r  mask_length padding_masks r)rKMarianPreTrainedModel._prepare_4d_causal_attention_mask_with_cache_position,s}>  %.*<*<*>!*C(K*' E*..I** 0Y\j\q\qK!##jjqA 5<< >S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r+cURRn[R"/SQSSSSU//URS9nUR U5UUS.nU$)N)r rr;r r;r)rrdecoder_input_ids)rsrrCtensorrVne)r7 pad_tokenr dummy_inputss r)r$"MarianPreTrainedModel.dummy_inputsdsZKK,, LL"2Q2q)4L!MVZVaVab 'll95"!* r+Nr)r^r_r`rar config_classbase_model_prefixsupports_gradient_checkpointing_supports_cache_class_supports_static_cacherrr{rr-rrCrfr rr  staticmethodrcr<rpropertyr$rgr&r+r)rrsL&*# !%E"))R\\Cf*f$g%.#( BellK78BllB B  B  BH4 444{{ 4  4  44lr+rc<^\rSrSrSrSS\S\\R4U4Sjjjr Sr Sr SS\\ RS \\ RS \\ RS \\ RS \\S \\S\\S\\\ R\44SjjrSrU=r$) MarianEncoderipz Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`MarianEncoderLayer`]. Args: config: MarianConfig embed_tokens (nn.Embedding): output embedding rs embed_tokensc>[TU]U5 URUlURUlUR nUR UlURUl UR(a[R"U5OSUl UbX lO0[R "UR"X0R5Ul[%URX0R5Ul[R("[+UR,5Vs/sHn[/U5PM sn5UlSUlUR55 gs snf)NrF)r5r6roencoder_layerdrop layerdroprrr1max_position_embeddingsmax_source_positionsscale_embeddingmathsqrt embed_scaler0rr vocab_sizer-embed_positions ModuleListrAencoder_layersrlayersgradient_checkpointing post_init)r7rsr0rmrr8s r)r6MarianEncoder.__init__zs  ~~ 11NN !..$*$B$B!393I3I499Y/s  # ,  " V->-> K[K[ \D B  * *I7G7G mmvOdOdIe$fIeA%7%?Ie$fg &+#  %gsEcUR$r4r0r7s r)get_input_embeddings"MarianEncoder.get_input_embeddings   r+cXlgr4rCr7values r)set_input_embeddings"MarianEncoder.set_input_embeddings!r+rr head_maskrroutput_hidden_states return_dictr2cUbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb7UR X5 UR 5nURSUS5nO"UbUR 5SSnO [ S5eUcURU5UR-nURU5n XI-n [RRXRURS9n Ub[X$R 5nU(aSOSn U(aSOSn Ub\UR 5S[#UR$5:Xd2S[#UR$5S UR 5SS 35e['UR$5HupU(aX4-n S nUR(a([(R*"/5nUUR,:aS nU(aS nO^UR.(a8UR(a'UR1UR2U UUbX=OSU5nOU"U UUbX=OSUS9nUSn U(dMU US4-n M U(aX4-n U(d[5SXU 455$[7XU S9$)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(encoder_layers, encoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzDYou cannot specify both input_ids and inputs_embeds at the same timer"z5You have to specify either input_ids or inputs_embedsrr&rz&The head_mask should be specified for  layers, but it is for .FT)NN)rrrc3.# UH ocMUv M g7fr4r&.0vs r) (MarianEncoder.forward..se$Sq$Ss last_hidden_stater attentions)rsrrOuse_return_dictr&%warn_if_padding_and_no_attention_maskrrr0r9r;rrrorrr<lenr> enumeraterCrandr3r?_gradient_checkpointing_func__call__tupler)r7rrrNrrrOrP input_shape embed_posrencoder_statesall_attentionsidx encoder_layerto_dropdropout_probability layer_outputss r)rYMarianEncoder.forwards\2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K!r;r?;I  &',,.s3KTU U   --i84;K;KKM((5 %1  --m||VZVcVc-d   %7H[H[\N30d  >>#A&3t{{+;< 8T[[9I8JJabkbpbpbrstbuavvwx <#,DKK"8 C#!/2B!BG}}&+jjn#&7"G , ..4==$($E$E%..%&+4+@d) %M%2%&;D;PVZ*; %M!.a 0   !/=3C2E!EA#9D +.>>Ne]N$See e+Vd  r+) ror;r9r0r?r3r>r5r1r4)NNNNNNN)r^r_r`rarbrrrrr6rErKrC LongTensorrfrErrrrrYrgrhris@r)r/r/ps|8BLL;Q2!" 1559,059,0/3&*~ E,,-~ !!1!12~ ELL) ~   1 12 ~ $D> ~ 'tn~ d^~  uU\\"O3 4~ ~ r+r/c^\rSrSrSrSS\S\\R4U4Sjjjr Sr Sr SS\\ RS \\ RS \\ RS \\ RS \\ RS \\ RS\\\\ RS\\ RS\\S\\S\\S\\S\\ RS\\\ R\44SjjrSrU=r$) MarianDecoderiz Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`MarianDecoderLayer`] Args: config: MarianConfig embed_tokens (nn.Embedding): output embedding rsr0c >[TU]U5 URUlURUlUR UlURUlUR(a [R"UR5OSUl UbX lO;[R "UR"URUR 5Ul[%URURUR 5Ul[R("[+UR,5Vs/sH n[/XS9PM sn5UlSUlUR55 gs snf)Nr)rtF)r5r6rodecoder_layerdropr3rr1r4max_target_positionsr6r7r8rr9r0rrdecoder_vocab_sizer-r;r<rAdecoder_layersrr>r?r@)r7rsr0ir8s r)r6MarianDecoder.__init__#s   ~~ 11!..$*$B$B!8>8N8N499V^^4TW  # ,  " V-F-FX\XhXh iD B  * *FNNDRAUUR@UR S 9nU (aS OSnU (aS OSnU (aUbS OSnSn[CXV/SS/5HjunnUcM UR%5S[EURF5:XaM7SUS[EURF5SUR%5SS35e [IURF5HunnU (aUU4- nUR (a([(RJ"/5nUURL:aMLUR (aGUR (a6UROURPUUUUUbUUOSUbUUOSSU U U 5 n OU"UUUUUbUUOSUbUUOSUU U U S9 n U SnU (aU U (aSOSnU (dMUU S4- nUcMUU S4- nGM U (aUU4- nU (aUOSn!U(aURS5n!U (d[USUU!UUU455$[WUU!UUUS9$)a| Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.LongTensor` of shape `(batch_size, encoder_sequence_length)`, *optional*): Mask to avoid performing cross-attention on padding tokens indices of encoder input_ids. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder to avoid performing cross-attention on hidden heads. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. NzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...FzTYou cannot specify both decoder_input_ids and decoder_inputs_embeds at the same timer"TzPassing a tuple of `past_key_values` is deprecated and will be removed in Transformers v4.58.0. You should pass an instance of `EncoderDecoderCache` instead, e.g. `past_key_values=EncoderDecoderCache.from_legacy_cache(past_key_values)`.rr)r)rTrr&rNr}zThe `z` should be specified for rRrS) rrrrrrrrrr rr;c30# UH nUcMUv M g7fr4r&rUs r)rX(MarianDecoder.forward..)smAms )r[rrr\cross_attentions),rsrrOrr]r?rryrzr&rr$r0r9rr r from_legacy_cacherrrCrWrVronesrr rr<r;rrrozipr_r>r`rar3rbrcto_legacy_cacherdr)"r7rrrrrNr}rrrrrOrPrreturn_legacy_cacher seq_lengthrSmask_seq_lengthself_attn_cacher rTrall_hidden_statesall_self_attnsall_cross_attentionsnext_decoder_cache attn_mask mask_nameri decoder_layerrlrm next_caches" r)rYMarianDecoder.forward?sh2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?I   --i8M&(8(88 $ Z??"&    \  2CCOTO!.!3!3!5cr!: JETE`!?!?!Afg  !"\\&(>(KTaThThN  !*B*D*D4zAO"ZZ OML`L`aN/+>??  0 0   ..        ! ,1G1S%?&##"& "++  $&<>, & 4  --mt||VZVcVc-d #7BD0d&7rtr1r4) NNNNNNNNNNNNN)r^r_r`rarbrrrrr6rErKrCrorfrErrrrrYrgrhris@r)rqrqs|8BLL;Q,!" 1515=A=A,07;EI59$(,0/3&*15u E,,-u !.u  ((9(9: u !))9)9 : u ELL) u 'u||4u "%e.?.?(@"ABu   1 12u D>u $D>u 'tnu d^u !.u  uU\\"$MM Nu u r+rqc$j^\rSrSrSS/rS\4U4SjjrSrSrSr S r S r S r S \ S \R4Sjr\S!S\\R(S\\R*S\\R(S\\R*S\\R*S\\R*S\\R*S\\\\R*\4S\\\\R2S\\R2S\\R2S\\S\\S\\S\\S\\R*S \4"Sjj5rS rU=r$)" MarianModeli7zencoder.embed_tokens.weightzdecoder.embed_tokens.weightrsc>[TU]U5 URURp2[R "X1R U5UlURR(aUR=pEOG[R"UR5n[R"UR5nSUl[X5Ul [X5UlUR!5 gr4)r5r6rr:rrrsharedrs share_encoder_decoder_embeddingscopydeepcopyr/encoderrqdecoderr@)r7rsr1r:encoder_embed_tokensdecoder_embed_tokensr8s r)r6MarianModel.__init__;s  "("5"5v7H7HZll:~~{K ;; 7 7:>++ E #7$(==#= #'==#= DK$VB $VB  r+c>UR5R5$r4) get_encoderrErDs r)rE MarianModel.get_input_embeddingsQs!6688r+cURR(a=XlURURlURUR lgXRlgr4)rsrrrr0rrIs r)rK MarianModel.set_input_embeddingsUs> ;; 7 7K(, DLL %(, DLL %(-LL %r+cURR(a [S5eUR5R 5$)Nz`get_decoder_input_embeddings` should not be called if `config.share_encoder_decoder_embeddings` is `True`. Please use `get_input_embeddings` instead.)rsrr& get_decoderrErDs r)get_decoder_input_embeddings(MarianModel.get_decoder_input_embeddings]s< ;; 7 7H !6688r+cpURR(a [S5eXRlg)Na `config.share_encoder_decoder_embeddings` is set to `True` meaning the decoder input embeddings are shared with the encoder. In order to set the decoder input embeddings, you should simply set the encoder input embeddings by calling `set_input_embeddings` with the appropriate embeddings.)rsrr&rr0rIs r)set_decoder_input_embeddings(MarianModel.set_decoder_input_embeddingses. ;; 7 7r  %* !r+cUR$r4)rrDs r)rMarianModel.get_encodern ||r+cUR$r4rrDs r)rMarianModel.get_decoderqrr+new_num_tokensr2c URR(a [S5eUR5nUR X!5nUR U5 UR5nUcU$XRlUR5 U$Nz`resize_decoder_token_embeddings` should not be called if `config.share_encoder_decoder_embeddings` is `True`. Please use `resize_token_embeddings` instead.)rsrr&r_get_resized_embeddingsrru tie_weights)r7rold_embeddingsnew_embeddings model_embedss r)resize_decoder_token_embeddings+MarianModel.resize_decoder_token_embeddingsts ;; 7 7K  ::<55nU )).988:  ! *8 & r+rrr decoder_attention_maskrNdecoder_head_maskr}encoder_outputsrrdecoder_inputs_embedsrrrOrPrcU bU OURRn UbUOURRnU bU OURRn UbUOURRnUcUR UUUU U UUS9nORU(aK[ U[5(d6[US[U5S:aUSOS[U5S:aUSOSS9nURUUUSUUUU U U U UUUS9 nU(dUU-$[URURURURURURURURS9$) a? decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Marian uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. Example: ```python >>> from transformers import AutoTokenizer, MarianModel >>> tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de") >>> model = MarianModel.from_pretrained("Helsinki-NLP/opus-mt-en-de") >>> inputs = tokenizer("Studies have been shown that owning a dog is good for you", return_tensors="pt") >>> decoder_inputs = tokenizer( ... " Studien haben gezeigt dass es hilfreich ist einen Hund zu besitzen", ... return_tensors="pt", ... add_special_tokens=False, ... ) >>> outputs = model(input_ids=inputs.input_ids, decoder_input_ids=decoder_inputs.input_ids) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 26, 512] ```N)rrrNrrrOrPrrr;rZ rrrrrNr}rrrrrOrPr)r[rdecoder_hidden_statesdecoder_attentionsrencoder_last_hidden_staterencoder_attentions)rsrrOrr]rrrr_rrr[rrr\r)r7rrr rrNrr}rrrrrrrOrPrdecoder_outputss r)rYMarianModel.forwards|2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKs $D>s 'tns d^!s "!.#s $ %s s r+rzX The Marian Model with a language modeling head. Can be used for summarization. ) custom_introc&B^\rSrSrSr/SQrSS/r/SQrS\4U4Sjjr S r S r S-S \ S \ \ S\S\R 4U4SjjjrS.S \ S\R 4SjjrSrS \ SS 4SjrSrS\R 4SjrSr\S/S\ \R4S\ \R6S\ \R4S\ \R6S\ \R6S\ \R6S\ \R6S\ \\\R6\4S \ \\\R>S!\ \R>S"\ \R>S#\ \R4S$\ \S%\ \S&\ \S'\ \S(\ \R6S\ 4$S)jj5r!S#\R64S*jr"\#S+5r$S,r%U=r&$)0 MarianMTModelir)final_logits_biaszencoder.embed_positions.weightzdecoder.embed_positions.weightz$model.encoder.embed_positions.weightz$model.decoder.embed_positions.weight)z!model.encoder.embed_tokens.weightz!model.decoder.embed_tokens.weightlm_head.weightrscP>[TU]U5 [U5UlUR(a UR O UR nURS[R"SU455 [R"URUSS9Ul UR5 g)NrrFrv)r5r6rrrr:ruregister_bufferrCzerosrr{rlm_headr@)r7rstarget_vocab_sizer8s r)r6MarianMTModel.__init__s~   ( 171X1XF--^d^w^w 0%++qBS>T2UVyy1BO  r+c6URR5$r4)rrrDs r)rMarianMTModel.get_encoderzz%%''r+c6URR5$r4)rrrDs r)rMarianMTModel.get_decoder rr+Nrpad_to_multiple_of mean_resizingr2c>[TU]XU5nURR(aUR U5 U$r4)r5resize_token_embeddingsrsr_resize_final_logits_bias)r7rrrrr8s r)r%MarianMTModel.resize_token_embeddings#s88]jk ;; 7 7  * *> :r+cUR5nURXAU5nURU5 URRSnUR R (aXR lUR R (a^UR5bMUR R(d2UR5nURXa5nURU5 UR5$)Nr) rErrKr>r$rsrruget_output_embeddingstie_word_embeddings_get_resized_lm_headset_output_embeddings)r7rrargsrr old_lm_head new_lm_heads r)_resize_token_embeddings&MarianMTModel._resize_token_embeddings,s22455nVhi !!.1'..44Q7 ;; 7 7-;KK * KK 8 8**,8KK33446K33KPK  & &{ 3((**r+c:URR(a [S5eURR 5nUR X!5nURR U5 UR5bMURR(d2UR5nURXA5nURU5 URR 5nUcU$XRl UR5 URU5 U$r)rsrr&rrrrrrrrrurr)r7rrrrrrs r)r-MarianMTModel.resize_decoder_token_embeddingsBs ;; 7 7K  @@B55nU //?  % % ' 3DKK<[<[446K33KPK  & &{ 3zz>>@  ! *8 &  &&~6r+c,URRSnX::aURSS2SU24nON[R"SX- 4URRS9n[R "URU/SS9nUR SU5 g)Nr"rrrr)rr$rCrrVcatr)r7rold_num_tokensnew_bias extra_biass r)r'MarianMTModel._resize_final_logits_biasbs//55b9  +--a..@AHa)H%IRVRhRhRoRopJyy$"8"8*!E1MH 0(;r+cUR$r4rrDs r)r#MarianMTModel.get_output_embeddingskrr+rcXlgr4rr7rs r)r#MarianMTModel.set_output_embeddingsn% r+crUR5nUbK[URSS5(a/UR5R 5nUR X5 [URSS5(a[URSS5(ag[ XR5(a[XR5nURURURURS5nX0l UR5H&n[ US5(dMUR5 M( g) z Tie the weights between the input embeddings and the output embeddings. If the `torchscript` flag is set in the configuration, can't handle parameter sharing so we are cloning the weights instead. NrTis_encoder_decoderFtie_encoder_decoderr _tie_weights)rgetattrrsrrE_tie_or_clone_weightshasattrr(_tie_encoder_decoder_weightsrr_dynamic_tied_weights_keysmodulesr)r7output_embeddingsword_embeddings tied_weightsrs r)rMarianMTModel.tie_weightsqs!668  (WT[[BWY]-^-^"..0EEGO  & &'8 J 4;; 4e < <VkmrAsAst3344t%;%;<<< dllD,B,BIL /; +llnFv~..##%%r+rrr rrNrr}rrrrlabelsrrrOrPrcUbUOURRnU bXU (a[RS5 Sn Uc7U c4[ XRR URR 5nURUUUUUUUUU U U U UUUUS9nURUS5UR-nSnU bF[5nU"URSURR5U RS55nU(dU4USS-nUbU4U-$U$[UUURURUR UR"UR$UR&UR(S9 $) u0 decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Marian uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MarianMTModel >>> src = "fr" # source language >>> trg = "en" # target language >>> model_name = f"Helsinki-NLP/opus-mt-{src}-{trg}" >>> model = MarianMTModel.from_pretrained(model_name) >>> tokenizer = AutoTokenizer.from_pretrained(model_name) >>> sample_text = "où est l'arrêt de bus ?" >>> batch = tokenizer([sample_text], return_tensors="pt") >>> generated_ids = model.generate(**batch) >>> tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] "Where's the bus stop?" ``` NzJThe `use_cache` argument is changed to `False` since `labels` is provided.F)rr rrrNrr}rrrrrrOrPrrr"r) losslogitsrrrrrrr)rsr]rywarningr*rr rrrrrrurrrrrrrr)r7rrr rrNrr}rrrrrrrrOrPrr lm_logitsmasked_lm_lossloss_fctoutputs r)rYMarianMTModel.forwardsF&1%# UH/oRSTRUR55v M1 g7fr] index_selectrrVrV past_statebeam_idxs r)rX/MarianMTModel._reorder_cache.. s1rcqU_--aZ=N=N1OPPcq7:r;rdrrreordered_past layer_pasts ` r)_reorder_cacheMarianMTModel._reorder_cachesO)J rcmnpopcqrrQR.! N* r+)rrr)NTr4)NNNNNNNNNNNNNNNNN)'r^r_r`rar(_keys_to_ignore_on_load_missing_keys_to_ignore_on_saverrr6rrrcrrrrrrrrrrrrrCrorfrrrrErrYrr,rrgrhris@r)rrs  '# FGmnu | ((dh!7?}\` +s+_a_k_k+,@<<<&BLL&&815158<9=,0487;QUEI59=A-1$(,0/3&*15%t E,,-t !.t $E$4$45 t !) 6 t ELL) t $ELL1t 'u||4t "%ell(;_(L"MNt "%e.?.?(@"ABt   1 12t  ((9(9:t ))*t D>t $D>t 'tn!t "d^#t $!.%t & 't t lhELLhr+rc2^\rSrSrSrU4SjrSrSrU=r$)MarianDecoderWrapperiz This wrapper class is a helper class to correctly load pretrained checkpoints when the causal language model is used in combination with the [`EncoderDecoderModel`] framework. cD>[TU]U5 [U5Ulgr4)r5r6rqrr7rsr8s r)r6MarianDecoderWrapper.__init__s  $V, r+c&UR"U0UD6$r4r)r7rrs r)rYMarianDecoderWrapper.forwards||T,V,,r+r) r^r_r`rarbr6rYrgrhris@r)r$r$s ---r+r$c"^\rSrSrS/rU4SjrSrSrSrSr Sr S r \ SS \ \RS \ \R S \ \R"S \ \R"S\ \R S\ \R S\ \\R"S\ \R"S\ \RS\ \S\ \S\ \S\ \S\ \RS\\\44Sjj5r\S5rSrU=r$)MarianForCausalLMi#rc >[R"U5nSUlSUl[TU]U5 [ U5Ul[R"URURSS9Ul UR5 g)NTFrv)rrrprr5r6r$rrr{ hidden_sizer:rr@r&s r)r6MarianForCausalLM.__init__&sfv& $)!  )&1 yy!3!3V5F5FUS  r+cBURRR$r4rrr0rDs r)rE&MarianForCausalLM.get_input_embeddings2szz!!...r+c8XRRlgr4r0rIs r)rK&MarianForCausalLM.set_input_embeddings5s*/ 'r+cUR$r4rrDs r)r'MarianForCausalLM.get_output_embeddings8rr+cXlgr4rrs r)r'MarianForCausalLM.set_output_embeddings;rr+c$XRlgr4rr)r7rs r) set_decoderMarianForCausalLM.set_decoder>s $ r+c.URR$r4r9rDs r)rMarianForCausalLM.get_decoderAszz!!!r+rrrrrNr}rrrrrrOrPrr2cU bU OURRn U bU OURRn U bU OURRn URR UUUUUUUUU U U U US9 nUR US5nSnU baU RUR5n [5nU"URSURR5U RS55nU (dU4USS-nUbU4U-$U$[UUURURURUR S9$)a  cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MarianForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-fr-en") >>> model = MarianForCausalLM.from_pretrained("Helsinki-NLP/opus-mt-fr-en", add_cross_attention=False) >>> assert model.config.is_decoder, f"{model.__class__} has to be configured as a decoder." >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> expected_shape = [1, inputs.input_ids.shape[-1], model.config.vocab_size] >>> list(logits.shape) == expected_shape True ```Nrrr"r)rrrrr\r)rsrrOr]rrrrrVrrr:rrrr\r)r7rrrrrNr}rrrrrrOrPrrrrr r s r)rYMarianForCausalLM.forwardDs[\2C1N-TXT_T_TqTq$8$D $++JjJj &1%# UH/oRSTRUR55v M1 g7fr]rrs r)rX3MarianForCausalLM._reorder_cache..s1ncmU_--aZ=N=N1OPPcmrrrs ` r)r MarianForCausalLM._reorder_caches8)J ncmnn N*r+)rr)NNNNNNNNNNNNNN)r^r_r`rarr6rErKrrr:rrrrCrorfrErrrrrrYr,rrgrhris@r)r+r+#s*+ /0&%"1515=A>B,07;=A59-1$(,0/3&*59W E,,-W !.W  ((9(9: W !)):): ; W ELL) W 'u||4W "$u'8'8"9:W   1 12W ))*W D>W $D>W 'tnW d^W !!1!12W u77 8!W W rr+r+)r+rrr)Arbrr7typingrrrrnumpyr?rCtorch.utils.checkpointrtorch.nnr activationsr cache_utilsr r generationr modeling_attn_mask_utilsrrmodeling_outputsrrrrrmodeling_utilsrutilsrrrrconfiguration_marianr!torch.nn.attention.flex_attentionrintegrations.flex_attentionr get_loggerr^ryrfrcr*rr-Modulerkrrrrr/rqrrr$r+__all__r&r+r)rUsD // %!5). / !!;J   H % %,, c [^  -",,-FZBbiiZB|DDN$_5ssl]O]]@g )g TZ )Z zH 'H H V H)?H  HX -0 -B-BJ Yr+