fTh 0SrSSKrSSKrSSKrSSKJrJrJrJr SSK r SSK r SSK J r SSK J r JrJr SSKJr SSKJrJr SS KJr SS KJrJrJr SS KJrJr SS KJrJ r J!r!J"r"J#r#J$r$J%r% SS K&J'r' SSK(J)r)J*r*J+r+J,r, SSK-J.r. \*"5(a SSK/J0r0 SSK1J2r2 \"5(aSSKJ3r3 \,Rh"\55r6S\ RnS\8S\84Sjr9"SS\ Rt5r;"SS\ Rt5r<"SS\ Rz5r>"SS\>5r?"S S!\>5r@\>\@\?S".rA"S#S$\ Rz5rB"S%S&\ Rz5rC"S'S(\ Rz5rD\)"S)S*\'55rE"S+S,\E5rF"S-S.\E5rG"S/S0\E5rH"S1S2\E5rI\)"S3S4\E55rJ\)"S5S69"S7S8\E\55rK\)"S9S69"S:S;\E55rL\)"S<S=\E55rM"S>S?\E5rN\)"S@S69"SASB\E\55rO/SCQrPg)DzPyTorch BART model.N)ListOptionalTupleUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)CacheEncoderDecoderCache)GenerationMixin)AttentionMaskConverter_prepare_4d_attention_mask#_prepare_4d_attention_mask_for_sdpa)!flash_attn_supports_top_left_maskis_flash_attn_available)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentions!CausalLMOutputWithCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutput#Seq2SeqQuestionAnsweringModelOutputSeq2SeqSequenceClassifierOutput)PreTrainedModel)auto_docstringis_torch_flex_attn_availableis_torchdynamo_compilinglogging) BartConfig) BlockMask)make_flex_block_causal_mask)_flash_attention_forward input_ids pad_token_iddecoder_start_token_idcURUR5nUSS2SS24R5USS2SS24'X#SS2S4'Uc [S5eUR US:HU5 U$)z) Shift input ids one token to the right. Nr!rz1self.model.config.pad_token_id has to be defined.i) new_zerosshapeclone ValueError masked_fill_)r&r'r(shifted_input_idss ^/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/bart/modeling_bart.pyshift_tokens_rightr2Dsz"++IOO<(CRC0668ae4adLMM""#4#SUl[TU] XR-U5 gN)offsetsuper__init__)selfr7r8 __class__s r1r>'BartLearnedPositionalEmbedding.__init__Ys"  ++5}Er3r&past_key_values_length position_idsc>Uc]URSSupE[R"X"U-[RURR S9R US5nOURS5n[TU]%X0R-5$)z3`input_ids' shape is expected to be [bsz x seqlen].Nr;)dtypedevicer*r) r,torcharangelongweightrFexpand unsqueezer=forwardr<)r?r&rBrCbszseq_lenr@s r1rM&BartLearnedPositionalEmbedding.forward_s  $??2A.LC <<&(HPUPZPZcgcncncucufS"o (11!4Lw|kk9::r3)r<rN) __name__ __module__ __qualname____firstlineno____doc__intr>rGTensorrM__static_attributes__ __classcell__r@s@r1r5r5TsHFsF3F ; ;s ;^c^j^j ; ;r3r5c r^\rSrSrSrS S\S\S\S\\4U4SjjjrS\ R4U4S jjr S r U=r $) BartScaledWordEmbeddingmzT This module overrides nn.Embeddings' forward by multiplying with embeddings scale. r7r8 padding_idx embed_scalec2>[TU]XU5 X@lgN)r=r>r`)r?r7r8r_r`r@s r1r> BartScaledWordEmbedding.__init__rs  D&r3r&c<>[TU]U5UR-$rb)r=rMr`)r?r&r@s r1rMBartScaledWordEmbedding.forwardvswy)D,<,<<rGrXrMrYrZr[s@r1r]r]msJ's'3'S'_ghm_n''===r3r]c^\rSrSrSrSS\S\S\S\S\S\S \\ S \\4U4S jjjr SS \ RS \\ RS\\ S\\ RS\\ RS\S\\ RS\\ R\\ R\\\ R44SjjrSrU=r$) BartAttentionzz=Multi-headed attention from 'Attention Is All You Need' paper embed_dim num_headsdropout is_decoderbias is_causalconfig layer_idxc t>[T U]5 XlX lX0lX-UlXplUR U-UR:wa[SURSUS35eUR S-UlX@l X`l Xl Uc>UR(a-[RSURRS35 [ R""XUS9Ul[ R""XUS9Ul[ R""XUS9Ul[ R""XUS9Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).gzInstantiating a decoder z without passing `layer_idx` is not recommended and will lead to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` when creating this class.rp)r=r>rlrmrnhead_dimrrr.scalingrorqrslogger warning_oncer@rRrLineark_projv_projq_projout_proj) r?rlrmrnrorprqrrrsr@s r1r>BartAttention.__init__}s  "" !.  MMI %$.. 8MdnnM]$YKr3 }}d* $""     *4>>+B+B*CD,,  ii 4@ ii 4@ ii 4@  )TB r3 hidden_stateskey_value_statespast_key_valueattention_masklayer_head_maskoutput_attentionscache_positionreturnc USLnUR5upn URU5RU SURUR5R SS5n XR -n Ub][U[5(aFURRUR5n U(a URnOURnOUnU(aUOUnU(a=Ub:W (a3WRURnURURnOUR!U5nUR#U5nURU SURUR5R SS5nURU SURUR5R SS5nUbNU(dUOSnWR%UUURSU05unnU(aSURUR'XR-SUR4nU R&"U6n UR&"U6nUR&"U6nURS5n[(R*"U UR SS55nUR5XR-U U4:wa.[-SXR-U U4SUR535eUb]USS2SS2SS2SUR.S 24nURXRU U5U-nURXR-U U5n[0R2R5USS 9nUbUR5UR4:wa*[-S UR4SUR535eURSSSS5URXRU U5-nURXR-U U5nU(a=URXRU U5nURXR-U U5nOSn[0R2R7UUR6UR8S 9n[(R*"UU5nUR5XR-XR4:wa7[-S XR-XR4SUR535eURXRXR5nUR SS5nUR'XUR:5nUR=U5nUUU4$)#Input shape: Batch x Time x ChannelNr*r!r;rTz$Attention weights should be of size z , but is dimz/Head mask for a single layer should be of size ptrainingz `attn_output` should be of size )sizer}viewrmrv transposerw isinstancer is_updatedgetrscross_attention_cacheself_attention_cache key_cache value_cacher{r|updatereshaperGbmmr.r,r functionalsoftmaxrnrrlr~)r?rrrrrrris_cross_attentionrNtgt_len_ query_statesrcurr_past_key_valuecurrent_states key_states value_states proj_shapesrc_len attn_weightsattn_weights_reshaped attn_probs attn_outputs r1rMBartAttention.forwards.T9',,.a{{=166sBPTP]P]^hhijlmn #ll2  %.*=>>+66::4>>J %*8*N*N'*8*M*M'&4#-?)] ."<,66t~~FJ.::4>>JL^4J;;~6L#b$..$--PZZ[\^_`J',,S"dnndmmT^^_`bcdL)7It+>+E+E dnn?OQ_>`,( L&@DN--dnn=NN*B > #++Z8 ''4 #++Z8 //!$yyz/C/CAq/IJ    3#7'"J J6nn8LgW^7_6`a %%'(*   %+Aq!5Kz7G7G7K5K,KLN',,S..'7SVddL',,S>>-A7GTL}},,\r,B  &##%$..):: Et~~FWEXY',,./1+//2q!<|?P?PQTVdVdfmov?wwL',,S>>-A7GTL  %1$5$5c>>7T[$\ !055cNN6JGU\]L$( !]]**<4<AAr3) rrrnrlrvrqror{rsrmr~r}rwr|)FTFNNNNNNFN)rRrSrTrUrVrWrhboolrr"r>rGrXr rrMrYrZr[s@r1rjrjzsYG  '+#'%C%C%C %C  %C  %C%C$%CC=%C%CT48*.1526"'15pB||pB#5<<0pB! pB !. pB "%,,/ pB pB!.pB u||Xell3XeELL>Q5RR SpBpBr3rjcB^\rSrSrSrU4SjrSS\RS\\RS\\ S\\RS\\RS \ S \\RS \ \R\\R\\ \R44S jjr S r U=r$)BartFlashAttention2ia4 Bart flash attention module. This module inherits from `BartAttention` as the weights of the module stays untouched. The only required change would be on the forward pass where it needs to correctly call the public API of flash attention and deal with padding tokens in case the input contains any of them. cD>[TU]"U0UD6 [5Ulgrb)r=r>r_flash_attn_uses_top_left_mask)r?argskwargsr@s r1r>BartFlashAttention2.__init__s# $)&) /P.Q+r3rrrrrrrrc U(a [S5eUSLnUR5upn URU5RU SURUR 5n Ub][ U[5(aFURRUR5n U(a URnOURnOUnU(aUOUnU(a=Ub:W (a3WRURnURURnOURU5nUR!U5nURU SURUR 5R#SS5nURU SURUR 5R#SS5nUbNU(dUOSnWR%UUURSU05unnU(aSURUR'UR#SS5nUR#SS5nU R&nU[(R*:Xa[(R,"5(a[(R."5nOR[1UR2S5(aUR2R4nO URR6R&n[8R;SUS 35 U R=U5n UR=U5nUR=U5n[?U UUUU UR@(a URBOS URDURFS 9nURIXS5nURKU5nUSU4$) NzBartSdpaAttention2 attention does not support `output_attentions`. Use the argument `attn_implementation='eager'` when loading the model.r*r!r;rT_pre_quantization_dtypezThe input hidden states seems to be silently casted in float32, this might be related to the fact you have upcasted embedding or layer norm layers in float32. We will cast back the input in .r)rnrquse_top_left_mask)&r.rr}rrmrvrrrrrsrrrrr{r|rrrErGfloat32is_autocast_enabledget_autocast_gpu_dtypehasattrrrrrJrxrytor%rrnrqrrr~)r?rrrrrrrrrNq_lenrrrrrrr input_dtype target_dtypers r1rMBartFlashAttention2.forward&s Y .T9%**, A{{=166sBPTP]P]^  %.*=>>+66::4>>J %*8*N*N'*8*M*M'&4#-?)] ."<,66t~~FJ.::4>>JL^4J;;~6L#b$..$--PZZ[\^_`J',,S"dnndmmT^^_`bcdL)7It+>+E+E dnn?OQ_>`,( L&@DN--dnn= ))!Q/ #--a3 #(( %-- '((**$;;= &?@@#{{BB #{{1177    >$  (??<8L#|4J'??<8L.     $(MMDLLsnn"AA  "))#b9 mmK0 D.00r3)rr)rRrSrTrUrVr>rGrXrr rrrMrYrZr[s@r1rrs R48*.1526"'15e1||e1#5<<0e1! e1 !. e1 "%,,/ e1 e1!.e1 u||Xell3XeELL>Q5RR Se1e1r3rc8^\rSrSrS S\R S\\R S\\S\\R S\\R S\S\\R S \ \R \\R \\ \R 44U4S jjjr S r U=r $) BartSdpaAttentionirrrrrrrrc >U(a'[RS5 [TU] UUUUUUS9$USLnUR 5upn UR U5R U SURUR5RSS5n Ub][U[5(aFURRUR5n U(a URnOUR nOUnU(aUOUnU(a=Ub:W (a3WR"URnUR$URnOUR'U5nUR)U5nUR U SURUR5RSS5nUR U SURUR5RSS5nUbNU(dUOSnWR+UUURSU05unnU(aSURUR'SnUbUSS2SS2SS2SUR,S 24nU R.R0S :Xa3Ub0U R35n UR35nUR35nUR4(a UcU S:aSOS n[6R8R:R=U UUUUR>(a UR@OS US 9nURSS5R35nUR XURB5nUREU5nUSU4$)raBartModel is using BartSdpaAttention, but `torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True` . Falling back to the manual attention implementation, but specifying the manual implementation will be required from Transformers version v5.0.0 onwards. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)rrrrrNr*r!r;rTrcudaFr) attn_mask dropout_prq)#rxryr=rMrr}rrmrvrrrrrrsrrrrr{r|rr,rFtype contiguousrqrGrrscaled_dot_product_attentionrrnrlr~)r?rrrrrrrrrNrrrrrrrr causal_maskrqrr@s r1rMBartSdpaAttention.forwards    l 7?!1--"3- # .T9',,.a{{=166sBPTP]P]^hhijlmn  %.*=>>+66::4>>J %*8*N*N'*8*M*M'&4#-?)] ."<,66t~~FJ.::4>>JL^4J;;~6L#b$..$--PZZ[\^_`J',,S"dnndmmT^^_`bcdL)7It+>+E+E dnn?OQ_>`,( L&@DN--dnn=  %(Aq2HJ4D4DR4H2H)HIK    # #v -+2I'224L#..0J'224L !NN{/BwQR{DX] hh))FF   !&*mmdll G "++Aq1<<> "&&sT^^D mmK0 D.00r3r) rRrSrTrUrGrXrr rrrMrYrZr[s@r1rrs48*.1526"'15e1||e1#5<<0e1! e1 !. e1 "%,,/ e1 e1!.e1 u||Xell3XeELL>Q5RR Se1e1r3r)eagersdpaflash_attention_2c^\rSrSrS S\S\\4U4SjjjrS S\RS\RS\RS\\ S \ \R\\R44 S jjr S r U=r$)BartEncoderLayerirrrsc>[TU]5 URUl[UR "URUR URUUS9Ul[R"UR5Ul URUl [URUlUR Ul[R""URUR$5Ul[R""UR$UR5Ul[R"UR5Ulg)N)rlrmrnrrrs)r=r>d_modelrlBART_ATTENTION_CLASSES_attn_implementationencoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normrnr activation_function activation_fnactivation_dropoutrzencoder_ffn_dimfc1fc2final_layer_normr?rrrsr@s r1r>BartEncoderLayer.__init__s /0K0KLnn44,,  %'LL$@!~~ #F$>$>?"(";";99T^^V-C-CD99V33T^^D " T^^ <r3rrrrrcUnURUUUUS9upn[RRXRURS9nXQ-nUR U5nUnUR URU55n[RRXRURS9nURU5n[RRXRURS9nXQ-nURU5nUR[R:Xa[R"U5R5(d)[R "U5R5(aC[R""UR5R$S- n[R&"X*US9nU4n U(aX4- n U $)aW Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )rrrrri)minmax)rrrrnrrrrrrrrErGfloat16isinfanyisnanfinforclamp) r?rrrrresidualrr clamp_valueoutputss r1rMBartEncoderLayer.forwards$!)-')+/ *8* & Q  --m||VZVcVc-d  0 11-@  **488M+BC  --m?V?Vaeanan-o /  --m||VZVcVc-d  0 --m<   %-- / KK & * * , , M0J0N0N0P0P++m&9&9:>>EK!KK <[YM "   &Gr3) rrrnrlrrrrrrbF)rRrSrTrUr"rrWr>rG FloatTensorrrrMrYrZr[s@r1rrs=z=hsm==0-2 0((0))0** 0 $D> 0 u  (5+<+<"== > 00r3rc^\rSrSrSS\S\\4U4SjjjrSS\RS\\RS\\RS\\RS \\RS \\RS \\ S \\ S \\ S\\RS\ \R\\ \R\R444SjjrSrU=r$)BartDecoderLayeriErrrsc T>[TU]5 URUl[UR "URUR URSSUUS9UlURUl [URUl URUl [R"UR5Ul[UR "URUR URSUUS9Ul[R"UR5Ul[R&"URUR(5Ul[R&"UR(UR5Ul[R"UR5Ulg)NT)rlrmrnrorqrrrs)rnrorrrs)r=r>rrlrrdecoder_attention_headsrrrnr rrrrrr encoder_attnencoder_attn_layer_normrzdecoder_ffn_dimrrrrs r1r>BartDecoderLayer.__init__Fs6 /0K0KLnn44,, ~~ #F$>$>?"(";";$&LL$@!263N3NO NN  * *,,  (*||DNN'C$99T^^V-C-CD99V33T^^D " T^^ <r3rrencoder_hidden_statesencoder_attention_maskrcross_attn_layer_head_maskrr use_cacherrc  Un URUUUUUU S9upn[RRXRURS9nX-nUR U5nSn Ub`Un UR UUUUUUS9upn[RRXRURS9nX-nURU5nUn URURU55n[RRXRURS9nURU5n[RRXRURS9nX-nURU5nU4nU(aXU 4- nU (aX4- nU$)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. encoder_hidden_states (`torch.FloatTensor`): cross attention input to the layer of shape `(batch, seq_len, embed_dim)` encoder_attention_mask (`torch.FloatTensor`): encoder attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. cross_attn_layer_head_mask (`torch.FloatTensor`): mask for cross-attention heads in a given layer of size `(decoder_attention_heads,)`. past_key_value (`Tuple(torch.FloatTensor)`): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. )rrrrrrrN)rrrrrr) rrrrnrrrrrrrrr)r?rrrrrrrrrrrself_attn_weightscross_attn_weightsrs r1rMBartDecoderLayer.forwardesD!<@>>'))+/) >G  ( (Gr3) rrrnrlrrrrrrrrb) NNNNNNFTN)rRrSrTrUr"rrWr>rGrXr rrrrMrYrZr[s@r1rrEs1=z=hsm==D268<9=26=A*.,1$(15S||S!.S ( 5 S !) 6 S "%,,/ S%-U\\$:S!S$D>SD>S!.S u  (51B1BEDUDU1U+V"WW XSSr3rcz^\rSrSrSrS\S\S\S\4U4SjjrS\RS \R4S jr S r U=r $) BartClassificationHeadiz-Head for sentence-level classification tasks. input_dim inner_dim num_classespooler_dropoutc>[TU]5 [R"X5Ul[R "US9Ul[R"X#5Ulg)N)r)r=r>rrzdenseDropoutrnr~)r?r r r r r@s r1r>BartClassificationHead.__init__s@ YYy4 zzN3  )9 r3rrcURU5nURU5n[R"U5nURU5nUR U5nU$rb)rnrrGtanhr~)r?rs r1rMBartClassificationHead.forwardsN ]3  =1  =1  ]3  m4 r3)rrnr~) rRrSrTrUrVrWrhr>rGrXrMrYrZr[s@r1rrsQ7 : : : :  :U\\ellr3rc ,\rSrSr\rSrSrSS/rSS/r Sr Sr Sr Sr SrS r\S 5rSS \\R(S 4S \R(S\R(S\S\4 Sjjr\S \R(S\S\S\R4S\R(S\4 Sj5rSrg)BartPreTrainedModelimodelTzencoder.versionzdecoder.versionrrpast_key_valuescURRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[R5(aJUR R RS5 URR R5 gg)Nr)meanstdrg)rrinit_stdrrrzrJdatanormal_rpzero_ Embeddingr_rfill_)r?modulers r1 _init_weights!BartPreTrainedModel._init_weightsskk"" fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>.  - - MM   $ $S ) KK   " " $.r3cURRn[R"/SQSSSSU//URS9nUR U5US.nU$)N)r r;r r;rF)rr&)rrr'rGtensorrFne)r? pad_tokenr& dummy_inputss r1r. BartPreTrainedModel.dummy_inputssWKK,, LL"2Q2q)4L!MVZVaVab 'll95" r3rr# input_tensorrrc URRS:XaUbUS:HR5(aU$gURRS:Xa,[U[R 5(a [ U5nU$UbUR5OSnUb UROSnURRS:Xa5U(d.U(d'[R"UUUURS9(agURnURSn U(aUR5n O5[U[R 5(aURS OXi-S-n URUU U UUURSS 9n URRS:XaZUbWUR R"S ;a=U(d6[R$"U5R&n [R("X5n U $) Nrrflex_attentionrFr) inputs_embedsrB is_trainingr!r*)sequence_length target_lengthrEr batch_size)rxpunpu)rrrrrrGrXr$get_seq_lengthis_compileabler_ignore_causal_mask_sdparrEr,get_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionrFrrr_unmask_unattended) r?rr0rrrpast_seen_tokensusing_compilable_cacherEr5r6r min_dtypes r1_update_causal_mask'BartPreTrainedModel._update_causal_masks ;; + +/B B)~/D.I.I.K.K%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`aCRC^!?!?di ;; + +v 5>T]n%>>*'7 MM  ""&,,Q/ !+??AMnell;;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCK[Kr3r5r6rEr7c UbUR5S:XaUnU$[R"U5Rn[R"X4XUR S9nUS:wa[R "USS9nU[R"X$R S9URSS5:-nUSSSS2SS24RUSSS5nUbUR5nURSn USS2SS2SS2SU 24USS2SSSS24RUR 5-n U S:Hn USS2SS2SS2SU 24RX5USS2SS2SS2SU 24'U$) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. Nr') fill_valuerErFr!)diagonalr*r*r)rrGrrfullrFtriurHrrKr-r,r masked_fill) rr5r6rErr7rrrB mask_length padding_masks r1r>IBartPreTrainedModel._prepare_4d_causal_attention_mask_with_cache_position=s}>  %.*<*<*>!*C(K*' E*..I** 0Y\j\q\qK!##jjqA 5<< >S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r3rNr)rRrSrTrUr" config_classbase_model_prefixsupports_gradient_checkpointing"_keys_to_ignore_on_load_unexpected_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdpa_supports_cache_class_supports_static_cacher"propertyr.rrGrXr rrC staticmethodrWrEr>rYrr3r1rrsL&*#*;=N)O&,.AB"3!N ! % #( BellK78BllB B  B  BH4 444{{ 4  4  44r3rc\rSrSrSrSrg)PretrainedBartModelivc:[R"S[5 gNz_The class `PretrainedBartModel` has been depreciated, please use `BartPreTrainedModel` instead.warningswarn FutureWarningr?s r1__init_subclass__%PretrainedBartModel.__init_subclass__w m  r3rNrRrSrTrUrcrYrr3r1r[r[v r3r[c\rSrSrSrSrg)BartPretrainedModeli~c:[R"S[5 gr]r^rbs r1rc%BartPretrainedModel.__init_subclass__rer3rNrfrr3r1riri~rgr3ric"^\rSrSrSrSS\S\\R4U4Sjjjr Sr Sr SS\\ RS \\ RS \\ RS \\ RS \\S \\S\\S\\\44SjjrSrU=r$) BartEncoderiz Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`BartEncoderLayer`]. Args: config: BartConfig embed_tokens (nn.Embedding): output embedding rr embed_tokensc >[TU]U5 URUlURUlUR nUR UlURUl UR(a[R"U5OSn[URX0RUS9UlUbUR URl[#URU5Ul[&R("[+UR,5Vs/sH n[/XS9PM sn5UlUR2S:HUlUR2S:HUl[&R8"U5UlSUlUR?5 gs snfNrgrf)rsrrF) r=r>rnencoder_layerdrop layerdroprr'r_max_position_embeddingsmax_source_positionsscale_embeddingmathsqrtr] vocab_sizernrJr5embed_positionsr ModuleListrangeencoder_layersrlayersr_use_flash_attention_2 _use_sdparlayernorm_embeddinggradient_checkpointing post_init)r?rrrnrlr`ir@s r1r>BartEncoder.__init__s?  ~~ 11NN !..$*$B$B!.4.D.Ddii *# 3   y*:*:    #'3':':D   $=  * *  mmTYZ`ZoZoTp$qTpq%5f%JTp$qr &,&A&AEX&X#44>#%<< #: &+# %rsFcUR$rbrnrbs r1get_input_embeddings BartEncoder.get_input_embeddings   r3cXlgrbrr?values r1set_input_embeddings BartEncoder.set_input_embeddings!r3r&r head_maskr3routput_hidden_states return_dictrc0UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb"UnUR SUR S5nOUb USS2SS2S4nO [ S5eUcURU5nURU5n U RUR5n XI-n URU 5n [RRXRURS9n UbbUR (a SU;aUOSnOFUR"(a UcU(d[%X$R&5nO[)X$R&5nU(aSOSn U(aSOSn Ub`UR+5S[-UR.5:wa6[ S[-UR.5S UR+5SS 35e[1UR.5HupU(aX4-n S nUR(a([2R4"/5nUUR6:aS nU(aS nO^UR8(a8UR(a'UR;UR<U UUbX=OSU5nOU"U UUbX=OSUS9nUSn U(dMU US4-n M U(aX4-n U(d[?SXU 455$[AXU S9$)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(encoder_layers, encoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzDYou cannot specify both input_ids and inputs_embeds at the same timer*z5You have to specify either input_ids or inputs_embedsrrrz&The head_mask should be specified for  layers, but it is for rFT)NN)rrr!c3.# UH ocMUv M g7frbr.0vs r1 &BartEncoder.forward..<se$Sq$Ss last_hidden_stater attentions)!rrrruse_return_dictr.rr,rnryrrFrrrrnrr~rrrErrlenr} enumeraterGrandrrr_gradient_checkpointing_func__call__tupler)r?r&rrr3rrrinput embed_posrencoder_statesall_attentionsidx encoder_layerto_dropdropout_probability layer_outputss r1rMBartEncoder.forwards \2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "E!r9??2+>?I  &!!Q(+ETU U   --i8M((/ LL!5!56 %1 00?  --m||VZVcVc-d   %**343FDI$5>O"E^UhUh!i">Ne]N$See e+Vd  r3) r~rrnryrnrrrrr}rtr_rb)NNNNNNN)rRrSrTrUrVr"rrrr>rrrG LongTensorrXrrrrrrMrYrZr[s@r1rmrmsz",,9O>!" 1515,059,0/3&*J E,,-J !.J ELL) J   1 12 J $D> J 'tnJ d^J  uo% &J J r3rmc^\rSrSrSrSS\S\\R4U4Sjjjr Sr Sr SS\\ RS \\ RS \\ RS \\ RS \\ RS \\ RS\\\ RS\\ RS\\S\\S\\S\\S\\ RS\\\44SjjrSrU=r$) BartDecoderiBz Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`BartDecoderLayer`] Args: config: BartConfig embed_tokens (nn.Embedding): output embedding rrrnc T>[TU]U5 URUlURUlUR UlURUlUR(a [R"UR5OSn[URURUR US9UlUbUR URl[#URUR5Ul[&R("[+UR,5Vs/sH n[/XS9PM sn5UlUR2S:HUlUR2S:HUl[&R8"UR5UlSUlUR?5 gs snfrp) r=r>rndecoder_layerdroprrr'r_rsmax_target_positionsrurvrwrr]rxrnrJr5ryrrzr{decoder_layersrr}rr~rrrrr)r?rrrnr`rr@s r1r>BartDecoder.__init__KsH  ~~ 11!..$*$B$B!393I3Idii/s 3   v~~t/?/?[   #'3':':D   $=  * * NN mmTYZ`ZoZoTp$qTpq%5f%JTp$qr &,&A&AEX&X#44>#%<<#? &+# %rs!F%cUR$rbrrbs r1r BartDecoder.get_input_embeddingshrr3cXlgrbrrs r1r BartDecoder.set_input_embeddingskrr3r&rrrrcross_attn_head_maskrr3rrrrrrc U bU OURRn U bU OURRn U bU OURRn U bU OURRn UR (a/UR (aU (a[RS5 Sn USLUSL- (a [S5eUbURSURS5nUcURU5nSnU (aB[U[5(d-Sn[RS5 [R "U5nUR#5SSunnUbUR%5OSnU c#[&R("UUU-UR*S 9n Uc4[-5(d%UU-n[&R."UUUR*S 9n[U[5(a UR0OUnUR3UUU UU 5nUbdUbaUR4(a SU;aUOSnOEUR6(a UcU (d[9UUR:US 9nO[=XHR:US 9nUR?[@UU S 9nURCUR*5nUU-nUREU5n[FRHRKUURJUR S 9nU (aS OSnU (aS OSnU (aUbS OSnSn[MXV/SS/5HnunnUcM UR#5S[OURP5:wdM7[SUS[OURP5SUR#5SS35e [SURP5HunnU (aUU4- nUR (a([&RT"/5nUURV:aMLUR (aGUR (a6URYURZUUUUUbUUOSUbUUOSSU U U 5 n OU"UUUUUbUUOSUbUUOSUU U U S9 n U SnU (aU U (aSOSnU (dMUU S4- nUcMUU S4- nGM U (aUU4- nU (aUOSn!U(aUR]5n!U (d[_SUU!UUU455$[aUU!UUUS9$)a| Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.LongTensor` of shape `(batch_size, encoder_sequence_length)`, *optional*): Mask to avoid performing cross-attention on padding tokens indices of encoder input_ids. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder to avoid performing cross-attention on hidden heads. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. NzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...FzTYou cannot specify both decoder_input_ids and decoder_inputs_embeds at the same timer*TzPassing a tuple of `past_key_values` is deprecated and will be removed in Transformers v4.58.0. You should pass an instance of `EncoderDecoderCache` instead, e.g. `past_key_values=EncoderDecoderCache.from_legacy_cache(past_key_values)`.rr*)r)rCrrrrzThe `z` should be specified for rr) rrrrrrrrrr r!r;c30# UH nUcMUv M g7frbrrs r1r&BartDecoder.forward..csmAms )rrrrcross_attentions)1rrrrrrrrrxryr.rr,rnrr rfrom_legacy_cacherr:rGrHrFronesrrCr~rrrErryrrrrrrnziprr}rrrrrrto_legacy_cacherr)"r?r&rrrrrrr3rrrrrreturn_legacy_cacher7 seq_lengthrBmask_seq_lengthself_attn_cacherrCrall_hidden_statesall_self_attnsall_cross_attentionsnext_decoder_cacher mask_namer decoder_layerrr next_caches" r1rMBartDecoder.forwardnsHh2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?I   --i8M$ Z??"&    \  2CCOTO!.!3!3!5cr!: JETE`!?!?!Afg  !"\\&(>(KTaThThN  !*B*D*D4zAO"ZZ OML`L`aN/+>??  0 0   ..        ! ,1G1S**CDH^C^)?dh&$8$@IZ*M*!''&*&*D*,?,?*& ++E3IXf+g #}';';< % 4 00?  --mt||VZVcVc-d #7BD0d&7>#A&3t{{+;<$ {*DSEUDVW%NN,Q/03%r#,DKK"8 C#!m%55!}}&+jjn#&7**t}} $ A A!**!)*&/&;IcN1E1Q(-W[%" ! !.!#.*?+A7@7LYs^RV5I5U,S1[_#2&7'#1 ! *!,M%28I1q%Q"  =#3"55(4(]1-=,??(_#9d -!1 1 +4'$ (88:J'5FXlm  9+&+%1   r3) r~rrnryrnrrrrr}rr_rb) NNNNNNNNNNNNN)rRrSrTrUrVr"rrrr>rrrGrrXrrrrrrrMrYrZr[s@r1rrBs}z",,9O:!" 1515=A=A,07;=A59$(,0/3&*59@ E,,-@ !.@  ((9(9: @ !))9)9 : @ ELL) @ 'u||4@ "$u'8'8"9:@   1 12@ D>@ $D>@ 'tn@ d^@ !!1!12@  u?? @@ @ r3rc&8^\rSrSrSS/rS\4U4SjjrSrSrSr S r S r \ SS \ \RS \ \R S \ \RS\ \RS\ \R S\ \R S\ \R S\ \\R$S\ \\R$S\ \R$S\ \R$S\ \S\ \S\ \S\ \S\ \RS\\\44"Sjj5rSrU=r$) BartModeliqencoder.embed_tokens.weightdecoder.embed_tokens.weightrrcr>[TU]U5 URURp2UR(a [ R "UR5OSn[X1RX$S9Ul [XR5Ul [XR5Ul UR5 g)Nrgrf)r=r>r'rxrurvrwrr]sharedrmencoderrdecoderr)r?rrr_rxr`r@s r1r>BartModel.__init__us}  "("5"5v7H7HZ393I3Idii/s -j..+o "6;;7 "6;;7  r3cURR(GaFURRR[ R"S5:XaUR RRR[ R"S5:wakURURRUR R5 URURUR R5 gURURRUR5 URUR RUR5 gg)Nmeta) rrtie_word_embeddingsrrJrFrGrrn_tie_or_clone_weightsrrbs r1 _tie_weightsBartModel._tie_weightss ;; * * *{{!!((ELL-,,++2299U\\&=QQ**4<<+D+DdllF_F_`**4;; 8Q8QR**4<<+D+DdkkR**4<<+D+DdkkR +r3cUR$rb)rrbs r1rBartModel.get_input_embeddingss {{r3c|XlURURlURURlgrb)rrrnrrs r1rBartModel.set_input_embeddingss' $(KK !$(KK !r3cUR$rb)rrbs r1 get_encoderBartModel.get_encoder ||r3cUR$rbrrbs r1 get_decoderBartModel.get_decoderrr3r&rdecoder_input_idsdecoder_attention_maskrdecoder_head_maskrencoder_outputsrr3decoder_inputs_embedsrrrrrrcTUcEU cBUc [S5e[XRRURR5nU bU OURR n UbUOURR nU bU OURRn UbUOURRnUcURUUUU U UUS9nORU(aK[U[5(d6[US[U5S:aUSOS[U5S:aUSOSS9nURUUUSUUUU U U U UUUS9 nU(dUU-$[URUR UR"UR$UR&URUR"UR$S 9$) t decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Bart uses the `eos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. If you want to change padding behavior, you should read [`modeling_bart._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. NzIf no `decoder_input_ids` or `decoder_inputs_embeds` are passed, `input_ids` cannot be `None`. Please pass either `input_ids` or `decoder_input_ids` or `decoder_inputs_embeds`.)r&rrr3rrrrr!r;r r&rrrrrrr3rrrrr)rrdecoder_hidden_statesdecoder_attentionsrencoder_last_hidden_staterencoder_attentions)r.r2rrr'r(rrrrrrrrrrrrrrr)r?r&rrrrrrrrr3rrrrrrdecoder_outputss r1rMBartModel.forwardsf  $)>)F  U !3;;33T[[5W5W! 2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKrrrrrrrrGrrXrrrrrrrMrYrZr[s@r1rrqs79VW z  S0 15158<=A,0487;=A=A59=A$(,0/3&*59#t E,,-t !.t $E$4$45 t !))9)9 : t ELL) t $ELL1t 'u||4t "$u'8'8"9:t "$u'8'8"9:t   1 12t  ((9(9:t D>t $D>t 'tnt d^!t "!!1!12#t $ u(( )%t t r3rzV The BART Model with a language modeling head. Can be used for summarization. ) custom_introc(^\rSrSrSr/SQrS/rS\4U4SjjrSr Sr S(S \ S \ \ S \ S \R4U4SjjjrS \ S S 4SjrSrSrSr\S)S\ \R.S\ \R0S\ \R.S\ \R.S\ \R0S\ \R0S\ \R0S\ \\R4S\ \\R4S\ \R4S\ \R4S\ \R.S\ \ S \ \ S!\ \ S"\ \ S#\ \R.S \\\44$S$jj5rS\R04S%jr\ S&5r!S'r"U=r#$)*BartForConditionalGenerationir)rrlm_head.weightfinal_logits_biasrrcv>[TU]U5 [U5UlUR S[ R "SURRR455 [R"URURRRSS9Ul UR5 g)Nrr!Fru)r=r>rrregister_bufferrGzerosrr7rrzrlm_headrr?rrr@s r1r>%BartForConditionalGeneration.__init__s  v&  0%++q$**BSBSBbBb>c2deyy1B1B1Q1QX]^  r3c6URR5$rb)rrrbs r1r(BartForConditionalGeneration.get_encoder'zz%%''r3c6URR5$rb)rrrbs r1r(BartForConditionalGeneration.get_decoder*rr3Nnew_num_tokenspad_to_multiple_of mean_resizingrcx>[TU]XU5nURURRS5 U$)Nr)r=resize_token_embeddings_resize_final_logits_biasrJr,)r?r r r new_embeddingsr@s r1r4BartForConditionalGeneration.resize_token_embeddings-s<8]jk &&~'<'<'B'B1'EFr3c,URRSnX::aURSS2SU24nON[R"SX- 4URRS9n[R "URU/SS9nUR SU5 g)Nr*r!r*rr)rr,rGrrFcatr)r?r old_num_tokensnew_bias extra_biass r1r6BartForConditionalGeneration._resize_final_logits_bias4s//55b9  +--a..@AHa)H%IRVRhRhRoRopJyy$"8"8*!E1MH 0(;r3cUR$rbrrbs r1get_output_embeddings2BartForConditionalGeneration.get_output_embeddings=rr3cXlgrbrr?rs r1set_output_embeddings2BartForConditionalGeneration.set_output_embeddings@% r3cURR(aKURR5 UR UR URR 5 ggrb)rrrrrrrrrbs r1r)BartForConditionalGeneration._tie_weightsCsB ;; * * JJ # # %  & &t||TZZ5F5F G +r3r&rrrrrrrrr3rlabelsrrrrrcRUbUOURRnU bXU (a[RS5 Sn Uc7U c4[ XRR URR 5nURUUUUUUUUU U U U UUUUS9nURUS5nUURRUR5-nSnU baU RUR5n [5nU"URSURR5U RS55nU(dU4USS-nUbU4U-$U$[UUUR UR"UR$UR&UR(UR*UR,S9 $) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Bart uses the `eos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. If you want to change padding behavior, you should read [`modeling_bart._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example summarization: ```python >>> from transformers import AutoTokenizer, BartForConditionalGeneration >>> model = BartForConditionalGeneration.from_pretrained("facebook/bart-large-cnn") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn") >>> ARTICLE_TO_SUMMARIZE = ( ... "PG&E stated it scheduled the blackouts in response to forecasts for high winds " ... "amid dry conditions. The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were " ... "scheduled to be affected by the shutoffs which were expected to last through at least midday tomorrow." ... ) >>> inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors="pt") >>> # Generate Summary >>> summary_ids = model.generate(inputs["input_ids"], num_beams=2, min_length=0, max_length=20) >>> tokenizer.batch_decode(summary_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] 'PG&E scheduled the blackouts in response to forecasts for high winds amid dry conditions' ``` Mask filling example: ```python >>> from transformers import AutoTokenizer, BartForConditionalGeneration >>> tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base") >>> model = BartForConditionalGeneration.from_pretrained("facebook/bart-base") >>> TXT = "My friends are but they eat too many carbs." >>> input_ids = tokenizer([TXT], return_tensors="pt")["input_ids"] >>> logits = model(input_ids).logits >>> masked_index = (input_ids[0] == tokenizer.mask_token_id).nonzero().item() >>> probs = logits[0, masked_index].softmax(dim=0) >>> values, predictions = probs.topk(5) >>> tokenizer.decode(predictions).split() ['not', 'good', 'healthy', 'great', 'very'] ``` NzJThe `use_cache` argument is changed to `False` since `labels` is provided.F)rrrrrrrrr3rrrrrrrr*r! losslogitsrrrrrrr)rrrrxwarningr2r'r(rrrrrFr rrxrrrrrrrr)r?r&rrrrrrrrr3rr$rrrrrr lm_logitsmasked_lm_lossloss_fctoutputs r1rM$BartForConditionalGeneration.forwardHs~&1%# UH/oRSTRUR55v M1 g7frQ index_selectrrFr past_statebeam_idxs r1r>BartForConditionalGeneration._reorder_cache..s1rcqU_--aZ=N=N1OPPcq7:r;rrr8reordered_past layer_pasts ` r1_reorder_cache+BartForConditionalGeneration._reorder_cachesO)J rcmnpopcqrrQR.! N* r3rr)NTNNNNNNNNNNNNNNNNN)$rRrSrTrUrOr_keys_to_ignore_on_load_missingr"r>rrrWrrrrrrrrrrrGrrXrrrrrrMr0rYr?rYrZr[s@r1rrsm  i':&;#z((dh!7?}\` <<<&H 15158<=A,0487;=A=A59=A-1$(,0/3&*59%S E,,-S !.S $E$4$45 S !))9)9 : S ELL) S $ELL1S 'u||4S "$u'8'8"9:S "$u'8'8"9:S   1 12S  ((9(9:S ))*S D>S $D>S 'tn!S "d^#S $!!1!12%S & uo% &'S S jhELLhr3rz Bart model with a sequence classification/head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. c&^\rSrSrSS/rS\4U4Sjjr\SS\\ RS\\ RS\\ RS \\ RS \\ RS \\ RS \\ RS \\ \ RS\\ RS\\ RS\\ RS\\S\\S\\S\\S\\ RS\\\44"Sjj5rSrU=r$)BartForSequenceClassificationirrrrc >[TU]"U40UD6 [U5Ul[ UR UR UR UR5UlUR5 grb) r=r>rrrr num_labelsclassifier_dropoutclassification_headr)r?rrrr@s r1r>&BartForSequenceClassification.__init__sZ *6*v& #9 NN NN     % % $   r3r&rrrrrrrr3rr$rrrrrrcUbUOURRnU bSn Uc%U b"[SURR35eUR UUUUUUUUU U U U UUUS9nUSnUR URR5RUR5n[[R"URS555S:a [S5eUUSS24RUR!S5SUR!S55SS2SSS24nUR#U5nSnU GbU RUR5n URR$cURR&S:XaS URlOyURR&S:aNU R([R*:XdU R([R,:XaS URlOS URlURR$S :XaT[/5nURR&S:Xa&U"UR15U R155nOU"UU 5nOURR$S :XaG[35nU"URSURR&5U RS55nO-URR$S :Xa[55nU"UU 5nU(dU4USS-nUbU4U-$U$[7UUUR8UR:UR<UR>UR@URBURDS 9 $) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Bart uses the `eos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. If you want to change padding behavior, you should read [`modeling_bart._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NFz8Passing input embeddings is currently not supported for rrrrrrrr3rrrrrrrr!z7All examples must have the same number of tokens.r* regressionsingle_label_classificationmulti_label_classificationr&)#rrrNotImplementedErrorr@rRreq eos_token_idrrFrrGunique_consecutivesumr.rrrI problem_typerGrErIrWr squeezer rrrrrrrrr)r?r&rrrrrrrr3rr$rrrrrrreos_masksentence_representationr(r'r,r-s r1rM%BartForSequenceClassification.forwards"h&1%>KbKbJcd ** )/#9/!5+'"7/!5#) "  << 8 89<<]=Q=QR u'' Q8 9A =VW W"/! "<"A"A-BTBTUVBWY[]j]o]opr]s"t r1H# ))*AB  YYv}}-F{{''/;;))Q./;DKK,[[++a/V\\UZZ5OSYS_S_chclclSl/LDKK,/KDKK,{{''<7"9;;))Q.#FNN$4fnn6FGD#FF3D))-JJ+- B 0F0F GUWY))-II,./Y,F)-)9TGf$ Ev E.#33")"?"?&99$55&-&G&G")"?"?&99  r3)rIrr)rRrSrTrUrr"r>rrrGrrXrrrrrrrMrYrZr[s@r1rErEs89VW z 15158<=A,0487;=A59=A-1$(,0/3&*59#} E,,-} !.} $E$4$45 } !))9)9 : } ELL) } $ELL1} 'u||4} "$u'8'8"9:}   1 12}  ((9(9:} ))*} D>} $D>} 'tn} d^!} "!!1!12#} $ u55 6%} } r3rEc(,^\rSrSrSS/rU4Sjr\SS\\RS\\RS\\RS\\RS \\RS \\RS \\RS \\ \RS \\RS\\RS\\RS\\RS\\ S\\ S\\ S\\ S\\RS\\\44$Sjj5rSrU=r$)BartForQuestionAnsweringirrc>[TU]U5 SUlURUl[U5Ul[ R "URUR5UlUR5 gr:) r=r>rGrrrrz hidden_size qa_outputsrrs r1r>!BartForQuestionAnswering.__init__s[   ++v& ))F$6$68I8IJ r3r&rrrrrrrstart_positions end_positionsr3rrrrrrrcUbUOURRnU bU bSn URUUUUUUUUU U U UUUUS9nUSnURU5nUR SSS9unnUR S5R 5nUR S5R 5nSnU bU b[U R55S:aU R S5n [U R55S:aU R S5n URS5nU RSU5n U RSU5n [US9nU"UU 5nU"UU 5nUU-S - nU(dUU4USS-nUbU4U-$U$[UUUURURURURUR UR"UR$S 9 $) rNFrLrr!r*r) ignore_indexr;) r' start_logits end_logitsrrrrrrr)rrrrr^splitrVrrrrr rrrrrrrr)r?r&rrrrrrrr`rar3rrrrrrrsequence_outputr(rdre total_loss ignored_indexr, start_lossend_lossr-s r1rM BartForQuestionAnswering.forwardsd&1%rrrGrXrrrrrrrrMrYrZr[s@r1r[r[s79VW -1158<=A,0487;=A6:4859=A$(,0/3&*59%q ELL)q !.q $E$4$45 q !))9)9 : q ELL) q $ELL1q 'u||4q "$u'8'8"9:q "%"2"23q   0 01q   1 12q  ((9(9:q D>q $D>q 'tn!q "d^#q $!!1!12%q & u99 :'q q r3r[c2^\rSrSrSrU4SjrSrSrU=r$)BartDecoderWrapperi z This wrapper class is a helper class to correctly load pretrained checkpoints when the causal language model is used in combination with the [`EncoderDecoderModel`] framework. cD>[TU]U5 [U5Ulgrb)r=r>rrrs r1r>BartDecoderWrapper.__init__s  "6* r3c&UR"U0UD6$rbr)r?rrs r1rMBartDecoderWrapper.forwards||T,V,,r3r) rRrSrTrUrVr>rMrYrZr[s@r1rnrn s +--r3rnzu BART decoder with a language modeling head on top (linear layer with weights tied to the input embeddings). c"^\rSrSrS/rU4SjrSrSrSrSr Sr S r \ SS \ \RS \ \R S \ \R"S \ \R"S\ \R S\ \R S\ \\R"S\ \R"S\ \RS\ \S\ \S\ \S\ \S\ \RS\\\44Sjj5r\S5rSrU=r$)BartForCausalLMirc >[R"U5nSUlSUl[TU]U5 [ U5Ul[R"URURSS9Ul UR5 g)NTFru)copydeepcopyrois_encoder_decoderr=r>rnrrrzr]rxrrrs r1r>BartForCausalLM.__init__sfv& $)!  '/ yy!3!3V5F5FUS  r3cBURRR$rbrrrnrbs r1r$BartForCausalLM.get_input_embeddings+szz!!...r3c8XRRlgrbr{rs r1r$BartForCausalLM.set_input_embeddings.s*/ 'r3cUR$rbrrbs r1r%BartForCausalLM.get_output_embeddings1rr3cXlgrbrrs r1r%BartForCausalLM.set_output_embeddings4r!r3c$XRlgrbrr)r?rs r1 set_decoderBartForCausalLM.set_decoder7s $ r3c.URR$rbrrbs r1rBartForCausalLM.get_decoder:szz!!!r3r&rrrrrrr3r$rrrrrrcU bU OURRn U bU OURRn U bU OURRn URR UUUUUUUUU U U U US9 nUR US5nSnU baU RUR5n [5nU"URSURR5U RS55nU (dU4USS-nUbU4U-$U$[UUURURURUR S9$)a cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, BartForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base") >>> model = BartForCausalLM.from_pretrained("facebook/bart-base", add_cross_attention=False) >>> assert model.config.is_decoder, f"{model.__class__} has to be configured as a decoder." >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> expected_shape = [1, inputs.input_ids.shape[-1], model.config.vocab_size] >>> list(logits.shape) == expected_shape True ```Nrrr*r!)r'r(rrrr)rrrrrrrrrrFr rrxrrrrr)r?r&rrrrrrr3r$rrrrrrr(r'r,r-s r1rMBartForCausalLM.forward=s[\2C1N-TXT_T_TqTq$8$D $++JjJj &1%# UH/oRSTRUR55v M1 g7frQr4r6s r1r1BartForCausalLM._reorder_cache..s1ncmU_--aZ=N=N1OPPcmr:r;r<s ` r1r?BartForCausalLM._reorder_caches8)J ncmnn N*r3rA)NNNNNNNNNNNNNN)rRrSrTrUrr>rrrrrrrrrGrrXrrrrrrrMrYr?rYrZr[s@r1rtrts ++ /0&%"1515=A>B,07;=A59-1$(,0/3&*59W E,,-W !.W  ((9(9: W !)):): ; W ELL) W 'u||4W "$u'8'8"9:W   1 12W ))*W D>W $D>W 'tnW d^W !!1!12W u77 8!W W rr3rt)rtrr[rErrrir[)QrVrvrvr_typingrrrrrGtorch.utils.checkpointrtorch.nnrr r activationsr cache_utilsr r generationrmodeling_attn_mask_utilsrrrmodeling_flash_attention_utilsrrmodeling_outputsrrrrrrrmodeling_utilsrutilsrrrr configuration_bartr"!torch.nn.attention.flex_attentionr#integrations.flex_attentionr$r% get_loggerrRrxrXrWr2rr5r]Modulerjrrrrrrrr[rirmrrrrEr[rnrt__all__rr3r1rs // AA!5) i. + !!;JJ   H % %,, c [^  ;R\\;2 =bll =ZBBIIZBzt1-t1nf1 f1T ,DryyDNsryyslRYY0_/__D -  - y %y xl %l ^ _ #_ _ D Q#6Q  Qh N $7N  N bA 2A A H -, - B)?B  BJ r3