fTh SrSSKrSSKrSSKrSSKrSSKJrJrJrJ r SSK r SSK r SSK J r SSK Jr SSKJr SSKJrJr SS KJr SS KJrJr SS KJrJrJrJrJr SS KJ r SS K!J"r"J#r#J$r$J%r% SSK&J'r'J(r( SSK)J*r* \#"5(a SSK+J,r, SSK-J.r. \%R^"\05r1S\ RdS\3S\34Sjr4"SS\ Rj5r6"SS\ Rj5r7"SS\ Rp5r9S\90r:"SS \ Rp5r;"S!S"\ Rp5r<\""S#S$\ 55r="S%S&\=5r>"S'S(\=5r?\""S)S*\=55r@\""S+S,9"S-S.\=\55rA"S/S0\=5rB"S1S2\=\5rC/S3QrDg)4zPyTorch Blenderbot model.N)ListOptionalTupleUnion)nn)CrossEntropyLoss)ACT2FN)CacheEncoderDecoderCache)GenerationMixin)AttentionMaskConverter_prepare_4d_attention_mask)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentions!CausalLMOutputWithCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutput)PreTrainedModel)auto_docstringis_torch_flex_attn_availableis_torchdynamo_compilinglogging)'BlenderbotSmallForConditionalGenerationBlenderbotSmallModel)BlenderbotConfig) BlockMask)make_flex_block_causal_mask input_ids pad_token_iddecoder_start_token_idcURUR5nUSS2SS24R5USS2SS24'X#SS2S4'Uc [S5eUR US:HU5 U$)z) Shift input ids one token to the right. Nrrz1self.model.config.pad_token_id has to be defined.i) new_zerosshapeclone ValueError masked_fill_)r!r"r#shifted_input_idss j/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/blenderbot/modeling_blenderbot.pyshift_tokens_rightr-?sz"++IOO<(CRC0668ae4adLMM""#4#[TU]X5 gN)super__init__)selfr2r3 __class__s r,r7-BlenderbotLearnedPositionalEmbedding.__init__Ts 7r.input_ids_shapepast_key_values_length position_idsc>UcBUSSupE[R"X"U-[RURRS9n[ TU]U5$)z3`input_ids_shape` is expected to be [bsz x seqlen].Nr)dtypedevice)torcharangelongweightr@r6forward)r8r;r<r=bszseq_lenr9s r,rE,BlenderbotLearnedPositionalEmbedding.forwardWsX  *2A.LC <<&(HPUPZPZcgcncncucuLw|,,r.rN)__name__ __module__ __qualname____firstlineno____doc__intr7rASizerTensorrE__static_attributes__ __classcell__r9s@r,r0r0OsT8s838tx -$zz -CF -ZbchcocoZp - -r.r0c r^\rSrSrSrS S\S\S\S\\4U4SjjjrS\ R4U4S jjr S r U=r $) BlenderbotScaledWordEmbeddingdzT This module overrides nn.Embeddings' forward by multiplying with embeddings scale. r2r3 padding_idx embed_scalec2>[TU]XU5 X@lgr5)r6r7rZ)r8r2r3rYrZr9s r,r7&BlenderbotScaledWordEmbedding.__init__is  D&r.r!c<>[TU]U5UR-$r5)r6rErZ)r8r!r9s r,rE%BlenderbotScaledWordEmbedding.forwardmswy)D,<,<<[T U]5 XlX lX0lX-UlXplUR U-UR:wa[SURSUS35eUR S-UlX@l X`l Xl Uc>UR(a-[RSURRS35 [ R""XUS9Ul[ R""XUS9Ul[ R""XUS9Ul[ R""XUS9Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).gzInstantiating a decoder z without passing `layer_idx` is not recommended and will lead to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` when creating this class.ri)r6r7rerfrghead_dimrkr)scalingrhrjrllogger warning_oncer9rKrLineark_projv_projq_projout_proj) r8rerfrgrhrirjrkrlr9s r,r7BlenderbotAttention.__init__us  "" !.  MMI %$.. 8MdnnM]$YKr3 }}d* $""     *4>>+B+B*CD,,  ii 4@ ii 4@ ii 4@  )TB r. hidden_stateskey_value_statespast_key_valueattention_masklayer_head_maskoutput_attentionscache_positionreturnc USLnUR5upn URU5RU SURUR5R SS5n XR -n Ub][U[5(aFURRUR5n U(a URnOURnOUnU(aUOUnU(a=Ub:W (a3WRURnURURnOUR!U5nUR#U5nURU SURUR5R SS5nURU SURUR5R SS5nUbNU(dUOSnWR%UUURSU05unnU(aSURUR'XR-SUR4nU R&"U6n UR&"U6nUR&"U6nURS5n[(R*"U UR SS55nUR5XR-U U4:wa.[-SXR-U U4SUR535eUb]USS2SS2SS2SUR.S 24nURXRU U5U-nURXR-U U5n[0R2R5USS 9nUbUR5UR4:wa*[-S UR4SUR535eURSSSS5URXRU U5-nURXR-U U5nU(a=URXRU U5nURXR-U U5nOSn[0R2R7UUR6UR8S 9n[(R*"UU5nUR5XR-XR4:wa7[-S XR-XR4SUR535eURXRXR5nUR SS5nUR'XUR:5nUR=U5nUUU4$)z#Input shape: Batch x Time x ChannelNr%rrrTz$Attention weights should be of size z , but is dimz/Head mask for a single layer should be of size ptrainingz `attn_output` should be of size )sizervviewrfro transposerp isinstancer is_updatedgetrlcross_attention_cacheself_attention_cache key_cache value_cachertruupdatereshaperAbmmr)r'r functionalsoftmaxrgrrerw)r8ryrzr{r|r}r~ris_cross_attentionrFtgt_len_ query_statesrcurr_past_key_valuecurrent_states key_states value_states proj_shapesrc_len attn_weightsattn_weights_reshaped attn_probs attn_outputs r,rEBlenderbotAttention.forwards.T9',,.a{{=166sBPTP]P]^hhijlmn #ll2  %.*=>>+66::4>>J %*8*N*N'*8*M*M'&4#-?)] ."<,66t~~FJ.::4>>JL^4J;;~6L#b$..$--PZZ[\^_`J',,S"dnndmmT^^_`bcdL)7It+>+E+E dnn?OQ_>`,( L&@DN--dnn=NN*B > #++Z8 ''4 #++Z8 //!$yyz/C/CAq/IJ    3#7'"J J6nn8LgW^7_6`a %%'(*   %+Aq!5Kz7G7G7K5K,KLN',,S..'7SVddL',,S>>-A7GTL}},,\r,B  &##%$..):: Et~~FWEXY',,./1+//2q!<|?P?PQTVdVdfmov?wwL',,S>>-A7GTL  %1$5$5c>>7T[$\ !055cNN6JGU\]L$( !]]**<4<AAr.) rkrgrerorjrhrtrlrfrwrvrpru)FTFNN)NNNNFN)rKrLrMrNrOrPraboolrrr7rArRr rrErSrTrUs@r,rcrcrsZG  -1#'%C%C%C %C  %C  %C%C)*%CC=%C%CT48*.1526"'15pB||pB#5<<0pB! pB !. pB "%,,/ pB pB!.pB u||Xell3XeELL>Q5RR SpBpBr.rceagerc ^\rSrSrS\4U4SjjrS S\RS\RS\RS\S\R4 S jjr S r U=r $) BlenderbotEncoderLayerirkc>[TU]5 URUl[UR "URUR URUS9Ul[R"UR5Ul URUl [URUlUR Ul[R""URUR$5Ul[R""UR$UR5Ul[R"UR5Ulg)N)rerfrgrk)r6r7d_modelreBLENDERBOT_ATTENTION_CLASSES_attn_implementationencoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normrgr activation_function activation_fnactivation_dropoutrsencoder_ffn_dimfc1fc2final_layer_normr8rkr9s r,r7BlenderbotEncoderLayer.__init__s 5f6Q6QRnn44,,   %'LL$@!~~ #F$>$>?"(";";99T^^V-C-CD99V33T^^D " T^^ <r.ryr|r}r~rcUnURU5nURUUUUS9upn[RR XRUR S9nXQ-nUnUR U5nURURU55n[RR XRUR S9nURU5n[RR XRUR S9nXQ-nUR[R:XaC[R"UR5RS- n[R "X*US9nU4n U(aX4- n U $)aW Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )ryr|r}r~ri)minmax)rrrrrgrrrrrrr?rAfloat16finforclamp) r8ryr|r}r~residualrr clamp_valueoutputss r,rEBlenderbotEncoderLayer.forward&sV$!11-@ )-')+/ *8* & Q  --m||VZVcVc-d  0  --m< **488M+BC  --m?V?Vaeanan-o /  --m||VZVcVc-d  0   %-- /++m&9&9:>>EK!KK <[YM "   &Gr.) rrrgrerrrrrF) rKrLrMrNrr7rArRrrErSrTrUs@r,rrs_=/=.#( .||. . .  .  ..r.rc`^\rSrSrSS\S\\4U4SjjjrSS\RS\\RS\\RS\\RS \\RS \\RS \\ S \\ S \\ S\\RS\R4Sjjr Sr U=r$)BlenderbotDecoderLayeriXrkrlc T>[TU]5 URUl[UR "URUR URSSUUS9UlURUl [URUl URUl [R"UR5Ul[UR "URUR URSUUS9Ul[R"UR5Ul[R&"URUR(5Ul[R&"UR(UR5Ul[R"UR5Ulg)NT)rerfrgrhrjrkrl)rgrhrkrl)r6r7rrerrdecoder_attention_headsrrrgr rrrrrr encoder_attnencoder_attn_layer_normrsdecoder_ffn_dimrrr)r8rkrlr9s r,r7BlenderbotDecoderLayer.__init__Ys6 5f6Q6QRnn44,, ~~ #F$>$>?"(";";$&LL$@!89T9TU NN  * *,,  (*||DNN'C$99T^^V-C-CD99V33T^^D " T^^ <r.ryr|encoder_hidden_statesencoder_attention_maskr}cross_attn_layer_head_maskr{r~ use_cacherrc  Un URU5nURUUUUUU S9upn[RR XRUR S9nX-nSn Ub`Un UR U5nURUUUUUUS9upn[RR XRUR S9nX-nUn URU5nURURU55n[RR XRUR S9nURU5n[RR XRUR S9nX-nU4nU(aXU 4- nU (aX4- nU$)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. encoder_hidden_states (`torch.FloatTensor`): cross attention input to the layer of shape `(batch, seq_len, embed_dim)` encoder_attention_mask (`torch.FloatTensor`): encoder attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. cross_attn_layer_head_mask (`torch.FloatTensor`): mask for cross-attention heads in a given layer of size `(decoder_attention_heads,)`. past_key_value (`Tuple(torch.FloatTensor)`): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. )ryr{r|r}r~rrN)ryrzr|r}r{r~) rrrrrgrrrrrrrr)r8ryr|rrr}rr{r~rrrself_attn_weightscross_attn_weightsrs r,rEBlenderbotDecoderLayer.forwardxsD!11-@ <@>>'))+/) >G  ( (Gr.) rrrgrerrrrrrrr5) NNNNNNFTN)rKrLrMrNrrrPr7rArRr rrErSrTrUs@r,rrXs =/=HSM==D268<9=26=A*.,1$(15T||T!.T ( 5 T !) 6 T "%,,/ T%-U\\$:T!T$D>TD>T!.T TTr.rc \rSrSr\rSrSrSrSr Sr \ S5r SS\ \RS4S\RS \RS \S \4 S jjr\S\RS \S\S\R*S \RS\4 Sj5rSrg)BlenderbotPreTrainedModelimodelTcURRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[R5(aJUR R RS5 URR R5 gg)Nr)meanstdr`)rkinit_stdrrrsrDdatanormal_rizero_ EmbeddingrYrfill_)r8modulers r, _init_weights'BlenderbotPreTrainedModel._init_weightsskk"" fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>.  - - MM   $ $S ) KK   " " $.r.cURRn[R"/SQSSSSU//URS9nUR U5UUS.nU$)N)r rr rr@)r|r!decoder_input_ids)rkr"rAtensorr@ne)r8 pad_tokenr! dummy_inputss r,r&BlenderbotPreTrainedModel.dummy_inputssZKK,, LL"2Q2q)4L!MVZVaVab 'll95"!* r.r|r input_tensorrpast_key_valuesr~c URRS:XaUbUS:HR5(aU$gURRS:Xa,[U[R 5(a [ U5nU$UbUR5OSnUb UROSnURRS:Xa5U(d.U(d'[R"UUUURS9(agURnURSn U(aUR5n O5[U[R 5(aURS OXi-S-n URUU U UUURSS 9n URRS:XaZUbWUR R"S ;a=U(d6[R$"U5R&n [R("X5n U $) Nflash_attention_2rflex_attentionrFsdpa) inputs_embedsr< is_trainingrr%)sequence_length target_lengthr?r batch_size)cudaxpunpu)rkranyrrArRr get_seq_lengthis_compileabler_ignore_causal_mask_sdparr?r'get_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionr@typerr_unmask_unattended) r8r|rrrr~past_seen_tokensusing_compilable_cacher?rr causal_mask min_dtypes r,_update_causal_mask-BlenderbotPreTrainedModel._update_causal_masks ;; + +/B B)~/D.I.I.K.K%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`aCRC^!?!?di ;; + +v 5>T]n%>>*'7 MM  ""&,,Q/ !+??AMnell;;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCK[Kr.rrr?rc UbUR5S:XaUnU$[R"U5Rn[R"X4XUR S9nUS:wa[R "USS9nU[R"X$R S9URSS5:-nUSSSS2SS24RUSSS5nUbUR5nURSn USS2SS2SS2SU 24USS2SSSS24RUR 5-n U S:Hn USS2SS2SS2SU 24RX5USS2SS2SS2SU 24'U$) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. Nr) fill_valuer?r@r)diagonalrr%r)rrArrfullr@triurBrexpandr(r'to masked_fill) r|rrr?rrkwargsr r  mask_length padding_masks r,rOBlenderbotPreTrainedModel._prepare_4d_causal_attention_mask_with_cache_position5s}>  %.*<*<*>!*C(K*' E*..I** 0Y\j\q\qK!##jjqA 5<< >S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r.rINr)rKrLrMrNr config_classbase_model_prefixsupports_gradient_checkpointing_supports_cache_class_supports_static_cacherpropertyrrrArRr rr staticmethodrPr?rrSrIr.r,rrs#L&*# ! %"#( BellK78BllB B  B  BH4 444{{ 4  4  44r.rcn^\rSrSrSrSS\S\\R4U4Sjjjr S Sjr Sr U=r $) BlenderbotEncoderinz Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`BlenderbotEncoderLayer`]. Args: config: BlenderbotConfig embed_tokens (nn.Embedding): output embedding rk embed_tokensc>[TU]U5 URUlURUlUR nUR UlURUl UR(a[R"U5OSnUbX l O$[URX0RUS9Ul [!URU5Ul[$R&"[)UR*5Vs/sHn[-U5PM sn5Ul[$R0"UR 5UlSUlUR75 gs snf)Nr`r_F)r6r7rgencoder_layerdrop layerdroprr"rYmax_position_embeddingsmax_source_positionsscale_embeddingmathsqrtr%rW vocab_sizer0embed_positionsr ModuleListrangeencoder_layersrlayersr layer_normgradient_checkpointing post_init)r8rkr%rerZrr9s r,r7BlenderbotEncoder.__init__xs  ~~ 11NN !..$*$B$B!.4.D.Ddii *#  # ,  =!!9.>.>K!D  D  * *  mmUSYShShMi$jMi%;F%CMi$jk ,,v~~6&+#  %ks:EcUbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb7UR X5 UR 5nURSUS5nO"UbUR 5SSnO [ S5eUcURU5nURU5n XI-n [RRXRURS9n Ub[X$R5nU(aSOSn U(aSOSn Ub`UR 5S[!UR"5:wa6[ S[!UR"5S UR 5SS 35e[%UR"5HupU(aX4-n S nUR(a([&R("/5nUUR*:aS nU(aS nO^UR,(a8UR(a'UR/UR0U UUbX=OSU5nOU"U UUbX=OSUS9nUSn U(dMU US4-n M UR3U 5n U(aX4-n U(d[5SXU 455$[7XU S9$)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(encoder_layers, encoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzDYou cannot specify both input_ids and inputs_embeds at the same timer%z5You have to specify either input_ids or inputs_embedsrrIrz&The head_mask should be specified for  layers, but it is for .FT)NN)r}r~rc3.# UH ocMUv M g7fr5rI.0vs r, ,BlenderbotEncoder.forward..se$Sq$Ss last_hidden_statery attentions)rkr~output_hidden_statesuse_return_dictr)%warn_if_padding_and_no_attention_maskrrr%r/rrrgrrr?lenr3 enumeraterArandr(r5_gradient_checkpointing_func__call__r4tupler)r8r!r| head_maskrr~rD return_dict input_shape embed_posryencoder_statesall_attentionsidx encoder_layerto_dropdropout_probability layer_outputss r,rEBlenderbotEncoder.forwards\2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K!r;r?;I  &',,.s3KTU U   --i8M((5 %1  --m||VZVcVc-d   %7H[H[\N30d  ~~"c$++&66 >Ne]N$See e+Vd  r.) rgr/r%r5r4r(r3r*rYr5)NNNNNNN) rKrLrMrNrOrrrrr7rErSrTrUs@r,r$r$nsM/x ?U>!C C r.r$c^\rSrSrSrS S\S\\R4U4Sjjjr Sr Sr S S\\ R4S jjrS rU=r$) BlenderbotDecoderiz Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`BlenderbotDecoderLayer`] Args: config: BlenderbotConfig embed_tokens (nn.Embedding): output embedding rkr%c >[TU]U5 URUlURUlUR UlURUlUR(a [R"UR5OSnUbX l O/[URURUR US9Ul [!URUR5Ul[$R&"[)UR*5Vs/sH n[-XS9PM sn5Ul[$R0"UR5UlSUlUR75 gs snf)Nr`r_)rlF)r6r7rgdecoder_layerdropr(r"rYr)max_target_positionsr+r,r-rr%rWr.r0r/rr0r1decoder_layersrr3rr4r5r6)r8rkr%rZir9s r,r7BlenderbotDecoder.__init__$s  ~~ 11!..$*$B$B!393I3Idii/s  # ,  =!!6>>43C3CQ\!D  D  * * NN mmBGH]H]B^ _B^Q #F 8B^ _  ,,v~~6&+#  `s E)cUR$r5r%r8s r,get_input_embeddings&BlenderbotDecoder.get_input_embeddings@s   r.cXlgr5rbr8values r,set_input_embeddings&BlenderbotDecoder.set_input_embeddingsCs!r.rc* U bU OURRn U bU OURRn U bU OURRn U bU OURRn UR (a/UR (aU (a[RS5 Sn USLUSL- (a [S5eUbURSURS5nUcURU5nSnU (aB[U[5(d-Sn[RS5 [ R""U5nUR%5SSunnUbUR'5OSnU c#[(R*"UUU-UR,S 9n Uc4[/5(d%UU-n[(R0"UUUR,S 9n[U[ 5(a UR2OUnUR5UUU UU 5nUbUb[7XHR8US 9nUR;UU4UU S 9nUU-n[<R>RAUUR@UR S 9nU (aS OSnU (aS OSnU (aUbS OSnSn[CXV/SS/5HnunnUcM UR%5S[EURF5:wdM7[SUS[EURF5SUR%5SS35e [IURF5HunnU (aUU4- nUR (a([(RJ"/5nUURL:aMLUR (aGUR (a6UROURPUUUUUbUUOSUbUUOSSU U U 5 n OU"UUUUUbUUOSUbUUOSUU U U S9 n U SnU (aU U (aSOSnU (dMUU S4- nUcMUU S4- nGM URSU5nU (aUU4- nU (aUOSn!U(aURU5n!U (d[WSUU!UUU455$[YUU!UUUS9$)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.LongTensor` of shape `(batch_size, encoder_sequence_length)`, *optional*): Mask to avoid performing cross-attention on padding tokens indices of encoder input_ids. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(encoder_layers, encoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules in the encoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder to avoid performing cross-attention on hidden heads. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. NzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...FzTYou cannot specify both decoder_input_ids and decoder_inputs_embeds at the same timer%TzPassing a tuple of `past_key_values` is deprecated and will be removed in Transformers v4.58.0. You should pass an instance of `EncoderDecoderCache` instead, e.g. `past_key_values=EncoderDecoderCache.from_legacy_cache(past_key_values)`.rr)r)r=rrIrMcross_attn_head_maskzThe `z` should be specified for r9r:) r|rrr}rr{r~rrr rrc30# UH nUcMUv M g7fr5rIr<s r,r?,BlenderbotDecoder.forward..1smAms )rBrryrCcross_attentions)-rkr~rDrrEr5rrqwarningr)rr'r%rr rrr from_legacy_cacherrrArBr@ronesrrrr?r/rrrgziprGr3rHrIr(rJrKr4to_legacy_cacherLr)"r8r!r|rrrMrlrrrr~rDrNrreturn_legacy_cacher seq_lengthr<mask_seq_lengthself_attn_cacher r=ryall_hidden_statesall_self_attnsall_cross_attentionsnext_decoder_cache attn_mask mask_namerS decoder_layerrVrW next_caches" r,rEBlenderbotDecoder.forwardFsj2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?I   --i8M$ Z??"&    \  2CCOTO!.!3!3!5cr!: JETE`!?!?!Afg  !"\\&(>(KTaThThN  !*B*D*D4zAO"ZZ OML`L`aN/+>??  0 0   ..        ! ,1G1S%?&(;(;Z& " ++  $&<>, & 4  --mt||VZVcVc-d #7BD0d&7>#A&#dkk*::$ {*DSEUDVW%NN,Q/03%r#,DKK"8 C#!m%55!}}&+jjn#&7**t}} $ A A!**!)*&/&;IcN1E1Q(-W[%" ! !.!#.*?+A7@7LYs^RV5I5U,S1[_#2&7'#1 ! *!,M%28I1q%Q"  =#3"55(4(]1-=,??(_#9d 6  -!1 1 +4'$ (88:J'5FXlm  9+&+%1   r.) rgr/r%r5r4r(r3r]rYr5) NNNNNNNNNNNNN)rKrLrMrNrOrrrrr7rdrirArRrErSrTrUs@r,rZrZs}/x ?U8!" "#!!15v !.v v r.rZc&~^\rSrSrSS/rS\4U4Sjjr\S\\ \ \ R44U4Sjj5r SrS rS rS r\SS \\R(S \\R*S\\R(S\\R(S\\R*S\\R*S\\R*S\\ \\4S\\\R2S\\R*S\\R2S\\S\\S\\S\\S\\R*S\ \\R2\44"Sjj5rSrU=r$) BlenderbotModeli?decoder.embed_tokens.weightencoder.embed_tokens.weightrkcr>[TU]U5 URURp2UR(a [ R "UR5OSn[X1RX$S9Ul [XR5Ul [XR5Ul UR5 g)Nr`r_)r6r7r"r.r+r,r-rrWsharedr$encoderrZdecoderr6)r8rkrYr.rZr9s r,r7BlenderbotModel.__init__Cs~  "("5"5v7H7HZ393I3Idii/s 3JP[u (= (=  r.pretrained_model_name_or_pathc>US:Xa1[R"S[5 [R"U5$[ [ U]"U/UQ70UD6$)Nfacebook/blenderbot-90MzThe checkpoint `facebook/blenderbot-90M` is deprecated. In the future, please use the identical checkpoint `facebook/small_blenderbot-90M` with `BlenderbotSmallModel.from_pretrained('facebook/small_blenderbot-90M')` instead.)warningswarn FutureWarningrfrom_pretrainedr6rclsr model_argsrr9s r,rBlenderbotModel.from_pretrainedOsV (,E E MMd   (778UV V_c:;Xp[epioppr.cUR$r5)rrcs r,rd$BlenderbotModel.get_input_embeddings\s {{r.c|XlURURlURURlgr5)rrr%rrgs r,ri$BlenderbotModel.set_input_embeddings_s' $(KK !$(KK !r.cUR$r5)rrcs r, get_encoderBlenderbotModel.get_encoderd ||r.cUR$r5rrcs r, get_decoderBlenderbotModel.get_decodergrr.r!r|rdecoder_attention_maskrMdecoder_head_maskrlencoder_outputsrrdecoder_inputs_embedsrr~rDrNrrcU bU OURRn UbUOURRnU bU OURRn UbUOURRnUcUR UUUU U UUS9nORU(aK[ U[5(d6[US[U5S:aUSOS[U5S:aUSOSS9nURUUUSUUUU U U U UUUS9 nU(dUU-$[URURURURURURURURS9$) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Blenderbot uses the `bos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. Example: ```python >>> from transformers import AutoTokenizer, BlenderbotModel >>> model = BlenderbotModel.from_pretrained("facebook/blenderbot-400M-distill") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/blenderbot-400M-distill") >>> inputs = tokenizer("Studies have been shown that owning a dog is good for you", return_tensors="pt") >>> decoder_input_ids = tokenizer("Studies show that", return_tensors="pt").input_ids # Batch size 1 >>> outputs = model(input_ids=inputs.input_ids, decoder_input_ids=decoder_input_ids) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 6, 1280] ```N)r!r|rMrr~rDrNrrrrA r!r|rrrMrlrrrr~rDrNr)rBrdecoder_hidden_statesdecoder_attentionsroencoder_last_hidden_staterencoder_attentions)rkr~rDrrErrrrGrrrBrryrCro)r8r!r|rrrMrrlrrrrrr~rDrNrdecoder_outputss r,rEBlenderbotModel.forwardjst2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK"?@o "$u'8'8"9:o   -o  ((9(9:o D>o $D>o 'tno d^!o "!.#o $ uU&&');; <%o o r.rz\ The Blenderbot Model with a language modeling head. Can be used for summarization. ) custom_introc( ^\rSrSrSrS/r/SQrS\4U4Sjjr\ S\ \ \ \ R44U4Sjj5rS rS rS(S \S \ \S\S\R*4U4SjjjrS \SS 4SjrSrSr\S)S\ \R8S\ \R:S\ \R8S\ \R8S\ \R:S\ \R:S\ \R:S\ \ \\4S\ \ \RBS\ \R:S\ \RBS\ \R8S \ \S!\ \S"\ \S#\ \S$\ \R:S\ \\RB\"44$S%jj5r#\$S&5r%S'r&U=r'$)*"BlenderbotForConditionalGenerationirfinal_logits_bias)rrlm_head.weightrkcv>[TU]U5 [U5UlUR S[ R "SURRR455 [R"URURRRSS9Ul UR5 g)NrrFrn)r6r7rrregister_bufferrAzerosrr2rrsrlm_headr6rs r,r7+BlenderbotForConditionalGeneration.__init__s  $V,  0%++q$**BSBSBbBb>c2deyy1B1B1Q1QX]^  r.rc>US:Xa1[R"S[5 [R"U5$[ [ U]"U/UQ70UD6$)NrzThe checkpoint `facebook/blenderbot-90M` is deprecated. In the future, please use the identical checkpoint `facebook/small_blenderbot-90M` with `BlenderbotSmallForConditionalGeneration.from_pretrained('facebook/small_blenderbot-90M')` instead.)rrrrrr6rrs r,r2BlenderbotForConditionalGeneration.from_pretraineds_ (,E E MMw   ;JJKhi i7M ) ,6 :@  r.c6URR5$r5)rrrcs r,r.BlenderbotForConditionalGeneration.get_encoderzz%%''r.c6URR5$r5)rrrcs r,r.BlenderbotForConditionalGeneration.get_decoderrr.Nnew_num_tokenspad_to_multiple_of mean_resizingrcx>[TU]XU5nURURRS5 U$)Nr)r6resize_token_embeddings_resize_final_logits_biasrDr')r8rrrnew_embeddingsr9s r,r:BlenderbotForConditionalGeneration.resize_token_embeddingss<8]jk &&~'<'<'B'B1'EFr.c,URRSnX::aURSS2SU24nON[R"SX- 4URRS9n[R "URU/SS9nUR SU5 g)Nr%rrrr)rr'rArr@catr)r8rold_num_tokensnew_bias extra_biass r,r>> from transformers import AutoTokenizer, BlenderbotForConditionalGeneration >>> mname = "facebook/blenderbot-400M-distill" >>> model = BlenderbotForConditionalGeneration.from_pretrained(mname) >>> tokenizer = AutoTokenizer.from_pretrained(mname) >>> UTTERANCE = "My friends are cool but they eat too many carbs." >>> print("Human: ", UTTERANCE) Human: My friends are cool but they eat too many carbs. >>> inputs = tokenizer([UTTERANCE], return_tensors="pt") >>> reply_ids = model.generate(**inputs) >>> print("Bot: ", tokenizer.batch_decode(reply_ids, skip_special_tokens=True)[0]) Bot: That's unfortunate. Are they trying to lose weight or are they just trying to be healthier? >>> REPLY = "I'm not sure" >>> print("Human: ", REPLY) Human: I'm not sure >>> NEXT_UTTERANCE = ( ... "My friends are cool but they eat too many carbs. That's unfortunate. " ... "Are they trying to lose weight or are they just trying to be healthier? " ... " I'm not sure." ... ) >>> inputs = tokenizer([NEXT_UTTERANCE], return_tensors="pt") >>> next_reply_ids = model.generate(**inputs) >>> print("Bot: ", tokenizer.batch_decode(next_reply_ids, skip_special_tokens=True)[0]) Bot: I see. Well, it's good that they're trying to change their eating habits. ``` NzJThe `use_cache` argument is changed to `False` since `labels` is provided.F)r|rrrrMrrlrrrrr~rDrNrrr%r) losslogitsrrrrorrr)rkrErqrpr-r"r#rrrrrr.rrrrrorrr)r8r!r|rrrMrrlrrrrrrr~rDrNrr lm_logitsmasked_lm_lossloss_fctoutputs r,rE*BlenderbotForConditionalGeneration.forwards^&1%# UH/oRSTRUR55v M1 g7frJ index_selectrr@r= past_statebeam_idxs r,r?DBlenderbotForConditionalGeneration._reorder_cache..s1rcqU_--aZ=N=N1OPPcq7:rrLrrreordered_past layer_pasts ` r,_reorder_cache1BlenderbotForConditionalGeneration._reorder_cachesO)J rcmnpopcqrrQR.! N* r.rr)NT)NNNNNNNNNNNNNNNNN)(rKrLrMrNr_keys_to_ignore_on_load_missingrrr7rrrrrrrrrrPrrrrrrrrrArrRrrrrrrEr"rrSrTrUs@r,rrs  ':&;#i/  HU3PRP[P[K[E\<]    ((dh!7?}\` <<<&15158<=A,0487;CG=A04=A-1$(,0/3&*15%@ E,,-@ !.@ $E$4$45 @ !))9)9 : @ ELL) @ $ELL1@ 'u||4@ "%(>"?@@ "$u'8'8"9:@   -@  ((9(9:@ ))*@ D>@ $D>@ 'tn!@ "d^#@ $!.%@ & uU&&'8 9'@ @ Dr.rc2^\rSrSrSrU4SjrSrSrU=r$)BlenderbotDecoderWrapperiz This wrapper class is a helper class to correctly load pretrained checkpoints when the causal language model is used in combination with the [`EncoderDecoderModel`] framework. cD>[TU]U5 [U5Ulgr5)r6r7rZrrs r,r7!BlenderbotDecoderWrapper.__init__s  (0 r.c&UR"U0UD6$r5r)r8argsrs r,rE BlenderbotDecoderWrapper.forwards||T,V,,r.r) rKrLrMrNrOr7rErSrTrUs@r,rrs 1--r.rc"^\rSrSrS/rU4SjrSrSrSrSr Sr S r \ SS \ \RS \ \R S \ \R"S \ \R"S\ \R S\ \R S\ \\R"S\ \R"S\ \RS\ \S\ \S\ \S\ \S\ \RS\\\44Sjj5r\S5rSrU=r$)BlenderbotForCausalLMirc >[R"U5nSUlSUl[TU]U5 [ U5Ul[R"URURSS9Ul UR5 g)NTFrn)copydeepcopyrhis_encoder_decoderr6r7rrrrs hidden_sizer.rr6rs r,r7BlenderbotForCausalLM.__init__sfv& $)!  -f5 yy!3!3V5F5FUS  r.cBURRR$r5rrr%rcs r,rd*BlenderbotForCausalLM.get_input_embeddingsszz!!...r.c8XRRlgr5rrgs r,ri*BlenderbotForCausalLM.set_input_embeddingss*/ 'r.cUR$r5rrcs r,r+BlenderbotForCausalLM.get_output_embeddingsrr.cXlgr5rrs r,r+BlenderbotForCausalLM.set_output_embeddingsrr.c$XRlgr5rr)r8rs r, set_decoder!BlenderbotForCausalLM.set_decoders $ r.c.URR$r5rrcs r,r!BlenderbotForCausalLM.get_decoderszz!!!r.r!r|rrrMrlrrrrr~rDrNrrcU bU OURRn U bU OURRn U bU OURRn URR UUUUUUUUU U U U US9 nUR US5nSnU baU RUR5n [5nU"URSURR5U RS55nU (dU4USS-nUbU4U-$U$[UUURURURUR S9$)a! cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, BlenderbotForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("facebook/blenderbot-400M-distill") >>> model = BlenderbotForCausalLM.from_pretrained("facebook/blenderbot-400M-distill", add_cross_attention=False) >>> assert model.config.is_decoder, f"{model.__class__} has to be configured as a decoder." >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> expected_shape = [1, inputs.input_ids.shape[-1], model.config.vocab_size] >>> list(logits.shape) == expected_shape True ```Nrrr%r)rrrryrCro)rkr~rDrErrrrr@rrr.rrryrCro)r8r!r|rrrMrlrrrrr~rDrNrrrrrrs r,rEBlenderbotForCausalLM.forwards[\2C1N-TXT_T_TqTq$8$D $++JjJj &1%# UH/oRSTRUR55v M1 g7frJrrs r,r?7BlenderbotForCausalLM._reorder_cache..:s1ncmU_--aZ=N=N1OPPcmrrrs ` r,r$BlenderbotForCausalLM._reorder_cache5s8)J ncmnn N*r.r)NNNNNNNNNNNNNN)rKrLrMrNrr7rdrirrrrrrrArrRrrrrrrrEr"rrSrTrUs@r,rrs*+ /0&%"1515=A>B,07;=A59-1$(,0/3&*59W E,,-W !.W  ((9(9: W !)):): ; W ELL) W 'u||4W "$u'8'8"9:W   1 12W ))*W D>W $D>W 'tnW d^W !!1!12W u77 8!W W rr.r)rrrr)ErOrr,rrtypingrrrrrAtorch.utils.checkpointrtorch.nnr activationsr cache_utilsr r generationr modeling_attn_mask_utilsrrmodeling_outputsrrrrrmodeling_utilsrutilsrrrrblenderbot_smallrrconfiguration_blenderbotr!torch.nn.attention.flex_attentionrintegrations.flex_attentionr get_loggerrKrqrRrPr-rr0rWModulercrrrrr$rZrrrr__all__rIr.r,r,s // %!5). ]6 !!;J   H % %,, c [^  -2<<-* =BLL =ZB"))ZBz!()<=ARYYAJtRYYtn[[[|j 1j Za 1a H Z /Z Z z E)BOE  ER -8 -B5BJ r.