fTh.ASSKJrJrJrJrJr SSKrSSKrSSKJr SSK J r J r J r SSK Jr SSKJr SSKJrJr SS KJr SS KJr SS KJrJr S S KJrJrJrJrJ r J!r!J"r"J#r#J$r$J%r%J&r& SSK'J(r( \"5(a SSK)J*r* SSK+J,r, \RZ"\.5r/Sr0"SS\"5r1"SS\5r2"SS\5r3"SS\$5r4"SS\#5r5"SS\5r6"SS \!5r7"S!S"\ 5r8"S#S$\5r9/S%Qr:g)&)CallableListOptionalTupleUnionN)nn)CacheSlidingWindowCache StaticCache)AttentionMaskConverter)FlashAttentionKwargs)BaseModelOutputWithPastQuestionAnsweringModelOutput)ALL_ATTENTION_FUNCTIONS)Unpack)is_torch_flex_attn_availablelogging) LlamaAttentionLlamaDecoderLayerLlamaForCausalLMLlamaForQuestionAnsweringLlamaForSequenceClassificationLlamaForTokenClassificationLlamaMLP LlamaModelLlamaPreTrainedModelapply_rotary_pos_embeager_attention_forward) MistralConfig) BlockMask)make_flex_block_causal_maskzmistralai/Mistral-7B-v0.1c(^\rSrSrU4SjrSrU=r$) MistralMLP)c>>[TU]U5 [R"URUR SS9Ul[R"URUR SS9Ul[R"UR URSS9Ulg)NFbias) super__init__rLinear hidden_sizeintermediate_size gate_projup_proj down_projselfconfig __class__s c/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/mistral/modular_mistral.pyr,MistralMLP.__init__*ss  4#3#3T5K5KRWXyy!1!143I3IPUV 4#9#94;K;KRWX)r2r0r1)__name__ __module__ __qualname____firstlineno__r,__static_attributes__ __classcell__r6s@r7r&r&)sYYr9r&cB^\rSrSrS\S\4U4SjjrSS\RS\ \R\R4S\ \RS\ \ S \ \RS \ \S \ \R\ \R\ \ \R44S jjrS rU=r$)MistralAttention1r5 layer_idxcf>[TU]5 [USS5=(d URUR-Ul[ R"URURUR -SS9Ul[ R"URURUR -SS9Ul [ R"URURUR -SS9Ul [ R"URUR -URSS9Ul g)Nhead_dimFr)) r+r,getattrr.num_attention_headsrFrr-q_projnum_key_value_headsk_projv_projo_projr4r5rDr6s r7r,MistralAttention.__init__2s  D9mV=O=OSYSmSm=m ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^ejk ii : :T]] JFL^L^ejk r9 hidden_statesposition_embeddingsattention_maskpast_key_valuecache_positionkwargsreturnc URSSn/UQSPURP7nURU5RU5R SS5n UR U5RU5R SS5n UR U5RU5R SS5n Uup[XX5upUb$XUS.nURXURU5up[nURRS:wadURRS:Xa-URSS5(a[RS 5 O[ URRnU"UU U U U4UR"(dS O UR$UR&[)URS S5S .UD6unnUR*"/UQSP76R-5nUR/U5nUU4$) Nr!r)sincosrTeagersdpaoutput_attentionsFz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.sliding_window)dropoutscalingr_)shaperFrIview transposerKrLrupdaterDr r5_attn_implementationgetlogger warning_oncertrainingattention_dropoutrarGreshape contiguousrM)r4rPrQrRrSrTrU input_shape hidden_shape query_states key_states value_statesrZrY cache_kwargsattention_interface attn_output attn_weightss r7forwardMistralAttention.forward:s$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &#7 RU#[  %#&nUL'5'<'d>d##L '>dkk>^>^&_#$7      % $}}C$2H2HLL"4;;0@$G %  % ! \"));;;;FFH kk+. L((r9)rFrKrMrIrL)NN)r:r;r<r=r"intr,torchTensorrrr LongTensorrrrwr>r?r@s@r7rBrB1sl}ll+/59 0)||0)#5<<#=>0)!. 0) ! 0) !!1!12 0)-.0) u||Xell3XeELL>Q5RR S0)0)r9rBc4^\rSrSrS\S\4U4SjjrSrU=r$)MistralDecoderLayermr5rDc`>[TU]X5 [XS9Ul[ U5Ulg)N)r5rD)r+r,rB self_attnr&mlprNs r7r,MistralDecoderLayer.__init__ns( +)Mf%r9)rr) r:r;r<r=r"ryr,r>r?r@s@r7r~r~ms&}&&&r9r~c\rSrSrSrg)MistralPreTrainedModeltNr:r;r<r=r>rr9r7rrtr9rc ^\rSrSrS\4U4SjjrSS\\RS4S\RS\RS\ S \ 4 S jjr \ S\RS \ S \ S \RS\RS\ S\S\ 4Sj5rSrU=r$) MistralModelxr5c >[TU]U5 [R"[ UR 5Vs/sHn[ X5PM sn5Ulgs snfN)r+r,r ModuleListrangenum_hidden_layersr~layersrNs r7r,MistralModel.__init__ysH  mmEJ6KcKcEd eEd  3Ed e  esArRr# input_tensorrTpast_key_valuesr]c URRS:Xa]UbNUbKUSS2S4R5R5UR 5S:gnU(a [ S5eUbSU;aU$gURRS:Xa,[ U[R5(a [U5nU$UbUR5OSn[ U[5n[ U[5n URRS:XaQU(dJU (dCU(d<[R"UUUURRUR S9(agUR"n [R$"U 5R&n UR(S n U (dU(aUR+5n O5[ U[R5(aUR(SOX|-S -n UR-UU U U UUR(SURUS 9nURRS:Xa:Ub7UR.R0S ;aU(d[R2"X5nU$) Nflash_attention_2rXrzYou are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Mistral. Make sure to call `tokenizer.padding_side = 'left'` before tokenizing the input. r^flex_attentionr\) inputs_embedspast_key_values_lengthr_ is_trainingr!)sequence_length target_lengthdtyperT batch_sizer5r)cudaxpunpu)r5rfsumitemsize ValueError isinstancerzr{r$get_seq_lengthr r r _ignore_causal_mask_sdpar_rjrfinfominrbget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positiondevicetype_unmask_unattended)r4rRrrTrr]is_padding_rightpast_seen_tokensusing_static_cacheusing_sliding_window_cacher min_dtyperr causal_masks r7_update_causal_mask MistralModel._update_causal_masks; ;; + +/B B)o.I#1!R%#8#<#<#>#C#C#EIZIZI\]^I_#_ #$a )c^.C%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`a'E%/AS%T" KK , , 6'+E%%>>*'7#{{99 MM ""KK&** &,,Q/ %);+??AM nell;;$$R(%7!; PP +')#))!,;;+Q   KK , , 6*%%**.DD% 1CCK[Kr9rrrrcUbUR5S:XaUnU$[R"U5Rn [R"X4XUR S9n[R "X$R S9URSS5:n UR5n [U SS5(avU Rbi[U[5(aX:aO[R "X$R S9URSS5U R- :*n U RU 5 X-nUSSSS2SS24RUSSS5nUbUR5nUR SU:a USS2SU24nUR Sn USS2SS2SS2SU 24USS2SSSS24R#UR 5-nUS :HnUSS2SS2SS2SU 24R%X5USS2SS2SS2SU 24'U$) aS Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. config (`MistralConfig`): The model's configuration class past_key_values (`Cache`): The cache class that is being used currently to generate N) fill_valuerr)rrXr!use_sliding_windowTr)dimrzrrfullrarangerlget_text_configrGr_rr bitwise_or_expandclonerbto masked_fill)rRrrrrTrr5rrrdiagonal_attend_mask text_configsliding_attend_mask mask_length padding_masks r7rBMistralModel._prepare_4d_causal_attention_mask_with_cache_positionsB  %.*<*<*>!*C(K@= E*..I** 0Y\j\q\qK$)<< F[F[#\_m_u_uA`$ !002K{$8$??KD^D^Dj"/3EFF/Ji*/,,}MbMb*c&..r15 8R8RR+')445HI  /K%dD!Q&67>>z1bRTUK))//1 !''+m;%3A~ ~4E%FN,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r9)r)F)r:r;r<r=r"r,rrzr{r boolr staticmethodryrrr>r?r@s@r7rrxs } #( TellK78TllT T  T  TlB BBB{{ B  B  BBBBr9rc\rSrSrSrg)MistralForCausalLMirNrrr9r7rrrr9rc\rSrSrSrg)MistralForTokenClassificationirNrrr9r7rrrr9rc\rSrSrSrg) MistralForSequenceClassificationi#rNrrr9r7rr#rr9rcR^\rSrSrSrU4SjrSrSrSS\\ RS\\ RS\\ RS \\ \ \\ R4S \\ RS \\ RS \\ RS \\S\\S\4SjjrSrU=r$)MistralForQuestionAnsweringi'modelcH>[TU]U5 [U5UlU?gr)r+r,rr transformerr3s r7r,$MistralForQuestionAnswering.__init__*s"  !&)  r9c.URR$rr embed_tokens)r4s r7get_input_embeddings0MistralForQuestionAnswering.get_input_embeddings/szz&&&r9c$XRlgrr)r4values r7set_input_embeddings0MistralForQuestionAnswering.set_input_embeddings2s "' r9 input_idsrR position_idsrrstart_positions end_positionsr]output_hidden_statesrVc URUUUUUUU S9n U Rn URU 5n U RSSS9upUR S5R 5nUR S5R 5nSnUbUbUR "XXg40U D6n[UUUU RU RS9$)a start_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the start of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. end_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the end of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. )rRrrrr]rr!rX)rN)loss start_logits end_logitsrP attentions) rlast_hidden_state qa_outputssplitsqueezerm loss_functionrrPr)r4rrRrrrrrr]rrUoutputssequence_outputlogitsrrrs r7rw#MistralForQuestionAnswering.forward5s0,0:: )%+'/!5,6, "331#)<<r<#: #++B/::< ''+668   &=+D%%libhiD+%!!//))   r9)r) NNNNNNNNN)r:r;r<r=base_model_prefixr,rrrrzr|r{rr r FloatTensorrrrwr>r?r@s@r7rr's '( 151537KO596:48,0/33 E,,-3 !.3 u//0 3 "%tE4E4E/F(F"GH 3   1 12 3 "%"2"233   0 013 $D>3 'tn3  &3 3 r9r)rrrrrr);typingrrrrrrztorch.utils.checkpointr cache_utilsr r r modeling_attn_mask_utilsr modeling_flash_attention_utilsrmodeling_outputsrrmodeling_utilsrprocessing_utilsrutilsrrllama.modeling_llamarrrrrrrrrrr configuration_mistralr"!torch.nn.attention.flex_attentionr#integrations.flex_attentionr$ get_loggerr:rh_CHECKPOINT_FOR_DOCr&rBr~rrrrrr__all__rr9r7rs99 AA>BU5&:    1 !!;J   H %1YY9)~9)x&+& 1 `:`F )  $?  'E A ";A H r9