fThGSSKJrJrJrJrJr SSKrSSKJr SSKJ r SSK J r J r J r Jr SSKJr SSKJr SS KJr SS KJr SS KJr SS KJrJrJrJrJr SS KJ r J!r! SSK"J#r#J$r$ SSK%J&r& SSK'J(r(J)r)J*r*J+r+J,r, SSK-J.r. \+"5(a SSK/J0r0 SSK1J2r2 \,Rf"\45r5"SS\Rl5r7Sr8S?Sjr9S\RtS\;S\Rt4Sjr<S@S\RlS\RtS\RtS \RtS!\\RtS"\=S#\=4S$jjr>"S%S&\Rl5r?\"S'5"S(S)\Rl55r@"S*S+\5rA\)"S,S-\$55rB"S.S/\Rl5rC\)"S0S1\B55rD"S2S3\\(5rE\)"S4S5\B\55rF\)"S6S7\B55rG\)"S8S99"S:S;\B55rH\)"S<S=\B55rI/S>QrJg)A)CallableListOptionalTupleUnionN)nn)ACT2FN)Cache DynamicCacheSlidingWindowCache StaticCache)GenerationMixin)use_kernel_forward_from_hub)AttentionMaskConverter)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPastTokenClassifierOutput)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) LossKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging) MistralConfig) BlockMask)make_flex_block_causal_maskc.^\rSrSrU4SjrSrSrU=r$) MistralMLP*c>[TU]5 XlURUlURUl[ R "URURSS9Ul[ R "URURSS9Ul[ R "URURSS9Ul [URUl gNFbias) super__init__config hidden_sizeintermediate_sizerLinear gate_projup_proj down_projr hidden_actact_fnselfr0 __class__s d/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/mistral/modeling_mistral.pyr/MistralMLP.__init__+s  !--!'!9!94#3#3T5K5KRWXyy!1!143I3IPUV 4#9#94;K;KRWXV../ cURURURU55URU5-5nU$N)r6r8r4r5)r:xr6s r<forwardMistralMLP.forward5s6NN4;;t~~a/@#ADLLQRO#ST r>)r8r0r6r4r1r2r5)__name__ __module__ __qualname____firstlineno__r/rB__static_attributes__ __classcell__r;s@r<r(r(*s0r>r(cUSSURSS-24nUSURSS-S24n[R"U*U4SS9$)z*Rotates half the hidden dims of the input..Ndim)shapetorchcat)rAx1x2s r< rotate_halfrU:sZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r>cURU5nURU5nX-[U5U--nX-[U5U--nXg4$)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezerU)qkcossin position_ids unsqueeze_dimq_embedk_embeds r<apply_rotary_pos_embr`AsS( -- &C -- &Cw;q>C/0Gw;q>C/0G  r> hidden_statesn_repreturncURup#pEUS:XaU$USS2SS2SSS2SS24RX#XU5nURX#U-XE5$)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) r#N)rPexpandreshape)rarbbatchnum_key_value_headsslenhead_dims r< repeat_kvrk\s_ 2?1D1D.E z!!Qa"23::5W\dlmM  e(CT TTr>modulequerykeyvalueattention_maskscalingdropoutc @[X R5n[X0R5n [R"XR SS55U-n Ub"USS2SS2SS2SUR S24n X-n [ RRU S[RS9RUR5n [ RRXURS9n [R"X5n U R SS5R5n X4$)NrMr rL)rOdtype)ptrainingr#)rknum_key_value_groupsrQmatmul transposerPr functionalsoftmaxfloat32torurrrw contiguous) rlrmrnrorprqrrkwargs key_states value_states attn_weights causal_mask attn_outputs r<eager_attention_forwardrhs3 ; ;cF^\rSrSrSrS\S\4U4SjjrSS\RS\ \R\R4S\ \RS \ \ S \ \RS \\S \ \R\ \R\ \ \R44S jjrSrU=r$)MistralAttentionz=Multi-headed attention from 'Attention Is All You Need' paperr0 layer_idxc>[TU]5 XlX l[ USS5=(d UR UR -UlUR UR-Ul URS-Ul URUl SUl [R"UR UR UR-SS9Ul[R"UR URUR-SS9Ul[R"UR URUR-SS9Ul[R"UR UR-UR SS9Ulg)NrjgTFr,)r.r/r0rgetattrr1num_attention_headsrjrhrxrqattention_dropout is_causalrr3q_projk_projv_projo_projr:r0rr;s r<r/MistralAttention.__init__s.  " D9mV=O=OSYSmSm=m $*$>$>&B\B\$\!}}d* !'!9!9ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^ejk ii : :T]] JFL^L^ejk r>raposition_embeddingsrppast_key_valuecache_positionrrcc URSSn/UQSPURP7nURU5RU5R SS5n UR U5RU5R SS5n UR U5RU5R SS5n Uup[XX5upUb$XUS.nURXURU5up[nURRS:wadURRS:Xa-URSS5(a[RS 5 O[ URRnU"UU U U U4UR"(dS O UR$UR&[)URS S5S .UD6unnUR*"/UQSP76R-5nUR/U5nUU4$) NrLr#rM)r[rZreagersdpaoutput_attentionsFz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.sliding_window)rrrqr)rPrjrviewrzrrr`updaterrr0_attn_implementationgetlogger warning_oncerrwrrqrrfrr)r:rarrprrr input_shape hidden_shape query_statesrrrZr[ cache_kwargsattention_interfacerrs r<rBMistralAttention.forwards$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &#7 RU#[  %#&nUL'5'<'d>d##L '>dkk>^>^&_#$7      % $}}C$2H2HLL"4;;0@$G %  % ! \"));;;;FFH kk+. L((r>) rr0rjrrrrxrrrqr)NN)rDrErFrG__doc__r$intr/rQTensorrrr LongTensorrrrBrHrIrJs@r<rrsG l} l l&+/59 0)||0)#5<<#=>0)!. 0) ! 0) !!1!12 0)-.0) u||Xell3XeELL>Q5RR S0)0)r>rRMSNormc8^\rSrSrSU4SjjrSrSrSrU=r$)MistralRMSNormc>[TU]5 [R"[R "U55UlX lg)z- MistralRMSNorm is equivalent to T5LayerNorm N)r.r/r ParameterrQonesweightvariance_epsilon)r:r1epsr;s r<r/MistralRMSNorm.__init__s/ ll5::k#:; #r>cURnUR[R5nUR S5R SSS9nU[R "X0R-5-nURURU5-$)NrMrLT)keepdim) rur~rQr}powmeanrsqrtrr)r:ra input_dtypevariances r<rBMistralRMSNorm.forwardsw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r>c^[URR5SUR3$)Nz, eps=)tuplerrPrr:s r< extra_reprMistralRMSNorm.extra_reprs* ))*+6$2G2G1HIIr>)rr)gư>) rDrErFrGr/rBrrHrIrJs@r<rrs$;JJr>rc^\rSrSrS\S\4U4SjjrSS\RS\ \RS\ \RS\ \ S \ \ S \ \ S \ \RS \ \ \R\R4S \\S\ \R \ \ \R \R 444SjjrSrU=r$)MistralDecoderLayerr0rc >[TU]5 URUl[XS9Ul[ U5Ul[URURS9Ul [URURS9Ul g)N)r0rr) r.r/r1r self_attnr(mlpr rms_norm_epsinput_layernormpost_attention_layernormrs r<r/MistralDecoderLayer.__init__sj !--)Mf%-f.@.@fFYFYZ(6v7I7IvObOb(c%r>rarpr\rr use_cacherrrrcc Un URU5nUR"SUUUUUUUUS.U D6upX-nUn URU5nURU5nX-nU4n U(aX4- n U $)N)rarpr\rrrrr)rrrr) r:rarpr\rrrrrrresidualself_attn_weightsoutputss r<rBMistralDecoderLayer.forwards!,,]; ,0>> , ')%)/) 3 ,  , ( !0 !55mD /  0  "  + +Gr>)r1rrrr)NNNFFNN)rDrErFrGr$rr/rQrrrr boolrrr FloatTensorrBrHrIrJs@r<rrsd}dd2637*.,1$)59KO'||'!.'u//0 ' ! ' $D> 'D>'!!1!12'&eELL%,,,F&GH'-.' u  (51B1BEDUDU1U+V"WW X''r>rcN\rSrSr\rSrSrS/rS/r Sr Sr Sr Sr SrSrSrSrSrg) MistralPreTrainedModelimodelTrpast_key_valuescURRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[5(a&UR R RS5 gg)Nr)rstdg?)r0initializer_range isinstancerr3rdatanormal_r-zero_ Embedding padding_idxrfill_)r:rlrs r< _init_weights$MistralPreTrainedModel._init_weightsskk++ fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>.  / / MM   $ $S )0r>rN)rDrErFrGr$ config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cache_supports_attention_backendrrHrr>r<rrsS L&*#./#4"5!N  $!"& *r>rcl^\rSrSrSS\4U4Sjjjr\R"5\S55r Sr U=r $)MistralRotaryEmbeddingi+r0c>[TU]5 [US5(aHURb;URR SURR S55UlOSUlUR UlUR UlXl [UR Ul URURU5uo0l URSUSS9 URUlg)N rope_scaling rope_typetypedefaultinv_freqF) persistent)r.r/hasattrrrrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr0r rope_init_fnattention_scalingregister_bufferroriginal_inv_freq)r:r0devicerr;s r<r/MistralRotaryEmbedding.__init__,s  6> * *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+cbURSSS2S4R5RURSSS5R UR 5nUSS2SSS24R5n[ UR R[5(a0UR RS:waUR ROSn[R"USS9 UR5UR5-RSS5n[R"Xf4SS 9nUR5UR-nUR5UR-n SSS5 WR UR S 9W R UR S 94$!,(df  N@=f) NrrLr#mpscpuF) device_typeenabledrMrN)ru)rfloatrerPr~rrrstrrQautocastrzrRrZrr[ru) r:rAr\inv_freq_expandedposition_ids_expandedr freqsembrZr[s r<rBMistralRotaryEmbedding.forward=sR!MM$4-8>>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E'E!((--[`J`ahhmmfk ^^ U C&,,.1F1L1L1NNYYZ[]^_E))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs $BF  F.)rr0rrrrrr@) rDrErFrGr$r/rQno_gradrrBrHrIrJs@r<rr+s6/}//" ]]_ < rc^\rSrSrS\4U4SjjrSrSr\\ SS\ \ RS\ \ RS\ \ RS \ \S \ \ RS \ \S \ \S \ \S\ \ RS\\S\4Sjj55rSS\\ RS4S\ RS\ RS \S \4 Sjjr\S\ RS\S\S\ R2S\ RS\S\S \4Sj5rSrU=r$) MistralModeliMr0c >[TU]U5 URUlURUl[ R "URURUR5Ul[ R"[UR5Vs/sHn[X5PM sn5Ul [URURS9Ul[#US9UlSUlUR)5 gs snf)Nr)r0F)r.r/ pad_token_idr vocab_sizerrr1 embed_tokens ModuleListrangenum_hidden_layersrlayersrrnormr rotary_embgradient_checkpointing post_initrs r<r/MistralModel.__init__Os  !.. ++LL):):FcXlgr@r(r:ros r<set_input_embeddings!MistralModel.set_input_embeddingsbs!r> input_idsrpr\r inputs_embedsrroutput_hidden_statesrflash_attn_kwargsrcc JUbUOURRnUbUOURRnUbUOURRnUSLUSL- (a [ S5eUR (a/UR (aU(a[RS5 Sn[U[S5[45(d [ S5eUcURU5nU(a Uc [5nU cDUbUR5OSn [R "XUR"S-UR$S9n UcU R'S5nUR)X%XU5n Un UR+X5nU(aSOSnU(aSOSnUR,SURR.H7nU(aX4- nU"U 4U UUUUU US .U D6nUSn U(dM.UUS4- nM9 UR1U 5n U(aX4- n[3U U(aUOSUUS 9$) Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FzBThe `past_key_values` should be either a `Cache` object or `None`.rr#rr)rpr\rrrrr)last_hidden_staterra attentions)r0rr1r ValueErrorr$rwrrrrr rr get_seq_lengthrQarangerPrrW_update_causal_maskr#r!r r"r)r:r/rpr\rr0rrr1rr2past_seen_tokensrrarall_hidden_statesall_self_attns decoder_layer layer_outputss r<rBMistralModel.forwardesI2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK??ab b   --i8M 0*nO  !CRC^==?de "\\ ]5H5H5K"KTaThThN  )33A6L.. >L] & #oomJ#7BD0d![[)H4;;+H+HIM#!%55!) *)."3#-$7 $ M*!,M  =#3"55'J* -0  !1 1 &+/8Od+%   r>r% input_tensorc URRS:Xa]UbNUbKUSS2S4R5R5UR 5S:gnU(a [ S5eUbSU;aU$gURRS:Xa,[ U[R5(a [U5nU$UbUR5OSn[ U[5n[ U[5n URRS:XaQU(dJU (dCU(d<[R"UUUURRUR S9(agUR"n [R$"U 5R&n UR(S n U (dU(aUR+5n O5[ U[R5(aUR(SOX|-S -n UR-UU U U UUR(SURUS 9nURRS:Xa:Ub7UR.R0S ;aU(d[R2"X5nU$) Nflash_attention_2rLrzYou are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Mistral. Make sure to call `tokenizer.padding_side = 'left'` before tokenizing the input. rflex_attentionr)r0past_key_values_lengthr is_trainingr#)sequence_length target_lengthrur batch_sizer0r)cudaxpunpu)r0rsumitemsizer7rrQrr&r8rr r_ignore_causal_mask_sdparrwrufinfominrPget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionrr_unmask_unattended)r:rprArrris_padding_rightr;using_static_cacheusing_sliding_window_cacheru min_dtyperGrHrs r<r: MistralModel._update_causal_masks; ;; + +/B B)o.I#1!R%#8#<#<#>#C#C#EIZIZI\]^I_#_ #$a )c^.C%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`a'E%/AS%T" KK , , 6'+E%%>>*'7#{{99 MM ""KK&** &,,Q/ %);+??AM nell;;$$R(%7!; PP +')#))!,;;+Q   KK , , 6*%%**.DD% 1CCK[Kr>rGrHrurIcUbUR5S:XaUnU$[R"U5Rn [R"X4XUR S9n[R "X$R S9URSS5:n UR5n [U SS5(avU Rbi[U[5(aX:aO[R "X$R S9URSS5U R- :*n U RU 5 X-nUSSSS2SS24RUSSS5nUbUR5nUR SU:a USS2SU24nUR Sn USS2SS2SS2SU 24USS2SSSS24R#UR 5-nUS :HnUSS2SS2SS2SU 24R%X5USS2SS2SS2SU 24'U$) aS Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. config (`MistralConfig`): The model's configuration class past_key_values (`Cache`): The cache class that is being used currently to generate N) fill_valuerurr4rLr#use_sliding_windowTr)rOrQrQrRfullrr9rfget_text_configrrrr bitwise_or_reclonerPr~ masked_fill)rprGrHrurrIr0rrrYdiagonal_attend_mask text_configsliding_attend_mask mask_length padding_masks r<rTBMistralModel._prepare_4d_causal_attention_mask_with_cache_positionsB  %.*<*<*>!*C(K@= E*..I** 0Y\j\q\qK$)<< F[F[#\_m_u_uA`$ !002K{$8$??KD^D^Dj"/3EFF/Ji*/,,}MbMb*c&..r15 8R8RR+')445HI  /K%dD!Q&67>>z1bRTUK))//1 !''+m;%3A~ ~4E%FN,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r>)rr$r!r"rr#r NNNNNNNNN)F)rDrErFrGr$r/r)r-r rrrQrrr rrrrrrBrr: staticmethodrrurTrHrIrJs@r<rrMs} !"151537+/59$(,0/359\ E,,-\ !.\ u//0 \ "% \   1 12 \ D>\ $D>\ 'tn\ !!1!12\ $$89\  !\ \ H#( TellK78TllT T  T  TlB BBB{{ B  B  BBBBr>rc\rSrSrSrg)KwargsForCausalLMiarN)rDrErFrGrHrr>r<rmrmas3r>rmc^\rSrSrS/rSS0rSS/S/40rU4SjrSrS r S r S r S r S r \\SS\\R$S\\R&S\\R$S\\S\\R*S\\R$S\\S\\S\\S\\R$S\\\R&4S\\S\4Sjj55rSrU=r$)MistralForCausalLMidzlm_head.weightlm_head colwise_repralogitsc>[TU]U5 [U5UlURUl[ R "URURSS9UlUR5 gr+) r.r/rrrrr3r1rpr%r9s r<r/MistralForCausalLM.__init__jsU  !&)  ++yy!3!3V5F5FUS  r>c.URR$r@rrrs r<r)'MistralForCausalLM.get_input_embeddingsszz&&&r>c$XRlgr@rvr,s r<r-'MistralForCausalLM.set_input_embeddingsv "' r>cUR$r@rprs r<get_output_embeddings(MistralForCausalLM.get_output_embeddingsys ||r>cXlgr@r})r:new_embeddingss r<set_output_embeddings(MistralForCausalLM.set_output_embeddings|s% r>cXlgr@r)r:decoders r< set_decoderMistralForCausalLM.set_decoders r>cUR$r@rrs r< get_decoderMistralForCausalLM.get_decoders zzr>r/rpr\rr0labelsrrr1rlogits_to_keeprrcc UbUOURRnU bU OURRn UR"SUUUUUUUU U S. U D6n U Rn[ U [ 5(a [U *S5OU nURUSS2USS245nSnUb)UR"SUX`RRS.U D6n[UUU RU RU RS9$)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MistralForCausalLM >>> model = MistralForCausalLM.from_pretrained("meta-mistral/Mistral-2-7b-hf") >>> tokenizer = AutoTokenizer.from_pretrained("meta-mistral/Mistral-2-7b-hf") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```N) r/rpr\rr0rrr1r)rrrrlossrrrrar6r)r0rr1rr5rrslicerp loss_functionrrrrar6)r:r/rpr\rr0rrrr1rrrrra slice_indicesrrrs r<rBMistralForCausalLM.forwardsN2C1N-TXT_T_TqTq$8$D $++JjJj  ,0:: , )%+'/!5) ,  ,  11 8B>SV8W8W~ot4]k mA}a,?@A  %%pVF{{OeOepiopD%#33!//))   r>)rprr) NNNNNNNNNNr)rDrErFrG_tied_weights_keys_tp_plan_pp_planr/r)r-r~rrrr rrrQrrr rrrrrrmrrBrHrIrJs@r<rorods*+=)H_-z:;H'(&151537+/59-1$(,0/35934G E,,-G !.G u//0 G "% G   1 12 G ))*G D>G $D>G 'tnG !!1!12G c5<</0G *+G  G G r>roc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$)MistralForTokenClassificationic>[TU]U5 URUl[U5Ul[ USS5b UR nO[ USS5b URnOSn[R"U5Ul [R"URUR5Ul UR5 g)Nclassifier_dropouthidden_dropoutg?)r.r/ num_labelsrrrrrrDropoutrrr3r1scorer%)r:r0rr;s r<r/&MistralForTokenClassification.__init__s   ++!&) 6/ 6 B!'!:!:  V-t 4 @!'!6!6 !$ zz"45 YYv1163D3DE  r>c.URR$r@rvrs r<r)2MistralForTokenClassification.get_input_embeddingsrxr>c$XRlgr@rvr,s r<r-2MistralForTokenClassification.set_input_embeddingsr{r>r/rpr\rr0rrrr1rcc  URUUUUUUUU S9n U Rn URU 5n URU 5n Sn UbUR XUR 5n [ U U U RU RS9$)e labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rpr\rr0rrr1N)rrrrar6) rr5rrrrr0rrar6)r:r/rpr\rr0rrrr1rsequence_outputrrrs r<rB%MistralForTokenClassification.forwards*,0:: )%+'/!5,6 , "33,,7O,  %%fdkkBD$!//))   r>)rrrrrrj)rDrErFrGr/r)r-r rrrQrrr rrrrBrHrIrJs@r<rrs '(151537+/59-1$(,0/3* E,,-* !.* u//0 * "% *   1 12 * ))** D>* $D>* 'tn*  * * r>ra The Mistral Model transformer with a sequence classification head on top (linear layer). [`MistralForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$) MistralForSequenceClassificationic>[TU]U5 URUl[U5Ul[ R "URURSS9UlUR5 gr+) r.r/rrrrr3r1rr%r9s r<r/)MistralForSequenceClassification.__init__'sS   ++!&) YYv114??O  r>c.URR$r@rvrs r<r)5MistralForSequenceClassification.get_input_embeddings0rxr>c$XRlgr@rvr,s r<r-5MistralForSequenceClassification.set_input_embeddings3r{r>r/rpr\rr0rrrr1rcc URUUUUUUUU S9n U Rn URU 5n UbURSn OURSn URR cU S:wa [ S5eURR cSnOUbXRR :gRU R[R5n[R"URSU R[RS9nUU-RS5nO.Sn[RURR S35 U [R"XRS 9U4nSnUbUR#XUURS 9n[%UUU R&U R(U R*S 9$) rrNrr#z=Cannot handle batch sizes > 1 if no padding token is defined.rL)rruz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r4)rrr pooled_logitsr0r)rr5rrPr0rr7r~rrQint32r9argmaxrrr;rDrrrrar6)r:r/rpr\rr0rrrr1transformer_outputsrarrrIlast_non_pad_token non_pad_mask token_indicesrrs r<rB(MistralForSequenceClassification.forward6s*8!F!Fr!J !#    >>**+,ZZ  u||J}}MOaab   %%VR_hlhshs%tD/ /??-;;*55   r>)rrrrj)rDrErFrGr/r)r-r rrrQrrr rrrrBrHrIrJs@r<rrs'(151537+/59-1$(,0/3A E,,-A !.A u//0 A "% A   1 12 A ))*A D>A $D>A 'tnA  *A A r>rcf^\rSrSrSrU4SjrSrSr\\ SS\ \ RS\ \ RS\ \ RS \ \\\\ R"4S \ \ R"S \ \ RS \ \ RS \ \S\ \S\4Sjj55rSrU=r$)MistralForQuestionAnsweringi|rc>[TU]U5 [R"URS5Ul[ U5UlUR5 g)NrM) r.r/rr3r1 qa_outputsrrr%r9s r<r/$MistralForQuestionAnswering.__init__s@  ))F$6$6:!&)  r>c.URR$r@rvrs r<r)0MistralForQuestionAnswering.get_input_embeddingsrxr>c$XRlgr@rvr,s r<r-0MistralForQuestionAnswering.set_input_embeddingsr{r>r/rpr\rr0start_positions end_positionsrr1rcc URUUUUUUU S9n U Rn URU 5n U RSSS9upUR S5R 5nUR S5R 5nSnUbUbUR "XXg40U D6n[UUUU RU RS9$)a start_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the start of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. end_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the end of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. )rpr\rr0rr1r#rLrNN)r start_logits end_logitsrar6) rr5rsplitsqueezerrrrar6)r:r/rpr\rr0rrrr1rrrrrrrrs r<rB#MistralForQuestionAnswering.forwards4,0:: )%+'/!5,6, "331#)<<r<#: #++B/::< ''+668   &=+D%%libhiD+%!!//))   r>)rrrj)rDrErFrGrr/r)r-r rrrQrrrr rrrrrBrHrIrJs@r<rr|s'(151537KO596:48,0/33 E,,-3 !.3 u//0 3 "%tE4E4E/F(F"GH 3   1 12 3 "%"2"233   0 013 $D>3 'tn3  &3 3 r>r)rorrrrr)Nr#)r)KtypingrrrrrrQr activationsr cache_utilsr r r r generationr integrationsrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrrmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrr r!r"configuration_mistralr$!torch.nn.attention.flex_attentionr%integrations.flex_attentionr& get_loggerrDrModuler(rUr`rrrkrrrrrrrrrmrorrr__all__rr>r<rsF:9 !OO)7>B9LF&hh0 !!;J   H %   (6 UU\\ U# U%,, U&% II% <<% % << % U\\* %  %%4A)ryyA)HY'JRYYJ(J(040f*_**8i /i i XC $:C C L  S '=S  S lF "8F F R r>