fThSSKJr SSKJrJrJrJr SSKrSSKJ r SSK J r SSK J r JrJr SSKJr SSKJr SS KJrJrJrJr SS KJrJr SS KJrJr SS KJ r SS K!J"r"J#r#J$r$J%r% SSK&J'r' SSK(J)r) \$"5(a SSK*J+r+ SSK,J-r- \%R\"\/5r0"SS\ Rb5r2"SS\ Rb5r3Sr4S9Sjr5S\RlS\7S\Rl4Sjr8S:S\ RbS\RlS\RlS \RlS!\\RlS"\9S#\\9S$\\9S\\Rl\Rl44S%jjr:"S&S'\ Rb5r;"S(S)\ Rb5r<"S*S+\ Rb5r=\""S,S-\55r>\""S.S/\>55r?\""S0S1\>\55r@\""S2S39"S4S5\>55rA\""S6S7\>55rB/S8QrCg);)partial)CallableOptionalTupleUnionN)ACT2FN)Cache HybridCache StaticCache)GenerationMixin)FlashAttentionKwargs)BaseModelOutputWithPastCausalLMOutputWithPast SequenceClassifierOutputWithPastTokenClassifierOutput)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)auto_docstringcan_return_tupleis_torch_flex_attn_availablelogging)deprecate_kwarg) Gemma2Config) BlockMask)make_flex_block_causal_maskcJ^\rSrSrS S\S\4U4SjjjrSrSrSr Sr U=r $) Gemma2RMSNorm7dimepsc>[TU]5 X l[R"[ R "U55UlgN)super__init__r%nn Parametertorchzerosweight)selfr$r% __class__s b/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/gemma2/modeling_gemma2.pyr)Gemma2RMSNorm.__init__8s, ll5;;s#34 cU[R"URS5RSSS9UR-5-$)NT)keepdim)r,rsqrtpowmeanr%)r/xs r1_normGemma2RMSNorm._norm=s45;;quuQx}}R}>IJJJr3cURUR55nUSURR5--nURU5$)N?)r<floatr.type_as)r/r;outputs r1forwardGemma2RMSNorm.forward@sCAGGI&3!2!2!445~~a  r3c^[URR5SUR3$)Nz, eps=)tupler.shaper%r/s r1 extra_reprGemma2RMSNorm.extra_reprGs' ))*+6$((<) __name__ __module__ __qualname____firstlineno__intr@r)r<rCrI__static_attributes__ __classcell__r0s@r1r"r"7s05C5e55 K!==r3r"c.^\rSrSrU4SjrSrSrU=r$) Gemma2MLPKc>[TU]5 XlURUlURUl[ R "URURSS9Ul[ R "URURSS9Ul[ R "URURSS9Ul [URUl gNFbias) r(r)config hidden_sizeintermediate_sizer*Linear gate_projup_proj down_projr hidden_activationact_fnr/rZr0s r1r)Gemma2MLP.__init__Ls  !--!'!9!94#3#3T5K5KRWXyy!1!143I3IPUV 4#9#94;K;KRWXV556 r3cURURURU55URU5-5nU$r')r`rbr^r_)r/r;r`s r1rCGemma2MLP.forwardVs6NN4;;t~~a/@#ADLLQRO#ST r3)rbrZr`r^r[r\r_)rKrLrMrNr)rCrPrQrRs@r1rTrTKs7r3rTcUSSURSS-24nUSURSS-S24n[R"U*U4SS9$)z*Rotates half the hidden dims of the input..Nr6r5r$)rGr,cat)r;x1x2s r1 rotate_halfrl[sZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r3cURU5nURU5nX-[U5U--nX-[U5U--nXg4$)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezerl)qkcossin position_ids unsqueeze_dimq_embedk_embeds r1apply_rotary_pos_embrwbsS( -- &C -- &Cw;q>C/0Gw;q>C/0G  r3 hidden_statesn_repreturncURup#pEUS:XaU$USS2SS2SSS2SS24RX#XU5nURX#U-XE5$)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) rN)rGexpandreshape)rxrybatchnum_key_value_headsslenhead_dims r1 repeat_kvr}s_ 2?1D1D.E z!!Qa"23::5W\dlmM  e(CT TTr3modulequerykeyvalueattention_maskdropoutscalingsoftcapc UcURS-n[X R5n [X0R5n [R"XR SS55U-n UbX- n [R "U 5n X-n Ub"USS2SS2SS2SU RS24n X-n [RRU S[RS9RUR5n [RRXURS9n [R"X5n U R SS5R!5n X4$) Nr5rr6)r$dtype)ptrainingr)rrnum_key_value_groupsr,matmul transposetanhrGr* functionalsoftmaxfloat32torrr contiguous)rrrrrrrrkwargs key_states value_states attn_weights causal_mask attn_outputs r1eager_attention_forwardrs///4'3 ; ;[TU]5 XlX l[ USUR UR -5UlUR UR-Ul URS-Ul URRUl SUl [R"UR UR UR-UR S9Ul[R"UR URUR-UR S9Ul[R"UR URUR-UR S9Ul[R"UR UR-UR UR S9UlURR*Ul[-US-5(dUR.UlgSUlg)NrrTrXr5)r(r)rZrgetattrr[num_attention_headsrrrquery_pre_attn_scalarrattention_dropout is_causalr*r]attention_biasq_projk_projv_projo_projattn_logit_softcappingboolsliding_windowr/rZrr0s r1r)Gemma2Attention.__init__s  " F4F4F&JdJd4de $*$>$>&B\B\$\!33T9 !%!>!>ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii  & & 68J8JQWQfQf  '+kk&H&H#;? A ;N;Nf33TXr3rxposition_embeddingsrpast_key_valuecache_positionrrzc URSSn/UQSPURP7nURU5RU5R SS5n UR U5RU5R SS5n UR U5RU5R SS5n Uup[XX5upUb}U U UURS.nURXURU5upUbJURRS:Xa0URSnU SS2SS2SU2SS24U SS2SS2SU2SS24p[nURRS:wadURRS:Xa-URSS 5(a[R!S 5 O["URRnU"UU U U U4UR$(a UR&OS UR(URUR*S .UD6unnUR,"/UQSP76R/5nUR1U5nUU4$) Nr6rr5)rrrqrrflash_attention_2eagersdpaoutput_attentionsFz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)rrrr)rGrrviewrrrrwrupdaterrZ_attn_implementationrgetlogger warning_oncerrrrrr}rr)r/rxrrrrr input_shape hidden_shape query_statesrrrqrr cache_kwargsseq_lenattention_interfacerrs r1rCGemma2Attention.forwardsS$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &#7 RU#[  %"0"&"5"5 L (6'<'d>d##L '>dkk>^>^&_#$7      % /3mmD**LL..// %  % ! \"));;;;FFH kk+. L((r3) rrrZrrrrrrrrrr)NN)rKrLrMrN__doc__rrOr)r,Tensorrrr LongTensorrrrCrPrQrRs@r1rrsGY|YY<+/59 ;)||;)#5<<#=>;)!. ;) ! ;) !!1!12 ;)-.;) u||Xell3XeELL>Q5RR S;);)r3rc^\rSrSrS\S\4U4Sjjr\"SSS9SS\RS \ \R\R4S \ \RS \ \RS \ \ S \ \S\ \S\ \RS\ \R\ \ \R\R444Sjj5rSrU=r$)Gemma2DecoderLayerirZrc>[TU]5 URUlXl[ US-5(+Ul[ XS9Ul[U5Ul [URURS9Ul [URURS9Ul [URURS9Ul[URURS9UlUR Ulg)Nr5)rZrr%)r(r)r[rZr is_slidingr self_attnrTmlpr" rms_norm_epsinput_layernormpost_attention_layernormpre_feedforward_layernormpost_feedforward_layernormrrs r1r)Gemma2DecoderLayer.__init__s !-- "9q=11(LV$,V-?-?VEXEXY(5f6H6HfNaNa(b%)6v7I7IvObOb)c&*78J8JPVPcPc*d'$33r3last_cache_positionz4.53.0)versionrxrrrsrr use_cacherrzc UR(Ga8UGb4[URSUR5n URR S:Xa USS2U *S24nO[ R"UR5Rn [ R"[ R"U[ RS9UR*S9n [ R"XU5nUSU - S-n [ R"U SS9n [ R"[XRS5UR S9nX- nUSS2SS2SS2U4nUnUR#U5nUR$"S UUUUUUUUS .U D6unnUR'U5nX-nUnUR)U5nUR+U5nUR-U5nX-nU4nU(aUU4- nU$) Nrrrdiagonalr6r)mindevice)rxrrrsrrrr)rmaxrGrrZrr,finforrtril ones_likerwhereclamparangerrrrrrr)r/rxrrrsrrrrreffective_seq_len min_dtypesliding_window_maskoffset mask_indexesresidualself_attn_weightsoutputss r1rCGemma2DecoderLayer.forwards ???~9 #N$8$8$;T=P=P Q {{//3FF!/4E3E3F0F!G"KK(<(<=AA &+jjOON%**EQUQdQdPd'#"'-@^!\'+.??!CV3 %||)+?+?+CD^MbMb  & !/1a0E!F ,,]; ,0>> , ' 3)%)/) ,  , ( (55mD  0  66}E / 77 F  0  "  )+ +Gr3) rZr[rrrrrrrr)NNNFFN)rKrLrMrNrrOr)rr,rrrrr r FloatTensorrCrPrQrRs@r1rrs 4| 4 4*H= 2637*.,1$)59E||E#5<<#=>E!. E u//0 E ! E$D>ED>E!!1!12E u  (51B1BEDUDU1U+V"WW XE>Er3rcl^\rSrSrSS\4U4Sjjjr\R"5\S55r Sr U=r $)Gemma2RotaryEmbeddingi^rZc>[TU]5 [US5(aHURb;URR SURR S55UlOSUlUR UlUR UlXl [UR Ul URURU5uo0l URSUSS9 URUlg)N rope_scaling rope_typetypedefaultinv_freqF) persistent)r(r)hasattrrrrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenrZr rope_init_fnattention_scalingregister_bufferroriginal_inv_freq)r/rZrrr0s r1r)Gemma2RotaryEmbedding.__init___s  6> * *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E'E!((--[`J`ahhmmfk ^^ U C&,,.1F1L1L1NNYYZ[]^_E))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs $BF  F.)rrZrrrrrr') rKrLrMrNrr)r,no_gradrrCrPrQrRs@r1rr^s6/|//" ]]_ < .  . . MM   $ $S )/r3rN)rKrLrMrNr config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cache_supports_attention_backendr$rPrr3r1rrsSL&*#-.#4"5!N  $!"& *r3rc6^\rSrSrS\4U4SjjrSrSr\\ SS\ \ RS\ \ RS\ \ RS \ \S \ \ RS \ \S \ \S \ \S\ \ RS\\S\4Sjj55r\ R*"5SS\\ RS4S\ RS\ RS \S \4 Sjj5r\S\ RS\S\S\ R4S\ RS\4 Sj5rSrU=r$) Gemma2ModelirZc >[TU]U5 URUlURUl[ R "URURUR5Ul[ R"[UR5Vs/sHn[X5PM sn5Ul [URURS9Ul[#US9UlSUlUR)5 gs snf)Nr)rZF)r(r) pad_token_idr" vocab_sizer*r!r[ embed_tokens ModuleListrangenum_hidden_layersrlayersr"rnormr rotary_embgradient_checkpointing post_initrs r1r)Gemma2Model.__init__s  !.. ++LL):):F&+#  esC?cUR$r'r7rHs r1get_input_embeddings Gemma2Model.get_input_embeddingss   r3cXlgr'rBr/rs r1set_input_embeddings Gemma2Model.set_input_embeddingss!r3 input_idsrrsr inputs_embedsrroutput_hidden_statesrflash_attn_kwargsrzc UbUOURRnUbUOURRnUbUOURRnUSLUSL- (a [ S5eUR (a/UR (aU(a[RS5 SnUcURU5nU(aNUcKUR (d:URupn [URU U URURS9nU cDUbUR5OSn[R "XURS-URS9n UcU R#S5nUR%X%XU5nUnUR'UU5n[R("URR*S-URS 9nUU-nU(aS OSnU(aS OSnUR,SURR.HnU(aUU4- nUR (a?UR (a.UR1[3UR440U D6UUUUUUUU 5 nOU"U4UUUUUUU S .U D6nUSnU(dMUUS4- nM UR7U5nU(aUU4- n[9UUUUS 9$) Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.F)max_batch_size max_cache_lenrrrrrg?rr)rrrsrrrr)last_hidden_staterrx attentions)rZrrKr ValueErrorr>rrrr7rGr rrget_seq_lengthr,rrn_update_causal_maskr=tensorr[r;r:_gradient_checkpointing_funcr__call__r<r)r/rIrrsrrJrrrKrrL batch_sizer_past_seen_tokensrrxr normalizerall_hidden_statesall_self_attns decoder_layer layer_outputss r1rCGemma2Model.forwards2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKL] & #oom\J \\$++"9"93">mFYFYZ % 2 #7BD0d![[)H4;;+H+HIM#!m%55!**t}} $ A AM22H6GH!' #%" ! !.! !(;#.!-#2&7'#1 !( ! *!,M  =#3"55AJD -0  -!1 1 &+++%   r3r input_tensorc URRS:XaU$URRS:Xa,[U[R5(a [ U5nU$UR URpvURSn[U[[45(aUR5n O!UbURSOURSn URUUU UUUURSS9n U $)Nrflex_attentionrr6rsequence_length target_lengthrrrrX) rZrrr,rr rrrGr r get_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_position) r/rrarrrrrrerfrs r1rTGemma2Model._update_causal_mask,s ;; + +/B B! ! ;; + +/? ?.%,,77!<^!L! !$**L,?,?v&,,Q/ o ['A B B+??AM8F8RN004XdXjXjklXmMPP +')#))!,Q r3rerfrrXc UbUR5S:XaUnU$[R"U5Rn[R"X4XUR S9nUS:wa[R "USS9nU[R"X$R S9URSS5:-nUSSSS2SS24RUSSS5nUbUR5nURSn USS2SS2SS2SU 24USS2SSSS24RUR 5-n U S:Hn USS2SS2SS2SU 24RX5USS2SS2SS2SU 24'U$) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuerrrrrr6r)r$r,rrfullrtriurr}r|clonerGr masked_fill) rrerfrrrXrrr mask_length padding_masks r1rhAGemma2Model._prepare_4d_causal_attention_mask_with_cache_positionSs}<  %.*<*<*>!*C(K*' E*..I** 0Y\j\q\qK!##jjqA 5<< >S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r3)r7r>r;r<r"r=r6 NNNNNNNNN)F)rKrLrMrNrr)rCrGrrrr,rrr rrrrrrCrrrT staticmethodrOrrhrPrQrRs@r1r3r3s| !"1515371559$(,0/359s E,,-s !.s u//0 s "+. s   1 12 s D>s $D>s 'tns !!1!12s $$89s  !s s j ]]_#( $ellK78$ll$ $ % $  $$L4 444{{ 4  4  44r3r3c^\rSrSrS/rSS0rSS/S/40rU4SjrSrS r S r S r S r S r \\SS\\R$S\\R&S\\R$S\\S\\R*S\\R$S\\S\\S\\S\\R$S\\\R&4S\4Sjj55rSU4SjjrSrU=r$)Gemma2ForCausalLMizlm_head.weightlm_head colwise_reprxlogitsc>[TU]U5 [U5UlURUl[ R "URURSS9UlUR5 grW) r(r)r3rr6r*r]r[rxr?rcs r1r)Gemma2ForCausalLM.__init__sU   (  ++yy!3!3V5F5FUS  r3c.URR$r'rr7rHs r1rC&Gemma2ForCausalLM.get_input_embeddingszz&&&r3c$XRlgr'r~rFs r1rG&Gemma2ForCausalLM.set_input_embeddings "' r3cUR$r'rxrHs r1get_output_embeddings'Gemma2ForCausalLM.get_output_embeddingss ||r3cXlgr'r)r/new_embeddingss r1set_output_embeddings'Gemma2ForCausalLM.set_output_embeddingss% r3cXlgr'r)r/decoders r1 set_decoderGemma2ForCausalLM.set_decoders r3cUR$r'rrHs r1 get_decoderGemma2ForCausalLM.get_decoders zzr3rIrrsrrJlabelsrrrKrlogits_to_keeprzc FUR(aGURRS:wa-[R SURRS35 UbUOURR nU bU OURR n UR"SUUUUUUUU U S. U D6n U Rn[U [5(a [U *S5OU nURUSS2USS245nURRbHUURR- n[R"U5nUURR-nSnUbUR "UX`R"40U D6n[%UUU R&U R(U R*S9$)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, Gemma2ForCausalLM >>> model = Gemma2ForCausalLM.from_pretrained("google/gemma-2-9b") >>> tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b") >>> prompt = "What is your favorite condiment?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "What is your favorite condiment?" ```rzhIt is strongly recommended to train Gemma2 models with the `eager` attention implementation instead of `zp`. Use `eager` with `AutoModelForCausalLM.from_pretrained('', attn_implementation='eager')`.N) rIrrsrrJrrrKrlossrzrrxrQr)rrZrrrrrKrrPrrOslicerxfinal_logit_softcappingr,r loss_functionr6rrrxrQ)r/rIrrsrrJrrrrKrr loss_kwargsrrx slice_indicesrzrs r1rCGemma2ForCausalLM.forwardsP ==T[[==H   #{{??@Aqr 2C1N-TXT_T_TqTq$8$D $++JjJj ,0:: , )%+'/!5) ,  ,  11 8B>SV8W8W~ot4]k mA}a,?@A ;; . . :dkkAAAFZZ'FdkkAAAF  %%ffooUUD%#33!//))   r3c >[TU]"U4UUUUUUUS.U D6n UcU RSS5n [U[5(aUR S:XaUR RS:XdU Sb"U SRupn U SRnO U SRupU SRnURRUU UR5URRRUUU S9nX:S'U $) N)rrrJrrsrrrr5rrJrIrdr)r(prepare_inputs_for_generationpoprr ndimrZrrGrrrhrgrxr.r)r/rIrrrJrrsrrr model_inputsrYrXrerr0s r1r/Gemma2ForCausalLM.prepare_inputs_for_generations2w<   +)')%)      !  !148A  4 4##q(KK448KKO,81=o1N1T1T. Q%o6==.:;.G.M.M+ %k299!ZZ]] /-AACll))//-%^N.<) *r3)rxrr6) NNNNNNNNNNr)NNNNNTN)rKrLrMrN_tied_weights_keys_tp_plan_pp_planr)rCrGrrrrrrrr,rrr rrrrOrrCrrPrQrRs@r1rwrws*+=)H_-z:;H'(&1515371559-1$(,0/35934P E,,-P !.P u//0 P "+. P   1 12 P ))*P D>P $D>P 'tnP !!1!12P c5<</0P  P P j44r3rwa The Gemma2 Model transformer with a sequence classification head on top (linear layer). [`Gemma2ForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$)Gemma2ForSequenceClassificationi7c>[TU]U5 URUl[U5Ul[ R "URURSS9UlUR5 grW) r(r) num_labelsr3rr*r]r[scorer?rcs r1r)(Gemma2ForSequenceClassification.__init__FsS   ++ ( YYv114??O  r3c.URR$r'r~rHs r1rC4Gemma2ForSequenceClassification.get_input_embeddingsOrr3c$XRlgr'r~rFs r1rG4Gemma2ForSequenceClassification.set_input_embeddingsRrr3rIrrsrrJrrrrKrzc URUUUUUUUU S9n U Rn URU 5n UbURSn OURSn URR cU S:wa [ S5eURR cSnOUbXRR :gRU R[R5n[R"URSU R[RS9nUU-RS5nO.Sn[RURR S35 U [R"XRS 9U4nSnUbUR#XUURS 9n[%UUU R&U R(U R*S 9$) e labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rrsrrJrrrKNrrz=Cannot handle batch sizes > 1 if no padding token is defined.r6)rrz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r)rzr pooled_logitsrZr)rrPrrGrZr5rRrrr,int32rargmaxrrr0rKrrrrxrQ)r/rIrrsrrJrrrrKtransformer_outputsrxrzrXlast_non_pad_token non_pad_mask token_indicesrrs r1rC'Gemma2ForSequenceClassification.forwardUs*8!F!Fr!J !#    >>**+,ZZ  u||J}}MOaab   %%VR_hlhshs%tD/ /??-;;*55   r3)rrrrt)rKrLrMrNr)rCrGrrrr,rrr rrrrCrPrQrRs@r1rr7s'(151537+/59-1$(,0/3A E,,-A !.A u//0 A "% A   1 12 A ))*A D>A $D>A 'tnA  *A A r3rc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$)Gemma2ForTokenClassificationic>[TU]U5 URUl[U5Ul[ USS5b UR nO[ USS5b URnOSn[R"U5Ul [R"URUR5Ul UR5 g)Nclassifier_dropouthidden_dropoutg?)r(r)rr3rrrrr*Dropoutrr]r[rr?)r/rZrr0s r1r)%Gemma2ForTokenClassification.__init__s   ++ ( 6/ 6 B!'!:!:  V-t 4 @!'!6!6 !$ zz"45 YYv1163D3DE  r3c.URR$r'r~rHs r1rC1Gemma2ForTokenClassification.get_input_embeddingsrr3c$XRlgr'r~rFs r1rG1Gemma2ForTokenClassification.set_input_embeddingsrr3rIrrsrrJrrrrKrzc  URUUUUUUUU S9n U Rn URU 5n URU 5n Sn UbUR XUR 5n [ U U U RU RS9$)rrN)rrzrxrQ) rrPrrrrZrrxrQ)r/rIrrsrrJrrrrKrsequence_outputrzrs r1rC$Gemma2ForTokenClassification.forwards*,0:: )%+'/!5,6 , "33,,7O,  %%fdkkBD$!//))   r3)rrrrrt)rKrLrMrNr)rCrGrrrr,rrr rrrrCrPrQrRs@r1rrs '(151537+/59-1$(,0/3* E,,-* !.* u//0 * "% *   1 12 * ))** D>* $D>* 'tn*  * * r3r)rwr3rrr)Nr)rNN)D functoolsrtypingrrrrr,torch.nnr* activationsr cache_utilsr r r generationr modeling_flash_attention_utilsrmodeling_outputsrrrrmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrrrutils.deprecationrconfiguration_gemma2r!torch.nn.attention.flex_attentionrintegrations.flex_attentionr get_loggerrKrModuler"rTrlrwrrOrr@rrrrrr3rwrr__all__rr3r1rs9,33 !::)B LF&\\0. !!;J   H %=BII=(   (6 UU\\ U# U%,, U$## % II % << %  % << % U\\* %  %e_ %e_ % 5<< %& %FW)biiW)tUUp