fThwSSKJrJrJrJrJr SSKrSSKJr SSKJ r SSK J r J r J r Jr SSKJr SSKJr SS KJr SS KJr SS KJrJrJrJr SS KJrJr SS KJ r J!r! SSK"J#r# SSK$J%r%J&r&J'r'J(r(J)r) SSK*J+r+ \("5(a SSK,J-r- SSK.J/r/ \)R`"\15r2"SS\Rf5r4Sr5S9Sjr6S\RnS\8S\Rn4Sjr9S:S\RfS\RnS\RnS\RnS \\RnS!\:S"\:4S#jjr;"S$S%\Rf5r<"S&S'\5r="S(S)\Rf5r>\&"S*S+\!55r?\&"S,S-\?55r@"S.S/\\%5rA\&"S0S1\?\55rB\&"S2S39"S4S5\?55rC\&"S6S7\?55rD/S8QrEg);)CallableListOptionalTupleUnionN)nn)ACT2FN)Cache DynamicCacheSlidingWindowCache StaticCache)GenerationMixin)AttentionMaskConverter)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPast SequenceClassifierOutputWithPastTokenClassifierOutput)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) LossKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging)Starcoder2Config) BlockMask)make_flex_block_causal_maskcv^\rSrSrS\4U4SjjrS\\\RS\R4Sjr Sr U=r $) Starcoder2MLP<configcD>[TU]5 URn[R"X!R UR S9Ul[R"UR X!R S9Ul[URUl URUl g)Nbias) super__init__ hidden_sizerLinearintermediate_sizeuse_biasc_fcc_projr hidden_actactresidual_dropout)selfr( embed_dim __class__s j/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/starcoder2/modeling_starcoder2.pyr-Starcoder2MLP.__init__=sq && IIi)A)AX ii 8 8)//Z &++, & 7 7 hidden_statesreturncURU5nURU5nURU5n[RR XR URS9nU$)Nptraining)r2r5r3r functionaldropoutr6rB)r7r=s r:forwardStarcoder2MLP.forwardEsX -0 /  M2  --m?T?T_c_l_l-m r<)r5r2r3r6) __name__ __module__ __qualname____firstlineno__r"r-rrtorch FloatTensorrE__static_attributes__ __classcell__r9s@r:r&r&<s>8/8XeE4E4E.F%GEL]L]r<r&cUSSURSS-24nUSURSS-S24n[R"U*U4SS9$)z*Rotates half the hidden dims of the input..Ndim)shaperKcat)xx1x2s r: rotate_halfrZMsZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r<cURU5nURU5nX-[U5U--nX-[U5U--nXg4$)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezerZ)qkcossin position_ids unsqueeze_dimq_embedk_embeds r:apply_rotary_pos_embreTsS( -- &C -- &Cw;q>C/0Gw;q>C/0G  r<r=n_repr>cURup#pEUS:XaU$USS2SS2SSS2SS24RX#XU5nURX#U-XE5$)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) r!N)rUexpandreshape)r=rfbatchnum_key_value_headsslenhead_dims r: repeat_kvrnos_ 2?1D1D.E z!!Qa"23::5W\dlmM  e(CT TTr<modulequerykeyvalueattention_maskscalingrDc @[X R5n[X0R5n [R"XR SS55U-n Ub"USS2SS2SS2SUR S24n X-n [ RRU S[RS9RUR5n [ RRXURS9n [R"X5n U R SS5R5n X4$)NrRr rQ)rTdtyper@r!)rnnum_key_value_groupsrKmatmul transposerUrrCsoftmaxfloat32torwrDrB contiguous) rorprqrrrsrtrDkwargs key_states value_states attn_weights causal_mask attn_outputs r:eager_attention_forwardr{s3 ; ;[TU]5 XlX l[ USS5=(d UR UR -UlUR UR-Ul URS-Ul URUl SUl [R"UR UR UR-URS9Ul[R"UR URUR-URS9Ul[R"UR URUR-URS9Ul[R"UR UR-UR URS9UlUR(Ulg)NrmgTr*)r,r-r(rgetattrr.num_attention_headsrmrkrxrtattention_dropout is_causalrr/r1q_projk_projv_projo_projr6r7r(rr9s r:r-Starcoder2Attention.__init__sT  " D9mV=O=OSYSmSm=m $*$>$>&B\B\$\!}}d* !'!9!9ii 2 2F4N4NQUQ^Q^4^eketetu ii 2 2F4N4NQUQ^Q^4^eketetu ii 2 2F4N4NQUQ^Q^4^eketetu ii : :T]] JFL^L^eketetu & 7 7r<r=position_embeddingsrspast_key_valuecache_positionrr>c TURSSn/UQSPURP7nURU5RU5R SS5n UR U5RU5R SS5n UR U5RU5R SS5n Uup[XX5upUb$XUS.nURXURU5up[nURRS:wadURRS:Xa-URSS5(a[RS 5 O[ URRnU"UU U U U4UR"(dS O UR$UR&[)URS S5S .UD6unnUR*"/UQSP76R-5nUR/U5n[0R2R5UUR6UR"S 9nUU4$)NrQr!rR)r`r_reagersdpaoutput_attentionsFz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.sliding_window)rDrtrr@)rUrmrviewrzrrreupdaterrr(_attn_implementationgetlogger warning_oncerrBrrtrrir~rrrCrDr6)r7r=rrsrrr input_shape hidden_shape query_statesrrr_r` cache_kwargsattention_interfacerrs r:rEStarcoder2Attention.forwards$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &#7 RU#[  %#&nUL'5'<'d>d##L '>dkk>^>^&_#$7      % $}}C$2H2HLL"4;;0@$G %  % ! \"));;;;FFH kk+. mm++ 4004==, L((r<) rr(rmrrrrxrrr6rtrN)NN)rGrHrIrJ__doc__r"rintr-rKTensorrr LongTensorrrrErMrNrOs@r:rrsG 8/ 8HSM 8 8(+/59 4)||4)#5<<#=>4)!. 4) ! 4) !!1!12 4)-.4) u||Xell3XeELL>Q5RR S4)4)r<rc^\rSrSrS\S\4U4SjjrSS\RS\ \RS\ \RS\ \ S \ \ S \ \ S \ \RS \ \ \R\R4S \\S\ \R \ \ \R \R 444SjjrSrU=r$)Starcoder2DecoderLayerr(rc8>[TU]5 URUl[XS9Ul[ U5Ul[R"URURS9Ul [R"URURS9Ul g)N)r(reps) r,r-r.r self_attnr&mlpr LayerNorm norm_epsiloninput_layernormpost_attention_layernormrs r:r-Starcoder2DecoderLayer.__init__sr !--,FP (!||F,>,>FDWDWX(* V5G5GVM`M`(a%r<r=rsrarr use_cacherrrr>c Un URU5nUR"SUUUUUUUUS.U D6upX-nUn URU5nURU5nX-nU4n U(aX4- n U $)N)r=rsrarrrrr)rrrr) r7r=rsrarrrrrrresidualself_attn_weightsoutputss r:rEStarcoder2DecoderLayer.forwards!,,]; ,0>> , ')%)/) 3 ,  , ( !0 !55mD /  0  "  + +Gr<)r.rrrr)NNNFFNN)rGrHrIrJr"rr-rKrrrr boolrrrrLrErMrNrOs@r:rrsb/bCb2637*.,1$)59KO'||'!.'u//0 ' ! ' $D> 'D>'!!1!12'&eELL%,,,F&GH'-.' u  (51B1BEDUDU1U+V"WW X''r<rcl^\rSrSrSS\4U4Sjjjr\R"5\S55r Sr U=r $)Starcoder2RotaryEmbeddingir(c>[TU]5 [US5(aHURb;URR SURR S55UlOSUlUR UlUR UlXl [UR Ul URURU5uo0l URSUSS9 URUlg)N rope_scaling rope_typetypedefaultinv_freqF) persistent)r,r-hasattrrrrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr(r rope_init_fnattention_scalingregister_bufferroriginal_inv_freq)r7r(devicerr9s r:r-"Starcoder2RotaryEmbedding.__init__s  6> * *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E'E!((--[`J`ahhmmfk ^^ U C&,,.1F1L1L1NNYYZ[]^_E))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs $BF  F.)rr(rrrrrr) rGrHrIrJr"r-rKno_gradrrErMrNrOs@r:rrs7////" ]]_ < .  - - MM   $ $S ) KK   " " $.r<rN)rGrHrIrJr" config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cache_supports_attention_backendrrMrr<r:rr3sS#L&*#12#4"5!N  $!"& %r<rc:^\rSrSrS\4U4SjjrSrSr\\ SS\ \ RS\ \ RS\ \ RS \ \\\\ R"4S \ \ R"S \ \S \ \S \ \S\ \ RS\\S\4Sjj55rSS\\ RS4S\ RS\ RS \S \4 Sjjr\S\ RS\S\S\ R4S\ RS\S\S \4Sj5rSrU=r$)Starcoder2ModeliQr(c @>[TU]U5 URUlURUl[ R "URURUR5Ul[ R"[UR5Vs/sHn[X5PM sn5Ul [ R"URURS9Ul[#US9UlSUlUR(UlUR+5 gs snf)Nr)r(F)r,r- pad_token_idr vocab_sizerrr. embed_tokens ModuleListrangenum_hidden_layersrlayersrrnormr rotary_embgradient_checkpointingembedding_dropout post_initrs r:r-Starcoder2Model.__init__Ss  !.. ++LL):):Fc XUbUOURRnUbUOURRnUbUOURRnUSLUSL- (a [ S5eUR (a/UR (aU(a[RS5 SnUcURU5nU(a Uc [5nU cDUbUR5OSn [R"XURS-URS9n UcU R!S5nUR#X%XU5n Un [$R&R)XR*UR S9n UR-X5nU(aSOSnU(aSOSnUR.SURR0H7nU(aX4- nU"U 4U UUUUU US .U D6nUSn U(dM.UUS4- nM9 UR3U 5n U(aX4- n[5U U(aUOSUUS 9$) Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.Frr!rr@r)rsrarrrrr)last_hidden_staterr= attentions)r(rrr ValueErrorrrBrrrr get_seq_lengthrKarangerUrr\_update_causal_maskrrCrDrrr r r r)r7rrsrarrrrrrrpast_seen_tokensrr=rall_hidden_statesall_self_attns decoder_layer layer_outputss r:rEStarcoder2Model.forwardjsM2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKL] &  -- 33dmm. #oomJ#7BD0d![[)H4;;+H+HIM#!%55!) *)."3#-$7 $ M*!,M  =#3"55'J* -0  !1 1 &+/8Od+%   r<r# input_tensorc URRS:Xa]UbNUbKUSS2S4R5R5UR 5S:gnU(a [ S5eUbSU;aU$gURRS:Xa,[ U[R5(a [U5nU$UbUR5OSn[ U[5n[ U[5n URRS:XaQU(dJU (dCU(d<[R"UUUURRUR S9(agUR"n [R$"U 5R&n UR(S n U (dU(aUR+5n O5[ U[R5(aUR(SOX|-S -n UR-UU U U UUR(SURUS 9nURRS:Xa:Ub7UR.R0S ;aU(d[R2"X5nU$) Nflash_attention_2rQrzYou are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Starcoder2. Make sure to call `tokenizer.padding_side = 'left'` before tokenizing the input. rflex_attentionr)rpast_key_values_lengthr is_trainingr!)sequence_length target_lengthrwr batch_sizer(r)cudaxpunpu)r(rsumitemsizer$rrKrr$r%rr r_ignore_causal_mask_sdparrBrwfinfominrUget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionrr_unmask_unattended)r7rsr.rrris_padding_rightr(using_static_cacheusing_sliding_window_cacherw min_dtyper4r5rs r:r'#Starcoder2Model._update_causal_masks; ;; + +/B B)o.I#1!R%#8#<#<#>#C#C#EIZIZI\]^I_#_ #$a )c^.C%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`a'E%/AS%T" KK , , 6'+E%%>>*'7#{{99 MM ""KK&** &,,Q/ %);+??AM nell;;$$R(%7!; PP +')#))!,;;+Q   KK , , 6*%%**.DD% 1CCK[Kr<r4r5rwr6cUbUR5S:XaUnU$[R"U5Rn [R"X4XUR S9n[R "X$R S9URSS5:n UR5n [U SS5(avU Rbi[U[5(aX:aO[R "X$R S9URSS5U R- :*n U RU 5 X-nUSSSS2SS24RUSSS5nUbUR5nUR SU:a USS2SU24nUR Sn USS2SS2SS2SU 24USS2SSSS24R#UR 5-nUS :HnUSS2SS2SS2SU 24R%X5USS2SS2SS2SU 24'U$) aV Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. config (`Starcoder2Config`): The model's configuration class past_key_values (`Cache`): The cache class that is being used currently to generate N) fill_valuerwrr!rQr!use_sliding_windowTr)rTrKr>r?fullrr&riget_text_configrrrr bitwise_or_rhclonerUr} masked_fill)rsr4r5rwrr6r(rrrFdiagonal_attend_mask text_configsliding_attend_mask mask_length padding_masks r:rAEStarcoder2Model._prepare_4d_causal_attention_mask_with_cache_positionsB  %.*<*<*>!*C(K@= E*..I** 0Y\j\q\qK$)<< F[F[#\_m_u_uA`$ !002K{$8$??KD^D^Dj"/3EFF/Ji*/,,}MbMb*c&..r15 8R8RR+')445HI  /K%dD!Q&67>>z1bRTUK))//1 !''+m;%3A~ ~4E%FN,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r<)rrrr r rrr NNNNNNNNN)F)rGrHrIrJr"r-rrrrrrKrrrr rrLrrrrrEr' staticmethodrrwrArMrNrOs@r:rrQs/"!"151537KO59$(,0/359[ E,,-[ !.[ u//0 [ "%tE4E4E/F(F"GH [   1 12 [ D>[ $D>[ 'tn[ !!1!12[ $$89[  ![ [ F#( TellK78TllT T  T  TlB BBB{{ B  B  B!BBBr<rc\rSrSrSrg)KwargsForCausalLMierN)rGrHrIrJrMrr<r:rZrZes3r<rZc^\rSrSrS/rSS0rSS/S/40rU4SjrSrS r S r S r S r S r \\SS\\R$S\\R&S\\R$S\\S\\R*S\\R$S\\S\\S\\S\\R$S\\\R&4S\\S\4Sjj55rSrU=r$)Starcoder2ForCausalLMihzlm_head.weightlm_head colwise_repr=logitsc>[TU]U5 [U5UlURUl[ R "URURSS9UlUR5 gNFr*) r,r-rrrrr/r.r]rr7r(r9s r:r-Starcoder2ForCausalLM.__init__nsU  $V,  ++yy!3!3V5F5FUS  r<c.URR$rrrrs r:r*Starcoder2ForCausalLM.get_input_embeddingswzz&&&r<c$XRlgrrers r:r*Starcoder2ForCausalLM.set_input_embeddingsz "' r<cUR$rr]rs r:get_output_embeddings+Starcoder2ForCausalLM.get_output_embeddings}s ||r<cXlgrrl)r7new_embeddingss r:set_output_embeddings+Starcoder2ForCausalLM.set_output_embeddingss% r<cXlgrr)r7decoders r: set_decoder!Starcoder2ForCausalLM.set_decoders r<cUR$rrtrs r: get_decoder!Starcoder2ForCausalLM.get_decoders zzr<rrsrarrlabelsrrrrlogits_to_keeprr>c UbUOURRnU bU OURRn UR"SUUUUUUUU U S. U D6n U Rn[ U [ 5(a [U *S5OU nURUSS2USS245nSnUb)UR"SUX`RRS.U D6n[UUU RU RU RS9$)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, Starcoder2ForCausalLM >>> model = Starcoder2ForCausalLM.from_pretrained("meta-starcoder2/Starcoder2-2-7b-hf") >>> tokenizer = AutoTokenizer.from_pretrained("meta-starcoder2/Starcoder2-2-7b-hf") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```N) rrsrarrrrrr)r_r{rlossr_rr=r#r)r(rrrr"rrslicer] loss_functionrrrr=r#)r7rrsrarrr{rrrrr|rrr= slice_indicesr_rs r:rEStarcoder2ForCausalLM.forwardsN2C1N-TXT_T_TqTq$8$D $++JjJj  ,0:: , )%+'/!5) ,  ,  11 8B>SV8W8W~ot4]k mA}a,?@A  %%pVF{{OeOepiopD%#33!//))   r<)r]rr) NNNNNNNNNNr)rGrHrIrJ_tied_weights_keys_tp_plan_pp_planr-rrrmrqrvryrrrrKrrr rLrrrrrZrrErMrNrOs@r:r\r\hs*+=)H_-z:;H'(&151537+/59-1$(,0/35934G E,,-G !.G u//0 G "% G   1 12 G ))*G D>G $D>G 'tnG !!1!12G c5<</0G *+G  G G r<r\a The Starcoder2 Model transformer with a sequence classification head on top (linear layer). [`Starcoder2ForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$)#Starcoder2ForSequenceClassificationic>[TU]U5 URUl[U5Ul[ R "URURSS9UlUR5 gra) r,r- num_labelsrrrr/r.scorerrbs r:r-,Starcoder2ForSequenceClassification.__init__sS   ++$V, YYv114??O  r<c.URR$rrers r:r8Starcoder2ForSequenceClassification.get_input_embeddingsrgr<c$XRlgrrers r:r8Starcoder2ForSequenceClassification.set_input_embeddingsrjr<rrsrarrr{rrrr>c URUUUUUUUU S9n U Rn URU 5n UbURSn OURSn URR cU S:wa [ S5eURR cSnOUbXRR :gRU R[R5n[R"URSU R[RS9nUU-RS5nO.Sn[RURR S35 U [R"XRS 9U4nSnUbUR#XUURS 9n[%UUU R&U R(U R*S 9$) e labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rsrarrrrrNrr!z=Cannot handle batch sizes > 1 if no padding token is defined.rQ)rrwz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r!)r_r{ pooled_logitsr(r~)rr"rrUr(rr$r}rrKint32r&argmaxrrr9rGrrrr=r#)r7rrsrarrr{rrrtransformer_outputsr=r_r6last_non_pad_token non_pad_mask token_indicesrrs r:rE+Starcoder2ForSequenceClassification.forwards*8!F!Fr!J !#    >>**+,ZZ  u||J}}MOaab   %%VR_hlhshs%tD/ /??-;;*55   r<)rrrrW)rGrHrIrJr-rrrrrrKrrr rLrrrErMrNrOs@r:rrs'(151537+/59-1$(,0/3A E,,-A !.A u//0 A "% A   1 12 A ))*A D>A $D>A 'tnA  *A A r<rc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$) Starcoder2ForTokenClassificationi9c>[TU]U5 URUl[U5Ul[ USS5b UR nO[ USS5b URnOSn[R"U5Ul [R"URUR5Ul UR5 g)Nclassifier_dropouthidden_dropoutg?)r,r-rrrrrrrDropoutrDr/r.rr)r7r(rr9s r:r-)Starcoder2ForTokenClassification.__init__;s   ++$V, 6/ 6 B!'!:!:  V-t 4 @!'!6!6 !$ zz"45 YYv1163D3DE  r<c.URR$rrers r:r5Starcoder2ForTokenClassification.get_input_embeddingsKrgr<c$XRlgrrers r:r5Starcoder2ForTokenClassification.set_input_embeddingsNrjr<rrsrarrr{rrrr>c  URUUUUUUUU S9n U Rn URU 5n URU 5n Sn UbUR XUR 5n [ U U U RU RS9$)rrN)rr_r=r#) rr"rDrrr(rr=r#)r7rrsrarrr{rrrrsequence_outputr_rs r:rE(Starcoder2ForTokenClassification.forwardQs*,0:: )%+'/!5,6 , "33,,7O,  %%fdkkBD$!//))   r<)rDrrrrW)rGrHrIrJr-rrrrrrKrrr rLrrrErMrNrOs@r:rr9s '(151537+/59-1$(,0/3* E,,-* !.* u//0 * "% *   1 12 * ))** D>* $D>* 'tn*  * * r<r)r\rrrr)Nr!)r)FtypingrrrrrrKr activationsr cache_utilsr r r r generationrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrrrr configuration_starcoder2r"!torch.nn.attention.flex_attentionr#integrations.flex_attentionr$ get_loggerrGrModuler&rZrerrrnrrrrrrrrZr\rr__all__rr<r:rs6:9 !OO)>B9 LF&hh6 !!;J   H %BII"(6 UU\\ U# U%,, U&% II% <<% % << % U\\* %  %%4F)"))F)R070f< i 5i i X  S *CS  S lC '@C C L r<