fThSSKJrJrJrJr SSKrSSKJr SSKJr SSK J r J r J r J r SSKJr SSKJr SS KJr SS KJr SS KJr SS KJrJrJrJrJr SS KJrJ r SSK!J"r"J#r# SSK$J%r% SSK&J'r'J(r(J)r)J*r*J+r+ SSK,J-r- \*"5(a SSK.J/r/ SSK0J1r1 \+Rd"\35r4\"S5"SS\Rj55r6"SS\Rj5r7Sr8S?Sjr9S\RtS\;S\Rt4Sjr<S@S \RjS!\RtS"\RtS#\RtS$\\RtS%\=S&\=4S'jjr>"S(S)\Rj5r?"S*S+\5r@\("S,S-\#55rA"S.S/\Rj5rB\("S0S1\A55rC"S2S3\\'5rD\("S4S5\A\55rE\("S6S79"S8S9\A55rF\("S:S;\A55rG\("S<S=\A55rH/S>QrIg)A)CallableOptionalTupleUnionN)nn)ACT2FN)Cache DynamicCacheSlidingWindowCache StaticCache)GenerationMixin)use_kernel_forward_from_hub)AttentionMaskConverter)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPastTokenClassifierOutput)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) LossKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging) Qwen3Config) BlockMask)make_flex_block_causal_maskRMSNormc8^\rSrSrSU4SjjrSrSrSrU=r$) Qwen3RMSNorm9c>[TU]5 [R"[R "U55UlX lg)z+ Qwen3RMSNorm is equivalent to T5LayerNorm N)super__init__r Parametertorchonesweightvariance_epsilon)self hidden_sizeeps __class__s `/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/qwen3/modeling_qwen3.pyr,Qwen3RMSNorm.__init__;s/ ll5::k#:; #cURnUR[R5nUR S5R SSS9nU[R "X0R-5-nURURU5-$)NT)keepdim) dtypetor.float32powmeanrsqrtr1r0)r2 hidden_states input_dtypevariances r6forwardQwen3RMSNorm.forwardCsw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r8c^[URR5SUR3$)Nz, eps=)tupler0shaper1r2s r6 extra_reprQwen3RMSNorm.extra_reprJs* ))*+6$2G2G1HIIr8)r1r0)gư>) __name__ __module__ __qualname____firstlineno__r,rFrL__static_attributes__ __classcell__r5s@r6r(r(9s$;JJr8r(c.^\rSrSrU4SjrSrSrU=r$)Qwen3MLPNc>[TU]5 XlURUlURUl[ R "URURSS9Ul[ R "URURSS9Ul[ R "URURSS9Ul [URUl gNFbias) r+r,configr3intermediate_sizerLinear gate_projup_proj down_projr hidden_actact_fnr2r\r5s r6r,Qwen3MLP.__init__Os  !--!'!9!94#3#3T5K5KRWXyy!1!143I3IPUV 4#9#94;K;KRWXV../ r8cURURURU55URU5-5nU$N)rarcr_r`)r2xras r6rFQwen3MLP.forwardYs6NN4;;t~~a/@#ADLLQRO#ST r8)rcr\rar_r3r]r`)rNrOrPrQr,rFrRrSrTs@r6rVrVNs0r8rVcUSSURSS-24nUSURSS-S24n[R"U*U4SS9$)z*Rotates half the hidden dims of the input..Nr;r:dim)rJr.cat)rhx1x2s r6 rotate_halfrp^sZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r8cURU5nURU5nX-[U5U--nX-[U5U--nXg4$)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezerp)qkcossin position_ids unsqueeze_dimq_embedk_embeds r6apply_rotary_pos_embr{esS( -- &C -- &Cw;q>C/0Gw;q>C/0G  r8rCn_repreturncURup#pEUS:XaU$USS2SS2SSS2SS24RX#XU5nURX#U-XE5$)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) r"N)rJexpandreshape)rCr|batchnum_key_value_headsslenhead_dims r6 repeat_kvrs_ 2?1D1D.E z!!Qa"23::5W\dlmM  e(CT TTr8modulequerykeyvalueattention_maskscalingdropoutc @[X R5n[X0R5n [R"XR SS55U-n Ub"USS2SS2SS2SUR S24n X-n [ RRU S[RS9RUR5n [ RRXURS9n [R"X5n U R SS5R5n X4$)Nr:rr;)rlr=)ptrainingr")rnum_key_value_groupsr.matmul transposerJr functionalsoftmaxr?r>r=rr contiguous) rrrrrrrkwargs key_states value_states attn_weights causal_mask attn_outputs r6eager_attention_forwardrs3 ; ;[TU]5 XlX l[ USUR UR -5UlUR UR-Ul URS-Ul URUl SUl [R"UR UR UR-URS9Ul[R"UR URUR-URS9Ul[R"UR URUR-URS9Ul[R"UR UR-UR URS9Ul[)URUR*S9Ul[)URUR*S9UlUR0UlURR2(a<[ URSS5b$URURR4:dSUlgg)NrgTrZr4sliding_window)r+r,r\rgetattrr3num_attention_headsrrrrattention_dropout is_causalrr^attention_biasq_projk_projv_projo_projr( rms_norm_epsq_normk_normruse_sliding_windowmax_window_layersr2r\rr5s r6r,Qwen3Attention.__init__s  " F4F4F&JdJd4de $*$>$>&B\B\$\!}}d* !'!9!9ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii  & & 68J8JQWQfQf  #4==f6I6IJ "4==f6I6IJ $33 KK * * %5t<H$++"?"??"&D @r8rCposition_embeddingsrpast_key_valuecache_positionrr}c URSSn/UQSPURP7nURURU5R U55R SS5n UR URU5R U55R SS5n URU5R U5R SS5n Uup[XX5upUb$XUS.nURXURU5up[nURRS:wadURRS:Xa-URSS5(a[ R#S 5 O[$URRnU"UU U U U4UR&(dS O UR(UR*UR,S .UD6unnUR."/UQSP76R15nUR3U5nUU4$) Nr;r"r:)rvrureagersdpaoutput_attentionsFz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)rrr)rJrrrviewrrrrr{updaterrr\_attn_implementationgetlogger warning_oncerrrrrrrr)r2rCrrrrr input_shape hidden_shape query_statesrrrurv cache_kwargsattention_interfacerrs r6rFQwen3Attention.forwards$))#2. 88b8$--8 {{4;;}#=#B#B<#PQ[[\]_`a [[]!;!@!@!NOYYZ[]^_ {{=166|DNNqRST &#7 RU#[  %#&nUL'5'<'d>d##L '>dkk>^>^&_#$7      % $}}C$2H2HLL.. %  % ! \"));;;;FFH kk+. L((r8)rr\rrrrrrrrrrrr)NN)rNrOrPrQ__doc__r#intr,r.Tensorrrr LongTensorrrrFrRrSrTs@r6rrsG'{'s'J+/59 0)||0)#5<<#=>0)!. 0) ! 0) !!1!12 0)-.0) u||Xell3XeELL>Q5RR S0)0)r8rc^\rSrSrS\S\4U4SjjrSS\RS\ \RS\ \RS\ \ S \ \ S \ \ S \ \RS \ \ \R\R4S \\S\ \R \ \ \R \R 444SjjrSrU=r$)Qwen3DecoderLayerr\rc>[TU]5 URUl[XS9Ul[ U5Ul[URURS9Ul [URURS9Ul UR(a5URS:wa$[RSURS35 ggg)N)r\rrflash_attention_2z=Sliding Window Attention is enabled but not implemented for `z)`; unexpected results may be encountered.)r+r,r3r self_attnrVmlpr(rinput_layernormpost_attention_layernormrrrrrs r6r,Qwen3DecoderLayer.__init__s !--'vKF#+F,>,>FDWDWX(4V5G5GVM`M`(a%  ! !f&A&AEX&X   OPVPkPkOlm99 'Y !r8rCrrwrr use_cacherrrr}c Un URU5nUR"SUUUUUUUUS.U D6upX-nUn URU5nURU5nX-nU4n U(aX4- n U $)N)rCrrwrrrrr)rrrr) r2rCrrwrrrrrrresidualself_attn_weightsoutputss r6rFQwen3DecoderLayer.forward s!,,]; ,0>> , ')%)/) 3 ,  , ( !0 !55mD /  0  "  + +Gr8)r3rrrr)NNNFFNN)rNrOrPrQr#rr,r.rrrr boolrrr FloatTensorrFrRrSrTs@r6rrs { s $2637*.,1$)59KO'||'!.'u//0 ' ! ' $D> 'D>'!!1!12'&eELL%,,,F&GH'-.' u  (51B1BEDUDU1U+V"WW X''r8rcN\rSrSr\rSrSrS/rS/r Sr Sr Sr Sr SrSrSrSrSrg) Qwen3PreTrainedModeli6modelTrpast_key_valuescURRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[5(a&UR R RS5 gg)Nr)rAstdg?)r\initializer_range isinstancerr^r0datanormal_r[zero_ Embedding padding_idxr(fill_)r2rrs r6 _init_weights"Qwen3PreTrainedModel._init_weightsEskk++ fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>.  - - MM   $ $S ).r8rN)rNrOrPrQr# config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cache_supports_attention_backendrrRrr8r6rr6sSL&*#,-#4"5!N  $!"& *r8rcl^\rSrSrSS\4U4Sjjjr\R"5\S55r Sr U=r $)Qwen3RotaryEmbeddingiSr\c>[TU]5 [US5(aHURb;URR SURR S55UlOSUlUR UlUR UlXl [UR Ul URURU5uo0l URSUSS9 URUlg)N rope_scaling rope_typetypedefaultinv_freqF) persistent)r+r,hasattrrrrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr\r rope_init_fnattention_scalingregister_bufferroriginal_inv_freq)r2r\devicerr5s r6r,Qwen3RotaryEmbedding.__init__Ts  6> * *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+r rrstrr.autocastrrmrur rvr=) r2rhrwinv_freq_expandedposition_ids_expandedrfreqsembrurvs r6rFQwen3RotaryEmbedding.forwardesR!MM$4-8>>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E'E!((--[`J`ahhmmfk ^^ U C&,,.1F1L1L1NNYYZ[]^_E))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs $BF  F.)r r\rr rr rrg) rNrOrPrQr#r,r.no_gradrrFrRrSrTs@r6rrSs6/{//" ]]_ < [TU]U5 URUlURUl[ R "URURUR5Ul[ R"[UR5Vs/sHn[X5PM sn5Ul [URURS9Ul[#US9UlSUlUR)5 gs snf)Nr)r\F)r+r, pad_token_idr vocab_sizerrr3 embed_tokens ModuleListrangenum_hidden_layersrlayersr(rnormr rotary_embgradient_checkpointing post_initrs r6r,Qwen3Model.__init__ws  !.. ++LL):):F??ab b   --i8M 0*nO  !CRC^==?de "\\ ]5H5H5K"KTaThThN  )33A6L.. >L] & #oomJ#7BD0d![[)H4;;+H+HIM#!%55!) *)."3#-$7 $ M*!,M  =#3"55'J* -0  !1 1 &+/8Od+%   r8r$ input_tensorc URRS:Xa]UbNUbKUSS2S4R5R5UR 5S:gnU(a [ S5eUbSU;aU$gURRS:Xa,[ U[R5(a [U5nU$UbUR5OSn[ U[5n[ U[5n URRS:XaQU(dJU (dCU(d<[R"UUUURRUR S9(agUR"n [R$"U 5R&n UR(S n U (dU(aUR+5n O5[ U[R5(aUR(SOX|-S -n UR-UU U U UUR(SURUS 9nURRS:Xa:Ub7UR.R0S ;aU(d[R2"X5nU$) Nrr;rzYou are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Qwen3. Make sure to call `tokenizer.padding_side = 'left'` before tokenizing the input. rflex_attentionr)r5past_key_values_lengthr is_trainingr")sequence_length target_lengthr=r batch_sizer\r)cudaxpunpu)r\rsumitemsizer<rr.rr%r=r r r_ignore_causal_mask_sdparrr=finfominrJget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionr r_unmask_unattended)r2rrFrrris_padding_rightr@using_static_cacheusing_sliding_window_cacher= min_dtyperKrLrs r6r?Qwen3Model._update_causal_masks; ;; + +/B B)o.I#1!R%#8#<#<#>#C#C#EIZIZI\]^I_#_ #$a )c^.C%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`a'E%/AS%T" KK , , 6'+E%%>>*'7#{{99 MM ""KK&** &,,Q/ %);+??AM nell;;$$R(%7!; PP +')#))!,;;+Q   KK , , 6*%%**.DD% 1CCK[Kr8rKrLr=rMcUbUR5S:XaUnU$[R"U5Rn [R"X4XUR S9n[R "X$R S9URSS5:n UR5n [U SS5(avU Rbi[U[5(aX:aO[R "X$R S9URSS5U R- :*n U RU 5 X-nUSSSS2SS24RUSSS5nUbUR5nUR SU:a USS2SU24nUR Sn USS2SS2SS2SU 24USS2SSSS24R#UR 5-nUS :HnUSS2SS2SS2SU 24R%X5USS2SS2SS2SU 24'U$) aQ Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. config (`Qwen3Config`): The model's configuration class past_key_values (`Cache`): The cache class that is being used currently to generate N) fill_valuer=r r9r;r"rTr)rlr.rUrVfullr r>rget_text_configrrrr bitwise_or_rclonerJr> masked_fill)rrKrLr=rrMr\rrr]diagonal_attend_mask text_configsliding_attend_mask mask_length padding_masks r6rX@Qwen3Model._prepare_4d_causal_attention_mask_with_cache_positionCsB  %.*<*<*>!*C(K@= E*..I** 0Y\j\q\qK$)<< F[F[#\_m_u_uA`$ !002K{$8$??KD^D^Dj"/3EFF/Ji*/,,}MbMb*c&..r15 8R8RR+')445HI  /K%dD!Q&67>>z1bRTUK))//1 !''+m;%3A~ ~4E%FN,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r8)r"r)r&r'rr(r! NNNNNNNNN)F)rNrOrPrQr#r,r.r2rrrr.rrr rrrrrrFrr? staticmethodrr=rXrRrSrTs@r6rrus{ !"151537+/59$(,0/359\ E,,-\ !.\ u//0 \ "% \   1 12 \ D>\ $D>\ 'tn\ !!1!12\ $$89\  !\ \ H#( TellK78TllT T  T  TlB BBB{{ B  B  BBBBr8rc\rSrSrSrg)KwargsForCausalLMirN)rNrOrPrQrRrr8r6rprps3r8rpc^\rSrSrS/rSS0rSS/S/40rU4SjrSrS r S r S r S r S r \\SS\\R$S\\R&S\\R$S\\S\\R*S\\R$S\\S\\S\\S\\R$S\\\R&4S\\S\4Sjj55rSrU=r$)Qwen3ForCausalLMizlm_head.weightlm_head colwise_reprClogitsc>[TU]U5 [U5UlURUl[ R "URURSS9UlUR5 grY) r+r,rrr!rr^r3rsr*rds r6r,Qwen3ForCausalLM.__init__sU  '  ++yy!3!3V5F5FUS  r8c.URR$rgrr"rKs r6r.%Qwen3ForCausalLM.get_input_embeddingszz&&&r8c$XRlgrgryr1s r6r2%Qwen3ForCausalLM.set_input_embeddings "' r8cUR$rgrsrKs r6get_output_embeddings&Qwen3ForCausalLM.get_output_embeddingss ||r8cXlgrgr)r2new_embeddingss r6set_output_embeddings&Qwen3ForCausalLM.set_output_embeddingss% r8cXlgrgr)r2decoders r6 set_decoderQwen3ForCausalLM.set_decoders r8cUR$rgrrKs r6 get_decoderQwen3ForCausalLM.get_decoders zzr8r4rrwrr5labelsrrr6rlogits_to_keeprr}c UbUOURRnU bU OURRn UR"SUUUUUUUU U S. U D6n U Rn[ U [ 5(a [U *S5OU nURUSS2USS245nSnUb)UR"SUX`RRS.U D6n[UUU RU RU RS9$)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, Qwen3ForCausalLM >>> model = Qwen3ForCausalLM.from_pretrained("Qwen/Qwen3-8B") >>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```N) r4rrwrr5rrr6r)rurr!lossrurrCr;r)r\rr6rr:rrslicers loss_functionr!rrrCr;)r2r4rrwrr5rrrr6rrrrrC slice_indicesrurs r6rFQwen3ForCausalLM.forwardsN2C1N-TXT_T_TqTq$8$D $++JjJj  ,0:: , )%+'/!5) ,  ,  11 8B>SV8W8W~ot4]k mA}a,?@A  %%pVF{{OeOepiopD%#33!//))   r8)rsrr!) NNNNNNNNNNr)rNrOrPrQ_tied_weights_keys_tp_plan_pp_planr,r.r2rrrrrrrr.rrr rrrrrrprrFrRrSrTs@r6rrrrs*+=)H_-z:;H'(&151537+/59-1$(,0/35934G E,,-G !.G u//0 G "% G   1 12 G ))*G D>G $D>G 'tnG !!1!12G c5<</0G *+G  G G r8rra The Qwen3 Model transformer with a sequence classification head on top (linear layer). [`Qwen3ForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$)Qwen3ForSequenceClassificationic>[TU]U5 URUl[U5Ul[ R "URURSS9UlUR5 grY) r+r, num_labelsrrrr^r3scorer*rds r6r,'Qwen3ForSequenceClassification.__init__sS   ++' YYv114??O  r8c.URR$rgryrKs r6r.3Qwen3ForSequenceClassification.get_input_embeddingsr{r8c$XRlgrgryr1s r6r23Qwen3ForSequenceClassification.set_input_embeddingsr~r8r4rrwrr5rrrr6r}c URUUUUUUUU S9n U Rn URU 5n UbURSn OURSn URR cU S:wa [ S5eURR cSnOUbXRR :gRU R[R5n[R"URSU R[RS9nUU-RS5nO.Sn[RURR S35 U [R"XRS 9U4nSnUbUR#XUURS 9n[%UUU R&U R(U R*S 9$) e labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rrwrr5rrr6Nrr"z=Cannot handle batch sizes > 1 if no padding token is defined.r;)r r=z will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r9)rur pooled_logitsr\r)rr:rrJr\r r<r>r r.int32r>argmaxrrr5rNrrrrCr;)r2r4rrwrr5rrrr6transformer_outputsrCrurMlast_non_pad_token non_pad_mask token_indicesrrs r6rF&Qwen3ForSequenceClassification.forwards*8!F!Fr!J !#    >>**+,ZZ  u||J}}MOaab   %%VR_hlhshs%tD/ /??-;;*55   r8)rrrrm)rNrOrPrQr,r.r2rrrr.rrr rrrrFrRrSrTs@r6rrs'(151537+/59-1$(,0/3A E,,-A !.A u//0 A "% A   1 12 A ))*A D>A $D>A 'tnA  *A A r8rc*^\rSrSrU4SjrSrSr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \ RS \ \S \ \S \ \S\4Sjj55rSrU=r$)Qwen3ForTokenClassificationi]c>[TU]U5 URUl[U5Ul[ USS5b UR nO[ USS5b URnOSn[R"U5Ul [R"URUR5Ul UR5 g)Nclassifier_dropouthidden_dropoutg?)r+r,rrrrrrrDropoutrr^r3rr*)r2r\rr5s r6r,$Qwen3ForTokenClassification.__init___s   ++' 6/ 6 B!'!:!:  V-t 4 @!'!6!6 !$ zz"45 YYv1163D3DE  r8c.URR$rgryrKs r6r.0Qwen3ForTokenClassification.get_input_embeddingsor{r8c$XRlgrgryr1s r6r20Qwen3ForTokenClassification.set_input_embeddingsrr~r8r4rrwrr5rrrr6r}c  URUUUUUUUU S9n U Rn URU 5n URU 5n Sn UbUR XUR 5n [ U U U RU RS9$)rrN)rrurCr;) rr:rrrr\rrCr;)r2r4rrwrr5rrrr6rsequence_outputrurs r6rF#Qwen3ForTokenClassification.forwardus*,0:: )%+'/!5,6 , "33,,7O,  %%fdkkBD$!//))   r8)rrrrrm)rNrOrPrQr,r.r2rrrr.rrr rrrrFrRrSrTs@r6rr]s '(151537+/59-1$(,0/3* E,,-* !.* u//0 * "% *   1 12 * ))** D>* $D>* 'tn*  * * r8rcB^\rSrSrSrU4SjrSrSr\\ SS\ \ RS\ \ RS\ \ RS \ \S \ \ RS \ \ RS \ \ RS \ \S\ \S\4Sjj55rSrU=r$)Qwen3ForQuestionAnsweringi transformerc>[TU]U5 [U5Ul[R "UR S5UlUR5 g)Nr:) r+r,rrrr^r3 qa_outputsr*rds r6r,"Qwen3ForQuestionAnswering.__init__sA  %f-))F$6$6: r8c.URR$rgrr"rKs r6r..Qwen3ForQuestionAnswering.get_input_embeddingss,,,r8c$XRlgrgrr1s r6r2.Qwen3ForQuestionAnswering.set_input_embeddingss(-%r8r4rrwrr5start_positions end_positionsrr6r}c URUUUUUUU S9n U Rn URU 5n U RSSS9upUR S5R 5nUR S5R 5nSnUbUbUR "XXg40U D6n[UUUU RU RS9$)N)rrwrr5rr6r"r;rk)r start_logits end_logitsrCr;) rr:rsplitsqueezerrrrCr;)r2r4rrwrr5rrrr6rrrrurrrs r6rF!Qwen3ForQuestionAnswering.forwards,0+;+; )%+'/!5,<, "331#)<<r<#: #++B/::< ''+668   &=+D%%libhiD+%!!//))   r8)rrrm)rNrOrPrQrr,r.r2rrrr.rrr rrrrFrRrSrTs@r6rrs%-.151537+/596:48,0/3( E,,-( !.( u//0 ( "% (   1 12 ( "%"2"23(   0 01( $D>( 'tn(  &( ( r8r)rrrrrrr)Nr")r)Jtypingrrrrr.r activationsr cache_utilsr r r r generationr integrationsrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrrmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrrr r!configuration_qwen3r#!torch.nn.attention.flex_attentionr$integrations.flex_attentionr% get_loggerrNrModuler(rVrpr{rrrrrrrrrrrprrrrr__all__rr8r6rsC,43 !OO)7>B9LF&hh, !!;J   H %Y'J299J(J( ryy  (6 UU\\ U# U%,, U&% II% <<% % << % U\\* %  %%4S)RYYS)l727t*?**8<299i +_i i X  S %9S  S lC "6C C L; 4; ; | r8