fTh SSKrSSKJr SSKJr SSKJr SSKJrJ r J r J r SSK r SSK Jr SSKJr SSKJrJrJr SS KJr SS KJr SS KJrJr SS KJrJr SS KJ r J!r! SSK"J#r# SSK$J%r%J&r&J'r'J(r(J)r)J*r* SSK+J,r, SSK-J.r. SSK/J0r0J1r1 \("5(a SSK2J3r3 SSK4J5r5 \*Rl"\75r8\"SS\55r9\"SS\%55r:"SS\Rv5r<"SS\Rz5r>"SS \Rz5r?"S!S"\Rz5r@S#rASFS$jrBS%\ RS&\DS'\ R4S(jrESGS)\RzS*\ RS+\ RS,\ RS-\ \ RS.\FS/\ \FS0\ \FS'\ \ R\ R44S1jjrG"S2S3\Rz5rH"S4S5\Rz5rI\&"S6S7\!55rJ\&"S8S9\J55rK\&"S:S;\J\55rL"S<S=\Rz5rM\&"S>S?9"S@SA\J55rN\&"SBS?9"SCSD\J\55rO/SEQrPg)HN)Callable) dataclass)partial)ListOptionalTupleUnion)ACT2FN)Cache HybridCache StaticCache)GenerationMixin)FlashAttentionKwargs)BaseModelOutputWithPastCausalLMOutputWithPast)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputauto_docstringcan_return_tupleis_torch_flex_attn_availableis_torchdynamo_compilinglogging)deprecate_kwarg) AutoModel) Gemma3ConfigGemma3TextConfig) BlockMask)make_flex_block_causal_maskcB\rSrSr%SrSr\\R\ S'Sr g)Gemma3ModelOutputWithPast=a Base class for Gemma3 outputs, with hidden states and attentions. Args: last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nimage_hidden_states) __name__ __module__ __qualname____firstlineno____doc__r)rtorch FloatTensor__annotations____static_attributes__r*b/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/gemma3/modeling_gemma3.pyr'r'=s88<%"3"34;r4r'c&\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \ \R\4\ S'Sr\\\R\ S'Sr\\\R\ S'Sr\\R\ S 'S rg) Gemma3CausalLMOutputWithPast^aB Base class for Gemma3 causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.text_config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder after projecting last hidden state. Nlosslogitspast_key_values hidden_states attentionsr)r*)r+r,r-r.r/r9rr0r1r2r:r;r rr r<rr=r)r3r*r4r5r7r7^s<)-D(5$$ %,*.FHU&& '.GKOXeD):):$;U$BCDK8[TU]XU5 URS[R"U5SS9 g)NrDF persistent)super__init__register_bufferr0tensor)selfrArBrCrD __class__s r5rI&Gemma3TextScaledWordEmbedding.__init__s1  D ]ELL,ERWXr4 input_idsc>[TU]U5URRURR 5-$N)rHforwardrDtoweightdtype)rLrOrMs r5rR%Gemma3TextScaledWordEmbedding.forwards2wy)D,<,<,?,? @Q@Q,RRRr4r*)?) r+r,r-r.r/intfloatrIr0TensorrRr3 __classcell__rMs@r5r?r?sMYsY3YSY_dYYSSSr4r?c6^\rSrSrS\4U4SjjrSrSrU=r$) Gemma3MLPconfigc>[TU]5 XlURUlURUl[ R "URURSS9Ul[ R "URURSS9Ul[ R "URURSS9Ul [URUl gNFbias) rHrIr` hidden_sizeintermediate_sizennLinear gate_projup_proj down_projr hidden_activationact_fnrLr`rMs r5rIGemma3MLP.__init__s  !--!'!9!94#3#3T5K5KRWXyy!1!143I3IPUV 4#9#94;K;KRWXV556 r4cURURURU55URU5-5nU$rQ)rkrmrirj)rLxrks r5rRGemma3MLP.forwards6NN4;;t~~a/@#ADLLQRO#ST r4)rmr`rkrirerfrj) r+r,r-r.r#rIrRr3r[r\s@r5r^r^s7/7r4r^cJ^\rSrSrS S\S\4U4SjjjrSrSrSr Sr U=r $) Gemma3RMSNormdimepsc>[TU]5 X l[R"[ R "U55UlgrQ)rHrIrwrg Parameterr0zerosrT)rLrvrwrMs r5rIGemma3RMSNorm.__init__s, ll5;;s#34 r4cU[R"URS5RSSS9UR-5-$)NrT)keepdim)r0rsqrtpowmeanrw)rLrqs r5_normGemma3RMSNorm._norms45;;quuQx}}R}>IJJJr4cURUR55nUSURR5--nURU5$)NrW)rrYrTtype_as)rLrqoutputs r5rRGemma3RMSNorm.forwardsCAGGI&3!2!2!445~~a  r4c^[URR5SUR3$)Nz, eps=)tuplerTshaperwrLs r5 extra_reprGemma3RMSNorm.extra_reprs' ))*+6$((<) r+r,r-r.rXrYrIrrRrr3r[r\s@r5rtrts05C5e55 K!==r4rtcl^\rSrSrSS\4U4Sjjjr\R"5\S55r Sr U=r $)Gemma3RotaryEmbeddingr`c>[TU]5 [US5(aHURb;URR SURR S55UlOSUlUR UlUR UlXl [UR Ul URURU5uo0l URSUSS9 URUlg)N rope_scaling rope_typetypedefaultinv_freqFrF)rHrIhasattrrgetrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr`r rope_init_fnattention_scalingrJroriginal_inv_freq)rLr`devicerrMs r5rIGemma3RotaryEmbedding.__init__s  6> * *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E'E!((--[`J`ahhmmfk ^^ U C&,,.1F1L1L1NNYYZ[]^_E))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs $BF  F.)rr`rrrrrrQ) r+r,r-r.r#rIr0no_gradrrRr3r[r\s@r5rrs7////" ]]_ <