fTh8m*SSKJr SSKJrJrJrJr SSKrSSKJr SSK J r SSK J r SSK Jr SS KJr SS KJrJr SS KJr SS KJr SS KJrJrJrJr SSKJr SSKJ r \"S5"SS\RB55r""SS\RB5r#"SS\RB5r$\"SS\55r%\"SS\55r&\"SS\55r'\"SS 9"S!S"\'55r("S#S$\\5r)\"S%S 9"S&S'\'\ 55r*/S(Qr+g))) dataclass)ListOptionalTupleUnionN)nn)ACT2FN)GenerationMixin)use_kernel_forward_from_hub)FlashAttentionKwargs)BaseModelOutputWithPast ModelOutput)PreTrainedModel)Unpack) LossKwargsauto_docstringcan_return_tupleis_torchdynamo_compiling) AutoModel)Mistral3ConfigRMSNormc8^\rSrSrSU4SjjrSrSrSrU=r$)Mistral3RMSNorm-c>[TU]5 [R"[R "U55UlX lg)z. Mistral3RMSNorm is equivalent to T5LayerNorm N)super__init__r Parametertorchonesweightvariance_epsilon)self hidden_sizeeps __class__s f/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/mistral3/modeling_mistral3.pyr Mistral3RMSNorm.__init__/s/ ll5::k#:; #cURnUR[R5nUR S5R SSS9nU[R "X0R-5-nURURU5-$)NrT)keepdim) dtypetor"float32powmeanrsqrtr%r$)r& hidden_states input_dtypevariances r*forwardMistral3RMSNorm.forward7sw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r,c^[URR5SUR3$)Nz, eps=)tupler$shaper%r&s r* extra_reprMistral3RMSNorm.extra_repr>s* ))*+6$2G2G1HIIr,)r%r$)gư>) __name__ __module__ __qualname____firstlineno__r r9r?__static_attributes__ __classcell__r)s@r*rr-s$;JJr,rc^\rSrSrSrS\4U4SjjrS\RS\RS\R4Sjr S r U=r $) Mistral3PatchMergerBz4 Learned merging of spatial_merge_size ** 2 patches configc>[TU]5 XlURRnUR UlURRR Ul[R"X R S--USS9Ul g)NrFbias) rr rK vision_configr'spatial_merge_size patch_sizerLinear merging_layer)r&rKr'r)s r*r Mistral3PatchMerger.__init__Gsn  **66 "(";";++33>>YY{5L5La5O'OQ\chir,image_features image_sizesreturncUVs/sH&o3SUR-USUR-4PM( nnUVVs/sH upEXE-PM nnnURSn/n[URU55HupX)upEU R XEU5R SSS5R S5n [RRRXRURS9n U R XpRS--S5R5n URU 5 M [R"USS9nURU5nU$s snfs snnf)Nrrr.r) kernel_sizestridedim)rQr= enumeratesplitviewpermute unsqueezer"r functionalunfoldrPtappendcatrS) r&rUrV image_sizehwtokens_per_imagedpermuted_tensor image_index image_tokens image_gridgrids r*r9Mistral3PatchMerger.forwardPs[cn cnU_]doo -z!}/O Pcn  /::kdaAEk:   $)2>3G3GHX3Y)Z %K+DA%**13;;Aq!DNNqQJ88&&--(?(?H_H_.D99Q!8!8!!;;R@BBDD  " "4 (*[?:++N;) ;s -EE!)rKrSrQrP) rArBrCrD__doc__rr r"Tensorr9rErFrGs@r*rIrIBsDj~jellRWR^R^r,rIcj^\rSrSrS\4U4SjjrS\RS\R4SjrSr U=r $)Mistral3MultiModalProjectorhrKc>[TU]5 [URRUR R S9Ul[U5Ul [UR[5(aSO[UR5n[R"URRU-UR RUR S9Ul[$UR&Ul[R"UR RUR RUR S9Ulg)N)r(rrM)rr rrOr' text_config rms_norm_epsnormrI patch_merger isinstancevision_feature_layerintlenrrRmultimodal_projector_biaslinear_1r projector_hidden_actactlinear_2)r&rKnum_feature_layersr)s r*r $Mistral3MultiModalProjector.__init__is #F$8$8$D$D&J\J\JiJij /7",V-H-H#"N"NQTWX^XsXsTt  , ,/A A    * *11  &556    * *F,>,>,J,JQWQqQq  r,rUrVcURU5nURX5nURU5nURU5nUR U5nU$N)rzr{rrr)r&rUrVr6s r*r9#Mistral3MultiModalProjector.forwardysP>2**>G n5 /  m4 r,)rrrrzr{) rArBrCrDrr r"rsr9rErFrGs@r*ruruhs/ ~ ellr,ruc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \R\ S'Sr \\\R\ S'Sr\\\R\ S'Sr\\R\ S 'S rg) Mistral3CausalLMOutputWithPasta@ Base class for Mistral3 causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nlosslogitspast_key_valuesr6 attentionsimage_hidden_states)rArBrCrDrrrrr" FloatTensor__annotations__rrrr6rrrrErr,r*rrs<)-D(5$$ %,*.FHU&& '.9=OXd5#4#456=8