fTh_rSrSSKJr SSKJrJrJrJr SSKrSSK rSSKJ r SSK J r J r Jr SSKJr SS KJr SS KJr SS KJr SS KJr SS KJrJrJrJrJrJr SSK J!r! SSK"J#r# \RH"\%5r&\"SS\55r'\"SS\55r("SS\ RR5r*\"SS\55r+\"SS9"SS\+55r,"SS\\5r-\"S S9"S!S"\+\55r./S#Qr/g)$zPyTorch PaliGemmamodel.) dataclass)ListOptionalTupleUnionN)nn)Cache HybridCache StaticCache)GenerationMixin)FlashAttentionKwargs)BaseModelOutputWithPast)PreTrainedModel)Unpack) LossKwargs ModelOutputauto_docstringcan_return_tupleis_torchdynamo_compilinglogging) AutoModel)PaliGemmaConfigcB\rSrSr%SrSr\\R\ S'Sr g)PaligemmaModelOutputWithPast&a Base class for Paligemma outputs, with hidden states and attentions. Args: last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nimage_hidden_states) __name__ __module__ __qualname____firstlineno____doc__rrtorch FloatTensor__annotations____static_attributes__r h/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/paligemma/modeling_paligemma.pyrr&s88<%"3"34;r*rc&\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \ \R\4\ S'Sr\\\R\ S'Sr\\\R\ S'Sr\\R\ S 'S rg) PaliGemmaCausalLMOutputWithPastGaE Base class for PaliGemma causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.text_config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder after projecting last hidden state. Nlosslogitspast_key_values hidden_states attentionsrr )r!r"r#r$r%r/rr&r'r(r0r1rrr r2rr3rr)r r*r+r-r-Gs<)-D(5$$ %,*.FHU&& '.GKOXeD):):$;U$BCDK8[TU]5 [R"URR URR SS9Ulg)NTbias)super__init__rLinear vision_config hidden_sizeprojection_dimlinearselfr7 __class__s r+r<%PaliGemmaMultiModalProjector.__init__ps; ii 4 4 @ @&BVBVBeBelpq r*c(URU5nU$NrA)rCimage_featuresr2s r+forward$PaliGemmaMultiModalProjector.forwardts N3 r*rH) r!r"r#r$rr<rJr) __classcell__rDs@r+r5r5osrrr*r5cH\rSrSr\rSrSrS/rSr Sr Sr Sr Sr SrSrSrSrg) PaliGemmaPreTrainedModelzTr5r1cb[URSURR5R5n[ U[ R 5(aWURRRSUS9 URb%URRR5 ggg)Ninitializer_range)meanstd) getattrr7get_text_configrS isinstancerr=weightdatanormal_r:zero_)rCmodulerVs r+ _init_weights&PaliGemmaPreTrainedModel._init_weightssdkk#6 8S8S8U8g8gh fbii ( ( MM   & &CS & 9{{&   &&(' )r*r N)r!r"r#r$r config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_cache_class_supports_quantized_cache_supports_static_cache_supports_flash_attn_2_supports_sdpa_supports_attention_backendr_r)r r*r+rOrOzsJ"L&*#78"3  $!!N"&)r*rOz{ The Base Paligemma model which consists of a vision backbone and a language model withou language modeling head., ) custom_introc#^\rSrSrSS0rS\4U4SjjrSrSrSS\ \ 4S jjr S \ R4S jr\\SS \ R"S \ RS \ \ R$S\ \ R"S\ \\\ R\4S\ \ R"S\ \ R"S\ \ RS\ \ R"S\ \ S\ \ S\ \ S\ \ S\\S\\\44Sjj55rSrU=r$)PaliGemmaModelzlanguage_model.modellanguage_modelr7c>[TU]U5 [R"URS9Ul[ U5UlURRUl [R"URS9nX l URRbURROSUl UR5 g)N)r7)r;r<r from_configr> vision_towerr5multi_modal_projector text_config vocab_sizerpr7 pad_token_id post_init)rCr7rprDs r+r<PaliGemmaModel.__init__s  %119M9MN%A&%I" ,,77"..f6H6HI,8< 8P8P8\DKK44bd r*c6URR5$rG)rpget_input_embeddingsrCs r+r|#PaliGemmaModel.get_input_embeddingss""7799r*c:URRU5 grG)rpset_input_embeddingsrCvalues r+r#PaliGemmaModel.set_input_embeddingss 007r* is_trainingcURRRS:Xa UbSU;aU$gUbUO URn[ U[ 5n[ R"UR5RnUcUnURSSupU(aUR5n O_[ U[5(aUR5n O9[ U[ R5(aURSO USU -S-n UbUR5S:XaU$[ R"X4XRUR S9n U S:wa(U(a[ R""U SS 9n O SU SS2SU 24'U [ R$"XR S 9UR'SS5:-n U SSSS2SS24R)U SSS5n UbU R+5n URSn U(aeUc [-S 5eU SS2SS2SS2SU 24R/USS2SSSS24R1U R 5S:HS5U SS2SS2SS2SU 24'U SS2SS2SS2SU 24USS2SSSS24R1U R 5-nUS:HnU SS2SS2SS2SU 24R/X5U SS2SS2SS2SU 24'U $) Nflash_attention_2rTrrrrr fill_valuedtypedevicediagonalrz/Token type ids must be provided during training)r7rv_attn_implementationtrainingrYr r&finforminshapeget_max_cache_shaper Tensordimfullrtriuarangereshapeexpandclone ValueError masked_fillto)rCattention_masktoken_type_idsr1cache_position input_tensorrusing_static_cache min_dtypeinputs_lead_dimsequence_length target_length causal_mask mask_length padding_masks r+_update_causal_mask"PaliGemmaModel._update_causal_masks ;; " " 7 7;N N)c^.C%%%0%!*C! !jj  ,**]k]r]r  a #jjqA 36 A///0u||M:O:OPSaSiSijlnoSppp !$a"23::?ArSUV  %%++-K(..r2K!)$%VWW5@Aq,;,AV5W5c5c"1dD!#3477 8J8JKqPRS6 Aq!\k\12 'q!Q  '<=qRVX\^_O_@`@c@cdodvdv@wwL'1,L1