fThbbSSKJr SSKJrJrJrJr SSKrSSKJr SSK J r SSK J r SSK JrJr SS KJr SS KJrJrJr S S KJr S SKJr \"SS\55r\"SS\55r"SS\R85r\"SS\55r\"SS9"SS\55r\"SS9"SS\\ 55r /SQr!g)) dataclass)ListOptionalTupleUnionN)nn)ACT2FN)GenerationMixin)BaseModelOutputWithPast ModelOutput)PreTrainedModel)auto_docstringcan_return_tupleis_torchdynamo_compiling) AutoModel)VipLlavaConfigcB\rSrSr%SrSr\\R\ S'Sr g)VipLlavaModelOutputWithPast%a Base class for VipLlava outputs, with hidden states and attentions. Args: last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nimage_hidden_states) __name__ __module__ __qualname____firstlineno____doc__rrtorch FloatTensor__annotations____static_attributes__rf/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/vipllava/modeling_vipllava.pyrr%s88<%"3"34;r$rc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \R\ S'Sr \\\R\ S'Sr\\\R\ S'Sr\\R\ S 'S rg) VipLlavaCausalLMOutputWithPastFa@ Base class for VipLlava causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nlosslogitspast_key_values hidden_states attentionsrr)rrrrrr)rr r!r"r*r+rr,rr-rr#rr$r%r'r'Fs<)-D(5$$ %,*.FHU&& '.9=OXd5#4#456=8[TU]5 [UR[5(aSO[ UR5n[ R"X!RR-URS9Ul [ R"X!RR-URRSS9Ul[UR Ul[ R"URRURRSS9Ulg)Nr)epsTbias)super__init__ isinstancevision_feature_layersintlenr LayerNorm vision_config hidden_sizeprojector_layernorm_epsprojector_layernormLinear text_configlinear_1r projector_hidden_actactlinear_2)selfr1num_feature_layers __class__s r%r7$VipLlavaMultiModalProjector.__init__os ",V-I-I3"O"OQUXY_YuYuUv#%<< !5!5!A!A AvGeGe$   !5!5!A!A A    * *  &556 &"4"4"@"@&BTBTB`B`gkl r$cURU5nURU5nURU5nURU5nU$N)r@rCrErF)rGr,s r%forward#VipLlavaMultiModalProjector.forward~sB00?  m4 /  m4 r$)rErCrFr@) rrrrrr7rMr# __classcell__rIs@r%r/r/ns m~ mr$r/cB\rSrSr\rSrSrSrSr Sr Sr Sr Sr SrSrSrg)VipLlavaPreTrainedModelTr+c4[URSURR5R5n[ U[ R 5(aWURRRSUS9 URb%URRR5 gg[ U[ R5(aJURRRS5 URRR5 gg)Ninitializer_rangeg)meanstdg?)getattrr1get_text_configrVr8rrAweightdatanormal_r5zero_r<fill_)rGmodulerXs r% _init_weights%VipLlavaPreTrainedModel._init_weightssdkk#6 8S8S8U8g8gh fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   $ $S ) KK   " " $.r$rN)rrrrr config_classbase_model_prefixsupports_gradient_checkpointing_skip_keys_device_placement_supports_cache_class_supports_flash_attn_2_supports_sdpa_supports_quantized_cache_supports_static_cache_supports_attention_backendrar#rr$r%rRrRsA!L&*#"3 !N $!"& %r$rRzx The VipLlava model which consists of a vision backbone and a language model, without a language modeling head. ) custom_introc^\rSrSrSS0rS\4U4SjjrSrSrS\ RS \ \ \ \ 44S jr\SS \ R S\ RS \\ R$S \\ R S\\ \ RS\\ RS \\ \ \ \ 4S\\S\\S\\S\\S\\ R S\ \\44Sjj5rSrU=r$) VipLlavaModelzlanguage_model.modellanguage_modelr1c>[TU]U5 [R"UR5Ul[ U5Ul[R"UR5Ul UR5 grL) r6r7r from_configr= vision_towerr/multi_modal_projectorrBrq post_initrGr1rIs r%r7VipLlavaModel.__init__sY  %11&2F2FG%@%H"'33F4F4FG r$c6URR5$rL)rqget_input_embeddingsrGs r%rz"VipLlavaModel.get_input_embeddingss""7799r$c:URRU5 grL)rqset_input_embeddingsrGvalues r%r~"VipLlavaModel.set_input_embeddingss 007r$ pixel_valuesr9c&URUSS9n[U[5(aURUSS2SS24nO