fThSSKrSSKrSSKJr SSKJrJrJrJrJ r SSK r SSK r SSK Jr SSKJs Jr SSK JrJrJr SSKJr SSKJr SSKJr SS KJr SS KJrJrJ r SS K!J"r"J#r# SS K$J%r%J&r&J'r'J(r( S SK)J*r*J+r+J,r, \(RZ"\.5r/\"SS\%55r0\"SS\%55r1\"SS\%55r2"SS\Rf5r4SIS\RfS\ RjS\ RjS\ RjS\\ RjS\6S\64Sjjr7"SS \Rf5r8"S!S"\Rf5r9"S#S$\5r:"S%S&\Rf5r;"S'S(\Rf5r<"S)S*\Rf5r=S+r>SJS,\ RjS-\6S.\6S/\6S0\6S1\ Rj4 S2jjr?SKS3jr@S4rAS5rB"S6S7\Rf5rC\&"S8S9\#55rD\&"S:S;9"S<S=\D55rE"S>S?\Rf5rF\&"S@S;9"SASB\D55rG\&"SCSD\D55rH\&"SES;9"SFSG\D55rI/SHQrJg)LN) dataclass)AnyCallableOptionalTupleUnion)BCEWithLogitsLossCrossEntropyLossMSELoss)_calculate_fan_in_and_fan_out)ACT2FN)_prepare_4d_attention_mask)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingImageClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel) ModelOutputauto_docstringcan_return_tuplelogging) Siglip2ConfigSiglip2TextConfigSiglip2VisionConfigc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \RS4\ S'Sr \\ \RS4\ S'S rg) Siglip2VisionOutput-a Base class for vision model's outputs that also contains image embeddings of the pooling of the last hidden states. Args: image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The image embeddings obtained by applying the projection layer to the pooler_output. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. N image_embedslast_hidden_state. hidden_states attentions)__name__ __module__ __qualname____firstlineno____doc__r!rtorch FloatTensor__annotations__r"r#rr$__static_attributes__r%d/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/siglip2/modeling_siglip2.pyrr-sr*15L(5,,-459x 1 129=AM8E%"3"3S"89:A:>Ju00#567>r/rc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \RS4\ S'Sr \\ \RS4\ S'S rg) Siglip2TextOutputJa Base class for text model's outputs that also contains a pooling of the last hidden states. Args: text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The text embeddings obtained by applying the projection layer to the pooler_output. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. N text_embedsr".r#r$r%)r&r'r(r)r*r4rr+r,r-r"r#rr$r.r%r/r0r2r2Jsr*04K%++,359x 1 129=AM8E%"3"3S"89:A:>Ju00#567>r/r2c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr\\ S 'Sr\\ S 'S \\4S jrS rg) Siglip2Outputga Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `return_loss` is `True`): Contrastive loss for image-text similarity. logits_per_image (`torch.FloatTensor` of shape `(image_batch_size, text_batch_size)`): The scaled dot product scores between `image_embeds` and `text_embeds`. This represents the image-text similarity scores. logits_per_text (`torch.FloatTensor` of shape `(text_batch_size, image_batch_size)`): The scaled dot product scores between `text_embeds` and `image_embeds`. This represents the text-image similarity scores. text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`Siglip2TextModel`]. image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`Siglip2VisionModel`]. text_model_output (`BaseModelOutputWithPooling`): The output of the [`Siglip2TextModel`]. vision_model_output (`BaseModelOutputWithPooling`): The output of the [`Siglip2VisionModel`]. Nlosslogits_per_imagelogits_per_textr4r!text_model_outputvision_model_outputreturncJ^[U4SjTR555$)Nc3n># UH*nUS;aTUO[TU5R5v M, g7f))r;r<N)getattrto_tuple).0kselfs r0 )Siglip2Output.to_tuple..s<   LLDGRYZ^`aRbRkRkRm m s25)tuplekeysrDs`r0rASiglip2Output.to_tuples# YY[   r/r%)r&r'r(r)r*r8rr+r,r-r9r:r4r!r;rr<rrrAr.r%r/r0r6r6gs()-D(5$$ %,48hu001837OXe//07/3K%++,304L(5,,-448186:3: %* r/r6c ^\rSrSrS\4U4Sjjr\S\RS\RS\ S\R4Sj5r S \RS\RS\R4S jr S rU=r$) Siglip2VisionEmbeddingsconfigc>[TU]5 XlURUlUR Ul[ R"URUR -UR -URS9Ul URUl [URS-5Ul [ R"URUR5Ulg)N) in_features out_featuresg?)super__init__rN hidden_size embed_dim patch_sizennLinear num_channelspatch_embedding num_patchesintposition_embedding_size Embeddingposition_embeddingrDrN __class__s r0rS Siglip2VisionEmbeddings.__init__s  ++ ++!yy++doo=O  "--'*4+;+;S+@'A$"$,,t/?/?"Pr/positional_embeddingsspatial_shapes max_lengthr=c NURSnURSnURn[R"X2U4URUS9nUR SSS5R S5nURRS:XaUR[R5n[U5HnnXup[R"UX4SSS S 9n U RXHU -5RSS5n U RU5n XUS X-24'U SXgX-S 24'Mp U$) a  Resize positional embeddings to image-specific size and pad to a fixed size. Args: positional_embeddings (`torch.Tensor`): Position embeddings of shape (height, width, embed_dim) spatial_shapes (`torch.LongTensor`): Spatial shapes of shape (batch_size, 2) to resize the positional embeddings to max_length (`int`): Maximum length of the positional embeddings to pad resized positional embeddings to Returns: `torch.Tensor`: Embeddings of shape (batch_size, max_length, embed_dim) r)devicedtypercpubilinearFT)sizemode align_corners antialiasN)shaperir+emptyrhpermute unsqueezetypetofloat32rangeF interpolatereshape transpose) rcrdre batch_sizerU source_dtyperesulted_positional_embeddingsiheightwidthresized_embeddingss r0resize_positional_embeddings4Siglip2VisionEmbeddings.resize_positional_embeddingssI($))!, )//3 ,22 ). Y /(//* &!6 = =aA F P PQR S ! ' ' , , 5$9$<$.>+> ?BTUVBW *fn.>+> ?%#(.-r/ pixel_valuesc:URRRnURURUS95nURRR UR UR S5nURXRURSS9nXF-nU$)a Args: pixel_values (`torch.FloatTensor`): Pixel values of shape (batch_size, max_num_patches, num_channels * patch_size * patch_size) spatial_shapes (`List[Tuple[int, int]]`): Spatial shapes of shape (batch_size, 2) to resize the positional embeddings to )rirgr)re) rZweightrirvr_r{r]rrq)rDrrd target_dtype patch_embedsrcresized_positional_embeddings embeddingss r0forwardSiglip2VisionEmbeddings.forwards++2288 ++LOO,O,OP !% 7 7 > > F F  ( ($*F*F! )-(I(I !l>P>PQR>S)J) % "A r/)rNrUr[rZrVr_r])r&r'r(r)rrS staticmethodr+Tensor LongTensorr\rr,rr. __classcell__ras@r0rLrLs Q2 Q8.$||8.((8.8.  8.8.tE$5$5uGWGW\a\h\hr/rLmodulequerykeyvalueattention_maskscalingdropoutc [R"XRSS55U-nUbX-n[RR US[R S9RUR5n[RRXURS9n[R"X5n U RSS5R5n X4$)Nrg)dimri)ptrainingrrj) r+matmulr|rW functionalsoftmaxrwrvrirr contiguous) rrrrrrrkwargs attn_weights attn_outputs r0eager_attention_forwardrs<<}}R'<=GL!#4 ==((2U]](SVVW\WbWbcL==((6??([L,,|3K''1-88:K  $$r/c ^\rSrSrSrS\\\44U4SjjrS S\ RS\ \ RS\ \ S\ \ R\ \ R44S jjrS rU=r$) Siglip2Attentioniz=Multi-headed attention from 'Attention Is All You Need' paperrNc >[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl SUl [R"URUR5Ul[R"URUR5Ul[R"URUR5Ul[R"URUR5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).F)rRrSrNrTrUnum_attention_heads num_headshead_dim ValueErrorscaleattention_dropoutr is_causalrWrXk_projv_projq_projout_projr`s r0rSSiglip2Attention.__init__ s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( // ii? ii? ii?  $..$..A r/r#routput_attentionsr=c URupEnURU5nURU5nURU5n UR XEUR UR 5RSS5nUR XEUR UR 5RSS5nU R XEUR UR 5RSS5n [n URRS:waTURRS:XaU(a[RS5 O[URRn U "UUUU UURURUR (dSO UR"S9upU R%XEU5R'5n UR)U 5n U(dSn X4$) z#Input shape: Batch x Time x Channelrrjeagersdpaz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)rrrN)rqrrrviewrrr|rrN_attn_implementationlogger warning_oncerrrrrr{rr) rDr#rrr} seq_lengthrUqueriesrHvaluesattention_interfacerrs r0rSiglip2Attention.forwards-:,?,?) ++m,{{=)]+,,zt~~t}}U__`acdeyyOYYZ[]^_ZT^^T]]S]]^_abc(? ;; + +w 6{{//69>O##L '>dkk>^>^&_#$7     nnJJ#}}C$,, % ! "))*)LWWY mmK0  L((r/) rNrrUrrrrrrrrNF)r&r'r(r)r*rrrrSr+rrboolrrr.rrs@r0rrsGBu%8:K%KLB.26,1 -)||-)!.-)$D> -) u||Xell33 4 -)-)r/rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $) Siglip2MLPiOc>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gN) rRrSrNr hidden_act activation_fnrWrXrTintermediate_sizefc1fc2r`s r0rSSiglip2MLP.__init__Psb  #F$5$5699V//1I1IJ99V55v7I7IJr/r#r=clURU5nURU5nURU5nU$r)rrr)rDr#s r0rSiglip2MLP.forwardWs4/ **=9 / r/)rrNrr) r&r'r(r)rSr+rrr.rrs@r0rrOs)KU\\ellr/rc ^\rSrSrS\\\44U4SjjrS S\RS\RS\ \ S\ \R4SjjrS rU=r$) Siglip2EncoderLayeri^rNc<>[TU]5 URUl[R "URUR S9Ul[U5Ul [R "URUR S9Ul [U5Ul g)Neps) rRrSrTrUrW LayerNormlayer_norm_eps layer_norm1r self_attn layer_norm2rmlpr`s r0rSSiglip2EncoderLayer.__init___sm ++<<F $ u  ! $$r/rc ^\rSrSrSrS\4U4Sjjr\S S\\ RS\\ S\\ S\ 4S jj5r S rU=r$) Siglip2Encoderiz Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`Siglip2EncoderLayer`]. Args: config: Siglip2Config rNc>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snfr) rRrSrNrW ModuleListrxnum_hidden_layersrlayersgradient_checkpointing)rDrN_ras r0rSSiglip2Encoder.__init__sT  mm%PVPhPhJi$jJiQ%8%@Ji$jk &+#%ksA&rroutput_hidden_statesr=cFUbUOURRnUbUOURRnU(aSOSnU(aSOSnUnURH-nU(aXW4-nU"UUUS9n U SnU(dM%XiS4-nM/ U(aXW4-n[ UUUS9$)a Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. Nr%)rrr)r"r#r$)rNrrrr) rD inputs_embedsrrrencoder_statesall_attentionsr# encoder_layer layer_outputss r0rSiglip2Encoder.forwards<2C1N-TXT_T_TqTq$8$D $++JjJj  40d% ![[M#!/2B!B)"3M *!,M  !/3C2E!E) +.>>N+(%  r/)rNrrNNN)r&r'r(r)r*rrSrrr+rrrrr.rrs@r0rrsl,},26,0/3 < !.< $D> < 'tn <  < < r/rc^\rSrSrS\4U4Sjjr\\S S\RS\RS\RS\ \ S\ \ S \4 S jj55rS rU=r$) Siglip2VisionTransformerirNc~>[TU]5 XlURn[ U5Ul[ U5Ul[R"X!RS9Ul [US5(dSO URUlUR(a[U5UlUR"S:HUlg)Nrvision_use_headTflash_attention_2)rRrSrNrTrLrrencoderrWrrpost_layernormhasattrruse_head$Siglip2MultiheadAttentionPoolingHeadheadr_use_flash_attention_2rDrNrUras r0rS!Siglip2VisionTransformer.__init__s  && 1&9%f-  ll9:O:OP$+F4E$F$FFLbLb == * 'tn *  $* * r/rc ^\rSrSrS\4U4SjjrS S\\RS\\RS\\RS\R4Sjjr S r U=r $) Siglip2TextEmbeddingsirNcN>[TU]5 URn[R"UR U5Ul[R"URU5UlURS[R"UR5RS5SS9 g)N position_ids)rrgF) persistent) rRrSrTrWr^ vocab_sizetoken_embeddingmax_position_embeddingsr_register_bufferr+arangeexpandrs r0rSSiglip2TextEmbeddings.__init__s && !||F,=,=yI"$,,v/M/My"Y  ELL)G)GHOOPWXej  r/ input_idsrrr=c<UbURSOURSnURRRSnXE:a[SUSU35eUcURSS2SU24nUcUR U5nURU5nX6-nU$)NrgrrzRSequence length must be less than max_position_embeddings (got `sequence length`: z and max_position_embeddings: )rqr_rrrr)rDrrrrmax_position_embeddingposition_embeddingsrs r0rSiglip2TextEmbeddings.forward(s -6,AY__R(}GZGZ[]G^ !%!8!8!?!?!E!Ea!H  .d,<=S.norm_cdfFs(dhhq499S>122c99r/rjzjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.) stacklevelrr )minmax) warningswarnuniform_erfinv_mul_r!r#add_clamp_)tensormeanstdabr%lus r0_trunc_normal_r8Cs: 1s7{1s7{ 2  ;  !(c!"A!(c!"A OOAEAIq1uqy) NN KKdiin$% KK MMaMr/r1r2r3r4r5r=c[R"5 [USSX45 URU5R U5 SSS5 g!,(df  g=f)a=Fills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}( ext{mean}, ext{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq ext{mean} \leq b`. NOTE: this 'tf' variant behaves closer to Tensorflow / JAX impl where the bounds [a, b] are applied when sampling the normal distribution with mean=0, std=1.0 and the result is subsequently scaled and shifted by the mean and std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value rrN)r+no_gradr8r.r/)r1r2r3r4r5s r0trunc_normal_tf_r;gs<* vq#q, Cd# s /A AcF[U5upEUS:XaUnOUS:XaUnO US:XaXE-S- nUW- nUS:Xa"[U[R"U5S- S9 gUS:XaB[R "5 UR [R"U5S9 SSS5 gUS :XaK[R"S U-5n[R "5 URU*U5 SSS5 g[S U35e!,(df  g=f!,(df  g=f) Nfan_infan_outfan_avgrjtruncated_normalg۶%?r3normaluniformr zinvalid distribution ) r r;r!r#r+r:normal_r,r) r1rrn distributionr=r>denomvariancebounds r0variance_scaling_rIs3F;OF x    !Q&u}H))TYYx%8;N%NO  ! ]]_ NNtyy2N 3_  " !h,' ]]_ OOUFE *_0?@@__s5$DD D D c[USSS9 g)Nr=r@rnrErIr1s r0 lecun_normal_rNsf8:LMr/c[USSS9 g)Nr=rBrKrLrMs r0default_flax_embed_initrPsf8(Cr/c^\rSrSrS\4U4Sjjr\\S S\\ RS\\ RS\\ RS\\ S\\ S \ 4 S jj55r S rU=r$) Siglip2TextTransformerirNc>>[TU]5 XlURn[ U5Ul[ U5Ul[R"X!RS9Ul [R"X!R5UlURS:HUlg)Nrr)rRrSrNrTr rrrrWrrfinal_layer_normrXprojection_sizerrrrs r0rSSiglip2TextTransformer.__init__sw  && /7%f- " Y . 'tn .  $. . r/rRc6\rSrSr\rSrSr/SQrSr Sr Sr Sr g)Siglip2PreTrainedModelisiglip2T)r rrLrrcV [U[5(a[UR[5(a URRR OURR n[ RRURRS[R"U5- S9 g[U[ R5(a[UR5 g[U[5(Ga[ RR!UR"R5 [ RR!UR$R5 [ RR!UR&R5 [ RR!UR(R5 [ RR+UR"R,5 [ RR+UR$R,5 [ RR+UR&R,5 [ RR+UR(R,5 g[U[.5(a[ RR!UR0R5 [ RR!UR2R5 [ RRUR0R,SS9 [ RRUR2R,SS9 g[U[45(a[ RR!UR6R85 [ RR!UR:R<R85 [ RR+UR:R>R85 g[U[@5(at[BRD"[BRF"S55nURHR8RKU5 URLR8RO5 g[U[P5(ak[ RRURRRURRR S-URRT-S9 g[U[ RV[ RX45(aM[[UR5 UR,b*[ RR+UR,5 gg[U[ R\5(aJUR,R8RO5 URR8RKS5 gg)zInitialize the weightsrrAgư>rrN)/ isinstancerLrNr vision_configrTrWinitrDr_rnpr#r^rPrxavier_uniform_rrrrzeros_biasrrrrprobedata attentionin_proj_weight in_proj_bias Siglip2Modelr+logr1 logit_scalefill_ logit_biaszero_Siglip2ForImageClassification classifierinitializer_factorrXConv2drNr)rDrrlogit_scale_inits r0 _init_weights$Siglip2PreTrainedModel._init_weightss f5 6 6dkk=99 ))55[[,,  GGOOF55<  - -$yyc):;     # # ) )*: ;    " " ( ( *  = > > GGOO!!((KK--994?$++B`B``  BII 6 7 7 &-- ({{&v{{+'  - - KK   " " $ MM   $ $S ).r/r%N) r&r'r(r)r config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_supports_flash_attn_2_supports_sdparwr.r%r/r0r]r]s- L!&*#"N,*r/r]zL The text model from Siglip2 without any head or projection on top. ) custom_introc^\rSrSr\rS\4U4SjjrS\R4Sjr Sr \ \ SS\ \RS\ \RS \ \RS \ \S \ \S\4 S jj55rS rU=r$)Siglip2TextModeli rNcd>[TU]U5 [U5UlUR 5 gr)rRrSrR text_model post_initr`s r0rSSiglip2TextModel.__init__(s&  08 r/r=cBURRR$rrrrrIs r0get_input_embeddings%Siglip2TextModel.get_input_embeddings.s))999r/c8XRRlgrr)rDrs r0set_input_embeddings%Siglip2TextModel.set_input_embeddings1s5:""2r/rrrrrc(URUUUUUS9$)aX Examples: ```python >>> from transformers import AutoTokenizer, Siglip2TextModel >>> model = Siglip2TextModel.from_pretrained("google/siglip2-base-patch16-224") >>> tokenizer = AutoTokenizer.from_pretrained("google/siglip2-base-patch16-224") >>> # important: make sure to set padding="max_length" as that's how the model was trained >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding="max_length", return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled (EOS token) states ```rrrrrr)rDrrrrrs r0rSiglip2TextModel.forward4s)6)%/!5   r/rr[)r&r'r(r)rryrSrWModulerrrrrr+rrrrr.rrs@r0rr s %L0 :bii:;-115/3,0/3  ELL) !. u||,  $D>  'tn   $  r/rc^\rSrSrSrS\4U4SjjrS S\RS\ \RS\R4Sjjr S r U=r $) riXzMultihead Attention Pooling.rNc>[TU]5 [R"[R "SSUR 55Ul[RRUR URSS9Ul [R"UR URS9Ul [U5UlURUlg)NrT) batch_firstr)rRrSrW Parameterr+randnrTrgMultiheadAttentionrrirr layernormrrrr`s r0rS-Siglip2MultiheadAttentionPoolingHead.__init__[s \\%++aF4F4F"GH 44V5G5GIcIcqu4vf&8&8f>S>STf%33r/ hidden_staterr=cURSnURRUSS5nUbcURSURSpe[X!RU5nURSUR US5nUR SXV5nURXAXS9SnUnURU5nXpRU5-nUSS2S4$)Nrrrg) attn_mask) rqrgrepeatrrirr{rirr)rDrrr}rg target_len source_lenrs r0r,Siglip2MultiheadAttentionPoolingHead.forwardds!''*  !!*a3  %%*[[^\5G5G5J 7HZHZ\fgN+221dnnjRSTN+33B ON~~e<~bcde ~~l3 ((<"88 AqD!!r/)rirrrrgr) r&r'r(r)r*rrSr+rrrr.rrs@r0rrXsF&424"ELL"(5<[TU]U5 [U5UlUR 5 gr)rRrSr vision_modelrr`s r0rSSiglip2VisionModel.__init__s)  4V< r/r=cBURRR$r)rrrZrIs r0r'Siglip2VisionModel.get_input_embeddingss  ++;;;r/pixel_attention_maskrdrrc(URUUUUUS9$)a pixel_attention_mask (`torch.Tensor` of shape `(batch_size, image_size, image_size)`, *optional*): Mask to avoid performing attention on padding pixel indices. spatial_shapes (`torch.LongTensor` of shape `(batch_size, 2)`): Tensor containing the spatial dimensions (height, width) of the input images. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, Siglip2VisionModel >>> model = Siglip2VisionModel.from_pretrained("google/siglip2-base-patch16-224") >>> processor = AutoProcessor.from_pretrained("google/siglip2-base-patch16-224") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled features ```rrrdrrr)rDrrrdrrs r0rSiglip2VisionModel.forwards,F  %/)/!5 !  r/rr )r&r'r(r)rrymain_input_namerSrWrrrrr+r,rrrrrrr.rrs@r0rrws 'L$O2 ' 'tn '  $' ' r/rc^\rSrSr\rS\4U4Sjjr\SS\\ RS\\ RS\\ RS\\ S\\ S \ R4 S jj5r \SS \\ RS \\ RS \\ RS\\ S\\ S \ R4 Sjj5r\\SS\\ RS \\ RS \\ RS \\ RS\\ RS\\ RS\\ S\\ S\\ S \4Sjj55rSrU=r$)rlirNc>[TU]U5 [UR[5(d"[ S[ UR5S35e[UR[5(d"[ S[ UR5S35eURnURn[RU5n[RU5nURUl URUl [R"[ R""S55Ul[R"[ R""S55UlUR)5 g)NzNconfig.text_config is expected to be of type Siglip2TextConfig but is of type .zRconfig.vision_config is expected to be of type Siglip2VisionConfig but is of type r)rRrSr` text_configr TypeErrorrurarr _from_configrrrrWrr+rrnrpr)rDrNrrarrras r0rSSiglip2Model.__init__s"  &,,.?@@++,-Q0  &..0CDD--./q2  (( ,, &22;? )66}E %//(55<< A7,,u{{1~6 r/rrrrrr=cUbUOURRnUbUOURRnURUUUUUS9nURnU$)a Returns: text_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`Siglip2TextModel`]. Examples: ```python >>> from transformers import AutoTokenizer, AutoModel >>> import torch >>> model = AutoModel.from_pretrained("google/siglip2-base-patch16-224") >>> tokenizer = AutoTokenizer.from_pretrained("google/siglip2-base-patch16-224") >>> # important: make sure to set padding="max_length" as that's how the model was trained >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding="max_length", return_tensors="pt") >>> with torch.no_grad(): ... text_features = model.get_text_features(**inputs) ```r)rNrrrr)rDrrrrr text_outputsrYs r0get_text_featuresSiglip2Model.get_text_featuressr:2C1N-TXT_T_TqTq$8$D $++JjJj 48??)%/!5 4C4 %22 r/rrrdcUbUOURRnUbUOURRnURUUUUUS9nURnU$)a< pixel_attention_mask (`torch.Tensor` of shape `(batch_size, image_size, image_size)`, *optional*): Mask to avoid performing attention on padding pixel indices. spatial_shapes (`torch.LongTensor` of shape `(batch_size, 2)`): Tensor containing the spatial dimensions (height, width) of the input images. Returns: image_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`Siglip2VisionModel`]. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AutoModel >>> import torch >>> model = AutoModel.from_pretrained("google/siglip2-base-patch16-224") >>> processor = AutoProcessor.from_pretrained("google/siglip2-base-patch16-224") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> with torch.no_grad(): ... image_features = model.get_image_features(**inputs) ``` r)rNrrrr)rDrrrdrrvision_outputsrYs r0get_image_featuresSiglip2Model.get_image_features suP2C1N-TXT_T_TqTq$8$D $++JjJj 6:5F5F%/)/!5 6G6 '44 r/ return_lossc UbUOURRnU bU OURRn URUUUUU S9n UR UUUUU S9n U R n U R n XR SSSS9- n XR SSSS9- n [R"XR5RU R55nURRU R5URRU R5nnXR5-U-nUR5nSnU(a[R"UR!S5URS 9n[R""U5*SU--n[R$R&R)UU-5n[R*"USS 9*nUR-5n[/UUUU U U U S 9$) a< pixel_attention_mask (`torch.Tensor` of shape `(batch_size, image_size, image_size)`, *optional*): Mask to avoid performing attention on padding pixel indices. spatial_shapes (`torch.LongTensor` of shape `(batch_size, 2)`): Tensor containing the spatial dimensions (height, width) of the input images. return_loss (`bool`, *optional*): Whether or not to return the contrastive loss. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AutoModel >>> import torch >>> model = AutoModel.from_pretrained("google/siglip2-base-patch16-224") >>> processor = AutoProcessor.from_pretrained("google/siglip2-base-patch16-224") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> texts = ["a photo of 2 cats", "a photo of 2 dogs"] >>> # important: we pass `padding=max_length` since the model was trained with this >>> inputs = processor(text=texts, images=image, padding="max_length", return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> logits_per_image = outputs.logits_per_image >>> probs = torch.sigmoid(logits_per_image) # these are the probabilities >>> print(f"{probs[0][0]:.1%} that image 0 is '{texts[0]}'") 31.9% that image 0 is 'a photo of 2 cats' ``` NrrrjrgT)rrkeepdimr)rhr)r8r9r:r4r!r;r<)rNrrrrrnormr+rtrvrhrnrpexpeyerm ones_likerWr logsigmoidsumr2r6)rDrrrrdrrrrrrrr!r4r:rnrpr9r8rm1_diag1logliknlls r0rSiglip2Model.forwardBsd2C1N-TXT_T_TqTq$8$D $++JjJj 6:5F5F%/)/!5 6G6 48??)%/!5 4C4 &33 "00 $&7&7!T&7&RR !$4$4qb$$4$OO  ,,{NN4D4G4G HZHZ4[\"&"2"2"5"5k6H6H"I4??K]K]^i^p^pKqZ )OO,== J*,,. ))O003O + 'tn +   ++Z597;59,0/3 6u0016'u||46!!1!12 6 $D> 6 'tn 6   66p15487;591537&*,0/3e E,,-e u001e 'u||4 e !!1!12 e !. e u//0e d^e $D>e 'tne  e e r/rlz Siglip2 vision encoder with an image classification head on top (a linear layer on top of the pooled final hidden states of the patch tokens) e.g. for ImageNet. c^\rSrSrSrS\SS4U4Sjjr\\SS\ \ RS\ \ RS\ \ RS \ \ RS \ \ S \ \ S\4S jj55rS rU=r$)rrirrNr=Nc>[TU]U5 URUl[R UR 5nUR UlURS:a5[R"UR RUR5O[R"5Ul UR5 g)Nr) rRrS num_labelsrrrarrWrXrTIdentityrsr)rDrNrras r0rS&Siglip2ForImageClassification.__init__s   ++*66v7K7KL (55OUN_N_bcNcBIIf**668I8I Jikititiv  r/rrdlabelsrrc4UbUOURRnUbUOURRnURUUUUUS9nURnUbLUSR UR 5n [R"X-SS9[R"U SS9- nO[R"USS9nURU5n Sn UGbUR U R 5nURRcURS:XaSURl OoURS:aNUR[R:XdUR[R:XaSURl OSURl URRS:XaI[!5n URS:Xa&U "U R#5UR#55n OU "X5n OURRS:Xa=[%5n U "U R'S UR5UR'S 55n O,URRS:Xa[)5n U "X5n [+U U UR,UR.S 9$) a pixel_attention_mask (`torch.Tensor` of shape `(batch_size, image_size, image_size)`, *optional*): Mask to avoid performing attention on padding pixel indices. spatial_shapes (`torch.LongTensor` of shape `(batch_size, 2)`): Tensor containing the spatial dimensions (height, width) of the input images. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoImageProcessor, Siglip2ForImageClassification >>> import torch >>> from PIL import Image >>> import requests >>> torch.manual_seed(3) # doctest: +IGNORE_RESULT >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> # note: we are loading a `Siglip2Model` from the hub here, >>> # so the head will be randomly initialized, hence the predictions will be random if seed is not set above. >>> image_processor = AutoImageProcessor.from_pretrained("google/siglip2-base-patch16-224") >>> model = Siglip2ForImageClassification.from_pretrained("google/siglip2-base-patch16-224") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> # model predicts one of the two classes >>> predicted_class_idx = logits.argmax(-1).item() >>> print("Predicted class:", model.config.id2label[predicted_class_idx]) Predicted class: LABEL_1 ``` N)rrdrr).Nrr regressionsingle_label_classificationmulti_label_classificationrg)r8logitsr#r$)rNrrrr"rvrhr+rr2rs problem_typerrilongr\r squeezer rr rr#r$) rDrrrdrrrrsequence_output pool_maskrr8loss_fcts r0r%Siglip2ForImageClassification.forwards2^2C1N-TXT_T_TqTq$8$D $++JjJj /3.?.? /)/!5 /@/ "33 +,Y7::?;Q;QRI#ii(CKeiiXaghNiiO#jja@O1  YYv}}-F{{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#F3D))-JJ+- B @&++b/R))-II,./$!//))   r/)rsrr)NNNNNN)r&r'r(r)rrrSrrrr+rrrrrr.rrs@r0rrrrs%O}$047;59)-,0/3d u||,d 'u||4d !!1!12 d & d $D> d 'tnd  d d r/rr)rlr]rrrr)r)rrgr )rr=rB)Kr!r* dataclassesrtypingrrrrrnumpyrcr+torch.nnrWtorch.nn.functionalrryr r r torch.nn.initr activationsrmodeling_attn_mask_utilsrmodeling_layersrmodeling_outputsrrrmodeling_utilsrrutilsrrrrconfiguration_siglip2rrr get_loggerr&rrr2r6rrLrfloatrrrrrrr r8r;rIrNrPrRr]rrrrlrr__all__r%r/r0rs* !88 AA7!B9bbFKKXX   H % ?+? ?8 ? ? ?8 ! K!  ! HbbiibX% II% <<% % << % U\\* %  %%.D)ryyD)N  -4-`M RYYM `: ryy: z%BII%P! J\_$ LL$ %$27$BG$SX$ \\$4A2ND< RYY< ~;*_;*;*| 0 -0  0 f"299"> 8 /8  8 vq )q q h { $:{  { | r/