fTh6SrSSKJr SSKJrJrJrJrJr SSK r SSK J r SSK J r J r Jr SSKJr SS KJrJr SS KJrJrJr SS KJrJr SS KJrJrJrJrJ r S SK!J"r"J#r#J$r$ \RJ"\&5r'S\ RPS\ RP4Sjr)S\ RPS\ RP4Sjr*S\ RPS\ RP4Sjr+\"SS\55r,\"SS\55r-\"SS\55r."SS\ R^5r0"SS\ R^5r1SHS \ R^S!\ RPS"\ RPS#\ RPS$\\ RPS%\2S&\2S'\34S(jjr4"S)S*\ R^5r5"S+S,\ R^5r6"S-S.\ R^5r7\"S/S0\55r8"S1S2\ R^5r9"S3S4\ R^5r:\"S5S69"S7S8\855r;"S9S:\ R^5r<\"S;S69"S<S=\855r=\"S>S?\855r>\"S@SA\855r?\"SBSC\855r@\"SDS69"SESF\855rA/SGQrBg)IzPyTorch CLIP model.) dataclass)AnyCallableOptionalTupleUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN) _create_4d_causal_attention_mask_prepare_4d_attention_mask)BaseModelOutputBaseModelOutputWithPoolingImageClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel) ModelOutputauto_docstringcan_return_tuplelogging torch_int) CLIPConfigCLIPTextConfigCLIPVisionConfiglogitsreturnc[RRU[R"[ U5UR S95$)Ndevice)r functional cross_entropytorcharangelenr#)rs ^/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/clip/modeling_clip.pycontrastive_lossr*%s/ == & &vu||CKPVP]P]/^ __ similaritycX[U5n[UR55nX-S- $)Ng@)r*t)r, caption_loss image_losss r) clip_lossr1)s*#J/L!*,,.1J  % ,,r+tensorc[R"US5n[R"USSS9n[R"US5nU$)z This method is equivalent to tensor.norm(p=2, dim=-1, keepdim=True) and used to make model `executorch` exportable. See issue https://github.com/pytorch/executorch/issues/3566 T)dimkeepdim?)r&powsum)r2 square_tensor sum_tensor normed_tensors r)_get_vector_normr>/s< IIfa(M=b$?JIIj#.M r+c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \RS4\ S'Sr \\ \RS4\ S'S rg) CLIPVisionModelOutput:a Base class for vision model's outputs that also contains image embeddings of the pooling of the last hidden states. Args: image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The image embeddings obtained by applying the projection layer to the pooler_output. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. N image_embedslast_hidden_state. hidden_states attentions)__name__ __module__ __qualname____firstlineno____doc__rBrr& FloatTensor__annotations__rCrDrrE__static_attributes__rFr+r)r@r@:sr*15L(5,,-459x 1 129=AM8E%"3"3S"89:A:>Ju00#567>r+r@c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \RS4\ S'Sr \\ \RS4\ S'S rg) CLIPTextModelOutputWa Base class for text model's outputs that also contains a pooling of the last hidden states. Args: text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The text embeddings obtained by applying the projection layer to the pooler_output. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. N text_embedsrC.rDrErF)rGrHrIrJrKrRrr&rLrMrCrDrrErNrFr+r)rPrPWsr*04K%++,359x 1 129=AM8E%"3"3S"89:A:>Ju00#567>r+rPc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr\\ S 'Sr\\ S 'S \\4S jrS rg) CLIPOutputta Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `return_loss` is `True`): Contrastive loss for image-text similarity. logits_per_image (`torch.FloatTensor` of shape `(image_batch_size, text_batch_size)`): The scaled dot product scores between `image_embeds` and `text_embeds`. This represents the image-text similarity scores. logits_per_text (`torch.FloatTensor` of shape `(text_batch_size, image_batch_size)`): The scaled dot product scores between `text_embeds` and `image_embeds`. This represents the text-image similarity scores. text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`CLIPTextModel`]. image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`CLIPVisionModel`]. text_model_output (`BaseModelOutputWithPooling`): The output of the [`CLIPTextModel`]. vision_model_output (`BaseModelOutputWithPooling`): The output of the [`CLIPVisionModel`]. Nlosslogits_per_imagelogits_per_textrRrBtext_model_outputvision_model_outputr cJ^[U4SjTR555$)Nc3n># UH*nUS;aTUO[TU5R5v M, g7f))rYrZN)getattrto_tuple).0kselfs r) &CLIPOutput.to_tuple..s<   LLDGRYZ^`aRbRkRkRm m s25)tuplekeysras`r)r^CLIPOutput.to_tuples# YY[   r+rF)rGrHrIrJrKrVrr&rLrMrWrXrRrBrYrrZrrr^rNrFr+r)rTrTts()-D(5$$ %,48hu001837OXe//07/3K%++,304L(5,,-448186:3: %* r+rTc^\rSrSrS\4U4SjjrS\RS\S\S\R4Sjr S S \RS\R4S jjr S r U=r $) CLIPVisionEmbeddingsconfigc>[TU]5 XlURUlUR UlUR Ul[R"[R"UR55Ul [R"URURUR UR SS9UlUR UR -S-UlURS-Ul[R""UR UR5UlUR'S[R("UR 5R+S5SS9 g)NF) in_channels out_channels kernel_sizestridebiasr4r position_idsrr5 persistent)super__init__rk hidden_size embed_dim image_size patch_sizer Parameterr&randnclass_embeddingConv2d num_channelspatch_embedding num_patches num_positions Embeddingposition_embeddingregister_bufferr'expandrark __class__s r)rwCLIPVisionEmbeddings.__init__s  ++ ++ ++!||EKK,GH!yy++??  !OOt>1D!--1"$,,t/A/A4>>"R ^U\\$:L:L-M-T-TU\-]jopr+ embeddingsheightwidthr cURSS- nURRRS5nURSS- n[R R 5(d%XF:Xa X#:XaURUR5$USS2SS24nUSS2SS24nURSn X R-n X0R-n [US-5n URSXU 5nURSSSS5n[RRUX4SS S 9nURSSSS5RSSU 5n[R "Xx4SS 9$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rrNr5r8r r4bicubicF)sizemode align_cornersr6)shaperweight unsqueezer&jit is_tracingrrr{rreshapepermuter r$ interpolateviewcat) rarrrrrrclass_pos_embedpatch_pos_embedr6 new_height new_widthsqrt_num_positionss r)interpolate_pos_encoding-CLIPVisionEmbeddings.interpolate_pos_encodingsi!&&q)A- !44;;EEaH*003a7 yy##%%+*F6?**4+<+<= =,QU3,QU3r". __, &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nyy/;CCr+ pixel_valuesc ^URup4pVU(dJXPR:wdX`R:wa,[SUSUSURSURS3 5eURRR nURUR US95nURS5RSS5nURRUSS5n [R"X/SS 9n U(aXRXU5-n U $XRUR5-n U $) NzInput image size (*z) doesn't match model ().)dtyper4rr5r)rrz ValueErrorrrrtoflatten transposer~rr&rrrrr) rarr batch_size_rr target_dtype patch_embeds class_embedsrs r)forwardCLIPVisionEmbeddings.forwards$'3'9'9$ v'V-F%SbSbJb$VHAeW4KDOOK\\]^b^m^m]nnpq ++2288 ++LOO,O,OP #++A.88A> ++22:q"E YY ;C ##&C&CJX]&^^J$&=&=d>O>O&PPJr+) r~rkryrzrrrr{rF)rGrHrIrJrrwr&TensorintrrLrrN __classcell__rs@r)ririsjq/q,'D5<<'D'DUX'D]b]i]i'DRE$5$5Z_ZfZfr+ric ^\rSrSrS\4U4SjjrS S\\RS\\RS\\RS\R4Sjjr S r U=r $) CLIPTextEmbeddingsrkcN>[TU]5 URn[R"UR U5Ul[R"URU5UlURS[R"UR5RS5SS9 g)NrrrsFrt) rvrwrxr r vocab_sizetoken_embeddingmax_position_embeddingsrrr&r'rrarkryrs r)rwCLIPTextEmbeddings.__init__s && !||F,=,=yI"$,,v/M/My"Y  ELL)G)GHOOPWXej  r+ input_idsrr inputs_embedsr c<UbURSOURSnURRRSnXE:a[SUSU35eUcURSS2SU24nUcUR U5nURU5nX6-nU$)Nr5rzRSequence length must be less than max_position_embeddings (got `sequence length`: z and max_position_embeddings: )rrrrrrr)rarrrr seq_lengthmax_position_embeddingposition_embeddingsrs r)rCLIPTextEmbeddings.forwards -6,AY__R(}GZGZ[]G^ !%!8!8!?!?!E!Ea!H  .d,<=S[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl SUl [R"URUR5Ul[R"URUR5Ul[R"URUR5Ul[R"URUR5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: rF)rvrwrkrxrynum_attention_heads num_headshead_dimrscaleattention_dropoutr is_causalr Lineark_projv_projq_projout_projrs r)rwCLIPAttention.__init__/s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( // ii? ii? ii?  $..$..A r+rDrcausal_attention_maskrr c URupVnURU5nURU5n URU5n UR XVSUR 5R SS5nU R XVSUR 5R SS5n U R XVSUR 5R SS5n URRS:Xa USLUl OUbUbX#-nOUbUn[n URRS:waTURRS:XaU(a[RS5 O[URRn U "UUU U UURURUR(dS O UR US 9 upU R#XVU5R%5n UR'U 5n U(dSn X4$) z#Input shape: Batch x Time x Channelr5rr4flash_attention_2Neagersdpaz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)rrrr)rrrrrrrrk_attn_implementationrrlogger warning_oncerrrrrrr)rarDrrrrrryqueriesrevaluesattention_interfacerrs r)rCLIPAttention.forwardCs-:,?,?) ++m,{{=)]+,,zr4==ISSTUWXYyyT]]CMMaQRSZRGQQRSUVW ;; + +/B B2$>DN).C.O!/!G&2!6(? ;; + +w 6{{//69>O##L '>dkk>^>^&_#$7     nnJJ#}}C$,,/ % ! "))*)LWWY mmK0  L((r+) rkrryrrrrrrrr)NNF)rGrHrIrJrKrrrrwr&rrboolrrrNrrs@r)rr,sGBu%5~%EFB.268<,1 7)||7)!.7) ( 5 7) $D> 7) u||Xell33 4 7)7)r+rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)CLIPMLPi}c>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gN) rvrwrkr hidden_act activation_fnr rrxintermediate_sizefc1fc2rs r)rwCLIPMLP.__init__~sb  #F$5$5699V//1I1IJ99V55v7I7IJr+rDr clURU5nURU5nURU5nU$r)rrr)rarDs r)rCLIPMLP.forwards4/ **=9 / r+)rrkrr) rGrHrIrJrwr&rrrNrrs@r)rr}s)KU\\ellr+rc ^\rSrSrS\\\44U4SjjrS S\RS\RS\RS\ \ S\ \R4 S jjrS rU=r$) CLIPEncoderLayerirkc<>[TU]5 URUl[ U5Ul[ R"URURS9Ul [U5Ul [ R"URURS9Ul gNeps) rvrwrxryr self_attnr LayerNormlayer_norm_eps layer_norm1rmlp layer_norm2rs r)rwCLIPEncoderLayer.__init__sl ++&v.<<F & u  ! &&r+r c.\rSrSr\rSrSrSrSr Sr Sr g)CLIPPreTrainedModeliclipTc" URRn[U[5(adURR R RSUS-S9 URR R RSUS-S9 GOF[U[5(aURRn[RRURSURS-U-S9 [RRURR URRU-S9 [RRURR URRU-S9 GON[U[ 5(Ga!URRnURS-SURR"-S--U-nURS-U-n[RRUR$R US9 [RRUR&R US9 [RRUR(R US9 [RRUR*R US9 GO[U[,5(aURRnURR.S-SURR"-S--U-nSURR.-S-U-n[RRUR0R US9 [RRUR2R US9 GO.[U[45(a[RRUR6R UR8S-URR-S9 [RRUR:R UR<S-URR-S9 GOk[U[>5(aa[RRUR:R URR.S-URR-S9 O[U[@5(aa[RRUR6R URR.S-URR-S9 O[U[B5(aj[RRURDR URRFR.S-URR-S9 [U[RH5(aIURJR RM5 UR R ROS5 [U[RP5(a3URJb%URJR RM5 ggg) zInitialize the weightsrg{Gz?)meanstdr)rr4g?N))rkinitializer_factor isinstancerrrdatanormal_rrir initr~ryrinitializer_rangernum_hidden_layersrrrrrrxrr CLIPModeltext_projectiontext_embed_dimvisual_projectionvision_embed_dimCLIPVisionModelWithProjectionCLIPTextModelWithProjectionCLIPForImageClassification classifier vision_configrrqzero_fill_r)rarfactor in_proj_std out_proj_stdfc_stds r) _init_weights!CLIPPreTrainedModel._init_weightss// f0 1 1  " " ) ) . . 6 6CVd] 6 S  % % , , 1 1 9 9sQU 9 V  4 5 5[[33F GGOOF22&BRBRTXBX[aBaO b GGOOF2299v}}?^?^ag?gO h GGOOF55<<&--BaBadjBjO k  . .[[33F!++T1q6==;Z;Z7Z_c6cdgmmK",,d2f > GGOO((//KK++T1DKK4R4RR   ; < < GGOO&&--KK++T1DKK4R4RR   : ; ; GGOO!!((KK--994?$++B`B``   fbll + + KK   " " $ MM   $ $S ) fbii ( (V[[-D KK   " " $.E (r+rFN) rGrHrIrJr config_classbase_model_prefixsupports_gradient_checkpointing_supports_sdpa_supports_flash_attn_2r7rNrFr+r)rrs#L&*#N!6%r+rc ^\rSrSrSrS\4U4Sjjr\S S\\ RS\\ RS\\ S\\ S \ 4 S jj5r S rU=r$) CLIPEncoderiz Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`CLIPEncoderLayer`]. Args: config: CLIPConfig rkc>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snf)NF) rvrwrkr ModuleListranger&r layersgradient_checkpointing)rarkrrs r)rwCLIPEncoder.__init__sS  mmuVMeMeGf$gGf!%5f%=Gf$gh &+#%hsA&rrroutput_hidden_statesr cUbUOURRnUbUOURRnU(aSOSnU(aSOSnUn[UR5HrupU(aXh4-nUR (a1UR (a URU RUUUU5n O U "UUUUS9n U SnU(dMjX{S4-nMt U(aXh4-n[UUUS9$)a Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) causal_attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Causal mask for the text model. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NrF)rrr)rCrDrE) rkrrF enumeraterCrDr_gradient_checkpointing_func__call__r) rarrrrrFencoder_statesall_attentionsrDidx encoder_layer layer_outputss r)rCLIPEncoder.forwards L2C1N-TXT_T_TqTq$8$D $++JjJj  40d% "+DKK"8 C#!/2B!B**t}} $ A A!**!")% ! !.!")&7 ! *!,M  !/3C2E!E-#90 +.>>N+(%  r+)rkrDrCNNNN)rGrHrIrJrKrrwrrr&rrrrrNrrs@r)r?r?s,z, 268<,0/3 M !.M  ( 5 M $D> M 'tn M  M M r+r?c^\rSrSrS\4U4Sjjr\\S S\\ RS\\ RS\\ RS\\ S\\ S \ 4 S jj55r S rU=r$) CLIPTextTransformeri_rkc>[TU]5 XlURn[ U5Ul[ U5Ul[R"X!RS9Ul URUl URS:HUlg)Nr r)rvrwrkrxrrr?encoderr rrfinal_layer_norm eos_token_idr_use_flash_attention_2rs r)rwCLIPTextTransformer.__init__`ss  && ,V4"6* " Y F 'tn F  $F F r+rSzI The text model from CLIP without any head or projection on top. ) custom_introc^\rSrSr\rSS/rS\4U4SjjrS\R4Sjr Sr \ \ SS \\R S \\R S \\R S \\S \\S\4 Sjj55rSrU=r$) CLIPTextModelirr rkcd>[TU]U5 [U5UlUR 5 gr)rvrwrS text_model post_initrs r)rwCLIPTextModel.__init__s&  -f5 r+r cBURRR$rrgrrrfs r)get_input_embeddings"CLIPTextModel.get_input_embeddings))999r+c8XRRlgrrkrars r)set_input_embeddings"CLIPTextModel.set_input_embeddings5:""2r+rrrrrrFc(URUUUUUS9$)a Examples: ```python >>> from transformers import AutoTokenizer, CLIPTextModel >>> model = CLIPTextModel.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled (EOS token) states ```rrrrrrFrg)rarrrrrrFs r)rCLIPTextModel.forwards)4)%/!5   r+rvrb)rGrHrIrJrr9_no_split_modulesrwr Modulerlrqrrrr&rrrrrNrrs@r)reres "L-/AB~ :bii:;-115/3,0/3  ELL) !. u||,  $D>  'tn   $  r+rec ^\rSrSrS\4U4Sjjr\\S S\\ RS\\ S\\ S\\ S\ 4 S jj55r S rU=r$) CLIPVisionTransformerirkc>[TU]5 XlURn[ U5Ul[ R"X!RS9Ul [U5Ul [ R"X!RS9Ul gr ) rvrwrkrxrirr rr pre_layrnormr?rUpost_layernormrs r)rwCLIPVisionTransformer.__init__sd  && .v6LL8M8MN"6*  ll9:O:OPr+rrrFrr c~UbUOURRnUbUOURRnUc [S5eUR XS9nUR U5nUR UUUS9nURnUSS2SSS24nURU5n[UUURURS9$)Nz You have to specify pixel_values)r)rrrFrr[) rkrrFrrr}rUrCr~rrDrE) rarrrFrrDr_rCr`s r)rCLIPVisionTransformer.forwards2C1N-TXT_T_TqTq$8$D $++JjJj   ?@ @ h ))-8 +/<<'/!5,8,  ,==)!Q'2 ++M: )/')77&11   r+)rkrrUr~r}NNNF)rGrHrIrJrrwrrrr&rLrrrrNrrs@r)r{r{sQ/Q59,0/338 ! u001! $D>! 'tn ! #+4. ! $ ! ! r+r{zK The vision model from CLIP without any head or projection on top. c ^\rSrSr\rSrS/rS\4U4SjjrS\ R4Sjr \ \ S S\\R S\\S \\S \S\4 S jj55rS rU=r$)CLIPVisionModeli#rr rkcd>[TU]U5 [U5UlUR 5 gr)rvrwr{ vision_modelrhrs r)rwCLIPVisionModel.__init__-s'  1&9 r+r cBURRR$rrrrrfs r)rl$CLIPVisionModel.get_input_embeddings3  ++;;;r+rrFrc&URUUUUS9$)ag Example: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, CLIPVisionModel >>> model = CLIPVisionModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled CLS states ```rrrFrr)rarrrFrs r)rCLIPVisionModel.forward6s(<  %/!5%= !  r+rr)rGrHrIrJrr9main_input_namerxrwr ryrlrrrr&rLrrrrNrrs@r)rr#s $L$O+,/ ! 'tn ! #' ! $ ! ! r+rc"^\rSrSr\r/SQrS\4U4Sjjr\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ S \ R4 S jj5r\SS \ \ RS\ \ S \ \ S \ S \ R4 Sjj5r\\SS\ \ R"S \ \ RS\ \ RS\ \ R"S\ \ S\ \ S \ \ S \ S \4Sjj55rSrU=r$)r'i\)rr rirkc>[TU]U5 [UR[5(d"[ S[ UR5S35e[UR[5(d"[ S[ UR5S35eURnURnURUl URUl URUl [RU5nURUl[ RU5nUR"Ul[$R&"URURSS9Ul[$R&"URURSS9Ul[$R,"[.R0"UR2R455UlUR95 g)NzKconfig.text_config is expected to be of type CLIPTextConfig but is of type .zOconfig.vision_config is expected to be of type CLIPVisionConfig but is of type Frq)rvrwr! text_configr TypeErrortyper0rprojection_dimrxr)r+re _from_configrgrrr rr*r(r|r&r2rklogit_scale_init_value logit_scalerh)rarkrr0rgrrs r)rwCLIPModel.__init__as~  &,,n==++,-Q0  &..0@AA--./q2  (( ,, $33)55 - 9 9"// < $//&33MB (55!#4+@+@$BUBU\a!b!yy)<)Q>QX]^<< T[[5W5W(XY r+rrrrrrFr cUbUOURRnUbUOURRnURUUUUUS9nURnUR U5nU$)aB Returns: text_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`CLIPTextModel`]. Examples: ```python >>> from transformers import AutoTokenizer, CLIPModel >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> text_features = model.get_text_features(**inputs) ```ru)rkrrFrgr\r() rarrrrrrF text_outputsr` text_featuress r)get_text_featuresCLIPModel.get_text_featuress42C1N-TXT_T_TqTq$8$D $++JjJj 48??)%/!5 4C4 %22 ,,]; r+rrcUbUOURRnUbUOURRnURUUUUS9nURnUR U5nU$)a Returns: image_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`CLIPVisionModel`]. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, CLIPModel >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> image_features = model.get_image_features(**inputs) ```r)rkrrFrr\r*)rarrrFrvision_outputsr`image_featuress r)get_image_featuresCLIPModel.get_image_featuress>2C1N-TXT_T_TqTq$8$D $++JjJj 6:5F5F%/!5%= 6G6 '44 // >r+ return_lossc UbUOURRnUbUOURRnURUUUUS9n UR UUUUUS9n U R n UR U 5n U R n URU 5n U [U 5- n U [U 5- n [R"XR5RU R55n XRR5RU R5-n U R5nSnU(a [!U 5n[#UUU U U U U S9$)aM return_loss (`bool`, *optional*): Whether or not to return the contrastive loss. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, CLIPModel >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor( ... text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True ... ) >>> outputs = model(**inputs) >>> logits_per_image = outputs.logits_per_image # this is the image-text similarity score >>> probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities ```Nrru)rVrWrXrRrBrYrZ)rkrrFrrgr\r*r(r>r&rr.rr#rexpr1rT)rarrrrrrrrFrrrrBrRrXrWrVs r)rCLIPModel.forwardsqN2C1N-TXT_T_TqTq$8$D $++JjJj 6:5F5F%/!5%= 6G6 48??)%/!5 4C4 &33 --l; "00 **;7 $&6|&DD !$4[$AA  ,,{NN4D4G4G HZHZ4[\),<,<,@,@,B,E,EkFXFX,YY*,,. _-D-+#%* .  r+)rrr)rgr(r+rr*rbr)NNNNNNNF)rGrHrIrJrr9rxrwrrr&rrrLrrrrrTrrNrrs@r)r'r'\sLZ!z!F-115/3,0/3 )ELL))!.)u||, ) $D> ) 'tn )   ))V59,0/3). -u001-$D>-'tn - #' -    --^15481537&*,0/3).U E,,-U u001U !. U u//0 U d^ U $D>U 'tnU #'U  U U r+r'c^\rSrSr\rSS/rS\4U4SjjrS\R4Sjr Sr \ \ SS \\R S \\R S \\R S \\S \\S\4 Sjj55rSrU=r$)r-i:rr rkc>[TU]U5 [RU5nURUl[ R "URURSS9Ul UR5 gNFr) rvrwrerrgr rrxrr(rh)rarkrgrs r)rw$CLIPTextModelWithProjection.__init__@s[  "//7 $//!yy););V=R=RY^_ r+r cBURRR$rrkrfs r)rl0CLIPTextModelWithProjection.get_input_embeddingsKrnr+c8XRRlgrrkrps r)rq0CLIPTextModelWithProjection.set_input_embeddingsNrsr+rrrrrrFcURUUUUUS9nURnURU5n[UURUR UR S9$)a Examples: ```python >>> from transformers import AutoTokenizer, CLIPTextModelWithProjection >>> model = CLIPTextModelWithProjection.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> outputs = model(**inputs) >>> text_embeds = outputs.text_embeds ```ru)rRrCrDrE)rgr\r(rPrCrDrE) rarrrrrrFrr`rRs r)r#CLIPTextModelWithProjection.forwardQso248??)%/!5 4C4 %22 **=9 "#*<<&44#..   r+)rgr(rb)rGrHrIrJrr9rxrwr ryrlrqrrrr&rrrPrrNrrs@r)r-r-:s!L-/AB ~ :bii:;-115/3,0/3 & ELL)& !.& u||, & $D> & 'tn &  & & r+r-c ^\rSrSr\rSrS\4U4SjjrS\R4Sjr \ \ S S\ \RS\ \S\ \S \S\4 S jj55rS rU=r$) r,i|rrkc>[TU]U5 [RU5nURUl[ R "URURSS9Ul UR5 gr) rvrwrrrr rrxrr*rhrarkrrs r)rw&CLIPVisionModelWithProjection.__init__s\  &33F; (55!#6+=+=v?T?T[`!a r+r cBURRR$rrrfs r)rl2CLIPVisionModelWithProjection.get_input_embeddingsrr+rrFrcURUUUUS9nURnURU5n[UURUR UR S9$)a; Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, CLIPVisionModelWithProjection >>> model = CLIPVisionModelWithProjection.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> image_embeds = outputs.image_embeds ```r)rBrCrDrE)rr\r*r@rCrDrE)rarrrFrrr`rBs r)r%CLIPVisionModelWithProjection.forwardsn:6:5F5F%/!5%= 6G6  '44 --m< $%,>>(66%00   r+)rr*r)rGrHrIrJrr9rrwr ryrlrrrr&rLrr@rrNrrs@r)r,r,|s#L$O / ) 'tn ) #' )  ) ) r+r,z CLIP vision encoder with an image classification head on top (a linear layer on top of the pooled final hidden states of the patch tokens) e.g. for ImageNet. c ^\rSrSrSrS\SS4U4Sjjr\\S S\ \ RS\ \ RS\ \ S \ \ S\ 4 S jj55rS rU=r$) r.irrkr Nc>[TU]U5 URUl[R UR 5nUR UlURS:a5[R"UR RUR5O[R"5Ul UR5 g)Nr) rvrw num_labelsrrr0rr rrxIdentityr/rhrs r)rw#CLIPForImageClassification.__init__s   ++&33F4H4HI (55OUN_N_bcNcBIIf**668I8I Jikititiv  r+labelsrrFcUbUOURRnUbUOURRnURUUUS9nURn[ R "USS2SS2SS24SS9nURU5nSnUGbURUR5nURRcURS:XaSURl OoURS:aNUR[ R:XdUR[ R:XaSURl OSURl URRS:XaI[5n URS:Xa&U "UR!5UR!55nOU "Xr5nOURRS:Xa=[#5n U "UR%SUR5UR%S55nO,URRS:Xa['5n U "Xr5n[)UUUR*UR,S 9$) ab labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). N)rrFrr regressionsingle_label_classificationmulti_label_classificationr5)rVrrDrE)rkrrFrrCr&rr/rr# problem_typerrlongrr squeezer rr rrDrE) rarrrrFrsequence_outputrrVloss_fcts r)r"CLIPForImageClassification.forwards2C1N-TXT_T_TqTq$8$D $++JjJj /3.?.? /!5/@/  "33 **_QAX%>AF1  YYv}}-F{{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#F3D))-JJ+- B @&++b/R))-II,./$!//))   r+)r/rrrQ)rGrHrIrJrrrwrrrr&rrrrrNrrs@r)r.r.s%O z d 04)-,0/3 = u||,= &= $D> = 'tn =  = = r+r.)r'rrer-rr,r.)rT)CrK dataclassesrtypingrrrrrr&r torch.nnr r r activationsrmodeling_attn_mask_utilsrrmodeling_outputsrrrmodeling_utilsrrutilsrrrrrconfiguration_cliprrr get_loggerrGrrr*r1r>r@rPrTryrirfloatrrrrr rr?rSrer{rr'r-r,r.__all__rFr+r)rs!88 AA!dbbFVVLL   H % `U\\`ell`-%,,-5<<- U\\ell ?K? ?8 ?+? ?8 ! !  ! HP299Pf%%^"% II% <<% % << % U\\* %  %%%0N)BIIN)b bii /ryy/d=%/=%=%@] "))] @W "))W t 1 '1  1 h. BII. b 1 )1  1 hZ #Z Z z> "5> > B= $7= = @ Q !4Q  Q h r+