eTh8SrSSKrSSKJr SSKJrJrJrJrJ r J r SSK r SSK J r SSKr SSKJr SSKJrJrJrJrJr SSKJrJr SS KJrJrJr SS KJrJ r J!r!J"r" S S K#J$r$J%r%J&r& \!RN"\(5r)S \ RTS\ RT4Sjr+S\ RTS\ RT4Sjr,\"SS\55r-"SS\ R\5r/"SS\ R\5r0"SS\ R\5r1S\00r2"SS\ R\5r3"SS\ R\5r4"SS \ R\5r5"S!S"\ R\5r6"S#S$\ R\5r7"S%S&\ R\5r8SIS'\ R\S(\ RTS)\ RTS*\ RTS+\\ RTS,\9S-\94S.jjr:"S/S0\ R\5r;"S1S2\ R\5r<"S3S4\ R\5r="S5S6\ R\5r>"S7S8\ R\5r?\ "S9S:\55r@"S;S<\ R\5rA"S=S>\@5rB\ "S?S@9"SASB\@55rC"SCSD\@5rD"SESF\@5rESJSGjrF/SHQrGg)KzPyTorch AltCLIP model.N) dataclass)AnyCallableListOptionalTupleUnion)ACT2FN)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentionsBaseModelOutputWithPooling,BaseModelOutputWithPoolingAndCrossAttentions'BaseModelOutputWithPoolingAndProjection)ALL_ATTENTION_FUNCTIONSPreTrainedModel)apply_chunking_to_forward find_pruneable_heads_and_indicesprune_linear_layer) ModelOutputauto_docstringlogging torch_int) AltCLIPConfigAltCLIPTextConfigAltCLIPVisionConfiglogitsreturnc[RRU[R"[ U5UR S95$)Ndevice)nn functional cross_entropytorcharangelenr")rs d/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/altclip/modeling_altclip.pycontrastive_lossr*,s/ == & &vu||CKPVP]P]/^ __ similaritycX[U5n[UR55nX-S- $)Ng@)r*t)r, caption_loss image_losss r) clip_lossr10s*#J/L!*,,.1J  % ,,r+c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr\\ S 'Sr\\ S 'S \\4S jrS rg) AltCLIPOutput6a Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `return_loss` is `True`): Contrastive loss for image-text similarity. logits_per_image (`torch.FloatTensor` of shape `(image_batch_size, text_batch_size)`): The scaled dot product scores between `image_embeds` and `text_embeds`. This represents the image-text similarity scores. logits_per_text (`torch.FloatTensor` of shape `(text_batch_size, image_batch_size)`): The scaled dot product scores between `text_embeds` and `image_embeds`. This represents the text-image similarity scores. text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`AltCLIPTextModel`]. image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`AltCLIPVisionModel`]. text_model_output (`BaseModelOutputWithPooling`): The output of the [`AltCLIPTextModel`]. vision_model_output (`BaseModelOutputWithPooling`): The output of the [`AltCLIPVisionModel`]. Nlosslogits_per_imagelogits_per_text text_embeds image_embedstext_model_outputvision_model_outputrcJ^[U4SjTR555$)Nc3n># UH*nUS;aTUO[TU5R5v M, g7f))r:r;N)getattrto_tuple).0kselfs r) )AltCLIPOutput.to_tuple..Vs<   LLDGRYZ^`aRbRkRkRm m s25)tuplekeysrBs`r)r?AltCLIPOutput.to_tupleUs# YY[   r+)__name__ __module__ __qualname____firstlineno____doc__r5rr& FloatTensor__annotations__r6r7r8r9r:rr;rrr?__static_attributes__rIr+r)r3r36s()-D(5$$ %,48hu001837OXe//07/3K%++,304L(5,,-448186:3: %* r+r3c>^\rSrSrSrU4SjrSSjrSrSrU=r $)AltRobertaEmbeddings]zN Same as BertEmbeddings with a tiny tweak for positional embeddings indexing. c>[TU]5 [R"URUR UR S9Ul[R"URUR 5Ul [R"URUR 5Ul [R"UR URS9Ul [R"UR5Ul[#USS5UlUR'S[(R*"UR5R-S5SS9 UR'S [(R."UR0R35[(R4S 9SS9 UR Ul[R"URUR UR6S9Ul g) N) padding_idxepsposition_embedding_typeabsolute position_idsrF persistenttoken_type_idsdtype)super__init__r# Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingstype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsDropouthidden_dropout_probdropoutr>rYregister_bufferr&r'expandzerosr[sizelongrVrBconfig __class__s r)rdAltRobertaEmbeddings.__init__csi !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c %'\\&2H2H&J\J\%]"f&8&8f>S>STzz&"<"<= '.v7PR\']$  ELL)G)GHOOPWXej    ekk$*;*;*@*@*B%**Ubg  "..#%<<  * *F,>,>DL\L\$  r+cUc+Ub[XRU5nOURU5nUbUR5nOUR5SSnUSnUcv[ US5(a-UR SS2SU24nUR USU5n U nO8[R"U[RURRS9nUcURU5nURU5n XJ-n URS:XaURU5n X- n UR!U 5n UR#U 5n U $)Nr]rr`rrbr"rZ)"create_position_ids_from_input_idsrV&create_position_ids_from_inputs_embedsrvhasattrr`rtr&rurwr[r"rirmrYrkrnrr) rB input_idsr`r[ inputs_embedspast_key_values_length input_shape seq_lengthbuffered_token_type_ids buffered_token_type_ids_expandedrm embeddingsrks r)forwardAltRobertaEmbeddings.forward|sM  $A)M]M]_uv #JJ=Y  #..*K',,.s3K ^  !t-..*.*=*=a*n*M'3J3Q3QR]^_R`bl3m0!A!&[ SWSdSdSkSk!l   00;M $ : :> J":  ' ': 5"&":":<"H   -J^^J/ \\*- r+cUR5SSnUSn[R"URS-X0R-S-[RUR S9nUR S5RU5$)z We are provided embeddings directly. We cannot infer which are padded so just generate sequential position ids. Args: inputs_embeds: torch.Tensor Returns: torch.Tensor Nr]rr}r)rvr&r'rVrwr" unsqueezert)rBrrsequence_lengthr[s r)r;AltRobertaEmbeddings.create_position_ids_from_inputs_embedss~$((*3B/ %a.||   q /4D4D"Dq"HPUPZPZcpcwcw %%a(// <[TU]5 URUR-S:wa7[ US5(d&[ SURSURS35eURUl[ URUR- 5UlURUR-Ul[R"URUR5Ul [R"URUR5Ul [R"URUR5Ul [R"UR5UlU=(d [#USS5UlUR$S:XdUR$S :XaGUR&Ul[R("S UR&-S - UR5UlUR,Ulg) Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()rYrZ relative_keyrelative_key_queryr)rcrdrgnum_attention_headsr ValueErrorintattention_head_size all_head_sizer#Linearquerykeyvaluerpattention_probs_dropout_probrrr>rYrjredistance_embedding is_decoderrBryrYrzs r)rd AltRobertaSelfAttention.__init__s     : : :a ?PVXhHiHi#F$6$6#78 445Q8  $*#=#= #&v'9'9F( ' -zC $  ' '> 9T=Y=Y]q=q+1+I+ID (&(ll1v7U7U3UXY3Y[_[s[s&tD # ++r+xrcUR5SSURUR4-nURU5nUR SSSS5$)Nr]rrrr )rvrrviewpermute)rBr new_x_shapes r)transpose_for_scores,AltRobertaSelfAttention.transpose_for_scoressLffhsmt'?'?AYAY&ZZ FF; yyAq!$$r+ hidden_statesattention_mask head_maskencoder_hidden_statesencoder_attention_maskpast_key_valueoutput_attentionscVURU5nUSLn U (aUbUSn USn UnGOU (aCURURU55n URURU55n UnOUbuURURU55n URURU55n [R "USU /SS9n [R "USU /SS9n O@URURU55n URURU55n URU5n USLn UR (aX4n[R"XRSS55nURS:XdURS:XGaU RSU RSnnU (aB[R"US- [RURS 9RSS5nO>[R"U[RURS 9RSS5n[R"U[RURS 9RSS5nUU- nUR!UUR"-S- 5nUR%U R&S 9nURS:Xa[R("S U U5nUU-nOHURS:Xa8[R("S U U5n[R("S U U5nUU-U-nU[*R,"UR.5- nUbX-n[0R2R5USS9nUR7U5nUbUU-n[R"UU 5nUR9SSSS 5R;5nUR=5SSUR>4-nURU5nU(aUU4OU4nUR (aUU4-nU$)Nrrrdimr]rrr}razbhld,lrd->bhlrzbhrd,lrd->bhlrr ) rrrrr&catrmatmul transposerYshapetensorrwr"rr'rrjtorbeinsummathsqrtrr#r$softmaxrrr contiguousrvr)rBrrrrrrrmixed_query_layeris_cross_attention key_layer value_layer query_layer use_cacheattention_scores query_length key_lengthposition_ids_lposition_ids_rdistancepositional_embeddingrelative_position_scoresrelative_position_scores_queryrelative_position_scores_keyattention_probs context_layernew_context_layer_shapeoutputss r)rAltRobertaSelfAttention.forwards!JJ}5 3$> ."<&q)I(+K3N 11$((;P2QRI33DJJ?T4UVK3N  '11$((=2IJI33DJJ}4MNK >!#4i"@aHI))^A%6 $D!LK11$((=2IJI33DJJ}4MNK//0AB "$. ??(5N!<< 5H5HR5PQ  ' '> 9T=Y=Y]q=q'2'8'8';Y__Q=O*L!&j1nEJJWdWkWk!l!q!q""'l%**UbUiUi!j!o!oprtu!v"\\*EJJ}OcOcdiijkmopN%6H#'#:#:8dFbFb;bef;f#g #7#:#:ARAR#:#S ++~=+0<<8H+Wk+l(#36N#N --1EE16>NP[]q1r./4||S"S%**+BC 6G=/2mM] ?? 11Gr+) rrrrrrrrjrrYrrNNNNNNF)rJrKrLrMrdr&TensorrrrOrboolrrQrrs@r)rrs,4%ell%u||%7;15=A>BDH,1c||c!!2!23cE--. c ((9(9: c !)):): ; c!uU->->'?!@Ac$D>c u|| ccr+rcz^\rSrSrU4SjrS\R S\R S\R 4SjrSrU=r $)AltRobertaSelfOutputi>c(>[TU]5 [R"URUR5Ul[R "URURS9Ul[R"UR5Ul gNrW) rcrdr#rrgdensernrorprqrrrxs r)rdAltRobertaSelfOutput.__init__?s` YYv1163E3EF f&8&8f>S>STzz&"<"<= r+r input_tensorrcpURU5nURU5nURX-5nU$rrrrrnrBrrs r)rAltRobertaSelfOutput.forwardE5 =1  ]3 }'CD r+rnrrr rJrKrLrMrdr&rrrQrrs@r)rr>6> U\\RWR^R^r+reagerc.^\rSrSrSU4SjjrSrSS\RS\\RS\\RS\\RS\\RS \\ \ \RS \\ S \ \R4S jjr S r U=r$)AltRobertaAttentioniRc>[TU]5 [UR"XS9Ul[ U5Ul[5Ulg)NrY) rcrd"ALT_ROBERTA_SELF_ATTENTION_CLASSES_attn_implementationrBroutputset pruned_headsrs r)rdAltRobertaAttention.__init__Ss@ 6v7R7RS   +62 Er+c6[U5S:Xag[XRRURRUR 5up[ URRU5URl[ URRU5URl[ URRU5URl [ URRUSS9URl URR[U5- URlURRURR-URl UR RU5Ulg)Nrrr)r(rrBrrrrrrrrrrunion)rBheadsindexs r) prune_headsAltRobertaAttention.prune_heads[s  u:? 7 9900$))2O2OQUQbQb  -TYY__eD *499==%@ ,TYY__eD .t{{/@/@%QO )- (E(EE (R %"&))"?"?$))B_B_"_  --33E:r+rrrrrrrrc pURUUUUUUU5nURUSU5n U 4USS-n U $)Nrr)rBr) rBrrrrrrr self_outputsattention_outputrs r)rAltRobertaAttention.forwardmsWyy    ! "    ;;|A F#% QR(88r+)rrrBrr)rJrKrLrMrdrr&rrrOrrrrQrrs@r)rrRs";*7;15=A>BDH,1||!!2!23E--.  ((9(9:  !)):): ; !uU->->'?!@A$D> u|| r+rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)AltRobertaIntermediateic>[TU]5 [R"URUR 5Ul[UR[5(a[URUl gURUl gr) rcrdr#rrgintermediate_sizer isinstance hidden_actstrr intermediate_act_fnrxs r)rdAltRobertaIntermediate.__init__s` YYv1163K3KL f'' - -'-f.?.?'@D $'-'8'8D $r+rrcJURU5nURU5nU$rrr rBrs r)rAltRobertaIntermediate.forwards& =1 00? r+r rrs@r)rrs(9U\\ellr+rcz^\rSrSrU4SjrS\R S\R S\R 4SjrSrU=r $)AltRobertaOutputic(>[TU]5 [R"URUR 5Ul[R"UR URS9Ul[R"UR5Ul gr) rcrdr#rrrgrrnrorprqrrrxs r)rdAltRobertaOutput.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= r+rrrcpURU5nURU5nURX-5nU$rrrs r)rAltRobertaOutput.forwardrr+rrrs@r)rrrr+rc*^\rSrSrU4SjrSS\R S\\RS\\RS\\RS\\RS\\ \ \RS \\ S \ \R 4S jjr S r S r U=r$)AltRobertaLayerict>[TU]5 URUlSUl[ U5UlUR UlURUlUR(a.UR (d[US35e[ USS9Ul [U5Ul [U5Ul g)Nrz> should be used as a decoder model if cross attention is addedrZr)rcrdchunk_size_feed_forward seq_len_dimr attentionradd_cross_attentionrcrossattentionr intermediaterrrxs r)rdAltRobertaLayer.__init__s '-'E'E$,V4 ++#)#=#=  # #?? D6)g!hii"5fV`"aD 26:&v. r+rrrrrrrrc UbUSSOSnURUUUUUS9n U Sn UR(a U SSn U Sn OU SSn Sn UR(aZUbW[US5(d[SUS35eUbUS SOSnUR U UUUUUU5nUSn XSS-n USn W U -n [ UR URURU 5nU4U -n UR(aU W 4-n U $) Nr)rrrrr]rz'If `encoder_hidden_states` are passed, z` has to be instantiated with cross-attention layers by setting `config.add_cross_attention=True`r) rrrrrrfeed_forward_chunkrr)rBrrrrrrrself_attn_past_key_valueself_attention_outputsrrpresent_key_valuecross_attn_present_key_valuecross_attn_past_key_valuecross_attention_outputs layer_outputs r)rAltRobertaLayer.forwards}:H9S>"1#5Y] !%   /3 "0" 2!4 ??,Qr2G 6r : ,QR0G'+$ ??4@4!122 =dVDDD @N?Yrs(;_c %&*&9&9 %&)!' # 7q9 " ==G,C2+F ( 14P P 0  # #T%A%A4CSCSUe  /G+ ??!2 44Gr+cJURU5nURX!5nU$r)rr)rBrintermediate_outputr's r)r "AltRobertaLayer.feed_forward_chunks)"//0@A{{#6I r+)rrrrrrrrr)rJrKrLrMrdr&rrrOrrrr rQrrs@r)rrs /"7;15=A>BDH,1?||?!!2!23?E--. ? ((9(9: ? !)):): ; ?!uU->->'?!@A?$D>? u|| ?Br+rcR^\rSrSrU4SjrSS\R S\\RS\\RS\\RS\\RS\\ \ \RS \\ S \\ S \\ S \\ S \ \ \R \ 44Sjjr SrU=r$)AltRobertaEncoderic>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snfNF) rcrdryr# ModuleListrangenum_hidden_layersrlayergradient_checkpointingrBry_rzs r)rdAltRobertaEncoder.__init__sR  ]]U6KcKcEd#eEdOF$;Ed#ef &+#$fA&rrrrrpast_key_valuesrroutput_hidden_states return_dictrc 8U (aSOSn U(aSOSn U(aURR(aSOSn UR(a/UR(aU(a[R S5 SnU(aSOSn[ UR5HunnU (aX4-n UbX?OSnUbXoOSnUR(a4UR(a#URURUUUUUUU5nOU"UUUUUUU5nUSnU(a UUS4- nU(dMU US4-n URR(dMU US4-n M U (aX4-n U (d[SUUU U U 455$[UUU U U S 9$) NrIzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...Frr]rrc30# UH nUcMUv M g7frrIr@vs r)rC,AltRobertaEncoder.forward..Es" As )last_hidden_stater9r attentionscross_attentions) ryrr4traininglogger warning_once enumerater3_gradient_checkpointing_func__call__rEr )rBrrrrrr9rrr:r;all_hidden_statesall_self_attentionsall_cross_attentionsnext_decoder_cachei layer_modulelayer_head_maskr layer_outputss r)rAltRobertaEncoder.forwards#7BD$5b4%64;;;Z;Zr`d  & &4==##p" #,R$(4OA|#$58H$H!.7.CilO3B3N_/TXN**t}} $ A A ))!"#)*"% ! !-!"#)*"%! *!,M"}R'8&::"  &9]1=MBEI$(,1/4&*S ||S !!2!23S E--. S ((9(9: S !)):): ; S "%e.?.?(@"ABS D>S $D>S 'tnS d^S  uU\\"$MM NS S r+r-cb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)AltRobertaPooleriZc>[TU]5 [R"URUR5Ul[R "5Ulgr)rcrdr#rrgrTanh activationrxs r)rdAltRobertaPooler.__init__[s9 YYv1163E3EF '')r+rrc\USS2S4nURU5nURU5nU$)Nr)rrW)rBrfirst_token_tensor pooled_outputs r)rAltRobertaPooler.forward`s6+1a40 #56  6 r+)rWrrrs@r)rTrTZs($ U\\ellr+rTmodulerrrrscalingrrc [R"XRSS55U-nUbX-n[RR US[R S9RUR5n[RRXURS9n[R"X5n U RSS5R5n X4$)Nr]r)rrb)prDrr) r&rrr#r$rfloat32rrbrrrDr) r]rrrrr^rrkwargs attn_weights attn_outputs r)eager_attention_forwardrejs<<}}R'<=GL!#4 ==((2U]](SVVW\WbWbcL==((6??([L,,|3K''1-88:K  $$r+c^\rSrSrSrU4SjrS S\RS\\RS\\RS\\ S\ \R\\R44 S jjr S r U=r $) AltCLIPAttentioniz=Multi-headed attention from 'Attention Is All You Need' paperc >[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl SUl [R"URUR5Ul[R"URUR5Ul[R"URUR5Ul[R"URUR5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: ).F)rcrdryrg embed_dimr num_headshead_dimrscaleattention_dropoutrr is_causalr#rk_projv_projq_projout_projrxs r)rdAltCLIPAttention.__init__s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( // ii? ii? ii?  $..$..A r+rrcausal_attention_maskrrc URupVnURU5nURU5n URU5n UR XVUR UR 5RSS5nU R XVUR UR 5RSS5n U R XVUR UR 5RSS5n URRS:waUbUbX#-nOUbUnO USLUl [n URRS:waTURRS:XaU(a[RS5 O[URRn U "UUU U UURURUR (dSO UR"S 9upU R%XVU5R'5n UR)U 5n U(dSn X4$) z#Input shape: Batch x Time x Channelrrflash_attention_2Nrsdpaz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)rpr^rr)rrsrqrrrrlrmrryrrprerErFrrnrDrrreshaperrt)rBrrrvr batch_sizerrkqueriesrFvaluesattention_interfacerdrcs r)rAltCLIPAttention.forwards-:,?,?) ++m,{{=)]+,,zt~~t}}U__`acdeyyOYYZ[]^_ZT^^T]]S]]^_abc ;; + +/B B).C.O!/!G&2!62$>DN(? ;; + +w 6{{//69>O##L '>dkk>^>^&_#$7     nnJJ#}}C$,, % ! "))*)LWWY mmK0  L((r+) ryrrrkrmrprqrlrtrsrnrr)NNF)rJrKrLrMrNrdr&rrrrrrQrrs@r)rgrgsGB.268<,1 5)||5)!.5) ( 5 5) $D> 5) u||Xell33 4 5)5)r+rgcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $) AltCLIPMLPic>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gr) rcrdryr r activation_fnr#rrgrfc1fc2rxs r)rdAltCLIPMLP.__init__sb  #F$5$5699V//1I1IJ99V55v7I7IJr+rrclURU5nURU5nURU5nU$r)rrrr s r)rAltCLIPMLP.forwards4/ **=9 / r+)rryrrrrs@r)rrs)KU\\ellr+rc ^\rSrSrS\4U4SjjrS S\RS\RS\RS\\ S\ \R4 S jjr S r U=r$) AltCLIPEncoderLayeriryc<>[TU]5 URUl[ U5Ul[ R"URURS9Ul [U5Ul [ R"URURS9Ul gr) rcrdrgrkrg self_attnr#rnro layer_norm1rmlp layer_norm2rxs r)rdAltCLIPEncoderLayer.__init__sm ++)&1<<F & u  ! &&r+rc^\rSrSrSrS\4U4SjjrS S\\RS\\RS\\ S\\ S \\ S \ \ \ 44 S jjrS rU=r$)AltCLIPEncoderiz Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`AltCLIPEncoderLayer`]. Args: config: AltCLIPConfig ryc>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snfr/) rcrdryr#r0r1r2rlayersr4r5s r)rdAltCLIPEncoder.__init__sT  mm%PVPhPhJi$jJiQ%8%@Ji$jk &+#%kr8rrvrr:r;rcLUbUOURRnUbUOURRnUbUOURRnU(aSOSnU(aSOSnUn [ UR 5HrupU(aXy4-nUR (a1UR(a URU RU UUU5n O U "U UUUS9n U Sn U(dMjXS4-nMt U(aXy4-nU(d[SXU455$[XUS9$)a Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) causal_attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Causal mask for the text model. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NrI)rrrc3.# UH ocMUv M g7frrIr>s r)rC)AltCLIPEncoder.forward..lse$Sq$Ss )rArrB) ryrr:use_return_dictrGrr4rDrHrIrEr ) rBrrrvrr:r;encoder_statesall_attentionsridx encoder_layerrQs r)rAltCLIPEncoder.forward s8L2C1N-TXT_T_TqTq$8$D $++JjJj &1%>Ne]N$See e+Vd  r+)ryr4r)NNNNN)rJrKrLrMrNrrdrr&rrr rr rrQrrs@r)rrs,},268<,0/3&*O !.O  ( 5 O $D> O 'tn O d^O  uo% &O O r+rc^\rSrSrS\4U4SjjrS\RS\S\S\R4Sjr S S \RS\R4S jjr S r U=r $) AltCLIPVisionEmbeddingsisryc>[TU]5 XlURUlUR UlUR Ul[R"[R"UR55Ul [R"URURUR UR SS9UlUR UR -S-UlURS-Ul[R""UR UR5UlUR'S[R("UR 5R+S5SS9 g)NF) in_channels out_channels kernel_sizestridebiasrrr[r\r^)rcrdryrgrk image_size patch_sizer# Parameterr&randnclass_embeddingConv2d num_channelspatch_embedding num_patches num_positionsreposition_embeddingrsr'rtrxs r)rd AltCLIPVisionEmbeddings.__init__ts  ++ ++ ++!||EKK,GH!yy++??  !OOt>1D!--1"$,,t/A/A4>>"R ^U\\$:L:L-M-T-TU\-]jopr+rheightwidthrcURSS- nURRRS5nURSS- n[R R 5(d%XF:Xa X#:XaURUR5$USS2SS24nUSS2SS24nURSn X R-n X0R-n [US-5n URSXU 5nURSSSS5n[RRUX4SS S 9nURSSSS5RSSU 5n[R "Xx4SS 9$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rrNr]g?r rbicubicF)rvmode align_cornersr)rrweightrr&jit is_tracingr[rrr{rr#r$ interpolaterr) rBrrrrrrclass_pos_embedpatch_pos_embedr new_height new_widthsqrt_num_positionss r)interpolate_pos_encoding0AltCLIPVisionEmbeddings.interpolate_pos_encodingsi!&&q)A- !44;;EEaH*003a7 yy##%%+*F6?**4+<+<= =,QU3,QU3r". __, &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nyy/;CCr+ pixel_valuesc ^URup4pVU(dJXPR:wdX`R:wa,[SUSUSURSURS3 5eURRR nURUR US95nURS5RSS5nURRUSS5n [R"X/SS 9n U(aXRXU5-n U $XRUR5-n U $) NzInput image size (*z) doesn't match model (rirarrr]r)rrrrrrbrflattenrrrtr&rrrr[) rBrrr|r6rr target_dtype patch_embeds class_embedsrs r)rAltCLIPVisionEmbeddings.forwards$'3'9'9$ v'V-F%SbSbJb$VHAeW4KDOOK\\]^b^m^m]nnpq ++2288 ++LOO,O,OP #++A.88A> ++22:q"E YY ;C ##&C&CJX]&^^J$&=&=d>O>O&PPJr+) rryrkrrrrrrr)rJrKrLrMrrdr&rrrrOrrQrrs@r)rrssjq2q,'D5<<'D'DUX'D]b]i]i'DRE$5$5Z_ZfZfr+rc*\rSrSr\rSrSr/rSr Sr g)AltCLIPPreTrainedModelialtclipTc6 URRn[U[5(aURRn[R R URSURS-U-S9 [R R URRURRU-S9 [R R URRURRU-S9 g[U[5(Ga URRnURS-SURR-S--U-nURS-U-n[R R URRUS9 [R R UR RUS9 [R R UR"RUS9 [R R UR$RUS9 g[U[&5(aURRnURR(S-SURR-S--U-nSURR(-S-U-n[R R UR*RUS9 [R R UR,RUS9 g[U[.5(a[R R UR0RUR2S-URR-S9 SUR0l[R R UR6RUR8S-URR-S9 SUR6lg[U[R:5(aJUR<R>RA5 URR>RCS5 g[U[RD5(akURR>R SURRS9 UR<b%UR<R>RA5 gg[U[RF5(axURR>R SURRS9 URHb2URR>URHRA5 ggg) zInitialize the weightsrzrj)meanstd)rrTg?N)%ryinitializer_factorrrr#initnormal_rrkrrinitializer_rangerrgr2rsrqrrrtrrgrr AltCLIPModeltext_projectiontext_embed_dim_is_hf_initializedvisual_projectionvision_embed_dimrnrdatazero_fill_rrerV)rBr]factor in_proj_std out_proj_stdfc_stds r) _init_weights$AltCLIPPreTrainedModel._init_weightss// f5 6 6[[33F GGOOF22&BRBRTXBX[aBaO b GGOOF2299v}}?^?^ag?gO h GGOOF55<<&--BaBadjBjO k  0 1 1[[33F!++T1q6==;Z;Z7Z_c6cdgmmK",,d2f..r+rIN) rJrKrLrMr config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modulerrQrIr+r)rrs L!&*#+?r+rc^\rSrSrS\4U4Sjjr\S S\\RS\\ S\\ S\\ S\\ S \ \ \ 44 S jj5rS rU=r$) AltCLIPVisionTransformeriryc>[TU]5 XlURn[ U5Ul[ R"X!RS9Ul [U5Ul [ R"X!RS9Ul gr) rcrdryrgrrr#rnro pre_layrnormrencoderpost_layernorm)rBryrkrzs r)rd!AltCLIPVisionTransformer.__init__sd  && 1&9LL8M8MN%f-  ll9:O:OPr+rrr:r;rrcUbUOURRnUbUOURRnUbUOURRnUc [ S5eUR XS9nUR U5nURUUUUS9nUSnUSS2SSS24n URU 5n U(d X4USS-$[UU URURS9$)Nz You have to specify pixel_values)r)rrr:r;rrrA pooler_outputrrB) ryrr:rrrrrrrrrB) rBrrr:r;rrencoder_outputsrAr[s r)r AltCLIPVisionTransformer.forwards 2C1N-TXT_T_TqTq$8$D $++JjJj &1%' 'tn ' d^ ' #+4. '  u00 1' ' r+rc^\rSrSr\rSrS\4U4SjjrS\R4Sjr \ S S\ \ RS\ \S\ \S \S \ \S\\\44 S jj5rS rU=r$)AltCLIPVisionModeli1rrycd>[TU]U5 [U5UlUR 5 gr)rcrdr vision_model post_initrxs r)rdAltCLIPVisionModel.__init__5s'  4V< r+rcBURRR$r)rrrrGs r)get_input_embeddings'AltCLIPVisionModel.get_input_embeddings;s  ++;;;r+rr:rr;c^UbUOURRnURUUUUUS9$)aN Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AltCLIPVisionModel >>> model = AltCLIPVisionModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled CLS states ```rrr:rr;)ryrr)rBrrr:rr;s r)rAltCLIPVisionModel.forward>sA:&1%$ 'tn $ #' $ d^ $  u00 1$ $ r+ra( The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between the self-attention layers, following the architecture described in *Attention is all you need*_ by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin. To behave as an decoder the model needs to be initialized with the `is_decoder` argument of the configuration set to `True`. To be used in a Seq2Seq model, the model needs to initialized with both `is_decoder` argument and `add_cross_attention` set to `True`; an `encoder_hidden_states` is then expected as an input to the forward pass. .. _*Attention is all you need*: https://arxiv.org/abs/1706.03762 ) custom_introc ^\rSrSr\rSU4SjjrSrSrSr \ SS\ \ RS\ \ RS\ \ RS \ \ RS \ \ RS \ \ RS \ \ RS \ \ RS\ \\ RS\ \S\ \S\ \S\ \S\\\ R\44Sjj5rSrU=r$)AltRobertaModelifc>[TU]U5 Xl[U5Ul[ U5UlU(a [U5OSUlUR5 g)z^ add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer N) rcrdryrSrr-rrTpoolerr)rBryadd_pooling_layerrzs r)rdAltRobertaModel.__init__xsL   .v6(0 2C&v.  r+c.URR$rrrirGs r)r$AltRobertaModel.get_input_embeddingss...r+c$XRlgrrrBrs r)set_input_embeddings$AltRobertaModel.set_input_embeddingss */'r+cUR5H7up#URRURR U5 M9 g)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsrr3rr)rBheads_to_pruner3rs r) _prune_headsAltRobertaModel._prune_headss< +002LE LL  u % / / ; ;E B3r+rrr`r[rrrrr9rrr:r;rc U bU OURRn U bU OURRn U bU OURRn URR(aU bU OURR n OSn UbUb [ S5eUb"URX5 UR5nO"UbUR5SSnO [ S5eUunnUb URO URnU bU SSRSOSnUc[R"UUU-4US9nUcs[URS5(a4URRSS2SU24nUR!UU5nUnO$[R""U[R$US 9nUR'X.5nURR(aEUbBUR5unnnUU4nUc[R"UUS9nUR)U5nOSnUR+XPRR,5nURUUUUUS 9nUR/UUUUUU U U U U S 9 nUSnUR0bUR1U5OSnU (d UU4US S-$[3UUUR4UR6UR8UR:S 9$)NFzDYou cannot specify both input_ids and inputs_embeds at the same timer]z5You have to specify either input_ids or inputs_embedsrrr!r`r})rr[r`rr) rrrrr9rrr:r;r)rArr9rrBrC)ryrr:rrrr%warn_if_padding_and_no_attention_maskrvr"rr&onesrrr`rtrurwget_extended_attention_maskinvert_attention_mask get_head_maskr2rr rr9rrBrC)rBrrr`r[rrrrr9rrr:r;rr|rr"rrrextended_attention_maskencoder_batch_sizeencoder_sequence_lengthr6encoder_hidden_shapeencoder_extended_attention_maskembedding_outputrsequence_outputr[s r)rAltRobertaModel.forwards$2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U!, J%.%:!! @T@TDSC^!3A!6!l $D>l 'tnl d^l  uU\\"$PP Ql l r+r c^\rSrSr\rU4SjrS\R4Sjr S\RSS4Sjr SS\ \ S\R4U4S jjjr\SS \ \R"S \ \R"S \ \R"S \ \R"S\ \R"S\ \R"S\ \R"S\ \R"S\ \S\ \S\ \S\\\44Sjj5rSrU=r$)AltCLIPTextModelic>[TU]U5 [USS9Ul[R "UR UR5Ul[R"UR URS9Ul UR5 g)NF)rrW) rcrdr robertar#rrg project_dimtransformationrnropre_LNrrxs r)rdAltCLIPTextModel.__init__ se  &vG  ii(:(:F[TU]U5$r)rcresize_token_embeddings)rBr7rzs r)r9(AltCLIPTextModel.resize_token_embeddingssw.~>>r+rrr`r[rrrrrr;r:c *U bU OURRn URUUUUUUUUU U U S9 n U Sn URU 5n UR U 5nUSS2S4nU (d X4U SS-$[ UUU R U RS9$)a Examples: ```python >>> from transformers import AutoProcessor, AltCLIPTextModel >>> model = AltCLIPTextModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> texts = ["it's a cat", "it's a dog"] >>> inputs = processor(text=texts, padding=True, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled CLS states ```N) rrr`r[rrrrrr:r;rrr)ryrr-r0r/rrrB)rBrrr`r[rrrrrr;r:rr(projection_staters r)rAltCLIPTextModel.forwardsB&1%B d^B 'tnB  u== >B B r+r+c`^\rSrSr\rS\4U4Sjjr\SS\\ RS\\ RS\\ RS\\ S\\ S \\ S \ R4S jj5r \SS \\ RS\\ S\\ S \ S \\ S \ R4 Sjj5r\SS\\ RS \\ RS\\ RS\\ RS\\ RS\\ S\\ S\\ S \ S \\ S \\\44Sjj5rSrU=r$)ri`ryc`>[TU]U5 [UR[5(d"[ S[ UR5S35e[UR[5(d"[ S[ UR5S35eURnURnURUl URUl URUl [U5Ul[!U5Ul[$R&"URURSS9Ul[$R&"URURSS9Ul[$R,"[.R0"UR2R455UlUR95 g)NzRconfig.vision_config is expected to be of type AltCLIPVisionConfig but is of type .zNconfig.text_config is expected to be of type AltCLIPTextConfig but is of type F)r)rcrdr vision_configr TypeErrortype text_configrprojection_dimr.rrgrr+ text_modelrrr#rrrrr&rrylogit_scale_init_value logit_scaler)rBryrErBrzs r)rdAltCLIPModel.__init__cs]  &..0CDD--./q2 &,,.?@@++,-Q0  (( ,, $33)55 - 9 9*;74]C!#4+@+@$BUBU\a!b!yy)<)Q>QX]^<< T[[5W5W(XY r+rrr[rr:r;rc UbUOURRnUbUOURRnUbUOURRnUR UUUUUUUS9nUSn UR U 5n U $)a/ Returns: text_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`AltCLIPTextModel`]. Examples: ```python >>> from transformers import AutoProcessor, AltCLIPModel >>> model = AltCLIPModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> inputs = processor(text=["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> text_features = model.get_text_features(**inputs) ```)rrr[r`rr:r;r)ryrr:rrGr) rBrrr[r`rr:r; text_outputsr[ text_featuress r)get_text_featuresAltCLIPModel.get_text_featuress62C1N-TXT_T_TqTq$8$D $++JjJj &1%>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AltCLIPModel >>> model = AltCLIPModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> image_features = model.get_image_features(**inputs) ```rr)ryrr:rrr) rBrrr:rr;vision_outputsr[image_featuress r)get_image_featuresAltCLIPModel.get_image_featuress:2C1N-TXT_T_TqTq$8$D $++JjJj &1%r+r` return_lossc UbUOURRnUbUOURRnU bU OURRn UR UUUUUUU S9n UR UUUU U S9n U Sn UR U 5n U SnURU5nXRSSSS9- n XRSSSS9- nURR5n[R"XR55U-nURnSnU(a [U5nU (dUUXX4nUbU4U-$U$[!UUUUU U U S 9$) a0 return_loss (`bool`, *optional*): Whether or not to return the contrastive loss. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AltCLIPModel >>> model = AltCLIPModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor( ... text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True ... ) >>> outputs = model(**inputs) >>> logits_per_image = outputs.logits_per_image # this is the image-text similarity score >>> probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities ```N)rrr`r[rr:r;rrrr]T)r`rkeepdim)r5r6r7r8r9r:r;)ryrr:rrGrrrnormrIexpr&rr.Tr1r3)rBrrrr[r`rUrr:rr;rLrQr9r8rIr7r6r5rs r)rAltCLIPModel.forwardsJ2C1N-TXT_T_TqTq$8$D $++JjJj &1%-115/3,0/3&*,ELL),!.,u||, , $D> ,'tn,d^,   ,,\59,0/3).&* -u001-$D>-'tn - #' - d^ -   --^1548153715&*,0/3).&*[ E,,-[ u001[ !. [ u//0 [ !. [ d^[ $D>[ 'tn[ #'[ d^[  um# $[ [ r+rcURU5R5n[R"USS9R U5U-U-nUR 5U-$)z Replace non-padding symbols with their position numbers. Position numbers begin at padding_idx+1. Padding symbols are ignored. This is modified from fairseq's `utils.make_positions`. Args: x: torch.Tensor x: Returns: torch.Tensor rr)nerr&cumsumtype_asrw)rrVrmaskincremental_indicess r)r~r~AsW << $ ( ( *D <<!4<rps !>> !GllDDXX   H % `U\\`ell`-%,,-5<<-  ! K!  ! JV=299V=tCbiiCN 299  $&" 0"))0h RYY  ryy SbiiSnZ  Z | ryy .% II% <<% % << % U\\* %  %%,L)ryyL)`  /"))/d^ RYY^ DPbiiPf1?_1?1?h3 ryy3 l2 /2 j  P ,P  P fV -V r] )] B 4 _r+