fThzNSrSSKJr SSKJrJrJr SSKrSSKJr SSK J r SSK J r SSK Jr S S KJrJrJr \R&"\5rS \R,S \R,S \S\S\S\\R,\R,44 SjrS \R,S \R,S\S\\R,\R,44SjrS \R,S\S\R,4Sjr\"SS\55r"SS\R85r"SS\R85r"SS\R85r"SS\R85r \"S S!\ 55r!\"S"S#9"S$S%\!55r"S%S!/r#g)&zPyTorch SuperPoint model.) dataclass)OptionalTupleUnionN)nn)PreTrainedModel)BaseModelOutputWithNoAttention)SuperPointConfig) ModelOutputauto_docstringlogging keypointsscoresborderheightwidthreturncUSS2S4U:USS2S4X2- :-nUSS2S4U:USS2S4XB- :-nXV-nXX4$)zPRemoves keypoints (and their associated scores) that are too close to the borderNr)rrrrrmask_hmask_wmasks j/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/superpoint/modeling_superpoint.pyremove_keypoints_from_bordersr&sf1o'IadOv,O PF1o'IadOu~,N OF ?D ?FL ((kc`U[U5:aX4$[R"XSS9upXU4$)z(Keeps the k keypoints with highest scorer)dim)lentorchtopk)rrrindicess rtop_k_keypointsr%0s7C N  jj2OF  v %%r nms_radiuscH^TS:a [S5eU4Sjn[R"U5nX"U5:Hn[S5HEnU"UR 55S:n[R "XcU5nXr"U5:HnXHU)--nMG [R "X@U5$)z)Applies non-maximum suppression on scoresrz'Expected positive values for nms_radiuscP>[RRUTS-S-STS9$)Nr kernel_sizestridepadding)r functional max_pool2d)xr&s rmax_poolsimple_nms..max_pool=s,}}''zA~7IRS]g'hhrr)) ValueErrorr" zeros_likerangefloatwhere) rr&r1zerosmax_mask_ supp_mask supp_scores new_max_masks ` r simple_nmsr>8sA~BCCi   V $E&))H 1XX^^-.2 kk)F; "h{&;;  z:;  ;;x //rc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr\\R\ S'Sr\\\R\ S 'S rg) #SuperPointKeypointDescriptionOutputJa[ Base class for outputs of image point description models. Due to the nature of keypoint detection, the number of keypoints is not fixed and can vary from image to image, which makes batching non-trivial. In the batch of images, the maximum number of keypoints is set as the dimension of the keypoints, scores and descriptors tensors. The mask tensor is used to indicate which values in the keypoints, scores and descriptors tensors are keypoint information and which are padding. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*): Loss computed during training. keypoints (`torch.FloatTensor` of shape `(batch_size, num_keypoints, 2)`): Relative (x, y) coordinates of predicted keypoints in a given image. scores (`torch.FloatTensor` of shape `(batch_size, num_keypoints)`): Scores of predicted keypoints. descriptors (`torch.FloatTensor` of shape `(batch_size, num_keypoints, descriptor_size)`): Descriptors of predicted keypoints. mask (`torch.BoolTensor` of shape `(batch_size, num_keypoints)`): Mask indicating which values in keypoints, scores and descriptors are keypoint information. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states (also called feature maps) of the model at the output of each stage. Nlossrr descriptorsr hidden_statesr)__name__ __module__ __qualname____firstlineno____doc__rBrr" FloatTensor__annotations__r IntTensorrrCr BoolTensorrDr__static_attributes__rrrr@r@Js2)-D(5$$ %,+/Ix(/*.FHU&& './3K%++,3'+D(5## $+8[TU]5 [R"UUSSSS9Ul[R"UUSSSS9Ul[R "SS9UlU(a[R"SSS9Ul gSUl g)Nr rr*Tinplacer)r+r,) super__init__rConv2dconv_aconv_bReLUrelu MaxPool2dpool)selfrRrSrTrU __class__s rr[SuperPointConvBlock.__init__ns| ii     ii     GGD) =HBLLQq9 d rrDcURURU55nURURU55nURbURU5nU$N)r`r]r^rb)rcrDs rforwardSuperPointConvBlock.forwardsN $++m"<=  $++m"<= 99  IIm4Mr)r]r^rbr`)F) rErFrGrHr intboolr[r"TensorrhrN __classcell__rds@rrPrPms_afS&S58SHKSZ^S SS*U\\ellrrPc l^\rSrSrSrS\SS4U4SjjrS S\\S\\S\ \ \ 44S jjr S r U=r$) SuperPointEncoderz SuperPoint encoder module. It is made of 4 convolutional layers with ReLU activation and max pooling, reducing the dimensionality of the image. rRrNc >[TU]5 SUl/nUR[ XRUR SSS95 [ S[UR 5S- 5H:nUR[ XR US- UR USS95 M< UR[ XR SUR SSS95 [R"U5Ul g)NrrT)rUF) rZr[ input_dimappendrPencoder_hidden_sizesr5r!r ModuleList conv_blocks)rcrRryirds rr[SuperPointEncoder.__init__s   8S8STU8Vdh i q#f99:Q>?A   #77A>@[@[\]@^lp @  33B79T9TUW9Xfk  ==5routput_hidden_states return_dictcU(aSOSnURHnU"U5nU(dMXA4-nM UnU(d[SXd455$[UUS9$)Nrc3.# UH ocMUv M g7frgr.0vs r ,SuperPointEncoder.forward..sQ$?q$? )last_hidden_staterD)rytupler )rcinputr|r}all_hidden_states conv_blockoutputs rrhSuperPointEncoder.forwardsj #7BD**Ju%E##$5$@!+QV$?QQ Q-$+  r)ryru)FT)rErFrGrHrIr r[rrkrrr rhrNrmrns@rrprps_ 6/6D6205&*  'tn d^  u44 5   rrpc"^\rSrSrSrS\SS4U4SjjrS\RS\ \R\R44Sjr S\RS\R4S jr S \RS\ \R\R44S jr S r U=r$) SuperPointInterestPointDecodera The SuperPointInterestPointDecoder uses the output of the SuperPointEncoder to compute the keypoint with scores. The scores are first computed by a convolutional layer, then a softmax is applied to get a probability distribution over the 65 possible keypoint classes. The keypoints are then extracted from the scores by thresholding and non-maximum suppression. Post-processing is then applied to remove keypoints too close to the image borders as well as to keep only the k keypoints with highest score. rRrNc>[TU]5 URUlURUlURUlUR Ul[ R"SS9Ul[ R"SSS9Ul [ R"URSURSSSS9Ul[ R"URURSSS S9Ulg NTrWr)rYrtr rr*r)rZr[keypoint_threshold max_keypointsr&border_removal_distancerr_r`rarbr\rwdecoder_hidden_size conv_score_akeypoint_decoder_dim conv_score_brcrRrds rr['SuperPointInterestPointDecoder.__init__s "(";";#11 ++'-'E'E$GGD) LLQq9 II  ' ' +  & &  II  & &(C(CQR[\fg rencodedcPURU5nURU5up2X24$rg)_get_pixel_scores_extract_keypoints)rcrrrs rrh&SuperPointInterestPointDecoder.forwards.''0 33F;   rcURURU55nURU5n[RR US5SS2SS24nUR up4pVURSSSS5RX5USS5nURSSSSS5RX5S-US-5n[X R5nU$) zKBased on the encoder output, compute the scores for each pixel of the imagerNrtrr)r ) r`rrrr.softmaxshapepermutereshaper>r&)rcrr batch_sizer:rrs rr0SuperPointInterestPointDecoder._get_pixel_scoress4,,W56""6*&&vq1!SbS&9'-||$ v1a+33JqRST1aA.66zA:uWXyYFOO4 rrcURup#n[R"USUR:5nUS[ UR 55n[ XQURUS-US-5upQURS:a[XQUR5upQ[R"US/5R5nXQ4$)z Based on their scores, extract the pixels that represent the keypoints that will be used for descriptors computation. The keypoints are in the form of relative (x, y) coordinates. rrr) rr"nonzerorrtrrrr%flipr6)rcrr:rrrs rr1SuperPointInterestPointDecoder._extract_keypointss "<<5MM&)d.E.E"EF 5/0: t;;VaZQR     " / 4CUCU V IJJy1#.446   r)rrrrrr&rbr`)rErFrGrHrIr r[r"rlrrhrrrNrmrns@rrrs / D (!u||!ellELL6P0Q!  %,, !!% ell@Z:[!!rrc^\rSrSrSrS\SS4U4SjjrS\RS\RS\R4S jr \ S S \ S\R4S jj5r S r U=r$)SuperPointDescriptorDecoderiaS The SuperPointDescriptorDecoder uses the outputs of both the SuperPointEncoder and the SuperPointInterestPointDecoder to compute the descriptors at the keypoints locations. The descriptors are first computed by a convolutional layer, then normalized to have a norm of 1. The descriptors are then interpolated at the keypoints locations. rRrNcR>[TU]5 [R"SS9Ul[R "SSS9Ul[R"URSURSSSS9Ul [R"URURSSS S9Ul gr) rZr[rr_r`rarbr\rwrconv_descriptor_adescriptor_decoder_dimconv_descriptor_brs rr[$SuperPointDescriptorDecoder.__init__s GGD) LLQq9 !#  ' ' +  & & " "$  & &  ) ) " rrrcURURURU555n[RR USSS9nUR USUSSS5Sn[R"USS5nU$)zXBased on the encoder output and the keypoints, compute the descriptors for each keypointr)rpr Nrr) rr`rrr. normalize_sample_descriptorsr" transpose)rcrrrCs rrh#SuperPointDescriptorDecoder.forward"s,,TYYt7M7Mg7V-WX mm--kQA-F ..y At@TVWXYZ[ ook1a8 rscalecURup4pVXS- - S-n[R"Xb-US- - S- XR-US- - S- //5nURU5nX-nUS-S- nSS0nUR USSS5n[ R R"X4SS0UD6nURX4S5n[ R RUSSS 9nU$) z-Interpolate descriptors at keypoint locationsr)g?r align_cornersTrtmodebilinearr) rr"tensortoviewrr. grid_samplerr) rrCrr num_channelsrrdivisorkwargss rr/SuperPointDescriptorDecoder._sample_descriptors.s3>2C2C/ & )C/ ,,%-%!)";c"AV^V[^_V_E_beEe ghi**Y' MA% !4(NN:q"a8 mm// bZb[ab !))*BG mm--kQA-F r)rrrbr`)r)rErFrGrHrIr r[r"rlrh staticmethodrjrrNrmrns@rrrsh / D ( u||   3u||rrc\rSrSr\rSrSrSrS\ \ R\ R\ R4SS4SjrS\R S\R 4S jrS rg) SuperPointPreTrainedModeliA superpoint pixel_valuesFmodulerNc [U[R[R45(akURR R SURRS9 URb%URR R5 gg[U[R5(aJURR R5 URR RS5 gg)zInitialize the weightsg)meanstdNg?) isinstancerLinearr\weightdatanormal_rRinitializer_rangebiaszero_ LayerNormfill_)rcrs r _init_weights'SuperPointPreTrainedModel._init_weightsHs fryy"))4 5 5 MM   & &CT[[5R5R & S{{&   &&('  - - KK   " " $ MM   $ $S ).rc:USS2SSS2SS24SS2SSS2SS24$)a Assuming pixel_values has shape (batch_size, 3, height, width), and that all channels values are the same, extract the first channel value to get a tensor of shape (batch_size, 1, height, width) for SuperPoint. This is a workaround for the issue discussed in : https://github.com/huggingface/transformers/pull/25786#issuecomment-1730176446 Args: pixel_values: torch.FloatTensor of shape (batch_size, 3, height, width) Returns: pixel_values: torch.FloatTensor of shape (batch_size, 1, height, width) Nrr)rcrs r extract_one_channel_pixel_values:SuperPointPreTrainedModel.extract_one_channel_pixel_valuesTs$Aq!QJ'4A 66rr)rErFrGrHr config_classbase_model_prefixmain_input_namesupports_gradient_checkpointingrrrr\rrr"rJrrNrrrrrAsd#L$$O&+# *E"))RYY *L$M *RV *7U=N=N7SXSdSd7rrz@ SuperPoint model outputting keypoints and descriptors. ) custom_introc^\rSrSrSrS\SS4U4Sjjr\S S\RS\ \RS \ \ S \ \ S\ \\44 S jj5rS rU=r$)SuperPointForKeypointDetectioniea SuperPoint model. It consists of a SuperPointEncoder, a SuperPointInterestPointDecoder and a SuperPointDescriptorDecoder. SuperPoint was proposed in `SuperPoint: Self-Supervised Interest Point Detection and Description `__ by Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. It is a fully convolutional neural network that extracts keypoints and descriptors from an image. It is trained in a self-supervised manner, using a combination of a photometric loss and a loss based on the homographic adaptation of keypoints. It is made of a convolutional encoder and two decoders: one for keypoints and one for descriptors. rRrNc>[TU]U5 Xl[U5Ul[ U5Ul[U5UlUR5 grg) rZr[rRrpencoderrkeypoint_decoderrdescriptor_decoder post_initrs rr['SuperPointForKeypointDetection.__init__tsE   (0 >v F"=f"E rrlabelsr|r}c PSnUb [S5eUbUOURRnUbUOURRnUR U5nUR upgpUR UUUS9n U Sn U V s/sHoRU S5PM n n U V s/sHoSPM nn U V s/sHoSPM nn [W U5V Vs/sHun nURU SUS5PM nn n[SU55n[R"UUS4URS 9n[R"UU4URS 9n[R"UUURR4URS 9n[R"UU4UR[RS 9n[![XU55HbununnnUUUSUR S24'UUUSUR S24'UUUSUR S24'SUUSUR S24'Md U[R""X/URS 9- nU(aU SOSnU(d[%S UUUUUU455$['UUUUUUS 9$s sn fs sn fs sn fs snn f) a. Examples: ```python >>> from transformers import AutoImageProcessor, SuperPointForKeypointDetection >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> processor = AutoImageProcessor.from_pretrained("magic-leap-community/superpoint") >>> model = SuperPointForKeypointDetection.from_pretrained("magic-leap-community/superpoint") >>> inputs = processor(image, return_tensors="pt") >>> outputs = model(**inputs) ```Nz-SuperPoint does not support training for now.)r|r}r)N.rc3># UHoRSv M g7f)rN)r)rrs rr9SuperPointForKeypointDetection.forward..s#W9OOA$6sr))device)rdtypec3.# UH ocMUv M g7frgrrs rrrsq$_q$_r)rBrrrCrrD)r3rRr|use_return_dictrrrrziprmaxr"r8rrrj enumeraterrr@)rcrrr|r}rBrr:rrencoder_outputsrlist_keypoints_scoreskeypoints_scoreslist_keypoints list_scoresrlist_descriptorsmaximum_num_keypointsrrCrrz _keypoints_scores _descriptorsrDs rrh&SuperPointForKeypointDetection.forwards 4  LM M%9$D $++JjJj &1% ?Yj ! G\\F[2B1-F[\CXYCX/?*CX Y144E~0V 0V,!9  # #$5i$@)IBV W0V  !$#W#W WKK-BA F|ObObc j*?@I\I\]kk . 0R0R S&& {{J(=>|GZGZbgbkbkl6?Naq@r6s 2A2 G\2r@ModulerPrprrrr__all__rrrrs !)) (U   H %)||)%*\\);>)HK)TW) 5<< %&)&u||&U\\&c&eTYT`T`bgbnbnTnNo&0u||000$ =+= =D"))n  n b ,-H Ir