o ZhI@shdZddlmZmZmZmZmZmZddlZ ddl m Z m Z ddl mZmZmZddlmZmZddlmZmZmZmZmZmZmZmZmZmZmZm Z dd l!m"Z"m#Z#m$Z$dd l%m&Z&e rcddl'Z'erkd d l(m)Z)e rrddl*Z*e#+e,Z- dd edeee.effddZ/ dd edeee.efdefddZ0defddZ1e&ddGdddeZ2dgZ3dS)z%Image processor class for SuperPoint.) TYPE_CHECKINGDictListOptionalTupleUnionN)is_torch_availableis_vision_available)BaseImageProcessor BatchFeature get_size_dict)resizeto_channel_dimension_format) ChannelDimension ImageInput ImageTypePILImageResamplingget_image_typeinfer_channel_dimension_format is_pil_imageis_scaled_imageis_valid_imageto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypeloggingrequires_backends)requires)KeypointMatchingOutputimageinput_data_formatcCs|tjkr$|jddkrdSt|d|dko#t|d|dkS|tjkrH|jddkr2dSt|d|d koGt|d |d kSdS) Nrr Tr.r ...r.r .r')rFIRSTshapenpallLAST)r"r#r1g/var/www/auris/lib/python3.10/site-packages/transformers/models/superglue/image_processing_superglue.py is_grayscale6s , ,r3returncCsttdgt|tjrZt||dr|S|tjkr7|dd|dd|dd}tj|gd d d }|S|tj krX|d d|d d|dd}tj|gd dd }|St|t j j sc|S| d}|S)ao Converts an image to grayscale format using the NTSC formula. Only support numpy and PIL Image. TODO support torch and tensorflow grayscale conversion This function is supposed to return a 1-channel image, but it returns a 3-channel image with the same value in each channel, because of an issue that is discussed in : https://github.com/huggingface/transformers/pull/25786#issuecomment-1730176446 Args: image (Image): The image to convert. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. Zvisionr#r$gŏ1w-!?r%gbX9?r&gv/?rr)Zaxisr)r*r+r(L) rconvert_to_grayscale isinstancer.ndarrayr3rr,stackr0PILZImageconvert)r"r#Z gray_imager1r1r2r7Es    $ $ r7imagescshd}ddt|tr0t|dkrtfdd|Dr|Stfdd|Dr0dd |DSt|) N)z-Input images must be a one of the following :z - A pair of PIL images.z - A pair of 3D arrays.z! - A list of pairs of PIL images.z - A list of pairs of 3D arrays.cSs,t|pt|ot|tjkot|jdkS)z$images is a PIL Image or a 3D array.r)rrrrr;lenr-)r"r1r1r2_is_valid_imagets"z8validate_and_format_image_pairs.._is_valid_imager'c3|]}|VqdSNr1.0r"r?r1r2 {z2validate_and_format_image_pairs..c3s<|]}t|tot|dkotfdd|DVqdS)r'c3r@rAr1rBrDr1r2rErFz..N)r8listr>r/)rC image_pairrDr1r2rE}s  cSsg|] }|D]}|qqSr1r1)rCrHr"r1r1r2 sz3validate_and_format_image_pairs..)r8rGr>r/ ValueError)r= error_messager1rDr2validate_and_format_image_pairsks " rL)torch)backendscsVeZdZdZdgZddejdddfdedee e e fded ed e d ed dffd d Z  ddejde e e fdeee efdeee effddZdddddddejdf deedee e e fded eed ee d eedeee efdedeee efd efddZ ddddeeeefde d ee e ejffddZZS) SuperGlueImageProcessorap Constructs a SuperGlue image processor. Args: do_resize (`bool`, *optional*, defaults to `True`): Controls whether to resize the image's (height, width) dimensions to the specified `size`. Can be overridden by `do_resize` in the `preprocess` method. size (`Dict[str, int]` *optional*, defaults to `{"height": 480, "width": 640}`): Resolution of the output image after `resize` is applied. Only has an effect if `do_resize` is set to `True`. Can be overridden by `size` in the `preprocess` method. resample (`PILImageResampling`, *optional*, defaults to `Resampling.BILINEAR`): Resampling filter to use if resizing the image. Can be overridden by `resample` in the `preprocess` method. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image by the specified scale `rescale_factor`. Can be overridden by `do_rescale` in the `preprocess` method. rescale_factor (`int` or `float`, *optional*, defaults to `1/255`): Scale factor to use if rescaling the image. Can be overridden by `rescale_factor` in the `preprocess` method. do_grayscale (`bool`, *optional*, defaults to `True`): Whether to convert the image to grayscale. Can be overridden by `do_grayscale` in the `preprocess` method. pixel_valuesTNgp? do_resizesizeresample do_rescalerescale_factor do_grayscaler4c s\tjdi||dur|nddd}t|dd}||_||_||_||_||_||_dS)Nii)heightwidthFZdefault_to_squarer1) super__init__r rQrRrSrTrUrV)selfrQrRrSrTrUrVkwargs __class__r1r2r[s   z SuperGlueImageProcessor.__init__r" data_formatr#cKs0t|dd}t|f|d|df||d|S)aL Resize an image. Args: image (`np.ndarray`): Image to resize. size (`Dict[str, int]`): Dictionary of the form `{"height": int, "width": int}`, specifying the size of the output image. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the output image. If not provided, it will be inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. FrYrWrX)rRr`r#)r r)r\r"rRr`r#r]r1r1r2rs zSuperGlueImageProcessor.resizereturn_tensorsc  sp|dur|n|j}|dur|n|j}|dur|n|j}|dur!|n|j}|dur*|n|j}|dur3|n|j}t|dd}t|}t|sHt dt |||||ddd|D}t |dre|ret d | durot|d} g|D]+} |r|j| ||| d } |r|j| || d } |rt| | d } t| | | d } | qsfddtdtdD} d| i}t||dS)a Preprocess an image or batch of images. Args: images (`ImageInput`): Image pairs to preprocess. Expects either a list of 2 images or a list of list of 2 images list with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`Dict[str, int]`, *optional*, defaults to `self.size`): Size of the output image after `resize` has been applied. If `size["shortest_edge"]` >= 384, the image is resized to `(size["shortest_edge"], size["shortest_edge"])`. Otherwise, the smaller edge of the image will be matched to `int(size["shortest_edge"]/ crop_pct)`, after which the image is cropped to `(size["shortest_edge"], size["shortest_edge"])`. Only has an effect if `do_resize` is set to `True`. resample (`PILImageResampling`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of `PILImageResampling`, filters. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image values between [0 - 1]. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_grayscale (`bool`, *optional*, defaults to `self.do_grayscale`): Whether to convert the image to grayscale. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. NFrYzkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.)rQrRrSrTrUcSsg|]}t|qSr1)rrBr1r1r2rI3sz6SuperGlueImageProcessor.preprocess..rzIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.)r"rRrSr#)r"scaler#r5)Zinput_channel_dimcsg|] }||dqS)r'r1)rCiZ all_imagesr1r2rINsr'rP)dataZ tensor_type)rQrSrTrUrVrRr rLrrJrrloggerZ warning_oncerrZrescaler7rappendranger>r )r\r=rQrRrSrTrUrVrar`r#r]r"Z image_pairsrer1rdr2 preprocesssN:      z"SuperGlueImageProcessor.preprocessoutputsr! target_sizes thresholdcCsp|jjdt|krtdtdd|Dstdt|tr*tj||jj d}n|jddks8|jddkr)rCZ target_sizer1r1r2rElszISuperGlueImageProcessor.post_process_keypoint_matching..zTEach element of target_sizes must contain the size (h, w) of each image of the batch)devicer r'r(N) keypoints0 keypoints1matching_scores)maskr-r>rJr/r8rrMZtensorrn keypointscloneflipZreshapetoZint32zipmatchesrq logical_andrg)r\rkrlrmZimage_pair_sizesrsresultsZ mask_pairZkeypoints_pairrxZscoresZmask0Zmask1rorpZmatches0Zscores0Z valid_matchesZmatched_keypoints0Zmatched_keypoints1rqr1r1r2post_process_keypoint_matchingTsF   &     z6SuperGlueImageProcessor.post_process_keypoint_matching)NN)rj)__name__ __module__ __qualname____doc__Zmodel_input_namesrZBILINEARboolrrstrintfloatr[r.r9rrrr,rr rirrrMZTensorr{ __classcell__r1r1r^r2rOs   *    zrOrA)4rtypingrrrrrrnumpyr.r r Zimage_processing_utilsr r r Zimage_transformsrrZ image_utilsrrrrrrrrrrrrutilsrrrZutils.import_utilsrrMZmodeling_supergluer!r;Z get_loggerr|rfrr3r7rLrO__all__r1r1r1r2sH 8     &