o Zh\@sddlmZmZmZmZmZddlZddlm Z m Z m Z m Z ddl mZmZmZmZmZddlmZmZmZmZmZmZmZmZmZmZddlmZm Z e !e"Z#dej$d e%d eej$fd d Z&Gd dde Z'dgZ(dS))IterableListOptionalTupleUnionN)BaseImageProcessor BatchFeatureget_patch_output_sizeselect_best_resolution) PaddingModeconvert_to_rgbpadresizeto_channel_dimension_format) ChannelDimension ImageInputPILImageResamplingget_image_sizeinfer_channel_dimension_formatis_scaled_imagemake_flat_list_of_imagesto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypeloggingimage patch_sizereturnc Csg}t||d\}}td||D]5}td||D],}|tjkr.|||||||f}n|dd||||||f}||qq|S)a Divides an image into patches of a specified size. Args: image (`np.array`): The input image. patch_size (`int`): The size of each patch. input_data_format (`ChannelDimension` or `str`): The channel dimension format of the input image. Returns: list: A list of np.array representing the patches.  channel_dimrN)rrangerZLASTappend) rrinput_data_formatpatchesheightwidthijpatchr+]/var/www/auris/lib/python3.10/site-packages/transformers/models/aria/image_processing_aria.pydivide_to_patches-s " r-c sVeZdZdZgdZddddddddddejf d eee d eee d e d e d eee e e fdee dee de de e e fdee deffdd Zdddddddddddejdf de eeefd ee e ee fd ee e ee fd ee d ee dee dee dee dee dee dedee eefdeedee eeffddZdejdededejfd d!Zdejdededejfd"d#Zejd$ddfdejd%e e e e e fee e e ffd&ed'e e ee fdee eefdee eefdejfd(d)Zdejd*ee e e fd+e dedededeejfd,d-ZZ S).AriaImageProcessoraG A vision processor for the Aria model that handles image preprocessing. Initialize the AriaImageProcessor. Args: image_mean (`list`, *optional*, defaults to [0.5, 0.5, 0.5]): Mean values for normalization. image_std (`list`, *optional*, defaults to [0.5, 0.5, 0.5]): Standard deviation values for normalization. max_image_size (`int`, *optional*, defaults to 980): Maximum image size. min_image_size (`int`, *optional*, defaults to 336): Minimum image size. split_resolutions (`list`, *optional*, defaults to a list of optimal,resolutions as tuples): The optimal resolutions for splitting the image. split_image (`bool`, *optional*, defaults to `False`): Whether to split the image. do_convert_rgb (`bool`, *optional*, defaults to `True`): Whether to convert the image to RGB. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image by the specified scale `rescale_factor`. Can be overridden by `do_rescale` in the `preprocess` method. rescale_factor (`int` or `float`, *optional*, defaults to `1/255`): Scale factor to use if rescaling the image. Can be overridden by `rescale_factor` in the `preprocess` method. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. resample (PILImageResampling, *optional*, defaults to `BICUBIC`): The resampling filter to use if resizing the image.  pixel_values pixel_mask num_cropsNiPFTgp? image_mean image_stdmax_image_sizemin_image_sizesplit_resolutions split_imagedo_convert_rgb do_rescalerescale_factor do_normalizeresamplec  stjdi| |durgd}|durgd}||_||_||_||_||_|dur7gd}dd|D}||_||_||_ | |_ | |_ | |_ dS)N)?r?r?)))r@r)r@)r@)r@)r@)r@)rArB)rAr)rArA)rAr@)rr@)rrA)rBr@)rBrA)rCr@)rDr@)rEr@)rFr@cSs$g|]}|dd|ddfqS)rr@r+).0elr+r+r, s$z/AriaImageProcessor.__init__..r+) super__init__r6r7r4r5r9r8r:r;r<r=r>) selfr4r5r6r7r8r9r:r;r<r=r>kwargs __class__r+r,rLks& zAriaImageProcessor.__init__ptimagesreturn_tensors data_formatr$c Cs|dur|n|j}|dur|n|j}|dur|n|j}|dur!|n|j}|dur*|n|j}|dur3|n|j}|dur<|n|j}| durE| n|j} | durN| n|j} | durW| n|j } |dvrbt dt |}t |snt dt | ||| || d|rdd|D}dd|D}|rt|d rtd |durt|d }g}g}d}|D]}|r|j||j|| ||d }n|g}|dust||krt|}|D]}t|\}}|t||}||krtt||||f}n |tt|||f}t||| ||d }||d ||d }}t|d |fd |ff||d }tj||ftd}d |d|d d|d f<|||r@|j|| |d}| r\|j||j|j||d }| durZt || |n|}||qqt!tj"|d dtj"|d d|d| dS)aI Process a list of images. Args: images (ImageInput or list of ImageInput): The input image or a list of images. image_mean (`list`, *optional*, defaults to [0.5, 0.5, 0.5]): Mean values for normalization. image_std (`list`, *optional*, defaults to [0.5, 0.5, 0.5]): Standard deviation values for normalization. max_image_size (`int`, *optional*, defaults to `self.max_image_size` (980)): Maximum image size. min_image_size (`int`, *optional*, defaults to `self.min_image_size` (336)): Minimum image size. split_image (`bool`, *optional*, defaults to `self.split_image` (False)): Whether to split the image. do_convert_rgb (`bool`, *optional*, defaults to `self.do_convert_rgb` (True)): Whether to convert the image to RGB. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize` (True)): Whether to normalize the image. resample (PILImageResampling, *optional*, defaults to `self.resample` (BICUBIC)): The resampling filter to use if resizing the image. return_tensors (`str` or `TensorType`, *optional*, defaults to "pt"): The type of tensor to return. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use same as the input image. input_data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. Returns: BatchFeature: A BatchFeature object containing: - 'pixel_values': Tensor of processed image pixel values. - 'pixel_mask': Boolean pixel mask. This mask is a 2D tensor of shape (max_image_size, max_image_size) where: - True (1) values indicate pixels that belong to the original resized image. - False (0) values indicate pixels that are part of the padding. The mask helps distinguish between actual image content and padded areas in subsequent processing steps. - 'num_crops': The maximum number of crops across all images. N)rGr3z(max_image_size must be either 490 or 980zkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.)r=r4r5r>r;r<cSg|]}t|qSr+)r rHrr+r+r,rJz1AriaImageProcessor.preprocess..cSrUr+)rrVr+r+r,rJrWrzIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.rTr$)r>rTr$r@)Zdtype)rscaler$)Zaxisr/)dataZ tensor_type)#r4r5r6r7r9r:r;r<r=r> ValueErrorrrrrloggerZ warning_oncerget_image_patchesr8lenrmaxintrrnpZzerosboolr#Zrescale normalizerr stack)rMrRr4r5r6r7r9r:r;r<r=r>rSrTr$r0Z pixel_masksr2rZ crop_imagesZ crop_imagehwrYnew_sizeZcrop_image_resizedpadding_bottom padding_rightZcrop_image_paddedr1r+r+r, preprocesssI       2  zAriaImageProcessor.preprocessrtarget_resolutionrcCs(t|||\}}t|||f||d}|S)aC Resizes an image to a target resolution while maintaining aspect ratio. Args: image (np.array): The input image. target_resolution (tuple): The target resolution (height, width) of the image. resample (`PILImageResampling`): Resampling filter to use if resizing the image. input_data_format (`ChannelDimension` or `str`): The channel dimension format of the input image. Returns: np.array: The resized and padded image. r>r$)r r)rMrrkr>r$ new_height new_width resized_imager+r+r,_resize_for_patchingUsz'AriaImageProcessor._resize_for_patchingc Csb|\}}t|||\}}t||d\}} t||d\} } |j|| | | f||| ffd} | S)zU Pad an image to a target resolution while maintaining aspect ratio. rA)padding)r divmodr) rMrrkr$Z target_heightZ target_widthrmrnZpaste_xZr_xZpaste_yZr_y padded_imager+r+r,_pad_for_patchingos "z$AriaImageProcessor._pad_for_patchinggrqmodeconstant_valuescCst|ts t|dkrt||||||S|durt|}tjdtjdtjdtj di}t j|||||d}|durAt |||}|S|}|S)a Pads the `image` with the specified `padding` and `mode`. Padding can be in the (`height`, `width`) dimension of in the (`num_patches`) dimension. In the second case an iterable if tuples is expected as input. Args: image (`np.ndarray`): The image to pad. padding (`int` or `Tuple[int, int]` or `Iterable[Tuple[int, int]]`): Padding to apply to the edges of the height, width axes. Can be one of three formats: - `((before_height, after_height), (before_width, after_width))` unique pad widths for each axis. - `((before, after),)` yields same before and after pad for height and width. - `(pad,)` or int is a shortcut for before = after = pad width for all axes. mode (`PaddingMode`): The padding mode to use. Can be one of: - `"constant"`: pads with a constant value. - `"reflect"`: pads with the reflection of the vector mirrored on the first and last values of the vector along each axis. - `"replicate"`: pads with the replication of the last value on the edge of the array along each axis. - `"symmetric"`: pads with the reflection of the vector mirrored along the edge of the array. constant_values (`float` or `Iterable[float]`, *optional*): The value to use for the padding if `mode` is `"constant"`. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use same as the input image. input_data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. Returns: `np.ndarray`: The padded image. rBNZconstantZreflectedgeZ symmetric)rurv) isinstancer`r^rrr CONSTANTZREFLECTZ REPLICATEZ SYMMETRICrar)rMrrqrurvrTr$Zpadding_mode_mappingr+r+r,rs 0zAriaImageProcessor.padgrid_pinpointsrc stt|ts td|}t|d}t||} |j|| |d} |j| | d} t| |d} fdd| D} | S)aY Process an image with variable resolutions by dividing it into patches. Args: image (`np.array`): The input image to be processed. grid_pinpoints (List[Tuple[int, int]]): A list of possible resolutions as tuples. patch_size (`int`): Size of the patches to divide the image into. resample (`PILImageResampling`): Resampling filter to use if resizing the image. data_format (`ChannelDimension` or `str`): The channel dimension format for the output image. input_data_format (`ChannelDimension` or `str`): The channel dimension format of the input image. Returns: `List[np.array]`: A list of NumPy arrays containing the processed image patches. z6grid_pinpoints must be a list of possible resolutions.r rl)r$)rr$csg|] }t|dqS))r!Zinput_channel_dim)r)rHr*rXr+r,rJs z8AriaImageProcessor.get_image_patches..)rxlist TypeErrorrr rprtr-) rMrrzrr>rTr$Zpossible_resolutionsZ image_sizeZbest_resolutionrorsr%r+rXr,r]s    z$AriaImageProcessor.get_image_patches)!__name__ __module__ __qualname____doc__Zmodel_input_namesrZBICUBICrrfloatr`rrbrrLrZFIRSTrstrrrjraarraytuplerprtr ryZndarrayrrr] __classcell__r+r+rOr,r.Is      '      G    Br.))typingrrrrrnumpyraZimage_processing_utilsrr r r Zimage_transformsr r rrrZ image_utilsrrrrrrrrrrutilsrrZ get_loggerr}r\rr`r-r.__all__r+r+r+r,s0  .