o Zhґ@sddlmZmZddlmZddlmZmZddlZ ddl m Z m Z m Z mZmZddlmZmZmZmZmZddlmZmZmZmZmZerQddlZdd l mZerXddlZer_ddlZ erhddl!mZ" dbd e j#d ee e$fd eee e$fd e j#fddZ%de j&dfd e j#de'dee de j(deee$e fd e j#f ddZ)ddZ*   dcd ee j#ddddfdee+dee$deee$e fd df ddZ,dbd e-e.e.ffd d!Z/ "  ddd#e j#d$ee.e-e.e.fe0e.e-e.fd%e+d&ee.deee$e fd e-f d'd(Z1    " ded e j#d$e-e.e.fd)d*d+ee.dee d,e+deee$e fd e j#fd-d.Z2  dfd e j#d/ee'ee'fd0ee'ee'fdee deee$e fd e j#f d1d2Z3  dfd e j#d$e-e.e.fdeee$e fdeee$e fd e j#f d3d4Z4dgd6d7Z5d5e j#d e j#fd8d9Z6dhd:d;Z7d5ed efde j#d e j#fdAdBZ:djdCdDZ;d>ed efdEdFZGdKdLdLeZ?e?j@dMddfd e j#dNee.e-e.e.fee-e.e.ffdOe?dPee'ee'fdeee$e fdeee$e fd e j#fdQdRZAd e d e fdSdTZB  dfd e j#dee deee$e fd e j#fdUdVZCdWdXZDdYe0dd e-eEe-e.e.fe0dfeEe.e-e-e.e.fe.ffffdZd[ZFd\eEe-e.e.fdfd]eEe.e-e.e.ffd e0dfd^d_ZGGd`dadaZHdS)k) CollectionIterable)ceil)OptionalUnionN)ChannelDimension ImageInputget_channel_dimension_axisget_image_sizeinfer_channel_dimension_format) ExplicitEnum TensorType is_jax_tensor is_tf_tensoris_torch_tensor)is_flax_availableis_tf_availableis_torch_availableis_vision_availablerequires_backends)PILImageResamplingimage channel_diminput_channel_dimreturncCst|tjstdt||durt|}t|}||kr!|S|tjkrDtt |j d|j d|j d|j dg}| |}|S|tj krgtt |j d|j d|j d|j dg}| |}|St d|)a Converts `image` to the channel dimension format specified by `channel_dim`. The input can have arbitrary number of leading dimensions. Only last three dimension will be permuted to format the `image`. Args: image (`numpy.ndarray`): The image to have its channel dimension set. channel_dim (`ChannelDimension`): The channel dimension format to use. input_channel_dim (`ChannelDimension`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred from the input image. Returns: `np.ndarray`: The image with the channel dimension set to `channel_dim`. ,Input image must be of type np.ndarray, got Nrz&Unsupported channel dimension format: ) isinstancenpndarray TypeErrortyper rFIRSTlistrangendim transposeLAST ValueError)rrrZtarget_channel_dimZaxesr+L/var/www/auris/lib/python3.10/site-packages/transformers/image_transforms.pyto_channel_dimension_format5s  .  . r-scale data_formatdtypeinput_data_formatcCsPt|tjstdt||tj|}|dur!t|||}||}|S)a Rescales `image` by `scale`. Args: image (`np.ndarray`): The image to rescale. scale (`float`): The scale to use for rescaling the image. data_format (`ChannelDimension`, *optional*): The channel dimension format of the image. If not provided, it will be the same as the input image. dtype (`np.dtype`, *optional*, defaults to `np.float32`): The dtype of the output image. Defaults to `np.float32`. Used for backwards compatibility with feature extractors. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred from the input image. Returns: `np.ndarray`: The rescaled image. rN)rr r!r"r#astypeZfloat64r-)rr.r/r0r1Zrescaled_imager+r+r,rescale`s   r3cCs|jtjkr d}|St||tr4td|kr%t|dkr%d}|Std|d| dtd|krFt|dkrFd}|Std |d| d) z Detects whether or not the image needs to be rescaled before being converted to a PIL image. The assumption is that if the image is of type `np.float` and all values are between 0 and 1, it needs to be rescaled. FrzZThe image to be converted to a PIL image contains values outside the range [0, 255], got [z, z%] which cannot be converted to uint8.rTzXThe image to be converted to a PIL image contains values outside the range [0, 1], got [) r0r uint8Zallcloser2intallr*minmax)r do_rescaler+r+r,_rescale_for_pil_conversions0  r;zPIL.Image.Image torch.Tensor tf.Tensorz jnp.ndarrayr: image_modecCsttdgt|tjjr|St|st|r|}nt|r&t |}nt|t j s5t dt |t|tj|}|jddkrJt j|ddn|}|durTt|n|}|r]t|d}|t j}tjj||dS) a Converts `image` to a PIL Image. Optionally rescales it and puts the channel dimension back as the last axis if needed. Args: image (`PIL.Image.Image` or `numpy.ndarray` or `torch.Tensor` or `tf.Tensor`): The image to convert to the `PIL.Image` format. do_rescale (`bool`, *optional*): Whether or not to apply the scaling factor (to make pixel values integers between 0 and 255). Will default to `True` if the image type is a floating type and casting to `int` would result in a loss of precision, and `False` otherwise. image_mode (`str`, *optional*): The mode to use for the PIL image. If unset, will use the default mode for the input image type. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If unset, will use the inferred format from the input. Returns: `PIL.Image.Image`: The converted image. visionz Input image type not supported: rZaxisNr4mode)r to_pil_imagerPILImagerrnumpyrr arrayr!r*r#r-rr)shapeZsqueezer;r3r2r5Z fromarray)rr:r>r1r+r+r,rDs       rDc Cs|\}}d}|dur.tt||f}tt||f}||||kr.|||}tt|}||kr6||ks>||krG||krG||}} || fS||krm|} |dura|durat|||}|| fSt|||}|| fS|}|dur|durt|||} || fSt|||} || fS)aC Computes the output image size given the input image size and the desired output size. Args: image_size (`Tuple[int, int]`): The input image size. size (`int`): The desired output size. max_size (`int`, *optional*): The maximum allowed output size. N)floatr8r9r6round) Z image_sizesizemax_sizeheightwidthZraw_sizeZmin_original_sizeZmax_original_sizeZohowr+r+r,get_size_with_aspect_ratios0      rQT input_imagerLdefault_to_squarerMc Cst|ttfr t|dkrt|St|dkr|d}ntd|r&||fSt||\}}||kr5||fn||f\}}|} | t| ||} } |duri|| krZtd|d|| |krit|| | |} } ||krq| | fS| | fS)a Find the target (height, width) dimension of the output image after resizing given the input image and the desired size. Args: input_image (`np.ndarray`): The image to resize. size (`int` or `Tuple[int, int]` or List[int] or `Tuple[int]`): The size to use for resizing the image. If `size` is a sequence like (h, w), output size will be matched to this. If `size` is an int and `default_to_square` is `True`, then image will be resized to (size, size). If `size` is an int and `default_to_square` is `False`, then smaller edge of the image will be matched to this number. i.e, if height > width, then image will be rescaled to (size * height / width, size). default_to_square (`bool`, *optional*, defaults to `True`): How to convert `size` when it is a single int. If set to `True`, the `size` will be converted to a square (`size`,`size`). If set to `False`, will replicate [`torchvision.transforms.Resize`](https://pytorch.org/vision/stable/transforms.html#torchvision.transforms.Resize) with support for resizing only the smallest edge and providing an optional `max_size`. max_size (`int`, *optional*): The maximum allowed for the longer edge of the resized image: if the longer edge of the image is greater than `max_size` after being resized according to `size`, then the image is resized again so that the longer edge is equal to `max_size`. As a result, `size` might be overruled, i.e the smaller edge may be shorter than `size`. Only used if `default_to_square` is `False`. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If unset, will use the inferred format from the input. Returns: `tuple`: The target (height, width) dimension of the output image after resizing. rrrz7size must have 1 or 2 elements if it is a list or tupleNz max_size = zN must be strictly greater than the requested size for the smaller edge size = )rtupler%lenr*r r6) rRrLrSrMr1rNrOZshortlongZrequested_new_shortZ new_shortZnew_longr+r+r,get_resize_output_image_sizes,%   rWresampler reducing_gap return_numpyc Csttdg|dur |ntj}t|dkstd|dur!t|}|dur'|n|}d}t|tj j s=t |}t |||d}|\}} |j| |f||d} |rqt | } | jdkr^t j| dd n| } t| |tjd } |rot| d n| } | S) a Resizes `image` to `(height, width)` specified by `size` using the PIL library. Args: image (`np.ndarray`): The image to resize. size (`Tuple[int, int]`): The size to use for resizing the image. resample (`int`, *optional*, defaults to `PILImageResampling.BILINEAR`): The filter to user for resampling. reducing_gap (`int`, *optional*): Apply optimization by resizing the image in two steps. The bigger `reducing_gap`, the closer the result to the fair resampling. See corresponding Pillow documentation for more details. data_format (`ChannelDimension`, *optional*): The channel dimension format of the output image. If unset, will use the inferred format from the input. return_numpy (`bool`, *optional*, defaults to `True`): Whether or not to return the resized image as a numpy array. If False a `PIL.Image.Image` object is returned. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If unset, will use the inferred format from the input. Returns: `np.ndarray`: The resized image. r?Nrzsize must have 2 elementsF)r:r1)rXrYr@rArgp?)rresizerZBILINEARrUr*r rrErFr;rDr rHr'Z expand_dimsr-rr)r3) rrLrXrYr/rZr1r:rNrOZ resized_imager+r+r,r\Bs* !  r\meanstdcCs2t|tjs td|durt|}t||d}|j|}t|jtj s+| tj }t|t rCt ||krBtd|dt |n|g|}tj||jd}t|t rht ||krgtd|dt |n|g|}tj||jd}|tjkr|||}n|j||j}|durt|||}|S|}|S)a Normalizes `image` using the mean and standard deviation specified by `mean` and `std`. image = (image - mean) / std Args: image (`np.ndarray`): The image to normalize. mean (`float` or `Collection[float]`): The mean to use for normalization. std (`float` or `Collection[float]`): The standard deviation to use for normalization. data_format (`ChannelDimension`, *optional*): The channel dimension format of the output image. If unset, will use the inferred format from the input. input_data_format (`ChannelDimension`, *optional*): The channel dimension format of the input image. If unset, will use the inferred format from the input. zimage must be a numpy arrayN)r1zmean must have z$ elements if it is an iterable, got r0zstd must have )rr r!r*r r rIZ issubdtyper0Zfloatingr2float32rrUrHrr)Tr-)rr]r^r/r1Z channel_axisZ num_channelsr+r+r, normalizes6           rbcCsttdgt|tjstdt|t|tr t|dkr$t d|dur,t |}|dur2|n|}t |t j |}t|t j \}}|\}}t|t|}}||d} | |} ||d} | |} | dkr| |kr| dkr| |kr|d| | | | f}t ||t j }|St||} t||}|jdd| |f}tj||d }t| |d}||}t||d}||}||d||||f<| |7} | |7} | |7} | |7} |dtd| t| | td| t|| f}t ||t j }|S) a Crops the `image` to the specified `size` using a center crop. Note that if the image is too small to be cropped to the size given, it will be padded (so the returned result will always be of size `size`). Args: image (`np.ndarray`): The image to crop. size (`Tuple[int, int]`): The target size for the cropped image. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. input_data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. Returns: `np.ndarray`: The cropped image. r?rrzOsize must have 2 elements representing the height and width of the output imageNr.)rI)r center_croprr r!r"r#rrUr*r r-rr$r r6r9rIZ zeros_likerr8)rrLr/r1Zoutput_data_formatZ orig_heightZ orig_widthZ crop_heightZ crop_widthtopbottomleftrightZ new_heightZ new_widthZ new_shapeZ new_imageZtop_padZ bottom_padZleft_padZ right_padr+r+r,rdsH       .rd bboxes_centercCsL|d\}}}}tj|d||d||d||d|gdd}|S)Nr@?dimZunbindtorchstack)ricenter_xcenter_yrOrNZ bbox_cornersr+r+r,_center_to_corners_format_torchs *rrcCsH|j\}}}}tj|d||d||d||d|gdd}|S)Nrjr@rArar rorirprqrOrNbboxes_cornersr+r+r,_center_to_corners_format_numpy$s *rvcCsPtj|dd\}}}}tj|d||d||d||d|gdd}|S)Nr@rArjtfZunstackrortr+r+r,_center_to_corners_format_tf.s *rycCFt|rt|St|tjrt|St|rt|Stdt |)a} Converts bounding boxes from center format to corners format. center format: contains the coordinate for the center of the box and its width, height dimensions (center_x, center_y, width, height) corners format: contains the coordinates for the top-left and bottom-right corners of the box (top_left_x, top_left_y, bottom_right_x, bottom_right_y) Unsupported input type ) rrrrr r!rvrryr*r#)rir+r+r,center_to_corners_format9s  r|rucCsD|d\}}}}||d||d||||g}tj|ddS)Nr@rrkrm)ru top_left_x top_left_ybottom_right_xbottom_right_ybr+r+r,_corners_to_center_format_torchNs  rcCs@|j\}}}}tj||d||d||||gdd}|S)Nrr@rArsrur}r~rrrir+r+r,_corners_to_center_format_numpyYs   rcCsHtj|dd\}}}}tj||d||d||||gdd}|S)Nr@rArrwrr+r+r,_corners_to_center_format_tfgs   rcCrz)a Converts bounding boxes from corners format to center format. corners format: contains the coordinates for the top-left and bottom-right corners of the box (top_left_x, top_left_y, bottom_right_x, bottom_right_y) center format: contains the coordinate for the center of the box and its the width, height dimensions (center_x, center_y, width, height) r{) rrrr r!rrrr*r#)rur+r+r,corners_to_center_formatus  rcCst|tjr>t|jdkr>|jtjkr|tj}|dddddfd|dddddfd|dddddfSt |dd|dd|dS)z* Converts RGB color to unique ID. rNrrir) rr r!rUrIr0r5r2Zint32r6)colorr+r+r, rgb_to_ids   J$rcCst|tjr1|}tt|jdg}tj|tjd}t dD]}|d|d|f<|d}q |Sg}t dD] }| |d|d}q7|S)z* Converts unique ID to RGB color. rr_r.) rr r!copyrTr%rIZzerosr5r&append)Zid_mapZ id_map_copyZ rgb_shapeZrgb_mapir_r+r+r, id_to_rgbs     rc@s eZdZdZdZdZdZdZdS) PaddingModezP Enum class for the different padding modes to use when padding images. constantreflectZ replicate symmetricN)__name__ __module__ __qualname____doc__CONSTANTREFLECT REPLICATE SYMMETRICr+r+r+r,rs rgpaddingrCconstant_valuescsdurtfdd}||}|tjkr&||}tj|d|dn1|tjkr4tj|ddn#|tjkrBtj|ddn|tjkrPtj|d dntd ||durct |SS) a Pads the `image` with the specified (height, width) `padding` and `mode`. Args: image (`np.ndarray`): The image to pad. padding (`int` or `Tuple[int, int]` or `Iterable[Tuple[int, int]]`): Padding to apply to the edges of the height, width axes. Can be one of three formats: - `((before_height, after_height), (before_width, after_width))` unique pad widths for each axis. - `((before, after),)` yields same before and after pad for height and width. - `(pad,)` or int is a shortcut for before = after = pad width for all axes. mode (`PaddingMode`): The padding mode to use. Can be one of: - `"constant"`: pads with a constant value. - `"reflect"`: pads with the reflection of the vector mirrored on the first and last values of the vector along each axis. - `"replicate"`: pads with the replication of the last value on the edge of the array along each axis. - `"symmetric"`: pads with the reflection of the vector mirrored along the edge of the array. constant_values (`float` or `Iterable[float]`, *optional*): The value to use for the padding if `mode` is `"constant"`. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use same as the input image. input_data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. Returns: `np.ndarray`: The padded image. Ncst|ttfr||f||ff}nOt|tr,t|dkr,|d|df|d|dff}n3t|trCt|dkrCt|dtrC||f}nt|trXt|dkrXt|dtrX|}ntd|tjkrjdg|Rng|dR}jdkr~dg|R}|S|}|S)za Convert values to be in the format expected by np.pad based on the data format. rrrzUnsupported format: )rr) rr6rJrTrUr*rr$r')valuesrr1r+r,_expand_for_data_formats"$ $$z$pad.._expand_for_data_formatr)rCrrrBedgerzInvalid padding mode: ) r rrr padrrrr*r-)rrrCrr/r1rr+rr,rs$+    rcCs:ttdgt|tjjs|S|jdkr|S|d}|S)z Converts an image to RGB format. Only converts if the image is of type PIL.Image.Image, otherwise returns the image as is. Args: image (Image): The image to convert. r?RGB)rconvert_to_rgbrrErFrCconvert)rr+r+r,r s   rcCsx|durt|n|}|tjkr|ddddf}n|tjkr(|ddddf}ntd||dur:t|||d}|S)a Flips the channel order of the image. If the image is in RGB format, it will be converted to BGR and vice versa. Args: image (`np.ndarray`): The image to flip. data_format (`ChannelDimension`, *optional*): The channel dimension format for the output image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use same as the input image. input_data_format (`ChannelDimension`, *optional*): The channel dimension format for the input image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. If unset, will use the inferred format of the input image. N.r@zUnsupported channel dimension: r[)r rr)r$r*r-)rr/r1r+r+r,flip_channel_order s  rcCs|r|S|SN)Zis_floating_pointrJ)xr+r+r,_cast_tensor_to_floatFsrimagescCsxi}i}t|D]&\}}|jdd}||vrg||<||||t||df||<qdd|D}||fS)a Groups images by shape. Returns a dictionary with the shape as key and a list of images with that shape as value, and a dictionary with the index of the image in the original list as key and the shape and index in the grouped list as value. rNcSs i|] \}}|tj|ddqS)rrk)rnro).0rIrr+r+r, ]s z)group_images_by_shape..) enumeraterIrrUitems)rZgrouped_imagesgrouped_images_indexrrrIr+r+r,group_images_by_shapeLsrprocessed_imagesrcsfddttDS)z> Reconstructs a list of images in the original order. cs(g|]}|d|dqS)rrr+)rrrrr+r, gsz"reorder_images..)r&rU)rrr+rr,reorder_imagesas  rc@s eZdZdZdejfddZdS) NumpyToTensorz4 Convert a numpy array to a PyTorch tensor. rcCst|dddS)Nrrr)rnZ from_numpyr( contiguous)selfrr+r+r,__call__rszNumpyToTensor.__call__N)rrrrr r!rr+r+r+r,rmsrr)NNN)TNN)NNNTN)NN)rir<rr<)rir=rr=)rur<rr<)rur=rr=)Icollections.abcrrmathrtypingrrrGr Z image_utilsrr r r r utilsr rrrrZutils.import_utilsrrrrrrErrnZ tensorflowrxZ jax.numpyZjnpr!strr-r`rJr0r3r;boolrDrTr6rQr%rWr\rbrdrrrvryr|rrrrrrrrrrrrdictrrrr+r+r+r,sh      . & 6+ F  K @  T      W &6