o Zhe@sddlmZddlmZmZddlmZmZmZm Z ddl Z ddl m Z mZmZddlmZmZmZmZmZddlmZmZmZmZmZmZmZmZmZm Z m!Z!dd l"m#Z#dd l$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+e*rqdd lm,Z,e'rxddl-Z-e(rdd lm.Z.e)rdd l/m0Z1n dd l2m0Z1ndZ.e+3e4Z5edddddddddddddddej6fdee7dee8dee7dee e8e9e8fdee e8e9e8fdee7dee:dee7deedee7deededdee e;e%fdeefdd Zd*e9d"d$e?e:fd+d,Z@d-e e jAd"fd.e:d$e9e e jAd"ffd/d0ZBGd1d2d2ed3d4ZCe&Gd5d6d6e ZDGd7d8d8ZEdS):)Iterable) lru_cachepartial)AnyOptional TypedDictUnionN)BaseImageProcessor BatchFeature get_size_dict)convert_to_rgbget_resize_output_image_sizeget_size_with_aspect_ratiogroup_images_by_shapereorder_images) ChannelDimension ImageInput ImageTypeSizeDictget_image_size#get_image_size_for_max_height_widthget_image_typeinfer_channel_dimension_formatmake_flat_list_of_imagesvalidate_kwargsvalidate_preprocess_arguments)Unpack) TensorTypeauto_docstringis_torch_availableis_torchvision_availableis_torchvision_v2_availableis_vision_availablelogging)PILImageResampling)pil_torch_interpolation_mapping) functional maxsize do_rescalerescale_factor do_normalize image_mean image_stddo_padsize_divisibilitydo_center_crop crop_size do_resizesizeresampler%return_tensors data_formatcCsNt|||||||||| | | d | dur| dkrtd| tjkr%tddS)z Checks validity of typically used arguments in an `ImageProcessorFast` `preprocess` method. Raises `ValueError` if arguments incompatibility is caught. ) r+r,r-r.r/r0r1r2r3r4r5r6Nptz6Only returning PyTorch tensors is currently supported.z6Only channel first data format is currently supported.)r ValueErrorrFIRST)r+r,r-r.r/r0r1r2r3r4r5r6r7r8r<W/var/www/auris/lib/python3.10/site-packages/transformers/image_processing_utils_fast.py"validate_fast_preprocess_argumentsIs& r>tensor torch.TensoraxisreturncCs6|dur|Sz|j|dWSty|YSw)zF Squeezes a tensor, but only if the axis specified has dim 1. N)rA)Zsqueezer:)r?rAr<r<r= safe_squeezets rCvaluescCsddt|DS)zO Return the maximum value across all indices of an iterable of values. cSsg|]}t|qSr<)max).0Zvalues_ir<r<r= z&max_across_indices..)zip)rDr<r<r=max_across_indicessrJimagescCs tdd|D\}}}||fS)zH Get the maximum height and width across all images in a batch. cSsg|]}|jqSr<)shape)rFZimgr<r<r=rGsz(get_max_height_width..)rJ)rK_ max_height max_widthr<r<r=get_max_height_widthsrPimage patch_sizecCsjg}t|tjd\}}td||D]!}td||D]}|dd||||||f}||qq|S)a6 Divides an image into patches of a specified size. Args: image (`Union[np.array, "torch.Tensor"]`): The input image. patch_size (`int`): The size of each patch. Returns: list: A list of Union[np.array, "torch.Tensor"] representing the patches. )Z channel_dimrN)rrr;rangeappend)rQrRZpatchesheightwidthijpatchr<r<r=divide_to_patchess" rZc@seZdZUeeed<eeeefed<eeed<ee ded<eeed<eeeefed<eeed<ee ee fed <eeed <ee e e e fed <ee e e e fed <eeed <ee ee fed<ee ed<ee ee fed<eded<dS)DefaultFastImageProcessorKwargsr4r5default_to_squarer%F.InterpolationModer6r2r3r+r,r-r.r/do_convert_rgbr7r8input_data_format torch.devicedeviceN)__name__ __module__ __qualname__rbool__annotations__dictstrintrfloatlistrrr<r<r<r=r[s"       r[F)totalcseZdZdZdZdZdZdZdZdZ dZ dZ dZ dZ dZdZejZdZdZdgZeZdZdeeddffdd Z  dHd d d ed d dedd f ddZd d dedd fddZd d de ee!efde ee!efdd fddZ"e#dd      dIde$ede$e ee%efde$e ee%efde$ede$ede$d de&fd!d"Z'd#d dededede ee%efde ee%efdd fd$d%Z(d d d e)e*e+fdd fd&d'Z,d e-de-fd(d)Z.de)fd*d+Z/d#e-de-fd,d-Z0   dJd e-d.e$ed/e$e e*efde$d dd f d0d1Z1   dJd#e-d.e$ed/e$e e*efde$d de%d f d2d3Z2      dId e$ed4e$ed5e$ede$e ee%efde$e ee%efd6e$ede)fd7d8Z3            dKde$ede$ede$ede$e ee&efde$e ee&efd9e$ed e$ed:e$ed4e$ed;e$e d<d=e$e e*e4fd6e$efd>d?Z5d#e-deede6fd@dAZ7e8d#e-deede6fdBdCZ9d#e%d d9ed ed e$d d:ed4edededede$e ee%efde$e ee%efd=e$e e*e4fde6fdDdEZ:fdFdGZ;Z|jdus |jdurtd|t||d|dfS)a Center crop an image to `(size["height"], size["width"])`. If the input size is smaller than `crop_size` along any edge, the image is padded with 0's and then center cropped. Args: image (`"torch.Tensor"`): Image to center crop. size (`Dict[str, int]`): Size of the output image. Returns: `torch.Tensor`: The center cropped image. Nz=The size dictionary must have keys 'height' and 'width'. Got rUrV)rUrVr:ryr center_crop)r}rQr5rpr<r<r=rtsz"BaseImageProcessorFast.center_cropcCt|S)a' Converts an image to RGB format. Only converts if the image is of type PIL.Image.Image, otherwise returns the image as is. Args: image (ImageInput): The image to convert. Returns: ImageInput: The converted image. )r )r}rQr<r<r=r z%BaseImageProcessorFast.convert_to_rgbcCsB|jdur|S|jD]}||vrtd|d||q |S)zJ Filter out the unused kwargs from the kwargs dictionary. Nz!This processor does not use the `z ` parameter. It will be ignored.) unused_kwargsloggerZ warning_oncerw)r}rp kwarg_namer<r<r=rvs   z/BaseImageProcessorFast.filter_out_unused_kwargscCr)z Prepare the images structure for processing. Args: images (`ImageInput`): The input images to process. Returns: `ImageInput`: The images with a valid nesting. )r)r}rKr<r<r=_prepare_images_structurerz0BaseImageProcessorFast._prepare_images_structurer_r`cCst|}|tjtjtjfvrtd||r||}|tjkr't|}n |tjkr3t | }|dur;t |}|t jkrI|ddd }|durR||}|S)NzUnsupported input image type rr )rrZPILZTORCHZNUMPYr:r rZ pil_to_tensorrZ from_numpy contiguousrrZLASTZpermuter)r}rQr_r`rbZ image_typer<r<r=_process_images       z%BaseImageProcessorFast._process_imagecCs<||}t|j|||d}g}|D] }|||q|S)z: Prepare the input images for processing. )r_r`rb)rrrrT)r}rKr_r`rbZprocess_image_fnprocessed_imagesrQr<r<r=_prepare_input_imagess z,BaseImageProcessorFast._prepare_input_imagesr3r\r8cKs|duri}|durtd it||d}|dur$td it|dd}t|tr-t|}t|tr6t|}|dur=tj}||d<||d<||d<||d<||d<||d <|S) z Update kwargs that need further processing before being validated Can be overridden by subclasses to customize the processing of kwargs. Nrqr3rrr5r\r.r/r8r<)rr isinstancerltuplerr;)r}r5r3r\r.r/r8rpr<r<r=_further_process_kwargss&  z.BaseImageProcessorFast._further_process_kwargsr4r2r6r]r7c Ks$t||||||||| | | | d dS)z@ validate the kwargs for the preprocess method. ) r+r,r-r.r/r4r5r2r3r6r7r8N)r>)r}r+r,r-r.r/r4r5r2r3r6r7r8rpr<r<r=_validate_preprocess_kwargss z2BaseImageProcessorFast._validate_preprocess_kwargscOs|j|g|Ri|SN) preprocess)r}rKargsrpr<r<r=__call__8szBaseImageProcessorFast.__call__c Ost||jd|jD] }||t||dq |d}|d}|d}|j||||d}|jd i|}|jd i||d}t |t t frQt |n||d<|d|d |j |g|Ri|S) N)Zcaptured_kwargsZvalid_processor_keysr_r`rb)rKr_r`rbr6rr\r8r<)rryr| setdefaultr{rwrrrrrjr%r& _preprocess) r}rKrrprr_r`rbr6r<r<r=r;s"       z!BaseImageProcessorFast.preprocessc  Kst|\}}i}|D]\}}|r|j|||d}|||<q t||}t|\}}i}|D]\}}|r<|||}||||| | | }|||<q0t||}| rYtj|ddn|}td|i| dS)N)rQr5rrdimro)dataZ tensor_type) ritemsrrrrrstackr )r}rKr4r5rr2r3r+r,r-r.r/r7rpZgrouped_imagesZgrouped_images_indexZresized_images_groupedrLZstacked_imagesZresized_imagesZprocessed_images_groupedrr<r<r=rcs&        z"BaseImageProcessorFast._preprocesscs&t}|dd|dd|S)NZ_valid_processor_keysr|)rtto_dictrw)r}Z encoder_dictrr<r=rs   zBaseImageProcessorFast.to_dict)NT)NNNNNN)NNN) NNNNNNNNNNNN)=rcrdrer6r.r/r5r\r3r4r2r+r,r-r_r7rr;r8r`rbZmodel_input_namesr[rxrrrurrfrrkrrrrrrrlrrrrhrirjrrr rvrrrrrrr rrrrr __classcell__r<r<rr=rns 1          %  &     "'      +rnc@s$eZdZddeeefddZdS)SemanticSegmentationMixinN target_sizescs|j}|dur@t|t|krtdgtt|D]"}tjjj||jdd||ddd}|dj dd} |qS|j ddfd d tj dDS) aD Converts the output of [`MobileNetV2ForSemanticSegmentation`] into semantic segmentation maps. Only supports PyTorch. Args: outputs ([`MobileNetV2ForSemanticSegmentation`]): Raw outputs of the model. target_sizes (`List[Tuple]` of length `batch_size`, *optional*): List of tuples corresponding to the requested final size (height, width) of each prediction. If unset, predictions will not be resized. Returns: semantic_segmentation: `List[torch.Tensor]` of length `batch_size`, where each item is a semantic segmentation map of shape (height, width) corresponding to the target_sizes entry (if `target_sizes` is specified). Each entry of each `torch.Tensor` correspond to a semantic class id. NzTMake sure that you pass in as many target sizes as the batch dimension of the logitsrrZbilinearF)r5modeZ align_cornersr csg|]}|qSr<r<)rFrWZsemantic_segmentationr<r=rGrHzPSemanticSegmentationMixin.post_process_semantic_segmentation..) logitslenr:rSrnnr'Z interpolateZ unsqueezeZargmaxrTrL)r}ZoutputsrridxZresized_logitsZ semantic_mapr<rr="post_process_semantic_segmentations"  zrCrJrrParrayrZr[rnrr<r<r<r=s 4 $         *   Z