o ZhvH @s2dZddlmZmZmZmZddlZddlm Z m Z m Z ddl m Z mZmZddlmZmZmZmZmZmZmZmZmZmZmZddlmZmZmZm Z m!Z!m"Z"dd l#m$Z$e r]ddl%Z%erdddl&Z&e!'e(Z)d d Z* dd ej+d ee,dee,deeee,ffddZ-e$ddGddde Z.dgZ/dS)z%Image processor class for LayoutLMv3.)DictIterableOptionalUnionN)BaseImageProcessor BatchFeature get_size_dict)resizeto_channel_dimension_format to_pil_image) IMAGENET_STANDARD_MEANIMAGENET_STANDARD_STDChannelDimension ImageInputPILImageResamplinginfer_channel_dimension_formatis_scaled_imagemake_list_of_imagesto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypefilter_out_non_signature_kwargsis_pytesseract_availableis_vision_availableloggingrequires_backends)requirescCsLtd|d|td|d|td|d|td|d|gS)Nirr)int)boxwidthheightr%i/var/www/auris/lib/python3.10/site-packages/transformers/models/layoutlmv3/image_processing_layoutlmv3.py normalize_box9s r'imagelangtesseract_configinput_data_formatcsRt||d}|j\}}tj||d|d}|d|d|d|d|df\}} } } } d d t|Dfd d t|D}fd d t| D} fd d t| D} fdd t| D} fdd t| D} g} t| | | | D]\}}}}||||||g}| |qsg}| D] }|t|||qt|t|ksJd||fS)zdApplies Tesseract OCR on a document image, and returns recognized words + normalized bounding boxes.r+dict)r) output_typeconfigtextlefttopr#r$cSsg|] \}}|s|qSr%)strip.0idxwordr%r%r& Qz#apply_tesseract..cg|] \}}|vr|qSr%r%r4Zirrelevant_indicesr%r&r8Rr9cr:r%r%r5r6Zcoordr;r%r&r8Sr9cr:r%r%r<r;r%r&r8Tr9cr:r%r%r<r;r%r&r8Ur9cr:r%r%r<r;r%r&r8Vr9z-Not as many words as there are bounding boxes) r size pytesseractZ image_to_data enumeratezipappendr'len)r(r)r*r+Z pil_imageZ image_widthZ image_heightdatawordsr1r2r#r$Z actual_boxesxywhZ actual_boxZnormalized_boxesr"r%r;r&apply_tesseractBs& , rI)Zvision)backendsc!seZdZdZdgZddejddddddddf dedee e e fd ed ed e d ed ee e ee fdee e ee fdedee dee ddffdd Zejddfdejde e e fd edee e efdee e efdejf ddZeddddddddddddejdfdedeedee e e fd eedee d eed ee e ee fdee e ee fdeedee dee dee e efdedee e efdejjfddZZS)LayoutLMv3ImageProcessora Constructs a LayoutLMv3 image processor. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image's (height, width) dimensions to `(size["height"], size["width"])`. Can be overridden by `do_resize` in `preprocess`. size (`Dict[str, int]` *optional*, defaults to `{"height": 224, "width": 224}`): Size of the image after resizing. Can be overridden by `size` in `preprocess`. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`): Resampling filter to use if resizing the image. Can be overridden by `resample` in `preprocess`. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image's pixel values by the specified `rescale_value`. Can be overridden by `do_rescale` in `preprocess`. rescale_factor (`float`, *optional*, defaults to 1 / 255): Value by which the image's pixel values are rescaled. Can be overridden by `rescale_factor` in `preprocess`. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. Can be overridden by the `do_normalize` parameter in the `preprocess` method. image_mean (`Iterable[float]` or `float`, *optional*, defaults to `IMAGENET_STANDARD_MEAN`): Mean to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`Iterable[float]` or `float`, *optional*, defaults to `IMAGENET_STANDARD_STD`): Standard deviation to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_std` parameter in the `preprocess` method. apply_ocr (`bool`, *optional*, defaults to `True`): Whether to apply the Tesseract OCR engine to get words + normalized bounding boxes. Can be overridden by the `apply_ocr` parameter in the `preprocess` method. ocr_lang (`str`, *optional*): The language, specified by its ISO code, to be used by the Tesseract OCR engine. By default, English is used. Can be overridden by the `ocr_lang` parameter in the `preprocess` method. tesseract_config (`str`, *optional*): Any additional custom configuration flags that are forwarded to the `config` parameter when calling Tesseract. For example: '--psm 6'. Can be overridden by the `tesseract_config` parameter in the `preprocess` method. pixel_valuesTNgp? do_resizer=resample do_rescale rescale_value do_normalize image_mean image_std apply_ocrocr_langr*returnc  stjdi| |dur|nddd}t|}||_||_||_||_||_||_|dur0|nt |_ |dur9|nt |_ | |_ | |_| |_dS)N)r$r#r%)super__init__r rNr=rOrPrescale_factorrRr rSrrTrUrVr*) selfrNr=rOrPrQrRrSrTrUrVr*kwargs __class__r%r&rZs z!LayoutLMv3ImageProcessor.__init__r( data_formatr+cKsTt|}d|vs d|vrtd||d|df}t|f||||d|S)a Resize an image to `(size["height"], size["width"])`. Args: image (`np.ndarray`): Image to resize. size (`Dict[str, int]`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`): `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BILINEAR`. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. Returns: `np.ndarray`: The resized image. r$r#zFThe `size` dictionary must contain the keys `height` and `width`. Got )r=rOr`r+)r ValueErrorkeysr )r\r(r=rOr`r+r]Z output_sizer%r%r&r s#zLayoutLMv3ImageProcessor.resizeimagesr[return_tensorsc s|dur|nj}durnjtdurnj|dur%|nj}dur.nj|dur7|nj}dur@njdurInj| durR| nj } | dur[| nj } | durd| nj } t |}t |sstdt|||ddd|D}|rt|drtddurt|d| rtdg}g}|D]}t|| | d \}}||||q|rχfd d|D}|r܇fd d|D}|rfd d|D}fd d|D}td|i| d}| r||d<||d<|S)a% Preprocess an image or batch of images. Args: images (`ImageInput`): Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`Dict[str, int]`, *optional*, defaults to `self.size`): Desired size of the output image after applying `resize`. resample (`int`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of the `PILImageResampling` filters. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image pixel values between [0, 1]. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to apply to the image pixel values. Only has an effect if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `Iterable[float]`, *optional*, defaults to `self.image_mean`): Mean values to be used for normalization. Only has an effect if `do_normalize` is set to `True`. image_std (`float` or `Iterable[float]`, *optional*, defaults to `self.image_std`): Standard deviation values to be used for normalization. Only has an effect if `do_normalize` is set to `True`. apply_ocr (`bool`, *optional*, defaults to `self.apply_ocr`): Whether to apply the Tesseract OCR engine to get words + normalized bounding boxes. ocr_lang (`str`, *optional*, defaults to `self.ocr_lang`): The language, specified by its ISO code, to be used by the Tesseract OCR engine. By default, English is used. tesseract_config (`str`, *optional*, defaults to `self.tesseract_config`): Any additional custom configuration flags that are forwarded to the `config` parameter when calling Tesseract. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. NzkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.)rPr[rRrSrTrNr=rOcSsg|]}t|qSr%)rr5r(r%r%r&r8Gsz7LayoutLMv3ImageProcessor.preprocess..rzIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.r>r,csg|] }j|dqS))r(r=rOr+)r re)r+rOr\r=r%r&r8^csg|] }j|dqS))r(scaler+)Zrescalere)r+r[r\r%r&r8dscsg|] }j|dqS))r(meanZstdr+) normalizere)rSrTr+r\r%r&r8jrfcsg|] }t|dqS))Zinput_channel_dim)r re)r`r+r%r&r8osrL)rCZ tensor_typerDboxes)rNr=r rOrPr[rRrSrTrUrVr*rrrarrloggerZ warning_oncerrrIrAr)r\rcrNr=rOrPr[rRrSrTrUrVr*rdr`r+Z words_batchZ boxes_batchr(rDrjrCr%)r`rSrTr+rOr[r\r=r& preprocesss|F      z#LayoutLMv3ImageProcessor.preprocess)__name__ __module__ __qualname____doc__Zmodel_input_namesrZBILINEARboolrrstrr!floatrrrZnpndarrayrr rZFIRSTrrPILZImagerl __classcell__r%r%r^r&rKhs&    $  0     rK)N)0rptypingrrrrnumpyrtZimage_processing_utilsrrr Zimage_transformsr r r Z image_utilsr rrrrrrrrrrutilsrrrrrrZutils.import_utilsrrvr>Z get_loggerrmrkr'rurrrIrK__all__r%r%r%r&s:4    &