o Zh1@sdZddlmZddlmZmZmZddlZddl m Z ddl m Z m Z mZmZddlmZmZmZmZdd lmZdd lmZerGddlZGd d d e ZGd dde ddZGdddeZdgZdS)z Processor class for SAMHQ. )deepcopy)ListOptionalUnionN) ImageInput) ImagesKwargsProcessingKwargsProcessorMixinUnpack) AudioInput BatchEncodingPreTokenizedInput TextInput)is_torch_available) VideoInputc@sfeZdZUeeed<eeeeed<eeeeed<eeeeeed<eeed<dS)SamHQImagesKwargssegmentation_maps input_points input_labels input_boxespoint_pad_valueN) __name__ __module__ __qualname__rr__annotations__rfloatintrrZ/var/www/auris/lib/python3.10/site-packages/transformers/models/sam_hq/processing_samhq.pyr#s rc@s"eZdZUeed<dddiiZdS)SamHQProcessorKwargs images_kwargsrN)rrrrr _defaultsrrrrr +s r F)totalcseZdZdZdgZdZgdZfddZ d)ddddd ee d ee e e e e e e fd eed eed eedef ddZ     d*ddZddZ d+dedejdejfddZd,ddZ   d-dd Zed!d"Zd#d$Zd%d&Zd+d'd(Z Z!S).SamHQProcessora Constructs a SAM HQ processor which wraps a SAM image processor and an 2D points & Bounding boxes processor into a single processor. [`SamHQProcessor`] offers all the functionalities of [`SamImageProcessor`]. See the docstring of [`~SamImageProcessor.__call__`] for more information. Args: image_processor (`SamImageProcessor`): An instance of [`SamImageProcessor`]. The image processor is a required input. image_processorZSamImageProcessor)rrrrcsDt|t|dstdt|jdstd|jjd|_dS)Nr%z,image_processor was not properly initializedsizezimage_processor.size is not set longest_edge)super__init__hasattr ValueErrorr%r& target_size)selfr% __class__rrr)Ks   zSamHQProcessor.__init__N)textaudiovideoimagesr0r1r2kwargsreturnc Os|jtfdii||j|}|ddd}|ddd} |ddd} |j|fi|d} | d} t| dr@| } |j|| | d \}} } |j| | || | |d  d |d d d } | S)z This method uses [`SamImageProcessor.__call__`] method to prepare image(s) for the model. It also prepares 2D points and bounding boxes for the model if they are provided. Ztokenizer_init_kwargsr!rNrroriginal_sizesnumpy)rrrZ common_kwargsreturn_tensorsr)rrrr8r) Z _merge_kwargsr Z'prepare_and_validate_optional_call_argspopr%r*r7_check_and_preprocess_points_normalize_and_convertget) r-r3r0r1r2argsr4Z output_kwargsrrrencoding_image_processorr6rrr__call__TsF     zSamHQProcessor.__call__ptcsdur'||tfddDs"|dur"|||\}t|dur0t|}|durA|j||dd}t|}|durN||d||d<dur[|d||d <|durh||d||d <|S) zZ Normalize and convert the image processor output to the expected format. Nc3s |] }|jdjkVqdS)rNshape.0pointrrr sz8SamHQProcessor._normalize_and_convert..Tis_bounding_boxrrrr)_normalize_batch_coordinatesall_pad_points_and_labelsnparray _to_tensor)r-r>r6rrrr8rrrGrr;s(    z%SamHQProcessor._normalize_and_convertcCstdd|D}g}t|D]0\}}|jd|kr:tj|t||jddf|gdd}t|||g||<||q|}||fS)zh The method pads the 2D points and labels to the maximum number of points in the batch. cSsg|]}|jdqS)rrBrDrrr sz9SamHQProcessor._pad_points_and_labels..r)Zaxis)max enumeraterCrOZ concatenateZzerosappend)r-rrrZexpected_nb_pointsZprocessed_input_pointsirFrrrrNs  z%SamHQProcessor._pad_points_and_labelsFr,coordsc Csx|\}}|jj||d\}}t|t}|r|ddd}|d|||d<|d|||d<|r:|dd}|S)z} Expects a numpy array of length 2 in the final dimension. Requires the original image size in (H,W) format. )r'rS).r).rK)r%Z_get_preprocess_shaperZastyperZreshape) r-r,rXZ original_sizerJZold_hZold_wZnew_hZnew_wrrr_normalize_coordinatess z%SamHQProcessor._normalize_coordinatesrZcs|durdSt|dr|}t|t}|}t|D]}|r"|s$n|o,t|dt}|r3|dnd}q|s.)r*r7tolist isinstancelistranger+)r-inputs error_messageexpected_nestingr]Zvalidcurrent_rr\r_preprocess_inputs     z SamHQProcessor._preprocess_inputcCs6||d}||d}|j|ddtjd}|||fS)a8 Check and preprocesses the 2D points, labels and bounding boxes. It checks if the input is valid and if they are, it converts the coordinates of the points and bounding boxes. If a user passes directly a `torch.Tensor`, it is converted to a `numpy.ndarray` and then to a `list`. z7Input points must be a list of list of floating points.z-Input labels must be a list of list integers.z>Input boxes must be a list of list of list of floating points.rS)rfr])rirOZfloat32)r-rrrrrrr:s  z+SamHQProcessor._check_and_preprocess_pointscCs|jj}tt|SN)r%model_input_namesrbdictfromkeys)r-Zimage_processor_input_namesrrrrksz SamHQProcessor.model_input_namescOs|jj|i|Srj)r%post_process_masks)r-r=r4rrrrn!sz!SamHQProcessor.post_process_maskscCs.|dkrt|}|j|kr|dS|S|S)a Convert numpy array to tensor and ensure proper dimensionality. Args: array: The numpy array to convert min_dim: The minimum number of dimensions the result should have return_tensors: The type of tensors to return (e.g., "pt" for PyTorch tensors) Returns: The converted array or tensor with proper dimensions r@rZ)torchZ from_numpyndimZ unsqueeze)r-rPZmin_dimr8rrrrQ$s zSamHQProcessor._to_tensorcs@tt|krfdd|DSfddt|DS)a0 Normalize coordinates based on original sizes. Args: inputs: List of coordinate arrays original_sizes: Original sizes of the images is_bounding_box: Whether inputs are bounding boxes Returns: Normalized coordinates as list cs$g|]}jj|ddqS)rrIr[r,r^rJr6r-rrrR?sz?SamHQProcessor._normalize_batch_coordinates..cs$g|]\}}jj||dqS)rIrq)rEr_r&)rJr-rrrREs)lenzip)r-rdr6rJrrrrrL3s  z+SamHQProcessor._normalize_batch_coordinatesrj)NNNr@rA)F)rZN)NNN)"rrr__doc__ attributesZimage_processor_classZoptional_call_argsr)rrrrrrr rr r r r?r;rNrrOZndarrayr[rir:propertyrkrnrQrL __classcell__rrr.rr$4sb         > , &  r$)rucopyrtypingrrrr7rOZ image_utilsrZprocessing_utilsrr r r Ztokenization_utils_baser r rrutilsrZ video_utilsrrorr r$__all__rrrrs"