o Zh2@sdZddlmZddlmZmZmZddlZddl m Z ddl m Z m Z mZddlmZmZmZmZdd lmZmZdd lmZerGddlZerNddlZGd d d e ZGd dde ddZGdddeZdgZdS)z Processor class for SAM. )deepcopy)ListOptionalUnionN) ImageInput) ImagesKwargsProcessingKwargsProcessorMixin) AudioInput BatchEncodingPreTokenizedInput TextInput)is_tf_availableis_torch_available) VideoInputc@sfeZdZUeeed<eeeeed<eeeeed<eeeeeed<eeed<dS)SamImagesKwargssegmentation_maps input_points input_labels input_boxespoint_pad_valueN) __name__ __module__ __qualname__rr__annotations__rfloatintrrU/var/www/auris/lib/python3.10/site-packages/transformers/models/sam/processing_sam.pyr&s rc@s"eZdZUeed<dddiiZdS)SamProcessorKwargs images_kwargsrN)rrrrr _defaultsrrrrr .s r F)totalc seZdZdZdgZdZgdZfddZ d!ddddd ee d ee e e e e e e fd eed eed ef ddZ     d"ddZddZ d#dedejd ejfddZ   d$ddZeddZdd ZZS)% SamProcessora Constructs a SAM processor which wraps a SAM image processor and an 2D points & Bounding boxes processor into a single processor. [`SamProcessor`] offers all the functionalities of [`SamImageProcessor`]. See the docstring of [`~SamImageProcessor.__call__`] for more information. Args: image_processor (`SamImageProcessor`): An instance of [`SamImageProcessor`]. The image processor is a required input. image_processorZSamImageProcessor)rrrrcst||jjd|_dS)N longest_edge)super__init__r&size target_size)selfr& __class__rrr)Ns zSamProcessor.__init__N)textaudiovideoimagesr/r0r1returnc Os|jtfdii||j|}|ddd}|ddd} |ddd} |ddd} |j|fi|d} | d} t| d rH| } |j|| | d \}} } |j| | || | |d  d | d } | S)z This method uses [`SamImageProcessor.__call__`] method to prepare image(s) for the model. It also prepares 2D points and bounding boxes for the model if they are provided. Ztokenizer_init_kwargsr!rNrrroriginal_sizesnumpy)rrrZ common_kwargsreturn_tensors)rrrr6r) Z _merge_kwargsr Z'prepare_and_validate_optional_call_argspopr&hasattrr5_check_and_preprocess_points_normalize_and_convertget)r,r2r/r0r1argskwargsZ output_kwargsrrrrencoding_image_processorr4rrr__call__RsH    zSamProcessor.__call__ptr"csRdur@ttkrfddDn fddtDtfddDs;|dur;||\}t|durIt|}|durqtt|kr`fdd|D}n fddt|D}t|}|dur|dkrt|}t|jd kr| d n|}n|d krt |}t|jd krt |d n|}| d |idur|dkrttjd krň d nn|d krt tjd krt d n| di|dur'|dkrt|}t|jd kr| d n|}n|d kr t |}t|jd krt |d n|}| d|i|S)Ncs g|] }j|dqSr_normalize_coordinatesr+.0pointr4r,rr sz7SamProcessor._normalize_and_convert..cs g|] \}}j||qSrrB)rErF original_sizer,rrrHsc3s |] }|jdjkVqdS)rNshaperD)rrr sz6SamProcessor._normalize_and_convert..cs$g|]}jj|dddqS)rTis_bounding_boxrBrEboxrGrrrHscs$g|]\}}jj||ddqS)TrNrB)rErQrIrJrrrHsr@rtfrrr)lenzipall_pad_points_and_labelsnparraytorchZ from_numpyrLZ unsqueezerSZconvert_to_tensorZ expand_dimsupdate)r,r>r4rrrr6rr)rr4r,rr:sd                   z#SamProcessor._normalize_and_convertcCstdd|D}g}t|D]0\}}|jd|kr:tj|t||jddf|gdd}t|||g||<||q|}||fS)zh The method pads the 2D points and labels to the maximum number of points in the batch. cSsg|]}|jdqSrArKrDrrrrHz7SamProcessor._pad_points_and_labels..r)Zaxis)max enumeraterLrYZ concatenateZzerosappend)r,rrrZexpected_nb_pointsZprocessed_input_pointsirFrrrrXs  z#SamProcessor._pad_points_and_labelsFr+coordsc Csx|\}}|jj||d\}}t|t}|r|ddd}|d|||d<|d|||d<|r:|dd}|S)z~ Expects a numpy array of length 2 in the final dimension. Requires the original image size in (H, W) format. )r'r^).r).rRrT)r&Z_get_preprocess_shaperastyperZreshape) r,r+rcrIrOZold_hZold_wZnew_hZnew_wrrrrCs z#SamProcessor._normalize_coordinatescCs|dur't|dr|}t|trt|dtstddd|D}nd}|durPt|dr8|}t|trDt|dtsHtddd|D}nd}|durt|dra|}t|trvt|dtrvt|ddtsztd d d|D}nd}|||fS) a8 Check and preprocesses the 2D points, labels and bounding boxes. It checks if the input is valid and if they are, it converts the coordinates of the points and bounding boxes. If a user passes directly a `torch.Tensor`, it is converted to a `numpy.ndarray` and then to a `list`. Nr5rz7Input points must be a list of list of floating points.cSg|]}t|qSrrYrZ)rEZ input_pointrrrrHr]z=SamProcessor._check_and_preprocess_points..z-Input labels must be a list of list integers.cSrfrrg)rElabelrrrrHr]z>Input boxes must be a list of list of list of floating points.cSsg|] }t|tjqSr)rYrZreZfloat32rPrrrrH)s)r8r5tolist isinstancelist ValueError)r,rrrrrrr9s6        z)SamProcessor._check_and_preprocess_pointscCs|jj}tt|SN)r&model_input_namesrkdictfromkeys)r,Zimage_processor_input_namesrrrrn/szSamProcessor.model_input_namescOs|jj|i|Srm)r&post_process_masks)r,r<r=rrrrq4szSamProcessor.post_process_masksrm)NNNr@r")F)NNN)rrr__doc__ attributesZimage_processor_classZoptional_call_argsr)rrrrr rr rr r?r:rXrrYZndarrayrCr9propertyrnrq __classcell__rrr-rr%7sX       ? N  / r%) rrcopyrtypingrrrr5rYZ image_utilsrZprocessing_utilsrr r Ztokenization_utils_baser r r rutilsrrZ video_utilsrr[Z tensorflowrSrr r%__all__rrrrs&