o Zh+ @sddlZddlmZddlmZddlmZddlmZm Z m Z m Z er+ddl m Z   dd ejd ejd ed ed ef ddZGdddeZGddde Z    dddZdS)N)center_to_corners_format)is_scipy_available)HungarianMatcher ImageLoss _set_aux_lossgeneralized_box_ioulinear_sum_assignment?inputstargets num_boxesalphagammac Csv|}tjj||dd}||d|d|}|d||}|dkr5||d|d|} | |}||S)aJ Loss used in RetinaNet for dense detection: https://arxiv.org/abs/1708.02002. Args: inputs (`torch.FloatTensor` of arbitrary shape): The predictions for each example. targets (`torch.FloatTensor` with the same shape as `inputs`) A tensor storing the binary classification label for each element in the `inputs` (0 for the negative class and 1 for the positive class). num_boxes (`int`): The total number of boxes in the batch. alpha (`float`, *optional*, defaults to 0.25): Optional weighting factor in the range (0,1) to balance positive vs. negative examples. gamma (`int`, *optional*, defaults to 2): Exponent of the modulating factor (1 - p_t) to balance easy vs hard examples. Returns: Loss tensor none)Z reductionrr)sigmoidnnZ functionalZ binary_cross_entropy_with_logitssum) r rrrrZprobZce_lossZp_tlossZalpha_trT/var/www/auris/lib/python3.10/site-packages/transformers/loss/loss_grounding_dino.pysigmoid_focal_losss rc@seZdZeddZdS)GroundingDinoHungarianMatchercCsd|djdd\}}|ddd}|ddd}|d}tdd t||D}||jd d d }td d |D}d} d} d| || d|d } | d|| |d } | | |} tj ||dd}t t |t | }|j ||j | |j|}|||d }dd |D}dd t||d D}dd |DS)a Args: outputs (`dict`): A dictionary that contains at least these entries: * "logits": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits * "pred_boxes": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates. * "label_maps": Tuple of tensors of dim [num_classes, hidden_dim]. targets (`List[dict]`): A list of targets (len(targets) = batch_size), where each target is a dict containing: * "class_labels": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth objects in the target) containing the class labels * "boxes": Tensor of dim [num_target_boxes, 4] containing the target box coordinates. Returns: `List[Tuple]`: A list of size `batch_size`, containing tuples of (index_i, index_j) where: - index_i is the indices of the selected predictions (in order) - index_j is the indices of the corresponding selected targets (in order) For each batch element, it holds: len(index_i) = len(index_j) = min(num_queries, num_target_boxes) logitsNrrr pred_boxes label_mapscSsg|] \}}||dqS) class_labelsr).0Z label_maptargetrrr bz9GroundingDinoHungarianMatcher.forward..T)dimZkeepdimcSsg|]}|dqSboxesrrvrrrr!gsr @g:0yE>)pcSsg|]}t|dqSr%lenr'rrrr!{scSsg|] \}}t||qSrr )ricrrrr!|r"cSs0g|]\}}tj|tjdtj|tjdfqS))dtype)torchZ as_tensorZint64)rr-jrrrr!}s0)shapeflattenrr0catziprlogtZcdistr r bbox_cost class_cost giou_costviewcpu enumeratesplit)selfoutputsrZ batch_sizeZ num_queriesZout_probZout_bboxrZ target_bboxrrZneg_cost_classZpos_cost_classr9r8r:Z cost_matrixsizesindicesrrrforwardDs&"z%GroundingDinoHungarianMatcher.forwardN)__name__ __module__ __qualname__r0Zno_gradrCrrrrrCsrc@s(eZdZdZddZddZddZdS) GroundingDinoImageLossa This class computes the losses for `GroundingDinoForObjectDetection`. The process happens in two steps: 1) we compute hungarian assignment between ground truth boxes and the outputs of the model 2) we supervise each pair of matched ground-truth / prediction (supervise class and box). Args: matcher (`GroundingDinoHungarianMatcher`): Module able to compute a matching between targets and proposals. focal_alpha (`float`): Alpha parameter in focal loss. losses (`List[str]`): List of all the losses to be applied. See `get_loss` for a list of all available losses. cCs"tj|||_||_||_dSN)rModule__init__matcher focal_alphalosses)r?rKrLrMrrrrJs  zGroundingDinoImageLoss.__init__c srd}tfddtt||D}tjddd}||}tj||jtjd}||tj||<|S)z> Create one_hot based on the matching indices rcsHg|] \}\}\}}|dkr|d|td|n|d|qS)rrrr+)rr-r _Jr@rrr!s.zFGroundingDinoImageLoss._get_target_classes_one_hot..rr)r$)devicer/) r0r4r=r5Z_get_source_permutation_idxZ zeros_likerQlongto) r?r@rrBrrridxtarget_classes_onehotrrPr_get_target_classes_one_hots   z2GroundingDinoImageLoss._get_target_classes_one_hotc Cs~d|vrtdd|vrtd||||}|d}|d}t||}t||}|}t||||jdd}d|i} | S)z Classification loss (Binary focal loss) targets dicts must contain the key "class_labels" containing a tensor of dim [nb_target_boxes] rz#No logits were found in the outputs text_maskz&No text_mask were found in the outputsr)r rrrrloss_ce)KeyErrorrVr0Z masked_selectfloatrrL) r?r@rrBrrUZ source_logitsrWrXrMrrr loss_labelss&  z"GroundingDinoImageLoss.loss_labelsN)rDrErF__doc__rJrVr[rrrrrGs  rGc spt|j|j|jd} gd} t| |j| d} | |i}||d<||d<||d<||d<d}|jrHt||}|D] }||d<||d<q9||d<| |||j rj| | ||d }| ||}d d | D} |d |j |j d |j rdd  D} ||jri}t|jdD]| fdd  Dq |tfddD}||fS)N)r9r8r:)labelsr&Z cardinality)rKrLrMrrrrWauxiliary_outputs)rrrrWcSi|] \}}|d|qSZ_encrrkr(rrr z7GroundingDinoForObjectDetectionLoss..r))rXZ loss_bboxZ loss_gioucSr_r`rrarrrrcrdrcs i|] \}}|d|qS)rNrra)r-rrrc s c3s(|]}|vr||VqdSrHr)rrb) loss_dict weight_dictrr s&z6GroundingDinoForObjectDetectionLoss..)rr9r8r:rGrLrSZauxiliary_lossrZ two_stageitemsupdateZbbox_loss_coefficientZgiou_loss_coefficientrangeZdecoder_layersrkeys)rr]rQrconfigrrWZ outputs_classZ outputs_coordZencoder_logitsZencoder_pred_boxesrKrM criterionZ outputs_lossr^Z aux_outputZencoder_outputs_lossZencoder_loss_dictZenc_weight_dictZaux_weight_dictrr)r-rerfr#GroundingDinoForObjectDetectionLosss\          rn)r r)NNNN)r0Ztorch.nnrZimage_transformsrutilsrZloss_for_object_detectionrrrr Zscipy.optimizer ZTensorintrZrrrGrnrrrrs6      '=Q