fThbR8SrSSKrSSKJr SSKJrJrJrJrJ r SSK r SSK J r J r SSK Jr SSKJr SS KJrJrJr SS KJr SS KJrJrJrJrJr SS KJr S SKJ r \"5(aSSK!J"r" \RF"\$5r%\"SS\55r&\"SS\55r'\"SS\55r(\"SS\55r)"SS\ RT5r+Sr,"SS\ RT5r-"SS\ RT5r."SS \ RT5r/"S!S"\ RT5r0S#r1"S$S%\ RT5r2"S&S'\ RT5r3"S(S)\ RT5r4\"S*S+\55r5"S,S-\55r6"S.S/\55r7\"S0S19"S2S3\555r8"S4S5\ RT5r9\"S6S19"S7S8\555r:\"S9S19"S:S;\555r;S<\<4S=jr="S>S?\ RT5r>"S@SA\ RT5r?/SBQr@g)CzPyTorch DETR model.N) dataclass)DictListOptionalTupleUnion)Tensornn)ACT2FN)_prepare_4d_attention_mask)BaseModelOutput"BaseModelOutputWithCrossAttentionsSeq2SeqModelOutput)PreTrainedModel) ModelOutputauto_docstringis_timm_availableloggingrequires_backends) load_backbone) DetrConfig) create_modelcB\rSrSr%SrSr\\R\ S'Sr g)DetrDecoderOutput.ab Base class for outputs of the DETR decoder. This class adds one attribute to BaseModelOutputWithCrossAttentions, namely an optional stack of intermediate decoder activations, i.e. the output of each decoder layer, each of them gone through a layernorm. This is useful when training the model with auxiliary decoding losses. Args: last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` and `config.add_cross_attention=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the weighted average in the cross-attention heads. intermediate_hidden_states (`torch.FloatTensor` of shape `(config.decoder_layers, batch_size, num_queries, hidden_size)`, *optional*, returned when `config.auxiliary_loss=True`): Intermediate decoder activations, i.e. the output of each decoder layer, each of them gone through a layernorm. Nintermediate_hidden_states __name__ __module__ __qualname____firstlineno____doc__rrtorch FloatTensor__annotations____static_attributes__r^/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/detr/modeling_detr.pyrr.s 2?C):): ;Br*rcB\rSrSr%SrSr\\R\ S'Sr g)DetrModelOutputLa Base class for outputs of the DETR encoder-decoder model. This class adds one attribute to Seq2SeqModelOutput, namely an optional stack of intermediate decoder activations, i.e. the output of each decoder layer, each of them gone through a layernorm. This is useful when training the model with auxiliary decoding losses. Args: last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the decoder of the model. decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the decoder at the output of each layer plus the initial embedding outputs. decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the self-attention heads. cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the weighted average in the cross-attention heads. encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder of the model. encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the encoder at the output of each layer plus the initial embedding outputs. encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the self-attention heads. intermediate_hidden_states (`torch.FloatTensor` of shape `(config.decoder_layers, batch_size, sequence_length, hidden_size)`, *optional*, returned when `config.auxiliary_loss=True`): Intermediate decoder activations, i.e. the output of each decoder layer, each of them gone through a layernorm. Nrrr rr*r+r-r-Ls!!F?C):): ;Br*r-c\rSrSr%SrSr\\R\ S'Sr \\ \ S'Sr \\R\ S'Sr \\R\ S'Sr\\\ \ S'Sr\\R\ S 'Sr\\\R\ S 'Sr\\\R\ S 'Sr\\\R\ S 'Sr\\R\ S 'Sr\\\R\ S'Sr\\\R\ S'Srg)DetrObjectDetectionOutputta Output type of [`DetrForObjectDetection`]. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` are provided)): Total loss as a linear combination of a negative log-likehood (cross-entropy) for class prediction and a bounding box loss. The latter is defined as a linear combination of the L1 loss and the generalized scale-invariant IoU loss. loss_dict (`Dict`, *optional*): A dictionary containing the individual losses. Useful for logging. logits (`torch.FloatTensor` of shape `(batch_size, num_queries, num_classes + 1)`): Classification logits (including no-object) for all queries. pred_boxes (`torch.FloatTensor` of shape `(batch_size, num_queries, 4)`): Normalized boxes coordinates for all queries, represented as (center_x, center_y, width, height). These values are normalized in [0, 1], relative to the size of each individual image in the batch (disregarding possible padding). You can use [`~DetrImageProcessor.post_process_object_detection`] to retrieve the unnormalized bounding boxes. auxiliary_outputs (`list[Dict]`, *optional*): Optional, only returned when auxiliary losses are activated (i.e. `config.auxiliary_loss` is set to `True`) and labels are provided. It is a list of dictionaries containing the two above keys (`logits` and `pred_boxes`) for each decoder layer. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the decoder of the model. decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the decoder at the output of each layer plus the initial embedding outputs. decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the self-attention heads. cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the weighted average in the cross-attention heads. encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder of the model. encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the encoder at the output of each layer plus the initial embedding outputs. encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the self-attention heads. Nloss loss_dictlogits pred_boxesauxiliary_outputslast_hidden_statedecoder_hidden_statesdecoder_attentionscross_attentionsencoder_last_hidden_stateencoder_hidden_statesencoder_attentionsr)r!r"r#r$r%r2rr&r'r(r3rr4r5r6rr7r8rr9r:r;r<r=r)rr*r+r0r0ts-^)-D(5$$ %, $Ix~$*.FHU&& '..2J**+2.2xT +259x 1 129@D8E%*;*;$<=D=Au'8'8!9:A;?huU%6%678?=Ax(9(9:A@D8E%*;*;$<=D=Au'8'8!9:Ar*r0c\rSrSr%SrSr\\R\ S'Sr \\ \ S'Sr \\R\ S'Sr \\R\ S'Sr\\R\ S'Sr\\\ \ S 'Sr\\R\ S 'Sr\\\R\ S 'Sr\\\R\ S 'Sr\\\R\ S 'Sr\\R\ S'Sr\\\R\ S'Sr\\\R\ S'Srg)DetrSegmentationOutputa Output type of [`DetrForSegmentation`]. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` are provided)): Total loss as a linear combination of a negative log-likehood (cross-entropy) for class prediction and a bounding box loss. The latter is defined as a linear combination of the L1 loss and the generalized scale-invariant IoU loss. loss_dict (`Dict`, *optional*): A dictionary containing the individual losses. Useful for logging. logits (`torch.FloatTensor` of shape `(batch_size, num_queries, num_classes + 1)`): Classification logits (including no-object) for all queries. pred_boxes (`torch.FloatTensor` of shape `(batch_size, num_queries, 4)`): Normalized boxes coordinates for all queries, represented as (center_x, center_y, width, height). These values are normalized in [0, 1], relative to the size of each individual image in the batch (disregarding possible padding). You can use [`~DetrImageProcessor.post_process_object_detection`] to retrieve the unnormalized bounding boxes. pred_masks (`torch.FloatTensor` of shape `(batch_size, num_queries, height/4, width/4)`): Segmentation masks logits for all queries. See also [`~DetrImageProcessor.post_process_semantic_segmentation`] or [`~DetrImageProcessor.post_process_instance_segmentation`] [`~DetrImageProcessor.post_process_panoptic_segmentation`] to evaluate semantic, instance and panoptic segmentation masks respectively. auxiliary_outputs (`list[Dict]`, *optional*): Optional, only returned when auxiliary losses are activated (i.e. `config.auxiliary_loss` is set to `True`) and labels are provided. It is a list of dictionaries containing the two above keys (`logits` and `pred_boxes`) for each decoder layer. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the decoder of the model. decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the decoder at the output of each layer plus the initial embedding outputs. decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the self-attention heads. cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the weighted average in the cross-attention heads. encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder of the model. encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the encoder at the output of each layer plus the initial embedding outputs. encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the self-attention heads. Nr2r3r4r5 pred_masksr6r7r8r9r:r;r<r=r)r!r"r#r$r%r2rr&r'r(r3rr4r5rAr6rr7r8rr9r:r;r<r=r)rr*r+r?r?s33j)-D(5$$ %, $Ix~$*.FHU&& '..2J**+2.2J**+2.2xT +259x 1 129@D8E%*;*;$<=D=Au'8'8!9:A;?huU%6%678?=Ax(9(9:A@D8E%*;*;$<=D=Au'8'8!9:Ar*r?c>^\rSrSrSrU4SjrU4SjrSrSrU=r $)DetrFrozenBatchNorm2dz BatchNorm2d where the batch statistics and the affine parameters are fixed. Copy-paste from torchvision.misc.ops with added eps before rqsrt, without which any other models than torchvision.models.resnet[18,34,50,101] produce nans. cR>[TU]5 URS[R"U55 URS[R "U55 URS[R "U55 URS[R"U55 g)Nweightbias running_mean running_var)super__init__register_bufferr&oneszeros)selfn __class__s r+rKDetrFrozenBatchNorm2d.__init__sn  Xuzz!}5 VU[[^4 ^U[[^< ]EJJqM:r*c B>US-nX;aX [T U]XX4XVU5 g)Nnum_batches_tracked)rJ_load_from_state_dict) rO state_dictprefixlocal_metadatastrict missing_keysunexpected_keys error_msgsnum_batches_tracked_keyrQs r+rU+DetrFrozenBatchNorm2d._load_from_state_dict s4#)+@"@ " 03 %  Wa r*c<URRSSSS5nURRSSSS5nURRSSSS5nURRSSSS5nSnX$U-R 5-nX5U-- nX-U-$)Nrgh㈵>)rFreshaperGrIrHrsqrt)rOxrFrGrIrHepsilonscales r+forwardDetrFrozenBatchNorm2d.forwards$$QAq1yy  B1-&&..q"a; ((00B1= /6688U**y4r*r) r!r"r#r$r%rKrUrfr) __classcell__rQs@r+rCrCs;     r*rCcUR5GHgup[U[R5(Ga [ UR 5nUR R[R"S5:XdUR RRUR 5 URRRUR5 URRRUR5 URRRUR5 X0RU'[[!UR#555S:dGM\[%U5 GMj g)z Recursively replace all `torch.nn.BatchNorm2d` with `DetrFrozenBatchNorm2d`. Args: model (torch.nn.Module): input model metarN)named_children isinstancer BatchNorm2drC num_featuresrFdevicer&datacopy_rGrHrI_moduleslenlistchildrenreplace_batch_norm)modelnamemodule new_modules r+rwrw"s,,.  fbnn - -.v/B/BCJ==''5<<+??!!&&,,V]];$$**6;;7'',,2263F3FG&&++11&2D2DE#-NN4 tFOO%& '! + v &/r*cf^\rSrSrSrU4SjrS\RS\R4SjrSr U=r $)DetrConvEncoderi:z Convolutional backbone, using either the AutoBackbone API or one from the timm library. nn.BatchNorm2d layers are replaced by DetrFrozenBatchNorm2d as defined above. c>[T U]5 XlUR(a[ US/5 [ US05nUc0OUR 5nURSS5nURSUR5nUR(aURSS5US'[UR4URSUUS .UD6nO [U5n[R "5 [#U5 SSS5 XPlUR(a$UR$R&R)5OUR$R(UlSnURb URnO/UR,bUR,R.nO [1S 5eS U;aUR$R35HjupxUR(a+S U;a#S U;aSU;aUR5S5 M;M=M?MASU;dMISU;dMQSU;dMYUR5S5 Ml gg!,(df  GN@=f)Ntimmbackbone_kwargs out_indices)rr in_chans output_strideT) pretrained features_onlyrrzGEither `backbone` or `backbone_config` should be provided in the configresnetlayer2layer3layer4Fzstage.1zstage.2zstage.3)rJrKconfiguse_timm_backbonergetattrcopypop num_channelsdilationgetrbackboneuse_pretrained_backbonerr&no_gradrwrx feature_infochannelsintermediate_channel_sizesbackbone_config model_type ValueErrornamed_parametersrequires_grad_) rOrkwargsrrrbackbone_model_typery parameterrQs r+rKDetrConvEncoder.__init__Bs    # # dVH -V%6;F!>Rv{{}F **]LAK!::j&2E2EFL*0**_b*I'#!99"'%   H%V,H]]_ x ( 282J2JDJJ # # , , .PTPZPZPcPc '# ?? &"(//   # # /"("8"8"C"C fg g * *#'::#>#>#@++t+0DY]I]!007J^0D+!,$1F9\`K`!007 $A +_s 0 H55 I pixel_values pixel_maskcURR(aURU5OURU5Rn/nUHqn[R R USR5URSSS9R[R5SnURXV45 Ms U$)N)sizer) rrrx feature_mapsr functional interpolatefloatshapetor&boolappend)rOrrfeaturesout feature_mapmasks r+rfDetrConvEncoder.forwardvs/3{{/L/L4::l+RVR\R\]iRjRwRw#K==,,Z-=-C-C-EKL]L]^`^aLb,cffglgqgqrstuD JJ * +$ r*)rrrx) r!r"r#r$r%rKr&r rfr)rhris@r+r}r}:s.28h ELL ell  r*r}c2^\rSrSrSrU4SjrSrSrU=r$) DetrConvModelizh This module adds 2D position embeddings to all intermediate feature maps of the convolutional encoder. c:>[TU]5 XlX lgN)rJrK conv_encoderposition_embedding)rOrrrQs r+rKDetrConvModel.__init__s ("4r*cURX5n/nUH>upVURURXV5RUR55 M@ X44$r)rrrrdtype)rOrrrposrrs r+rfDetrConvModel.forwardsU 9!$ K JJt..{ADD[EVEVW X"%xr*)rr r!r"r#r$r%rKrfr)rhris@r+rrs5 r*rc6^\rSrSrSrSU4SjjrSrSrU=r$)DetrSinePositionEmbeddingiz This is a more standard version of the position embedding, very similar to the one used by the Attention is all you need paper, generalized to work on images. c>[TU]5 XlX lX0lUbUSLa [ S5eUcS[ R-nX@lg)NFz+normalize should be True if scale is passedr) rJrK embedding_dim temperature normalizermathpire)rOrrrrerQs r+rK"DetrSinePositionEmbedding.__init__sO *&"  e!3JK K =KE r*c NUc [S5eURS[RS9nURS[RS9nUR(a@X3SS2SS2SS24S-- UR -nXDSS2SS2SS24S-- UR -n[R "UR[RURS9R5nURS[R"USSS 9-UR- -nUSS2SS2SS2S4U- nUSS2SS2SS2S4U- n[R"USS2SS2SS2S SS24R5USS2SS2SS2SSS24R54S S 9R!S 5n[R"USS2SS2SS2S SS24R5USS2SS2SS2SSS24R54S S 9R!S 5n[R""Xv4S S 9R%S S SS5nU$)NzNo pixel mask providedr)rrr`gư>)rrpfloor) rounding_moderrdimr )rcumsumr&float32rrearangerint64rprrdivstacksincosflattencatpermute) rOrry_embedx_embeddim_tpos_xpos_yrs r+rf!DetrSinePositionEmbedding.forwards  56 6##AU]]#;##AU]]#; >>BC!3d!:;djjHGArs!3d!:;djjHG T//u{{[TU]5 [R"SU5Ul[R"SU5Ulg)N2)rJrKr Embeddingrow_embeddingscolumn_embeddings)rOrrQs r+rK%DetrLearnedPositionEmbedding.__init__s4  ll2}=!#b-!@r*cURSSup4[R"XARS9n[R"X1RS9nUR U5nUR U5n[R "URS5RUSS5URS5RSUS5/SS9n U RSSS5n U RS5n U RURSSSS5n U $)Nrrprrr`rr) rr&rrprrr unsqueezerepeatr) rOrrheightwidth width_values height_valuesx_emby_embrs r+rf$DetrLearnedPositionEmbedding.forwards$**23/ ||E2E2EF  V4G4GH &&|4##M2ii+2261a@%//RSBTB[B[\]_dfgBhioqrkk!Q"mmAjj++A.1a8 r*)rr)rrris@r+rrsA  r*rcURS-nURS:Xa [USS9nU$URS:Xa [U5nU$[ SUR35e)NrsineT)rlearnedzNot supported )d_modelposition_embedding_typerrr)rn_stepsrs r+build_position_encodingrsqnn!G %%/6w$O   ' '9 49'B >&*H*H)IJKKr*c^\rSrSrSrSS\S\S\S\4U4SjjjrS\ RS \S \4S jr S\ RS \ \ 4S jr SS\ RS\ \ RS \ \ RS\ \ RS\ \ RS\S\\ R\ \ R\ \\ R44SjjrSrU=r$) DetrAttentioniz Multi-headed attention from 'Attention Is All You Need' paper. Here, we add position embeddings to the queries and keys (as explained in the DETR paper). embed_dim num_headsdropoutrGc>[TU]5 XlX lX0lX-UlUR U-UR:wa[ SURSUS35eUR S-Ul[R"XUS9Ul [R"XUS9Ul [R"XUS9Ul [R"XUS9Ul g)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).࿩rG)rJrKrrrhead_dimrscalingr Lineark_projv_projq_projout_proj)rOrrrrGrQs r+rKDetrAttention.__init__s "" !. ==9 $ 6MdnnM]^;b" }}d* ii 4@ ii 4@ ii 4@  )TB r*tensorseq_len batch_sizecURX2URUR5RSS5R 5$)Nrr)viewrr transpose contiguous)rOr rrs r+_shapeDetrAttention._shapes5{{: NXXYZ\]^iikkr*object_queriescUcU$X-$rr)rOr rs r+with_pos_embedDetrAttention.with_pos_embeds'/vLV5LLr* hidden_statesattention_maskkey_value_statesspatial_position_embeddingsoutput_attentionsreturnczUSLnUR5upn UbUn URX5nUbUn URXE5nURU5UR-n U(aEUR UR U5SU5nUR UR W 5SU5nODUR UR U5SU5nUR UR W 5SU5nXR-SUR4nUR XU5R"U6n UR"U6nUR"U6nURS5n[R"XRSS55nUR5XR-U U4:wa.[SXR-U U4SUR535eUbzUR5USU U4:wa#[SUSU U4SUR535eURXRU U5U-nURXR-U U5n[RR!USS9nU(a=URXRU U5nURXR-U U5nOSn[RR#UUR"UR$S 9n[R"UU5nUR5XR-XR4:wa5[S XRXR4SUR535eURXRXR5nURSS5nUR'XU 5nUR)U5nUU4$) z#Input shape: Batch x Time x ChannelNr`rrz$Attention weights should be of size z , but is z!Attention mask should be of size rptrainingz `attn_output` should be of size )rrr rrrr rrrr&bmmrrr rsoftmaxrr#rar )rOrrrrrris_cross_attentionr target_lenrhidden_states_originalkey_value_states_original query_states key_states value_states proj_shape source_len attn_weightsattn_weights_reshaped attn_probs attn_outputs r+rfDetrAttention.forwards.T9,9,>,>,@)   %%2 " // NM ' 2(8 %#223Ca {{=1DLL@ T[[1A%BB SJ;;t{{3L'MrS]^LT[[%?ZPJ;;t{{3I'JBPZ >>12t}}E {{ J"W W6 ^^8SU_ak7l6mn %%'(*   %""$Q J(OO 7Q T^8_7`a&++-.0(,,ZU_`cqqL',,Z..-H*V`aL}},,\r,B  %1$5$5j..R\^h$i !055j>>6QS]_ijL$( !]]**<4<>:}}] !++Aq1 !))*)L mmK0 111r*) rrrrrr r rr )T)NNNNF)r!r"r#r$r%intrrrKr&r rrrrrfr)rhris@r+rrs> CCC C  CC0lU\\lClSlMU\\M8FCSM 261537>B"'Y2||Y2!.Y2!. Y2 #5<<0 Y2 &.ell%; Y2 Y2 u||Xell3XeELL>Q5RR SY2Y2r*rc ^\rSrSrS\4U4SjjrS S\RS\RS\\RS\ 4Sjjr S r U=r $) DetrEncoderLayeriarch>[TU]5 URUl[ URUR UR S9Ul[R"UR5Ul URUl [URUlURUl[R "URUR"5Ul[R "UR"UR5Ul[R"UR5Ulg)Nrrr)rJrKrrrencoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normrr activation_function activation_fnactivation_dropoutrencoder_ffn_dimfc1fc2final_layer_normrOrrQs r+rKDetrEncoderLayer.__init__bs &nn44,,  %'LL$@!~~ #F$>$>?"(";";99T^^V-C-CD99V33T^^D " T^^ <r*rrrrcUnURUUUUS9up[RRXRURS9nXQ-nUR U5nUnUR URU55n[RRXRURS9nURU5n[RRXRURS9nXQ-nURU5nUR(a[R"U5R5(d)[R"U5R5(aC[R"UR 5R"S- n[R$"X*US9nU4nU(aX4- nU$)as Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, target_len, source_len)` where padding elements are indicated by very large negative values. object_queries (`torch.FloatTensor`, *optional*): Object queries (also called content embeddings), to be added to the hidden states. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )rrrrr!i)minmax)r<r rrr#r>r@rCrArDrEr&isinfanyisnanfinforrJclamp) rOrrrrresidualr/ clamp_valueoutputss r+rfDetrEncoderLayer.forwardrsy&!&*nn'))/ '5' #  --m||VZVcVc-d  0 11-@  **488M+BC  --m?V?Vaeanan-o /  --m||VZVcVc-d  0 --m< =={{=)--//5;;}3M3Q3Q3S3S#kk-*=*=>BBTI % M|Q\ ]  "   &Gr*) rAr@rrrCrDrEr<r>NF r!r"r#r$rrKr&r rrrfr)rhris@r+r7r7asX=z=(26"' 3||3 3!. 3  33r*r7c^\rSrSrS\4U4SjjrS S\RS\\RS\\RS\\RS\\RS \\RS \\ 4S jjr S r U=r $)DetrDecoderLayerirc>[TU]5 URUl[ URUR UR S9UlURUl[URUl URUl [R"UR5Ul[ URUR UR S9Ul[R"UR5Ul[R$"URUR&5Ul[R$"UR&UR5Ul[R"UR5Ulg)Nr9)r)rJrKrrrdecoder_attention_headsr;r<rr r?r@rAr r=r> encoder_attnencoder_attn_layer_normrdecoder_ffn_dimrCrDrErFs r+rKDetrDecoderLayer.__init__s  &nn44,,  ~~ #F$>$>?"(";";$&LL$@!) NN  * *,,  (*||DNN'C$99T^^V-C-CD99V33T^^D " T^^ <r*rrrquery_position_embeddingsr<encoder_attention_maskrc UnURUUUUS9up[RRXRURS9nX-nUR U5nSn Ub_UnUR UUUUUUS9up[RRXRURS9nX-nURU5nUnURURU55n[RRXRURS9nURU5n[RRXRURS9nX-nURU5nU4n U(aXU 4- n U $)ag Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, target_len, source_len)` where padding elements are indicated by very large negative values. object_queries (`torch.FloatTensor`, *optional*): object_queries that are added to the hidden states in the cross-attention layer. query_position_embeddings (`torch.FloatTensor`, *optional*): position embeddings that are added to the queries and keys in the self-attention layer. encoder_hidden_states (`torch.FloatTensor`): cross attention input to the layer of shape `(batch, seq_len, embed_dim)` encoder_attention_mask (`torch.FloatTensor`): encoder attention mask of size `(batch, 1, target_len, source_len)` where padding elements are indicated by very large negative values. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )rrrrr!N)rrrrrr) r<r rrr#r>rZr[r@rCrArDrE) rOrrrr^r<r_rrPself_attn_weightscross_attn_weightsrRs r+rfDetrDecoderLayer.forwards>!,0>>'4)/ ,:, (  --m||VZVcVc-d  0 11-@ " ,$H040A0A+8!65,:"3 1B1 -MMM11-< >Gr*) rAr@rrrZr[rCrDrEr<r>)NNNNNFrUris@r+rWrWs=z=62615<@8<9=,1M||M!.M!. M $,ELL#9 M ( 5 M!) 6M$D>MMr*rWc.\rSrSr\rSrSr/SQrSr Sr g)DetrPreTrainedModelirxr)r}r7rWcBURRnURRn[U[5(a[ R RURR5 [ R RURR5 [ R RURRUS9 [ R RURRUS9 O{[U[5(af[ R RURR5 [ R RUR R5 [U[ R"[ R$[ R&45(aWURR(R+SUS9 URb%URR(R-5 gg[U[ R.5(adURR(R+SUS9 UR0b2URR(UR0R-5 ggg)N)gainr4)meanstd)rinit_stdinit_xavier_stdrmDetrMHAttentionMapr initzeros_k_linearrGq_linearxavier_uniform_rFruniform_rrrConv2drnrqnormal_zero_r padding_idx)rOrzri xavier_stds r+ _init_weights!DetrPreTrainedModel._init_weightsskk""[[00 f0 1 1 GGNN6??// 0 GGNN6??// 0 GG # #FOO$:$: # L GG # #FOO$:$: # L  < = = GG  V2299 : GG  V55<< = fryy"))R^^D E E MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>..r*rN) r!r"r#r$r config_classbase_model_prefixmain_input_name_no_split_modulesrxr)rr*r+reresL$OV?r*recJ^\rSrSrSrS\4U4SjjrSSjrSrU=r $) DetrEncoderi0a5 Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`DetrEncoderLayer`]. The encoder updates the flattened feature map through multiple self-attention layers. Small tweak for DETR: - object_queries are added to the forward pass. Args: config: DetrConfig rc">[TU]U5 URUlURUl[ R "[UR5Vs/sHn[U5PM sn5Ul UR5 gs snfr) rJrKrencoder_layerdrop layerdropr ModuleListrangeencoder_layersr7layers post_initrOr_rQs r+rKDetrEncoder.__init__?sl  ~~ 11mmuVMbMbGc$dGc!%5f%=Gc$de  %esB cUbUOURRnUbUOURRnUbUOURRnUn[R R XpR URS9nUb[X!R5nU(aSOSnU(aSOSn [UR5HtupU(aX4-nSn UR(a'[R"/5n XR:aSn U (aSnOU "UUUUS9nUSnU(dMlXS 4-n Mv U(aX4-nU(d[S XxU 455$[!XxU S 9$) a Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Flattened feature map (output of the backbone + projection layer) that is passed to the encoder. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding pixel features. Mask values selected in `[0, 1]`: - 1 for pixel features that are real (i.e. **not masked**), - 0 for pixel features that are padding (i.e. **masked**). [What are attention masks?](../glossary#attention-mask) object_queries (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Object queries that are added to the queries in each self-attention layer. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. Nr!rFTNN)rrrrc3.# UH ocMUv M g7frr.0vs r+ &DetrEncoder.forward..se$Sq$Ss r7r attentions)rroutput_hidden_statesuse_return_dictr rrr#r r enumeraterr&randrtupler)rO inputs_embedsrrrr return_dictrencoder_statesall_attentionsi encoder_layerto_dropdropout_probability layer_outputss r+rfDetrEncoder.forwardLsqD2C1N-TXT_T_TqTq$8$D $++JjJj &1%>Ne]N$See e+Vd  r*)rrr)NNNNNN r!r"r#r$r%rrKrfr)rhris@r+rr0s4  z !S S r*rcP^\rSrSrSrS\4U4SjjrSSjrSrU=r $) DetrDecoderia Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`DetrDecoderLayer`]. The decoder updates the query embeddings through multiple self-attention and cross-attention layers. Some small tweaks for DETR: - object_queries and query_position_embeddings are added to the forward pass. - if self.config.auxiliary_loss is set to True, also returns a stack of activations from all decoding layers. Args: config: DetrConfig rcz>[TU]U5 URUlURUl[ R "[UR5Vs/sHn[U5PM sn5Ul [ R"UR5Ul SUlUR5 gs snfrT)rJrKrdecoder_layerdroprr rrdecoder_layersrWrr=r layernormgradient_checkpointingrrs r+rKDetrDecoder.__init__s  ~~ 11mmuVMbMbGc$dGc!%5f%=Gc$de fnn5&+#  %esB8c UbUOURRnUbUOURRnU bU OURRn UbUn UR 5SSn Sn UbU bU [ X!R W SS9-n UbUb[ XAR W SS9nURR(aSOSn U(aSOSnU(aSOSnU(aUbSOSn[UR5HunnU(aUW 4- nUR(a([R"/5nUUR:aMLUR(a2UR(a!URUR W U UUS5nO U"W U UUUUUS9nUSn URR(aUR#U 5n X4- n U(dMUUS4- nUcMUUS4- nM UR#W 5n U(aX4- nURR(a[R$"U 5n U (d['S XUUU 455$[)U UUUU S 9$) a Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): The query embeddings that are passed into the decoder. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on certain queries. Mask values selected in `[0, 1]`: - 1 for queries that are **not masked**, - 0 for queries that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.LongTensor` of shape `(batch_size, encoder_sequence_length)`, *optional*): Mask to avoid performing cross-attention on padding pixel_values of the encoder. Mask values selected in `[0, 1]`: - 1 for pixels that are real (i.e. **not masked**), - 0 for pixels that are padding (i.e. **masked**). object_queries (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Object queries that are added to the queries and keys in each cross-attention layer. query_position_embeddings (`torch.FloatTensor` of shape `(batch_size, num_queries, hidden_size)`): , *optional*): Position embeddings that are added to the values and keys in each self-attention layer. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. Nr`)tgt_lenr)rrr^r<r_rrrrc30# UH nUcMUv M g7frrrs r+r&DetrDecoder.forward..DsoAos )r7rrr:r)rrrrrr rauxiliary_lossrrr#r&rrr_gradient_checkpointing_func__call__rrrr)rOrrr<r_rr^rrrr input_shapecombined_attention_mask intermediateall_hidden_statesall_self_attnsall_cross_attentionsidx decoder_layerrrs r+rfDetrDecoder.forwards`2C1N-TXT_T_TqTq$8$D $++JjJj &1%[TU]U5 [U5n[U5n[ X#5Ul[ R"URSURSS9Ul [ R"URUR5Ul [U5Ul[!U5UlUR%5 g)Nr`r) kernel_size)rJrKr}rrrr rsrrinput_projectionr num_queriesr^rencoderrdecoderr)rOrrrrQs r+rKDetrModel.__init__Ys  #6*08%h? !# (*M*Mb*QSYSaSaop q)+f6H6H&..)Y&"6* "6*  r*cUR$r)rrOs r+ get_encoderDetrModel.get_encoderl ||r*cUR$r)rrs r+ get_decoderDetrModel.get_decoderorr*cURRRR5HupUR S5 M grTrrrxrrrOryparams r+freeze_backboneDetrModel.freeze_backboners6==55;;LLNKD   'Or*cURRRR5HupUR S5 M g)NTrrs r+unfreeze_backboneDetrModel.unfreeze_backbonevs6==55;;LLNKD   &Or*rrdecoder_attention_maskencoder_outputsrdecoder_inputs_embedsrrrrc UbUOURRnUbUOURRnU bU OURRn URuppUR nUc[ R"XU 4US9nURX5unnUSunnUc [S5eURU5nURS5RSSS5nUSRS5RSSS5nURS5nUcURUUUUUU S9nORU (aK[U[5(d6[US[!U5S:aUSOS[!U5S:aUSOSS 9nUR"R$R'S5R)U SS5n[ R*"U5nUR-USUUUSUUUU S 9 nU (dUU-$[/UR0UR2UR4UR6UR0UR2UR4UR8S 9$) a decoder_attention_mask (`torch.FloatTensor` of shape `(batch_size, num_queries)`, *optional*): Not used by default. Can be used to mask object queries. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing the flattened feature map (output of the backbone + projection layer), you can choose to directly pass a flattened representation of an image. decoder_inputs_embeds (`torch.FloatTensor` of shape `(batch_size, num_queries, hidden_size)`, *optional*): Optionally, instead of initializing the queries with a tensor of zeros, you can choose to directly pass an embedded representation. Examples: ```python >>> from transformers import AutoImageProcessor, DetrModel >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("facebook/detr-resnet-50") >>> model = DetrModel.from_pretrained("facebook/detr-resnet-50") >>> # prepare image for the model >>> inputs = image_processor(images=image, return_tensors="pt") >>> # forward pass >>> outputs = model(**inputs) >>> # the last hidden states are the final query embeddings of the Transformer decoder >>> # these are of shape (batch_size, num_queries, hidden_size) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 100, 256] ```Nrr`z/Backbone does not return downsampled pixel maskrrrrrrrrrr rrrr^r<r_rrr)r7r8r9r:r;r<r=r)rrrrrrpr&rMrrrrrrrmrrtr^rFrr zeros_likerr-r7rrr:r)rOrrrrrrrrrrrrrrprobject_queries_listrrprojected_feature_mapflattened_featuresrflattened_maskr^queriesdecoder_outputss r+rfDetrModel.forwardzs`2C1N-TXT_T_TqTq$8$D $++JjJj &1%2D2D/ &$$  j%%@&QJ )- l(O%%%RL T <NO O!% 5 5k B3::1=EEaAN,R088;CCAq!La  ""ll0--"3%9' +OO_!M!M-"1!"4474H14Loa0RV14_1E1I?1-tO%)$B$B$I$I$S$STU$V$]$]^hjkmn$o!""#<=,,!)&?"1!"4#1/!5#'  "_4 4-??"1"?"?.99,==&5&G&G"1"?"?.99'6'Q'Q  r*)rrrrr^)NNNNNNNN)r!r"r#r$rrKrrrrrr&r'r LongTensorrrrr-rfr)rhris@r+rrRsz&('26>B7;59=A,0/3&*A ''A U--.A !)):): ; A "%"3"34 A   1 12 A  ((9(9:A $D>A 'tnA d^A  uU&&'8 9A A r*rc2^\rSrSrSrU4SjrSrSrU=r$)DetrMLPPredictionHeadiz Very simple multi-layer perceptron (MLP, also called FFN), used to predict the normalized center coordinates, height and width of a bounding box w.r.t. an image. Copied from https://github.com/facebookresearch/detr/blob/master/models/detr.py c>[TU]5 X@lU/US- -n[R"S[ U/U-XS/-555Ulg)Nrc3R# UHup[R"X5v M g7fr)r r)rrPks r+r1DetrMLPPredictionHead.__init__.. s#g@fBIIaOO@fs%')rJrK num_layersr rzipr)rO input_dim hidden_dim output_dimrhrQs r+rKDetrMLPPredictionHead.__init__ sN $ LJN +mm#gYKRSOUVYeUe@f#gg r*c[UR5HDup#X RS- :a%[RR U"U55OU"U5nMF U$)Nr)rrrr rrelu)rOrcrlayers r+rfDetrMLPPredictionHead.forwardsI!$++.HA01OOa4G0G ""58,USTXA/r*)rrrris@r+rrsh r*rz DETR Model (consisting of a backbone and encoder-decoder Transformer) with object detection heads on top, for tasks such as COCO detection. c^^\rSrSrS\4U4Sjjr\SS\RS\ \RS\ \RS\ \RS\ \RS \ \RS \ \ \ S \ \ S \ \ S \ \ S\\\R\44Sjj5rSrU=r$)DetrForObjectDetectionirc>[TU]U5 [U5Ul[R "UR URS-5Ul[UR UR SSS9Ul UR5 g)Nrrr )rrrr) rJrKrrxr rr num_labelsclass_labels_classifierrbbox_predictorrrFs r+rKDetrForObjectDetection.__init__sr  v& (*yy NNF--1( $4nnAZ[  r*rrrrrrlabelsrrrrc  U bU OURRn URUUUUUUUU U S9 n U Sn URU 5n UR U 5R 5nSunnnUbSunnURR (aHU (a U ROU SnURU5nUR U5R 5nURXURXRUU5unnnU (dUb X4U-U -nOX4U -nUbUU4U-$U$[UUU UUU RU RU RU RU RU R U R"S9 $)a decoder_attention_mask (`torch.FloatTensor` of shape `(batch_size, num_queries)`, *optional*): Not used by default. Can be used to mask object queries. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing the flattened feature map (output of the backbone + projection layer), you can choose to directly pass a flattened representation of an image. decoder_inputs_embeds (`torch.FloatTensor` of shape `(batch_size, num_queries, hidden_size)`, *optional*): Optionally, instead of initializing the queries with a tensor of zeros, you can choose to directly pass an embedded representation. labels (`List[Dict]` of len `(batch_size,)`, *optional*): Labels for computing the bipartite matching loss. List of dicts, each dictionary containing at least the following 2 keys: 'class_labels' and 'boxes' (the class labels and bounding boxes of an image in the batch respectively). The class labels themselves should be a `torch.LongTensor` of len `(number of bounding boxes in the image,)` and the boxes a `torch.FloatTensor` of shape `(number of bounding boxes in the image, 4)`. Examples: ```python >>> from transformers import AutoImageProcessor, DetrForObjectDetection >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("facebook/detr-resnet-50") >>> model = DetrForObjectDetection.from_pretrained("facebook/detr-resnet-50") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> # convert outputs (bounding boxes and class logits) to Pascal VOC format (xmin, ymin, xmax, ymax) >>> target_sizes = torch.tensor([image.size[::-1]]) >>> results = image_processor.post_process_object_detection(outputs, threshold=0.9, target_sizes=target_sizes)[ ... 0 ... ] >>> for score, label, box in zip(results["scores"], results["labels"], results["boxes"]): ... box = [round(i, 2) for i in box.tolist()] ... print( ... f"Detected {model.config.id2label[label.item()]} with confidence " ... f"{round(score.item(), 3)} at location {box}" ... ) Detected remote with confidence 0.998 at location [40.16, 70.81, 175.55, 117.98] Detected remote with confidence 0.996 at location [333.24, 72.55, 368.33, 187.66] Detected couch with confidence 0.995 at location [-0.02, 1.15, 639.73, 473.76] Detected cat with confidence 0.999 at location [13.24, 52.05, 314.02, 470.93] Detected cat with confidence 0.999 at location [345.4, 23.85, 640.37, 368.72] ```)rrrrrrrrrNNNrr) r2r3r4r5r6r7r8r9r:r;r<r=)rrrxrrsigmoidrr loss_functionrpr0r7r8r9r:r;r<r=)rOrrrrrrrrrrrRsequence_outputr4r5r2r3r6 outputs_class outputs_coordroutputs r+rfDetrForObjectDetection.forward-s@&1%((9AAC -=*i*  +5 (M={{))EPwAAV]^_V` $ < <\ J $ 3 3L A I I K 151C1C ZmUb2 .D). , -0AAGK -7373CT9%. O O(!/%77")"?"?&99$55&-&G&G")"?"?&99  r*)rrrxr)r!r"r#r$rrKrr&r'rrrdictrrrr0rfr)rhris@r+rrs!z"26>B7;59=A'+,0/3&*s ''s U--.s !)):): ; s "%"3"34 s   1 12 s  ((9(9:s d$s $D>s 'tns d^s  uU&&')BB Cs s r*rz DETR Model (consisting of a backbone and encoder-decoder Transformer) with a segmentation head on top, for tasks such as COCO panoptic. c^^\rSrSrS\4U4Sjjr\SS\RS\ \RS\ \RS\ \RS\ \RS \ \RS \ \ \ S \ \ S \ \ S \ \ S\\\R\44Sjj5rSrU=r$)DetrForSegmentationircj>[TU]U5 [U5UlURUR p2URR RRRn[X#-USSS2SSU5Ul [X"USURS9UlUR5 g)Nr`r4)rri)rJrKrdetrrr:rxrrrDetrMaskHeadSmallConv mask_headrlrkbbox_attentionr)rOr hidden_sizenumber_of_headsrrQs r+rKDetrForSegmentation.__init__s  +62 (.~~v7U7U_%)YY__%=%=%J%J%e%e".  )+Edd+KBC+PR] 1 osH^H^  r*rrrrrrrrrrrc U bU OURRn URuppURnUc[R "XU4US9nUR RRXS9unnUSunnURuppUR RRU5nURS5RSSS5nUSRS5RSSS5nURS5nUc)UR RRUUUUU U S9nORU (aK[U[5(d6[US[U5S:aUSOS[U5S:aUSOSS 9nUR RR R"R%S5R'U SS5n[R("U5nUR RR+USUUUSUUU U S 9 nUSnUR R-U5nUR R/U5R15nUSRSSS5R3XRR4X5nUR3XU5nUR7UUU)S 9nUR9UUUSSUSSUSS/5n U R3XR RR:U RS U RS5n!S un"n#n$UbSun%n&URR<(a\U (a UR>OUSn'UR R-U'5n%UR R/U'5R15n&URAUXUU!URU%U&5un"n#n$U (d)U$bUUU!4U$-U-U-n(O UUU!4U-U-n(U"bU"U#4U(-$U($[CU"U#UUU!U$URDURFURHURJURDURFURHS9 $)a` decoder_attention_mask (`torch.FloatTensor` of shape `(batch_size, num_queries)`, *optional*): Not used by default. Can be used to mask object queries. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing the flattened feature map (output of the backbone + projection layer), you can choose to directly pass a flattened representation of an image. decoder_inputs_embeds (`torch.FloatTensor` of shape `(batch_size, num_queries, hidden_size)`, *optional*): Optionally, instead of initializing the queries with a tensor of zeros, you can choose to directly pass an embedded representation. labels (`List[Dict]` of len `(batch_size,)`, *optional*): Labels for computing the bipartite matching loss, DICE/F-1 loss and Focal loss. List of dicts, each dictionary containing at least the following 3 keys: 'class_labels', 'boxes' and 'masks' (the class labels, bounding boxes and segmentation masks of an image in the batch respectively). The class labels themselves should be a `torch.LongTensor` of len `(number of bounding boxes in the image,)`, the boxes a `torch.FloatTensor` of shape `(number of bounding boxes in the image, 4)` and the masks a `torch.FloatTensor` of shape `(number of bounding boxes in the image, height, width)`. Examples: ```python >>> import io >>> import requests >>> from PIL import Image >>> import torch >>> import numpy >>> from transformers import AutoImageProcessor, DetrForSegmentation >>> from transformers.image_transforms import rgb_to_id >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("facebook/detr-resnet-50-panoptic") >>> model = DetrForSegmentation.from_pretrained("facebook/detr-resnet-50-panoptic") >>> # prepare image for the model >>> inputs = image_processor(images=image, return_tensors="pt") >>> # forward pass >>> outputs = model(**inputs) >>> # Use the `post_process_panoptic_segmentation` method of the `image_processor` to retrieve post-processed panoptic segmentation maps >>> # Segmentation results are returned as a list of dictionaries >>> result = image_processor.post_process_panoptic_segmentation(outputs, target_sizes=[(300, 500)]) >>> # A tensor of shape (height, width) where each value denotes a segment id, filled with -1 if no segment is found >>> panoptic_seg = result[0]["segmentation"] >>> # Get prediction score and segment_id to class_id mapping of each segment >>> panoptic_segments_info = result[0]["segments_info"] ```Nr)rr`rrrrrr)rrrr) r2r3r4r5rAr6r7r8r9r:r;r<r=)&rrrrpr&rMrrxrrrrrrmrrtr^rFrrrrrrrrrrrrrrrr?r7rrr:))rOrrrrrrrrrrrrrrrprrrrrrrrr^rrrr4r5memory bbox_mask seg_masksrAr2r3r6rrrrs) r+rfDetrForSegmentation.forwardszB&1%2D2D/ &$$  Z$?OJ)- (@(@(@(e%%%RL T2=2C2C/ & $  @ @ M3::1=EEaAN,R088;CCAq!La  ""iioo550--"3%9' 6OO_!M!M-"1!"4474H14Loa0RV14_1E1I?1-tO%)IIOO$M$M$T$T$^$^_`$a$h$h 1% !""#<=))//11!)&?"1!"4#1/!5#2  *!,22?CYY--o>FFH  #++Aq!499*kkFYFY[ai"":u= ''te'L NN#8)hqkRSnV^_`VabcVdfnopfqrsftEuv ^^J 0@0@0L0Lioo^`Naclcrcrsucvw -=*i*  +5 (M={{))MXII^mnp^q $ A A, O $ 8 8 F N N P 151C1C J ]\i2 .D). , *j9B7;59=A'+,0/3&*p ''p U--.p !)):): ; p "%"3"34 p   1 12 p  ((9(9:p d$p $D>p 'tnp d^p  uU&&')?? @p p r*r lengthc|URS5RS[U5SSS5RSS5$)Nrr)rrr5r)r rs r+_expandrss7   A  % %aVaA > F Fq! LLr*cH^\rSrSrSrU4SjrS\S\S\\4SjrSr U=r $) rixzV Simple convolutional head, using group norm. Upsampling is done using a FPN approach c|>[TU]5 US-S:wa [S5eXS-US-US-US-US-/n[R"XSS S 9Ul[R "SU5Ul[R"XS SS S 9Ul[R "[SUS 5US 5Ul [R"US USSS S 9Ul [R "[SUS5US5Ul [R"USUSSS S 9Ul [R "[SUS5US5Ul[R"USUSSS S 9Ul[R "[SUS5US5Ul[R"USS SS S 9UlXl[R"USUS S 5Ul[R"US USS 5Ul[R"USUSS 5UlUR-5Hvn[/U[R5(dM$[R0R3UR4S S 9 [R0R7UR8S5 Mx g) NrzsThe hidden_size + number of attention heads must be divisible by 8 as the number of groups in GroupNorm is set to 8rrrrr r)padding)a)rJrKrr rslay1 GroupNormgn1lay2rIgn2lay3gn3lay4gn4lay5gn5out_layradapter1adapter2adapter3modulesrmrmkaiming_uniform_rF constant_rG)rOrfpn_dims context_dim inter_dimsmrQs r+rKDetrMaskHeadSmallConv.__init__}s=  7a<)  !+[A-={a?OQ\`bQbdosuduv IIc15 <<3'IIca=!Q? <<Az!} 5z!}EIIjmZ]AqI <<Az!} 5z!}EIIjmZ]AqI <<Az!} 5z!}EIIjmZ]AqI <<Az!} 5z!}EyyA1a@  (1+z!}a@  (1+z!}a@  (1+z!}a@ A!RYY''((Q(7!!!&&!, r*rcrfpnsc[R"[XRS5UR SS5/S5nUR U5nUR U5n[RRU5nURU5nURU5n[RRU5nURUS5nURS5URS5:wa,[XARS5URS5-5nU[RRXRSSSS9-nURU5nUR!U5n[RRU5nUR#US5nURS5URS5:wa,[XARS5URS5-5nU[RRXRSSSS9-nUR%U5nUR'U5n[RRU5nUR)US5nURS5URS5:wa,[XARS5URS5-5nU[RRXRSSSS9-nUR+U5nUR-U5n[RRU5nUR/U5nU$)Nrrrnearest)rmoder)r&rrrrr#r%r rrr&r'r/rrr(r)r0r*r+r1r,r-r.)rOrcrr:cur_fpns r+rfDetrMaskHeadSmallConv.forwards IIwq//!"45y7H7HA7NOQR S IIaL HHQK MM  q ! IIaL HHQK MM  q !--Q( <<?affQi 'gvvayGLLO'CDG bmm// bc8JQZ/[ [ IIaL HHQK MM  q !--Q( <<?affQi 'gvvayGLLO'CDG bmm// bc8JQZ/[ [ IIaL HHQK MM  q !--Q( <<?affQi 'gvvayGLLO'CDG bmm// bc8JQZ/[ [ IIaL HHQK MM  q ! LLOr*)r/r0r1rr%r'r)r+r-r#r&r(r*r,r.) r!r"r#r$r%rKr rrfr)rhris@r+rrxs1 -D&&F&$v,&&r*rcH^\rSrSrSrSU4SjjrSS\\4SjjrSr U=r $) rlizdThis is a 2D attention module, which only returns the attention softmax (no multiplication by value)c>[TU]5 X0lX l[R "U5Ul[R"XUS9Ul[R"XUS9Ul [X R- 5S-Ul g)Nrr) rJrKrrr Dropoutrrrprornormalize_fact)rO query_dimrrrrGrirQs r+rKDetrMHAttentionMap.__init__se "$zz'*  )dC  )dC #J$?@DHr*rcURU5n[RRX RR R S5R S5URR5nURURSURSURURUR-5nURURSURURUR-URSURS5n[R"SX@R-U5nUbXURUR S5R S5[R "UR"5R$5n[RR'UR)S5SS9RUR+55nUR-U5nU$)Nr`rrrzbqnc,bnchw->bqnhwrr)rpr rconv2drorFrrGrrrrr&einsumrC masked_fillrNrrIr%rrr)rOqrrqueries_per_head keys_per_headweightss r+rfDetrMHAttentionMap.forwards| MM!  MM MM$8$8$B$B2$F$P$PQS$TVZVcVcVhVh i66!''!*aggaj$..$//]a]k]kJklqwwqz4>>4??dnn;\^_^e^efh^iklkrkrsukvw ,,24DGZGZ4Z\ij  ))$..*;*E*Ea*H%++V]VcVcJdJhJhiG--''(:'CHHX,,w'r*)rrrorCrrp)r4TNr) r!r"r#r$r%rKrr rfr)rhris@r+rlrls"n I (6"2  r*rl)rr rre)Ar%r dataclassesrtypingrrrrrr&r r activationsr modeling_attn_mask_utilsr modeling_outputsrrrmodeling_utilsrutilsrrrrrutils.backbone_utilsrconfiguration_detrrrr get_loggerr!loggerrr-r0r?ModulerCrwr}rrrrrr7rWrerrrrrr r5rrrl__all__rr*r+r\sv !55 !Bgg-2*!   H % C:C C: $C($C $CN ;B ;B ;B| BB[BB BBN$ BII$ N'0EbiiEPBII*" "J299. ~2BII~2BDryyDNfryyfR?/??