fTh~SrSSKrSSKJr SSKJrJrJrJr SSK r SSK r SSK J r SSK J r SSKJr SS KJr SS KJrJrJr SS KJr S S KJr \R4"\5r\"SS\55r"SS\ R<5r"SS\ R<5r "SS\ R<5r!"SS\ R<5r""SS\ R<5r#"SS\ R<5r$"SS\ R<5r%S@Sjr&"SS \ R<5r'"S!S"\ R<5r("S#S$\ R<5r)\ RTRVSAS%\,S&\-4S'jj5r."S(S)\ R<5r/"S*S+\ R<5r0"S,S-\ R<5r1"S.S/\ R<5r2"S0S1\ R<5r3"S2S3\ R<5r4"S4S5\ R<5r5"S6S7\ R<5r6"S8S9\ R<5r7\"S:S;\55r8\"SS?\855r9S?S;/r:g)BzPyTorch ZoeDepth model.N) dataclass)ListOptionalTupleUnion)nn)ACT2FN)DepthEstimatorOutput)PreTrainedModel) ModelOutputauto_docstringlogging) load_backbone)ZoeDepthConfigc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\ \RS4\ S'Sr\\ \RS4\ S 'S rg) ZoeDepthDepthEstimatorOutput$a Extension of `DepthEstimatorOutput` to include domain logits (ZoeDepth specific). Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Classification (or regression if config.num_labels==1) loss. predicted_depth (`torch.FloatTensor` of shape `(batch_size, height, width)`): Predicted depth for each pixel. domain_logits (`torch.FloatTensor` of shape `(batch_size, num_domains)`): Logits for each domain (e.g. NYU and KITTI) in case multiple metric heads are used. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, num_channels, height, width)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, patch_size, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. Nlosspredicted_depth domain_logits. hidden_states attentions)__name__ __module__ __qualname____firstlineno____doc__rrtorch FloatTensor__annotations__rrrrr__static_attributes__rf/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/zoedepth/modeling_zoedepth.pyrr$s2)-D(5$$ %,37OXe//0715M8E--.5=AM8E%"3"3S"89:A:>Ju00#567>r%rcr^\rSrSrSrU4SjrS\\RS\\R4Sjr Sr U=r $)ZoeDepthReassembleStageFa This class reassembles the hidden states of the backbone into image-like feature representations at various resolutions. This happens in 3 stages: 1. Map the N + 1 tokens to a set of N tokens, by taking into account the readout ([CLS]) token according to `config.readout_type`. 2. Project the channel dimension of the hidden states according to `config.neck_hidden_sizes`. 3. Resizing the spatial dimensions (height, width). Args: config (`[ZoeDepthConfig]`): Model configuration class defining the model architecture. c V>[TU]5 URUl[R"5Ul[ URUR5H(up#UR R[XUS95 M* URS:Xa[R"5Ul URnURH\nURR[R"[R"SU-U5[UR 55 M^ gg)N)channelsfactorproject)super__init__ readout_typer ModuleListlayerszipneck_hidden_sizesreassemble_factorsappendZoeDepthReassembleLayerreadout_projectsbackbone_hidden_size SequentialLinearr hidden_act)selfconfigneck_hidden_sizer, hidden_size_ __class__s r&r0 ZoeDepthReassembleStage.__init__Vs "//mmo (+F,D,DfF_F_(` $  KK  6vagh i)a   ) +$&MMOD ! 55K--%%,,MM"))A O["I6RXRcRcKde. ,r%rreturnc:USRSn[R"USS9nUSS2S4USS2SS24pURupgnURXbX85nUR SSSS5R 5nUR S:XaWURS5R S5nURSS9RU5n [R"X4S 5nO#UR S :XaXRS 5-n/n [URUSS95HtupUR S:XaURU "U 5n U R SSS5RUS X#5n URU "U 5n U RU 5 Mv U $) z Args: hidden_states (`List[torch.FloatTensor]`, each of shape `(batch_size, sequence_length + 1, hidden_size)`): List of hidden states from the backbone. rdimNrr r.r-)rr.radd)shaper!catreshapepermute contiguousr1flatten unsqueeze expand_as enumeratesplitr9r3r7) r>r patch_height patch_width batch_size cls_tokentotal_batch_sizesequence_length num_channelsreadoutout stage_idx hidden_states r&forwardZoeDepthReassembleStage.forwardgs #1%++A.  -Q7 #0A#6 ae8L=:G:M:M7<%--.>kh %--aAq9DDF    ))11!4<CM   % '),?,?,CCM'01D1DZUV1D1W'X #I  I-#44Y? M (//1a8@@RQ]kL;;y1,?L JJ| $(Y r%)r3r9r1 rrrrr r0rr!Tensorr`r$ __classcell__rCs@r&r(r(Fs; "&T%,,%7&W[\a\h\hWi&&r%r(c.^\rSrSrU4SjrSrSrU=r$)r8c R>[TU]5 URn[R"XBSS9UlUS:a[R "X"X3SS9UlgUS:Xa[R"5UlgUS:a)[R"X"S[SU- 5SS9Ulgg)Nr) in_channels out_channels kernel_sizerrkstridepaddingr ) r/r0r:rConv2d projectionConvTranspose2dresizeIdentityint)r>r?r+r,rArCs r&r0 ZoeDepthReassembleLayer.__init__s 11 )) `ab A:,,XVlmnDK q[++-DK aZ))HAcRSV\R\oghiDKr%cJURU5nURU5nU$Nrprrr>r_s r&r`ZoeDepthReassembleLayer.forwards$|4 {{<0 r%rxrrrrr0r`r$rdres@r&r8r8s j r%r8c.^\rSrSrU4SjrSrSrU=r$)ZoeDepthFeatureFusionStagec>[TU]5 [R"5Ul[ [ UR55H'nURR[U55 M) grw) r/r0rr2r3rangelenr5r7ZoeDepthFeatureFusionLayer)r>r?rBrCs r&r0#ZoeDepthFeatureFusionStage.__init__sM mmo s63345A KK  9&A B6r%cUSSS2n/nSn[XR5H*upEUc U"U5nOU"X45nURU5 M, U$)NrI)r4r3r7)r>rfused_hidden_statesfused_hidden_stater_layers r&r`"ZoeDepthFeatureFusionStage.forwardsg%dd+  !#&}kk#B L!)%*<%8"%*+=%L"  & &'9 : $C#"r%)r3r{res@r&r}r}sC ##r%r}cf^\rSrSrSrU4SjrS\RS\R4SjrSr U=r $)ZoeDepthPreActResidualLayerz ResidualConvUnit, pre-activate residual unit. Args: config (`[ZoeDepthConfig]`): Model configuration class defining the model architecture. c >[TU]5 URUlURb UROUR(+n[ R "5Ul[ R"URURSSSUS9Ul [ R "5Ul [ R"URURSSSUS9Ul UR(a][ R"URURS9Ul[ R"URURS9Ulgg)Nr r)rkrmrnbias)eps)r/r0!use_batch_norm_in_fusion_residualuse_batch_normuse_bias_in_fusion_residualrReLU activation1rofusion_hidden_size convolution1 activation2 convolution2 BatchNorm2dbatch_norm_eps batch_norm1 batch_norm2)r>r?rrCs r&r0$ZoeDepthPreActResidualLayer.__init__s $FF11=  . .((( $ 779II  % %  % %,  779II  % %  % %,     !~~f.G.GVMbMbcD !~~f.G.GVMbMbcD  r%r_rEcUnURU5nURU5nUR(aURU5nUR U5nUR U5nUR(aUR U5nX-$rw)rrrrrrrr>r_residuals r&r`#ZoeDepthPreActResidualLayer.forwards'' 5 ((6   ++L9L'' 5 ((6   ++L9L&&r%)rrrrrrr) rrrrr r0r!rcr`r$rdres@r&rrs/ dD'ELL'U\\''r%rc:^\rSrSrSrSU4SjjrSSjrSrU=r$)ra Feature fusion layer, merges feature maps from different stages. Args: config (`[ZoeDepthConfig]`): Model configuration class defining the model architecture. align_corners (`bool`, *optional*, defaults to `True`): The align_corner setting for bilinear upsample. c>[TU]5 X l[R"UR UR SSS9Ul[U5Ul[U5Ul g)NrT)rkr) r/r0 align_cornersrrorrprresidual_layer1residual_layer2)r>r?rrCs r&r0#ZoeDepthFeatureFusionLayer.__init__ sR *))F$=$=v?X?Xfgnrs:6B:6Br%ctUbhURUR:wa;[RRX!RSURS4SSS9nXR U5-nUR U5n[RRUSSUR S9nURU5nU$)Nr.r bilinearFsizemoder scale_factorrr)rKr functional interpolaterrrrprs r&r`"ZoeDepthFeatureFusionLayer.forwards  !!X^^3==44$6$6q$9<;M;Ma;P#QXbrw5(*>*>x*HHL++L9 }}00 qzI[I[1 |4 r%)rrprr)Trw rrrrr r0r`r$rdres@r&rrsCr%rcr^\rSrSrSrU4SjrS\\RS\\R4Sjr Sr U=r $) ZoeDepthNecki%a3 ZoeDepthNeck. A neck is a module that is normally used between the backbone and the head. It takes a list of tensors as input and produces another list of tensors as output. For ZoeDepth, it includes 2 stages: * ZoeDepthReassembleStage * ZoeDepthFeatureFusionStage. Args: config (dict): config dict. c >[TU]5 XlURb"URRS;aSUlO[ U5Ul[R"5Ul URH=nURR[R"X!RSSSS95 M? [U5Ulg)N)swinv2r rF)rkrnr)r/r0r?backbone_config model_typereassemble_stager(rr2convsr5r7rorr} fusion_stage)r>r?channelrCs r&r0ZoeDepthNeck.__init__2s    ! ! -&2H2H2S2SWa2a$(D !$;F$CD !]]_ //G JJ  bii1J1JXYcdkpq r07v>r%rrEc[U[[45(d [S5e[ U5[ UR R 5:wa [S5eURbURXU5n[U5VVs/sHupEURU"U5PM nnnURU5nXvS4$s snnf)z Args: hidden_states (`List[torch.FloatTensor]`, each of shape `(batch_size, sequence_length, hidden_size)` or `(batch_size, hidden_size, height, width)`): List of hidden states from the backbone. z2hidden_states should be a tuple or list of tensorszOThe number of hidden states should be equal to the number of neck hidden sizes.rI) isinstancetuplelist TypeErrorrr?r5 ValueErrorrrSrr)r>rrUrVifeaturefeaturesoutputs r&r`ZoeDepthNeck.forwardCs -%77PQ Q } T[[%B%B!C Cno o  , 11-{[M=F}=UV=UzqDJJqM'*=UV""8,|## Ws !C)r?rrrrbres@r&rr%s; ?"$T%,,%7$W[\a\h\hWi$$r%rcl^\rSrSrSrU4SjrS\\RS\R4Sjr Sr U=r $)#ZoeDepthRelativeDepthEstimationHeadi[a Relative depth estimation head consisting of 3 convolutional layers. It progressively halves the feature dimension and upsamples the predictions to the input resolution after the first convolutional layer (details can be found in DPT's paper's supplementary material). c>[TU]5 URUlSUlUR(a[ R "SSSSSS9UlURn[ R "X"S-SSSS9Ul[ R"SSS S 9Ul [ R "US-URSSSS9Ul [ R "URSSSS S9Ul g) Nr)r r )rrrlr.r rrTrr)r/r0 head_in_indexrpadd_projectionrrorconv1Upsampleupsamplenum_relative_featuresconv2conv3)r>r?rrCs r&r0,ZoeDepthRelativeDepthEstimationHead.__init__bs #11   iiSfV]cdDO,,YYxQAaYZ[  SWX YYx1}f.J.JXYbcmno YYv;;QAVWabc r%rrEcXRnURb,URU5n[R"5"U5nUR U5nUR U5nUR U5n[R"5"U5nUnURU5n[R"5"U5nURSS9nX24$)NrrG) rrprrrrrrsqueeze)r>rrrs r&r`+ZoeDepthRelativeDepthEstimationHead.forwardqs%&8&89 ?? & OOM:MGGIm4M =1  m4  =1  -0   =1  -0 '//A/6((r%)rrrrrprrbres@r&rr[s3 d)T%,,%7)ELL))r%rcX-nX-nU[R"U5-U[R"U5-- X- [R"X- U-5-- $)z%log(nCk) using stirling approximation)r!log)nkrs r& log_binomrsP A A uyy| a%))A,. .!%599QUS[;Q1Q QQr%cN^\rSrSrS\R 4U4SjjrSSjrSrU=r $)LogBinomialSoftmaxirc 4>[TU]5 XlX lUR S[ R "SU5RSSSS5SS9 UR S[ R"URS- /5RSSSS5SS9 g) aCompute log binomial distribution for n_classes Args: n_classes (`int`, *optional*, defaults to 256): Number of output classes. act (`torch.nn.Module`, *optional*, defaults to `torch.softmax`): Activation function to apply to the output. k_idxrrrIF) persistent k_minus_1N) r/r0ractregister_bufferr!arangeviewtensor)r> n_classesrrCs r&r0LogBinomialSoftmax.__init__s  Well1i&@&E&EaQPQ&R_de [%,, |*D*I*I!RQRTU*Vchir%cURS:XaURS5n[R"SU- US5n[R"XS5n[ UR UR 5UR [R"U5--UR UR - [R"U5--nURXR- SS9$)aXCompute the log binomial distribution for probabilities. Args: probabilities (`torch.Tensor` of shape `(batch_size, num_channels, height, width)`): Tensor containing probabilities of each class. temperature (`float` or `torch.Tensor` of shape `(batch_size, num_channels, height, width)`, *optional*, defaults to 1): Temperature of distribution. eps (`float`, *optional*, defaults to 1e-4): Small number for numerical stability. Returns: `torch.Tensor` of shape `(batch_size, num_channels, height, width)`: Log binomial distribution logbinomial(p;t). r rrG) ndimrQr!clamprrrrr)r> probabilities temperaturerone_minus_probabilitiesys r&r`LogBinomialSoftmax.forwards    ")33A6M"'++a-.?a"H M: dnndjj 1jj599]33 4~~ *eii8O.PP Q xxQx//r%)rr)?-C6?) rrrrr!softmaxr0r`r$rdres@r&rrs!$%-- j00r%rc6^\rSrSrSU4SjjrSrSrU=r$)%ZoeDepthConditionalLogBinomialSoftmaxic >[TU]5 X#-U-n[R"[R"X#-USSSS9[R "5[R"USSSSS9[R "55UlSUlURUl URUl [U[RS9Ulg)aPer-pixel MLP followed by a Conditional Log Binomial softmax. Args: in_features (`int`): Number of input channels in the main feature. condition_dim (`int`): Number of input channels in the condition feature. n_classes (`int`, *optional*, defaults to 256): Number of classes. bottleneck_factor (`int`, *optional*, defaults to 2): Hidden dim factor. rrrlr)rN)r/r0rr;roGELUSoftplusmlpp_epsmax_tempmin_temprr!rlog_binomial_transform)r>r? in_features condition_dimrbottleneck_factor bottleneckrCs r&r0.ZoeDepthConditionalLogBinomialSoftmax.__init__s* !16GG == IIk1:1UV`a b GGI IIj%Qq! L KKM     &8 &V#r%cUR[R"X4SS95nUSS2SS2S4USS2SS2S4pTX@R-nUSS2SS4USS2SS4USS2SS4-- nXPR-nUSS2SS4USS2SS4USS2SS4-- nUR S5nUR UR - U-UR -nURXE5$)a2 Args: main_feature (`torch.Tensor` of shape `(batch_size, num_channels, height, width)`): Main feature. condition_feature (torch.Tensor of shape `(batch_size, num_channels, height, width)`): Condition feature. Returns: `torch.Tensor`: Output log binomial distribution rrGNr..r)rr!concatrrQrrr)r> main_featurecondition_featureprobabilities_and_temperaturerrs r&r`-ZoeDepthConditionalLogBinomialSoftmax.forwards )-|>_ef1g(h% )!RaR* 5 )!QR* 5# & 2 %aCi0M!Q)4L}]^`acf]fOg4gh !JJ. !!Q), Aq#I0FUVXY[^U^I_0_` !++A. }}t}}4 CdmmS **=FFr%)rrrrr)rr.r{res@r&rrs  #WJGGr%rc2^\rSrSrSU4SjjrSrSrU=r$)ZoeDepthSeedBinRegressoric>[TU]5 URUlURUlX@lXPl[R"URUSSS5Ul [R"SS9Ul [R"X2SSS5Ul URS:Xa[R"SS9Ulg[R"5Ulg)aBin center regressor network. Can be "normed" or "unnormed". If "normed", bin centers are bounded on the (min_depth, max_depth) interval. Args: config (`int`): Model configuration. n_bins (`int`, *optional*, defaults to 16): Number of bin centers. mlp_dim (`int`, *optional*, defaults to 256): Hidden dimension. min_depth (`float`, *optional*, defaults to 1e-3): Min depth value. max_depth (`float`, *optional*, defaults to 10): Max depth value. rrTinplacenormedN)r/r0bottleneck_featuresrbin_centers_type min_depth max_depthrrorract1rract2)r>r?n_binsmlp_dimrrrCs r&r0!ZoeDepthSeedBinRegressor.__init__s" !55 & 7 7""YYt//!QB GGD) YYw1a8 -1-B-Bh-NBGGD) TVT_T_Ta r%cURU5nURU5nURU5nURU5nURS:XaUS-nX"R SSS9- nUR UR- U-n[RRUSSURS9n[R"USS 9nS US S 2S S 2S 4US S 2SS 2S 4--nX24$X"4$)zM Returns tensor of bin_width vectors (centers). One vector b for every pixel rMbP?rTrHkeepdim)rrrrrrconstant)rvaluerGg?NrI.) rrrrrsumrrrrpadr!cumsum)r>x bin_centersbin_widths_normed bin_widths bin_edgess r&r` ZoeDepthSeedBinRegressor.forwards JJqM IIaL JJqMiil  H ,%,K +oo!To.R R ..4>>9=NNJ**:7IPZbfbpbp*qJ ZQ7I1crc3;!7)Aqr3J:O!OPK$1 1+ +r%)rrrrrrrr)rr r{res@r&r r sb:,,r%r alphagammacLURSXRU5--5$)a Inverse attractor: dc = dx / (1 + alpha*dx^gamma), where dx = a - c, a = attractor point, c = bin center, dc = shift in bin center This is the default one according to the accompanying paper. Args: dx (`torch.Tensor`): The difference tensor dx = Ai - Cj, where Ai is the attractor point and Cj is the bin center. alpha (`float`, *optional*, defaults to 300): Proportional Attractor strength. Determines the absolute strength. Lower alpha = greater attraction. gamma (`int`, *optional*, defaults to 2): Exponential Attractor strength. Determines the "region of influence" and indirectly number of bin centers affected. Lower gamma = farther reach. Returns: torch.Tensor: Delta shifts - dc; New bin centers = Old bin centers + dc r)divpow)dxr+r,s r& inv_attractorr15s#" 66!effUm++ ,,r%c>^\rSrSrSU4SjjrSSjrSrU=r$)ZoeDepthAttractorLayeriIc>[T U]5 URUlURUlUR UlX0lX l X@l XPl X`l UR=px[R"XxSSS5Ul[R""SS9Ul[R"XS-SSS5Ul[R""SS9Ulg)za Attractor layer for bin centers. Bin centers are bounded on the interval (min_depth, max_depth) rrTrr.N)r/r0attractor_alphar+attractor_gammagemmaattractor_kindkind n_attractorsrrrmemory_efficientbin_embedding_dimrrorrrrr r>r?rr:rrr;rrrCs r&r0ZoeDepthAttractorLayer.__init__Js ++ ++ )) ( "" 0!' 8 88 YY{Q1= GGD) YYwq(8!QB GGD) r%cUb7U(a,[RRX1RSSSSS9nX-nUR U5nUR U5nUR U5nURU5nUS-nURupgpURX`RSX5nUSS2SS2SS 4n [RRX(U 4SSS9n UR(d`[R[RS .URn U "[U R!S5U R!S 5- 5S S 9n O[R""XR$S 9n ['UR5H+nU [U SS2US 4R!S 5U - 5- n M- URS:XaXR- n X-nUR(UR*- U-UR*-n [R,"U S S 9up[R."XR*UR(5n X4$)a The forward pass of the attractor layer. This layer predicts the new bin centers based on the previous bin centers and the attractor points (the latter are predicted by the MLP). Args: x (`torch.Tensor` of shape `(batch_size, num_channels, height, width)`): Feature block. prev_bin (`torch.Tensor` of shape `(batch_size, prev_number_of_bins, height, width)`): Previous bin centers normed. prev_bin_embedding (`torch.Tensor`, *optional*): Optional previous bin embeddings. interpolate (`bool`, *optional*, defaults to `True`): Whether to interpolate the previous bin embeddings to the size of the input features. Returns: `Tuple[`torch.Tensor`, `torch.Tensor`]: New bin centers normed and scaled. NrTrrrr.r.meanr rrGdevicerC)rrrrKrrrrrr:r;r!rCr r9r1rQ zeros_likerErrrsortclip)r>r#prev_binprev_bin_embeddingr attractorsrWrBheightwidthattractors_normedr$funcdelta_crbin_new_centerss r&r`ZoeDepthAttractorLayer.forwardis&  )%']]%>%>& :UY&?&"&A JJqM IIaL JJqMYYq\ $& '1'7'7$ v__Z1B1BAvU 'q!Q|4mm//5/PZjn/o $$!JJuyy9$))DD=):)D)DQ)G+J_J_`aJb)bcijkG&&{;M;MNG4,,-=):1a9)E)O)OPQ)RU`)`aa.yyF"!$5$55%/~~6/IDNNZ KQ7 jjnndnnM ++r%) rrr+rrr7r9rr;rr:r)r)rr*FNTr{res@r&r3r3Is *><,<,r%r3c>^\rSrSrSU4SjjrSSjrSrU=r$)ZoeDepthAttractorLayerUnnormedic>[T U]5 X0lX lX@lXPlUR UlUR UlURUl X`l UR=px[R"XxSSS5Ul[R "SS9Ul[R"XSSS5Ul[R&"5Ulg)z< Attractor layer for bin centers. Bin centers are unbounded rrTrN)r/r0r:rrrr5r+r,r8r9r;r<rrorrrrrrr=s r&r0'ZoeDepthAttractorLayerUnnormed.__init__s ( ""++ ++ )) 0 & 8 88 YY{Q1= GGD) YYwaA> KKM r%cVUb7U(a,[RRX1RSSSSS9nX-nUR U5nUR U5nUR U5nURU5nURSSupg[RRX&U4SSS9nUR(d`[R[RS.URn U "[URS5URS5- 5SS 9n O[R"XR S 9n [#UR$5H+n U [USS2U S 4RS5U- 5- n M- URS :XaXR$- n X-n U nX4$) a, The forward pass of the attractor layer. This layer predicts the new bin centers based on the previous bin centers and the attractor points (the latter are predicted by the MLP). Args: x (`torch.Tensor` of shape (batch_size, num_channels, height, width)`): Feature block. prev_bin (`torch.Tensor` of shape (batch_size, prev_num_bins, height, width)`): Previous bin centers normed. prev_bin_embedding (`torch.Tensor`, *optional*): Optional previous bin embeddings. interpolate (`bool`, *optional*, defaults to `True`): Whether to interpolate the previous bin embeddings to the size of the input features. Returns: `Tuple[`torch.Tensor`, `torch.Tensor`]: New bin centers unbounded. Two outputs just to keep the API consistent with the normed version. Nr@rTrArBr.rrGrD.rC)rrrrKrrrrr;r!rCr r9r1rQrFrErr:) r>r#rIrJrrKrLrMr$rOrPrrQs r&r`&ZoeDepthAttractorLayerUnnormed.forwards&  )%']]%>%>& :UY&?&"&A JJqM IIaL JJqMYYq\ "((- mm//5/PZjn/o $$!JJuyy9$))DD=)=)=a)@;CXCXYZC[)[\bcdG&&{;M;MNG4,,-=Aq#I)>)H)H)Kk)YZZ.yyF"!$5$55%/% ++r%) rrr+rrr,r9rr;rr:r)r)rr*TrSr{res@r&rUrUs ":3,3,r%rUcf^\rSrSrSU4SjjrS\R S\R 4SjrSrU=r $)ZoeDepthProjectoric>[TU]5 [R"XSSS5Ul[R "SS9Ul[R"X2SSS5Ulg)zProjector MLP. Args: in_features (`int`): Number of input channels. out_features (`int`): Number of output channels. mlp_dim (`int`, *optional*, defaults to 128): Hidden dimension. rrTrN)r/r0rrorrrr)r>r out_featuresrrCs r&r0ZoeDepthProjector.__init__sL YY{Q1= 774(YYwaA> r%r_rEclURU5nURU5nURU5nU$rw)rrrrys r&r`ZoeDepthProjector.forwards2zz,/ xx - zz,/ r%)rrr)) rrrrr0r!rcr`r$rdres@r&r[r[s(?"ELLU\\r%r[c^\rSrSrSrU4SjrS\RS\R4SjrSS\RS\RS \RS \ \RS \ \ S\ \R4 S jjr S rU=r$)ZoeDepthMultiheadAttentionizKEquivalent implementation of nn.MultiheadAttention with `batch_first=True`.c >[TU]5 X-S:wa[SUSUS35eX l[ X- 5UlURUR -Ul[R"XR 5Ul [R"XR 5Ul [R"XR 5Ul [R"X5Ul [R"U5Ulg)NrzThe hidden size (z6) is not a multiple of the number of attention heads ())r/r0rnum_attention_headsrtattention_head_size all_head_sizerr<querykeyrout_projDropoutdropout)r>rArfrmrCs r&r0#ZoeDepthMultiheadAttention.__init__s   , 1#K=1-.a1  $7 #&{'H#I !558P8PPYY{,>,>? 99[*<*<=YY{,>,>?  +; zz'* r%r#rEcUR5SSURUR4-nURU5nUR SSSS5$)NrIrr.rr )rrfrgrrN)r>r# new_x_shapes r&transpose_for_scores/ZoeDepthMultiheadAttention.transpose_for_scores/sLffhsmt'?'?AYAY&ZZ FF; yyAq!$$r%querieskeysvaluesattention_maskoutput_attentionscURURU55nURURU55nURURU55n[R "XgR SS55n U [R"UR5- n UbX-n [RRU SS9n URU 5n [R "X5n U RSSSS5R5n U R!5SSUR"4-n U R%U 5n UR'U 5n U(aX4n U $U 4n U $)NrIr@rGrr.rr )rqrirjrr!matmul transposemathsqrtrgrrrrmrNrOrrhrrk)r>rsrtrurvrw query_layer key_layer value_layerattention_scoresattention_probs context_layernew_context_layer_shapeoutputss r&r`"ZoeDepthMultiheadAttention.forward4sQ// 70CD --dhhtn= // 60BC !<< 5H5HR5PQ+dii8P8P.QQ  %/@ --//0@b/I,,7 _B %--aAq9DDF "/"4"4"6s";t?Q?Q>S"S%**+BC  m4 6G=2O\M]r%)rhrgrmrjrfrkrir)NF)rrrrr r0r!rcrqrr"boolrr`r$rdres@r&rcrcsU+(%ell%u||%7;,1 %%ll% % !!2!23 % $D> % u|| %%r%rccZ^\rSrSrSU4SjjrSS\\R4SjjrSr U=r $)ZoeDepthTransformerEncoderLayeri\c>[TU]5 URnURnURn[ XFUS9Ul[R"XE5Ul [R"U5Ul [R"XT5Ul [R"U5Ul[R"U5Ul[R"U5Ul[R"U5Ul[$UUlg)N)rm)r/r0patch_transformer_hidden_size#patch_transformer_intermediate_size%patch_transformer_num_attention_headsrc self_attnrr<linear1rlrmlinear2 LayerNormnorm1norm2dropout1dropout2r activation)r>r?rmrrAintermediate_sizerfrCs r&r0(ZoeDepthTransformerEncoderLayer.__init__]s :: "FF$JJ3K^efyy@ zz'* yy!2@ \\+. \\+.  7+  7+  ,r%src_maskc >U=p4URX4XS9SnXRU5-nURU5nURUR UR UR U5555nXRU5-nURU5nU$)N)rsrtrurvr) rrrrrmrrrr)r>srcrrsrtsrc2s r&r`'ZoeDepthTransformerEncoderLayer.forwardqs ~~g~^_`aMM$''jjo||DLLc9J)KLMMM$''jjo r%) rrmrrrrrrr)g?relurw) rrrrr0rr!rcr`r$rdres@r&rr\s*-.,0 5<<(  r%rcP^\rSrSrU4SjrS\R 4SjrSrSr U=r $)ZoeDepthPatchTransformerEncoderic >[TU]5 URn[R"[ UR 5Vs/sHn[U5PM sn5Ul[R"X!RSSSS9Ul gs snf)zViT-like transformer block Args: config (`ZoeDepthConfig`): Model configuration class defining the model architecture. rrrlN) r/r0rrr2rnum_patch_transformer_layersrtransformer_encoderrorembedding_convPxP)r>r?rirBrCs r&r0(ZoeDepthPatchTransformerEncoder.__init__s{ 00 #%==>CFDgDg>h i>h ,V 4>h i$  "$ ==1UV`a"  jsB cpuc [R"SX%US9RS5n[R"SUSXTS9RS5n[R"U[R"[R "SUS95*U- -5nXh-n [R "[R"U 5[R"U 5/SS9n U RSS9RUSS5n U $)zGenerate positional encodings Args: sequence_length (int): Sequence length embedding_dim (int): Embedding dimension Returns: torch.Tensor: Positional encodings. r)dtyperErr.g@rDrG) r!rrQexprrrLsincosrepeat) r>rWrZ embedding_dimrErpositionindexdiv_term pos_encodings r&positional_encoding_1d6ZoeDepthPatchTransformerEncoder.positional_encoding_1ds<<?OYYZ[\ Q qMWWXYZ99Uuyygf1U'V&VYf&fgh* yy%))L"9599\;R!SYZ[ #--!-4;;J1M r%c fURU5RS5n[RR US5nUR SSS5nUR up4nX RX4XRRURS9-n[S5HnURU"U5nM U$)zForward pass Args: x (torch.Tensor - NCHW): Input feature tensor Returns: torch.Tensor - Transformer output embeddings of shape (batch_size, sequence_length, embedding_dim) r.)rrrr)rErr) rrPrrr!rNrKrrErrr)r>r# embeddingsrWrZrrs r&r`'ZoeDepthPatchTransformerEncoder.forwards++A.66q9 ]]&&z6: ''1a0 5?5E5E2 ]"="= ?P?PXbXhXh#>#  qA11!4Z@Jr%)rr) rrrrr0r!float32rr`r$rdres@r&rrs' &Y^ejerer$r%rc2^\rSrSrSU4SjjrSrSrU=r$)ZoeDepthMLPClassifieric>[TU]5 Un[R"X5Ul[R "5Ul[R"X25Ulgrw)r/r0rr<rrrr)r>rr]hidden_featuresrCs r&r0ZoeDepthMLPClassifier.__init__s@ %yy> '')yy? r%clURU5nURU5nURU5nU$rw)rrr)r>r_rs r&r`ZoeDepthMLPClassifier.forwards2||L1 |4  \2 r%)rrr)rENr{res@r&rrs@r%rc2^\rSrSrSrU4SjrSrSrU=r$)*ZoeDepthMultipleMetricDepthEstimationHeadsizf Multiple metric depth estimation heads. A MLP classifier is used to route between 2 different heads. c>[T U]5 URnURnURUlUR UlUR n[R"XDSSSS9Ul [U5Ul [SSS9Ul UR S:Xa[nOUR S:Xa[n[R "URVs0sH nUS [#UUS US-US US S 9_M" sn5Ul['XBUS-S9Ul[R*"[-S5Vs/sHn['UR.UUS-S9PM sn5Ul[R "URVV s0sHOnUS [R*"[-[3U55V s/sHn W"UX9US US S9PM sn 5_MQ sn n5UlUR6n [R "URVs0sHnUS [9UU UUS SS9_M sn5Ulgs snfs snfs sn fs sn nfs snf)Nrrrlrar.rr]rsoftplusnamerrr)rrrr)rr]rrrrr)r)r/r0r<num_attractorsbin_configurationsrrrrorrpatch_transformerrmlp_classifierr3rU ModuleDictr seed_bin_regressorsr[seed_projectorr2rr projectorsrrKrrconditional_log_binomial) r>r?r<r:r AttractorconfrB configurationrlast_inrCs r&r03ZoeDepthMultipleMetricDepthEstimationHeads.__init__s "44,, "(";"; & 7 7%88YY2UV_`jkl "A!H3RST  H ,.I  " "j 06I$&==#55 6DV 6>-2";/";/ 6  $  0+UfjkUk --q   "A " & 9 9!2-2 "    --&,%>%> &?Mf%r}}"'s<'8!9":A ""#/?&3K&@&3K&@  ": ( &?  "..(* &,%>%> &?Mf%'L%!(+&' (&?  ) %]    & s*'I>"I 0I ;I  I I I cbURU5nURU5SS2SSS24nURU5n[R"UR SSS9SS9nUR V s/sHoSPM n n U [R"USS9R5R5n UR V s/sHoSU :XdM U PM sn Sn U S nU S nURU nU"U5unnURS ;a UU- X- - nOUnURU5nURU n[!UR"UU5HunnnU"U5nU"UUUSS 9unnUnUnM! Un[$R&R)WUR*S SSSS9n[$R&R)WUR*S SSSS9nUR,U nU"UU5n[R "UU-SSS9nUU4$s sn fs sn f![a [SU S35ef=f)NrTrrIrGrzbin_configurations_name z! not found in bin_configurationssrrrhybrid2rr@rrAr)rrrr!rr rargmaxritem IndexErrorrrrrrKr4rrrrrKr)r>outconv_activationrfeature_blocksrelative_depthr# embeddingr domain_voternamesbin_configurations_namer?rrrseed_bin_regressorrBseed_bin_centersrIrJrK projector attractorr bin_embeddingbinr$lastrr]s r&r`2ZoeDepthMultipleMetricDepthEstimationHeads.forward)su JJz "**1-aAg6 ++I6 mmM$5$5!T$5$JPRS =A[TU]5 URSnUSnUSnUSnURnURnUR nX@lXPlXlURn [R"XSSSS9Ul UR S:Xa[n OUR S:Xa[n [XXES 9Ul[!XS 9Ul[R$"['S 5V s/sHn [!UR(US 9PM sn 5Ul[R$"['S 5V s/sHn W "UUX|UUS 9PM sn 5UlUR.S-n [1UU UUS 9Ulgs sn fs sn f)Nrrrrrrlrrrrr)rr:rr)r)r/r0rr<rrrrrrrorr3rUr rr[rr2rrrrKrrr)r>r?bin_configurationrrrr<r:rrrrBrrrCs r&r0*ZoeDepthMetricDepthEstimationHead.__init__^s "55a8"8,%k2 %k2 "44,, !22"" 0%88YY2UV_`jkl   H ,.I  " "j 06I": Y# 0rrrrr#rBrrIrJrrrrrr$rrelative_conditioningr]s r&r`)ZoeDepthMetricDepthEstimationHead.forwards JJz ""55a8  $9 9(>>9dnnt~~>]^H'H!003.1$//Sa-b )I'%g.M(BTbfg Cyy{H!.!4!4!6  .c "!/ 8 8 ; " 9 9 ! 12ZW[!:! yy$ 56A> 11-BCWaqu1v  ) )$ >mm// QWWRS\PZjn/o iiQ=Dyr%) rKrrrrrrrrr{res@r&rr]s9 v""r%rc*\rSrSr\rSrSrSrSr Sr g)ZoeDepthPreTrainedModelizoedepth pixel_valuesTc([U[R[R[R45(akUR R RSURRS9 URb%URR R5 gg[U[R5(aJURR R5 UR R RS5 gg)zInitialize the weightsg)rCstdNr)rrr<rorqweightdatanormal_r?initializer_rangerzero_rfill_)r>modules r& _init_weights%ZoeDepthPreTrainedModel._init_weightss fryy"))R5G5GH I I MM   & &CT[[5R5R & S{{&   &&('  - - KK   " " $ MM   $ $S ).r%rN) rrrrr config_classbase_model_prefixmain_input_namesupports_gradient_checkpointingrr$rr%r&rrs!L"$O&*# *r%rzU ZoeDepth model with one or multiple metric depth estimation head(s) on top. ) custom_introc^\rSrSrU4Sjr\S S\RS\\RS\\ S\\ S\\ S\ \ \R\44 S jj5rS rU=r$) ZoeDepthForDepthEstimationicL>[TU]U5 [U5Ul[ URR S5(ap[ URR S5(aKURR R UlURR RUlO [S5e[U5Ul [U5Ul [UR5S:a [!U5O [#U5UlUR'5 g)NrA patch_sizezXZoeDepth assumes the backbone's config to have `hidden_size` and `patch_size` attributesr)r/r0rbackbonehasattrr?rAr:r rrneckr relative_headrrrr metric_head post_init)r>r?rCs r&r0#ZoeDepthForDepthEstimation.__init__s  %f- 4=='' 7 7GDMMDXDXZf*>*J*JF '"mm22==DOj !( @H6,,-1 7v >26:  r%rlabelsrwoutput_hidden_states return_dictrEcSnUb [S5eUbUOURRnUbUOURRnUbUOURRnUR R XUS9nURnURu pn URn X-n X-nURXU5upU/U-nURU5unnU/U-nURUSUSUSSUS9unnURSS9nU(d"Ub UU4USS-nO U4USS-nUbU4U-$U$[UUUURUR S 9$) a labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth depth estimation maps for computing the loss. Examples: ```python >>> from transformers import AutoImageProcessor, ZoeDepthForDepthEstimation >>> import torch >>> import numpy as np >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("Intel/zoedepth-nyu-kitti") >>> model = ZoeDepthForDepthEstimation.from_pretrained("Intel/zoedepth-nyu-kitti") >>> # prepare image for the model >>> inputs = image_processor(images=image, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> # interpolate to original size >>> post_processed_output = image_processor.post_process_depth_estimation( ... outputs, ... source_sizes=[(image.height, image.width)], ... ) >>> # visualize the prediction >>> predicted_depth = post_processed_output[0]["predicted_depth"] >>> depth = predicted_depth * 255 / predicted_depth.max() >>> depth = depth.detach().cpu().numpy() >>> depth = Image.fromarray(depth.astype("uint8")) ```NzTraining is not implemented yet)rrwrrr.)rrrrrG)rrrrr)NotImplementedErrorr?use_return_dictrrwr forward_with_filtered_kwargs feature_mapsrKr r r rrrrr)r>rrrwrrrrrrBrLrMr rUrVrr]r metric_depthrrs r&r`"ZoeDepthForDepthEstimation.forwardsZ  %&GH H%0% [ 'tn [ d^ [  uU\\"$88 9[ [ r%r)gHz>)i,r.);r r{ dataclassesrtypingrrrrr!torch.utils.checkpointr activationsr modeling_outputsr modeling_utilsr utilsr rrutils.backbone_utilsrconfiguration_zoedepthr get_loggerrloggerrModuler(r8r}rrrrrrrr jitscriptfloatrtr1r3rUr[rcrrrrrrr__all__rr%r&r,s5 !// !4-9912   H % ?;? ?BGbiiGTbii0##2;'"));'~""J3$2993$l))"))))XR)0)0X@GBII@GF5,ryy5,p-U---&\,RYY\,~Q,RYYQ,h 6BBJ!bii!H<bii<~BII"J"J"Z^ ^F*o**& v !8v  v r ()B Cr%