o Zh T@sdZddlZddlmZmZmZddlZddlZddlmZddl m Z m Z m Z ddl mZddlmZmZmZdd lmZdd lmZmZd d lmZeeZd edefddZd/deeefdefddZ Gdddej!Z"Gdddej#Z$Gdddej!Z%Gdddej!Z&Gdddej!Z'Gd d!d!ej!Z(Gd"d#d#ej!Z)Gd$d%d%ej!Z*eGd&d'd'eZ+eGd(d)d)e+Z,ed*d+Gd,d-d-e+Z-gd.Z.dS)0zPyTorch EfficientNet model.N)OptionalTupleUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)BaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)auto_docstringlogging)EfficientNetConfigconfig num_channelscCsJ|j}||j9}t|t||d||}|d|kr!||7}t|S)z< Round number of filters based on depth multiplier. g?)Z depth_divisorZwidth_coefficientmaxint)rrZdivisorZnew_dimre/var/www/auris/lib/python3.10/site-packages/transformers/models/efficientnet/modeling_efficientnet.py round_filters's   rT kernel_sizeadjustcCsnt|tr ||f}|dd|ddf}|r)|dd|d|dd|dfS|d|d|d|dfS)aJ Utility function to get the tuple padding value for the depthwise convolution. Args: kernel_size (`int` or `tuple`): Kernel size of the convolution layers. adjust (`bool`, *optional*, defaults to `True`): Adjusts padding value to apply to right and bottom sides of the input. rrr) isinstancer)rrZcorrectrrr correct_pad6s $rcs<eZdZdZdeffdd ZdejdejfddZZ S) EfficientNetEmbeddingszL A module that corresponds to the stem module of the original work. rcshtt|d|_tjdd|_tj|j|jddddd|_ tj |j|j |j d |_ t|j|_dS) N )rrrrpaddingr rvalidFrstrider"bias)epsmomentum)super__init__rout_dimr ZeroPad2dr"Conv2dr convolution BatchNorm2dbatch_norm_epsbatch_norm_momentum batchnormr hidden_act activationselfr __class__rrr*Os  zEfficientNetEmbeddings.__init__ pixel_valuesreturncCs,||}||}||}||}|SN)r"r.r2r4)r6r9featuresrrrforwardZs    zEfficientNetEmbeddings.forward) __name__ __module__ __qualname____doc__rr*torchTensorr= __classcell__rrr7rrJs rcs,eZdZ       dfdd ZZS) EfficientNetDepthwiseConv2drr rTzerosc s*||} tj|| |||||||d dS)N) in_channels out_channelsrr%r"dilationgroupsr& padding_mode)r)r*) r6rGZdepth_multiplierrr%r"rIr&rKrHr7rrr*ds  z$EfficientNetDepthwiseConv2d.__init__)rr rrrTrF)r>r?r@r*rDrrr7rrEcsrEcsHeZdZdZdedededeffdd Zdejd ej fd d Z Z S) EfficientNetExpansionLayerz_ This corresponds to the expansion phase of each block in the original implementation. rin_dimr+r%csBttj||dddd|_tj||jd|_t|j |_ dS)NrsameFrGrHrr"r&) num_featuresr') r)r*rr- expand_convr/r0 expand_bnr r3 expand_act)r6rrMr+r%r7rrr*s z#EfficientNetExpansionLayer.__init__ hidden_statesr:cCs"||}||}||}|Sr;)rQrRrSr6rTrrrr=s   z"EfficientNetExpansionLayer.forward) r>r?r@rArrr*rB FloatTensorrCr=rDrrr7rrL}s rLc sLeZdZdZdededededef fdd Zd ej d ej fd d Z Z S) EfficientNetDepthwiseLayerzk This corresponds to the depthwise convolution phase of each block in the original implementation. rrMr%radjust_paddingcsvt||_|jdkrdnd}t||d}tj|d|_t||||dd|_tj ||j |j d|_ t |j|_dS) Nrr#rN)rr!Fr$rPr'r()r)r*r%rrr,depthwise_conv_padrEdepthwise_convr/r0r1depthwise_normr r3 depthwise_act)r6rrMr%rrXZconv_padr"r7rrr*s    z#EfficientNetDepthwiseLayer.__init__rTr:cCs6|jdkr ||}||}||}||}|S)Nr)r%rZr[r\r]rUrrrr=s     z"EfficientNetDepthwiseLayer.forward r>r?r@rArrboolr*rBrVrCr=rDrrr7rrWsrWc sJeZdZdZd dedededeffdd Zd ej d ej fd d Z Z S)EfficientNetSqueezeExciteLayerzl This corresponds to the Squeeze and Excitement phase of each block in the original implementation. FrrM expand_dimexpandcst|r |n||_tdt||j|_tjdd|_ tj |j|jddd|_ tj |j|jddd|_ t |j|_t|_dS)Nr)Z output_sizerN)rGrHrr")r)r*dimrrZsqueeze_expansion_ratioZdim_serZAdaptiveAvgPool2dsqueezer-reducerbr r3 act_reduceZSigmoid act_expand)r6rrMrarbr7rrr*s$  z'EfficientNetSqueezeExciteLayer.__init__rTr:cCsF|}||}||}||}||}||}t||}|Sr;)rdrerfrbrgrBmul)r6rTZinputsrrrr=s      z&EfficientNetSqueezeExciteLayer.forward)Fr^rrr7rr`s r`c sVeZdZdZdedededededef fdd Zd e j d e j d e j fd dZ Z S)EfficientNetFinalBlockLayerz[ This corresponds to the final phase of each block in the original implementation. rrMr+r% drop_rateid_skipcsXt|dko | |_tj||dddd|_tj||j|jd|_ tj |d|_ dS)NrrNFrOrYp) r)r* apply_dropoutrr- project_convr/r0r1 project_bnDropoutdropout)r6rrMr+r%rjrkr7rrr*s  z$EfficientNetFinalBlockLayer.__init__ embeddingsrTr:cCs0||}||}|jr||}||}|Sr;)rorprnrr)r6rsrTrrrr=s   z#EfficientNetFinalBlockLayer.forwardr>r?r@rArrfloatr_r*rBrVrCr=rDrrr7rris $rics\eZdZdZdededededededed ed effd d Zd e j de j fddZ Z S)EfficientNetBlocka This corresponds to the expansion and depthwise convolution phase of each block in the original implementation. Args: config ([`EfficientNetConfig`]): Model configuration class. in_dim (`int`): Number of input channels. out_dim (`int`): Number of output channels. stride (`int`): Stride size to be used in convolution layers. expand_ratio (`int`): Expand ratio to set the output dimensions for the expansion and squeeze-excite layers. kernel_size (`int`): Kernel size for the depthwise convolution layer. drop_rate (`float`): Dropout rate to be used in the final phase of each block. id_skip (`bool`): Whether to apply dropout and sum the final hidden states with the input embeddings during the final phase of each block. Set to `True` for the first block of each stage. adjust_padding (`bool`): Whether to apply padding to only right and bottom side of the input kernel before the depthwise convolution operation, set to `True` for inputs with odd input sizes. rrMr+r% expand_ratiorrjrkrXc st||_|jdkrdnd|_||} |jr"t||| |d|_t||jr)| n|||| d|_t||| |jd|_ t ||jrB| n|||||d|_ dS)NrTF)rrMr+r%)rrMr%rrX)rrMrarb)rrMr+r%rjrk) r)r*rwrbrL expansionrWr[r`squeeze_exciteri projection) r6rrMr+r%rwrrjrkrXZ expand_in_dimr7rrr*!s4     zEfficientNetBlock.__init__rTr:cCs<|}|jdkr ||}||}||}|||}|S)Nr)rwrxr[ryrz)r6rTrsrrrr=Js     zEfficientNetBlock.forwardrtrrr7rrvs,  )rvc sPeZdZdZdeffdd Z  d dejdee d ee d e fd d Z Z S)EfficientNetEncoderz Forward propagates the embeddings through each EfficientNet block. Args: config ([`EfficientNetConfig`]): Model configuration class. rc s~t|_|j_fddt|j}tfdd|jD}d}g}t|D]k}t ||j|}t ||j |}|j |} |j |} |j |} t|j|D]@} | dkr]dnd} | dkredn| } | dkrm|n|}||jvrvdnd}|j||}t|||| | | || |d }|||d7}qUq+t|_tj|t |d dd dd _tj|j|j|jd _t|j_dS)Ncsttj|Sr;)rmathceildepth_coefficient)Zrepeats)r6rr round_repeatsesz3EfficientNetEncoder.__init__..round_repeatsc3s|]}|VqdSr;r).0n)rrr jsz/EfficientNetEncoder.__init__..rTFr) rrMr+r%rrwrjrkrXirNrOrY)r)r*rr~lenrGsumZnum_block_repeatsrangerrHstridesZ kernel_sizesZ expand_ratiosZdepthwise_paddingZdrop_connect_ratervappendrZ ModuleListblocksr-top_convr/ hidden_dimr0r1top_bnr r3top_activation)r6rZnum_base_blocksZ num_blocksZcurr_block_numrirMr+r%rrwjrkrXrjblockr7)rr6rr*`s\          zEfficientNetEncoder.__init__FTrToutput_hidden_states return_dictr:cCst|r|fnd}|jD] }||}|r||f7}q ||}||}||}|s4tdd||fDSt||dS)Ncss|] }|dur|VqdSr;r)rvrrrrsz.EfficientNetEncoder.forward..)last_hidden_staterT)rrrrtupler )r6rTrrZall_hidden_statesrrrrr=s     zEfficientNetEncoder.forward)FT) r>r?r@rArr*rBrVrr_r r=rDrrr7rr{Ws:r{c@s$eZdZeZdZdZgZddZdS)EfficientNetPreTrainedModel efficientnetr9cCstt|tjtjfr#|jjjd|jjd|j dur!|j j dSdSt|tj r8|j j |jj ddSdS)zInitialize the weightsg)meanZstdNg?) rrLinearr-weightdataZnormal_rZinitializer_ranger&Zzero_Z LayerNormZfill_)r6modulerrr _init_weightss   z)EfficientNetPreTrainedModel._init_weightsN) r>r?r@rZ config_classZbase_model_prefixZmain_input_nameZ_no_split_modulesrrrrrrs  rc s^eZdZdeffdd Ze   d deejdee dee de e e ffd d Z ZS) EfficientNetModelrcs~t|||_t||_t||_|jdkr"tj |j dd|_ n|jdkr1tj |j dd|_ nt d|j|dS)NrT)Z ceil_moderz2config.pooling must be one of ['mean', 'max'] got )r)r*rrrsr{encoderZ pooling_typerZ AvgPool2drpoolerZ MaxPool2d ValueErrorZpooling post_initr5r7rrr*s      zEfficientNetModel.__init__Nr9rrr:cCs|dur|n|jj}|dur|n|jj}|durtd||}|j|||d}|d}||}||jdd}|sH||f|ddSt |||j dS)Nz You have to specify pixel_valuesrrrrr)r pooler_outputrT) rruse_return_dictrrsrrZreshapeshaper rT)r6r9rrZembedding_outputZencoder_outputsr pooled_outputrrrr=s*  zEfficientNetModel.forward)NNN)r>r?r@rr*rrrBrVr_rrr r=rDrrr7rrs rz EfficientNet Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. )Z custom_introcsdeZdZfddZe    d deejdeejdee dee de e e ff d d Z ZS) "EfficientNetForImageClassificationcsdt||j|_||_t||_tj|jd|_ |jdkr't |j |jnt |_ |dS)Nrlr)r)r* num_labelsrrrrrqZ dropout_raterrrrZIdentity classifierrr5r7rrr*s  $ z+EfficientNetForImageClassification.__init__Nr9labelsrrr:c Csl|dur|n|jj}|j|||d}|r|jn|d}||}||}d}|dur|jjdurU|jdkr;d|j_n|jdkrQ|jt j ksL|jt j krQd|j_nd|j_|jjdkrst } |jdkrm| | | }n+| ||}n%|jjdkrt} | |d|j|d}n|jjdkrt} | ||}|s|f|dd} |dur|f| S| St|||jd S) a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NrrZ regressionZsingle_label_classificationZmulti_label_classificationr)losslogitsrT)rrrrrrrZ problem_typerZdtyperBlongrrrdrviewrr rT) r6r9rrrZoutputsrrrZloss_fctoutputrrrr=s@     "       z*EfficientNetForImageClassification.forward)NNNN)r>r?r@r*rrrBrVZ LongTensorr_rrr r=rDrrr7rrs$   r)rrr)T)/rAr|typingrrrrBZtorch.utils.checkpointrZtorch.nnrrrZ activationsr Zmodeling_outputsr r r Zmodeling_utilsrutilsrrZconfiguration_efficientnetrZ get_loggerr>loggerrrr_rModulerr-rErLrWr`rirvr{rrr__all__rrrrs@     ''!QZ8 E