fTh|uSSKrSSKrSSKrSSKJr SSKJr SSKJr \R"\ 5r "SS\R5r "SS \R5r"S S \R5r"S S \R5r"SS\R5r"SS\R5rSSjrSSjrSSjr"SS\5r"SS\5r"SS\5rSSjr"SS\5rg) N)nn)Function)loggingcL^\rSrSrSrSU4SjjrSSjrSrU=r$)QuantEmbeddinga Quantized version of `torch.nn.Embedding`. Adds quantization-specific arguments on top of `torch.nn.Embedding`. Args: weight_bit (`int`, *optional*, defaults to `8`): Bitwidth for the quantized weight. momentum (`float`, *optional*, defaults to `0.95`): Momentum for updating the activation quantization range. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. c >[T U]5 XlX lX0lX@lXPlX`lXpl[R"[R"X/55Ul URS[R"S55 URS[R"UR55 XlXlXlSUl[(R*Ulg)Nweight_scaling_factorweight_integerF)super__init__num_dim padding_idxmax_norm norm_typescale_grad_by_freqsparser Parametertorchzerosweightregister_buffer zeros_like weight_bitmomentum quant_modepercentile_modeSymmetricQuantFunctionapplyweight_function) selfnum_embeddings embedding_dimrrrrr_weightrrr __class__s _/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/ibert/quant_modules.pyrQuantEmbedding.__init__,s "  & ""4 ll5;;/N#OP  4ekk!nE -u/?/? /LM$ $$5;;c bUR(dc[RRUURUR UR URURUR5S4$URnURR5nUR5RS5nUR5RS5n[UR XgS5UlUR%URUR UR&UR"5Ul[RRUUR(UR UR URURUR5nXR"-UR"4$)Nr F)rr functional embeddingrrrrrrdatadetachminexpandmax$symmetric_linear_quantization_paramsrr r#r r ) r$x positionsincremental_statew w_transformw_minw_maxemb_ints r)forwardQuantEmbedding.forwardMsO ''KK$$MMNN++KK   KKffmmo !((+!((+%I$//[`in%o""22 KK$*>*>@Z@Z --))       MM NN  # # KK 333T5O5OOOr+)rrrrrrr rrrrrr#r r ) NN@FFNffffff?FNN) __name__ __module__ __qualname____firstlineno____doc__rr=__static_attributes__ __classcell__r(s@r)rrs7   [TU]5 XlX lXPlX0lSUl[RUl UR (dURS[R"S55 URS[R"S55 URS[R"S55 U=RS-sl U=RS- slg[S5e)NFx_minr x_maxact_scaling_factorgh㈵>;per-channel mode is not currently supported for activation.)rractivation_bitact_range_momentumr per_channel percentiler!r" act_functionrrrrOrPNotImplementedError)r$rSrTrU channel_lenrr(s r)rQuantAct.__init__s ,"4$&288  %++a. 9  %++a. 9  !5u{{1~ F JJ$ J JJ$ J%&cd dr+c URRSURSURSURR 5SSUR R 5SS3 $)Nz(activation_bit=z, quant_mode: z , Act_min: z.2fz , Act_max: ))r(rCrSrrOitemrP)r$s r)__repr__QuantAct.__repr__si~~&&''78K8K7LM??+;tzz7H6MN )#.a 1 r+cUcUOX1-nUR(GaUR(aS5eUR(aS5eURR 5nURR 5n U R 5R5S:Xa"UR 5R5S:XdS5eURR 5S:aGURR 5S:a)URU-UlURU -Ul OURS:XaM[R"URU5Ul[R "URU 5Ul ObURUR-USUR- --UlURUR-U SUR- --Ul UR(dUS4$Uc UROUnUc UROUn [URXURS 9UlUc2UR!XRURUR5n O.["R%UUURURUU5n URR'S5n X-UR4$) Nz:percentile mode is not currently supported for activation.rRrz5NaN detected when computing min/max of the activationg&|g&|>r )rU)trainingrVrUr/r1r3isnansumrOrPrTrrr4rSrQrW FixedPointMulr"view) r$r5pre_act_scaling_factoridentityidentity_scaling_factor specified_min specified_maxx_actrOrP quant_act_intcorrect_output_scales r)r=QuantAct.forwardsT%8< === d(d d&'' f)f f'JJNN$EJJNN$E;;=$$&!+ 0A0A0Cq0H G H zz~~')djjnn.>.G!ZZ%/ !ZZ%/ ((B."YYtzz59 "YYtzz59 !ZZ$*A*AAEQQUQhQhMhDii !ZZ$*A*AAEQQUQhQhMhDii $; +3 +3 "F   4;K;K#  " ) --a1D1DdooW[WnWnoM)//&##''' M $66;;B?3T5L5LLLr+) rWrTrQrSrUrVrrPrO)rAFNF)NNNNN rCrDrErFrGrr^r=rHrIrJs@r)rLrLrs0 e&  $ $<M<Mr+rLcH^\rSrSrSrSU4SjjrU4SjrSSjrSrU=r $) QuantLineara  Quantized version of `torch.nn.Linear`. Adds quantization-specific arguments on top of `torch.nn.Linear`. Args: weight_bit (`int`, *optional*, defaults to `8`): Bitwidth for the quantized weight. bias_bit (`int`, *optional*, defaults to `32`): Bitwidth for the quantized bias. per_channel (`bool`, *optional*, defaults to `False`): Whether or not to use channel-wise quantization. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. c>[TU]5 XlX l[R "[ R"X!/55UlURS[ R"UR55 URS[ R"UR55 U(a_[R "[ R"U55Ul URS[ R"UR55 X@l Xpl X`lXPlXpl SUl["R$Ulg)Nr fc_scaling_factor bias_integerF)rr in_features out_featuresrrrrrrrbiasrrrUbias_bitr r!r"r#) r$rwrxryrrzrUrr(s r)rQuantLinear.__init__s &(ll5;; /J#KL  -u/?/? /LM 0%++d>O>O2PQ  U[[%>?DI  1A1A$))1L M$$& $$5;;r+cd>[TU]5nSUSURSURS3nU$)N(z weight_bit=z , quant_mode=r\)rr^rr)r$sr(s r)r^QuantLinear.__repr__s9 G  s,t/}T__UR5RS5nUR5RS5n[URXWUR5UlUR#URURUR$UR 5UlUR U-nUR b-UR#UR UR(SU5UlUR-SS5nX- n [RRXR&UR*S9U-U4$)N)rry)r zInput activation to the QuantLinear layer should be globally (non-channel-wise) quantized. Please add a QuantAct layer with `per_channel = True` before this QuantAct layerr )routFra)rrr-linearrryshaper/r0rUrr1r3r2r4rrur#r r rzrvrf) r$r5prev_act_scaling_factorr8r9r:_r;bias_scaling_factorx_ints r)r=QuantLinear.forwards==''++DII'NPTT T'27N7T7TX\7\  _ \ KKffmmo   yy!>HEyy!>HE1OO%,,Q/EOO%,,Q/E!EdooW\eieueu!v"22 KK$*>*>@V@V #447NN 99 $ 4 4TYY uVi jD "9">">q""E+ MM /B/BIZIZ [^q q   r+) ryrzrvrurwrxrUr rrrr#r )Tr@ FFNrprJs@r)rrrrs# ns<, # # r+rrc@^\rSrSrSrSU4SjjrSrSSjrSrU=r $) IntGELUi,aa Quantized version of `torch.nn.GELU`. Adds quantization-specific arguments on top of `torch.nn.GELU`. Args: quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. force_dequant (`str`, *optional*, defaults to `"none"`): Force dequantize the layer if either "gelu" or "nonlinear" is given. c>>[TU]5 XlUS;a[R S5 SUlUR(d[ R "5UlSUlSUl /SQUl URS==URS-ss'g) N) nonlineargeluzForce dequantize geluFg -?)g]m{ҿgMr r) rrrloggerinforGELU activation_fnkconstcoeff)r$r force_dequantr(s r)rIntGELU.__init__7st $ 1 1 KK/ 0#DO!#D  )  1 A& r+c[R"URSU- 5n[R"URSUS-- 5n[R"U5n[R"[R "U5U*5nXVU-S-U--nUS-URS-n[ RUSUR-- 5nUSUR--nXr4$Nr rr) rfloorrsignr1abs floor_ster"r)r$rscaling_factorb_intc_intrabs_inty_ints r)int_erfIntGELU.int_erfGs DJJqMN:; DJJqMNA,==>zz% ))EIIe,uf55Q.67'*TZZ]:4:: 56'!TZZ-7$$r+cUR(dURU5S4$X- nURX2UR- 5upESU-nX4U--nX%-S- nX2-U4$)N?r)rrrr)r$r5rr sigmoid_intsigmoid_scaling_factor shift_ints r)r=IntGELU.forwardVsw%%a($. .".2ll5SWSYSYBY.Z+ 11 y01'@1D%~55r+)rrrrr)Tnoner) rCrDrErFrGrrr=rHrIrJs@r)rr,s' % 6 6r+rcB^\rSrSrSrSU4SjjrSrSrSrSr U=r $) IntSoftmaxiea Quantized version of `torch.nn.Softmax`. Adds quantization-specific arguments on top of `torch.nn.Softmax`. Args: output_bit (`int`): Bitwidth for the layer output activation. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. force_dequant (`str`, *optional*, defaults to `"none"`): Force dequantize the layer if either "softmax" or "nonlinear" is given. c|>[TU]5 XlSUlX lUS;a[ R S5 SUl[SURS9UlSUl SUl /S QUl URS ==URS -ss'URS ==URS -ss'g) Nr)rsoftmaxzForce dequantize softmaxFrgvq -)gN$?g'|:?rr rr) rr output_bitmax_bitrrrrLactx0rcoef)r$rrrr(s r)rIntSoftmax.__init__rs $ $ 4 4 KK2 3#DOB4??; 1  !  ! $  !  ! $ r+cD[R"5 [R"URSU- 5n[R"URSUS-- 5nSSS5 UW-U-W-nURSUS--nXR4$!,(df  N1=fr)rno_gradrr)r$rrrrzs r)int_polynomialIntSoftmax.int_polynomials ]]_KK ! ~ =>EKK ! ~q/@ @AEU]e #e +1(99  _s AB Bc[R"5 [R"URU- 5nSSS5 [R"XR W-5n[ RX- 5nXU-- nURXR5upg[R"[ RUSUR U- --5SS9nUSUR -- nXb4$!,(df  N=f)Nrrr1) rrrrr3rrr"rclamp)r$rrx0_intqrexp_intexp_scaling_factors r)int_expIntSoftmax.int_exps ]]_[[>!9:F %f!45 OOEN + QJ &*&9&9!&L#++ioogdjj1n8M.MNTUV+am;&&_s $C&& C4cUR(d [RRUSS9S4$X- nUR SSS9upEX4- nUR X25upgUR Xg5upX- nURSSS9n [RSUR-U - 5n [RXj-SURUR- -- 5nSSUR-- nXb-U4$)NrarT)rkeepdimrr ) rrr-rr3rrrdrr"rr) r$r5rr x_int_maxrrrexp exp_int_sumfactors r)r=IntSoftmax.forwards==(((3T9 9"yyRy6 !&*ll5&I##'((7"G*kkb$k7 DLL;!>?//'"2Q4<<$//;Y5Z"Z[Q//'77r+)rrrrrrr)Fr) rCrDrErFrGrrrr=rHrIrJs@r)rres! %"! '88r+rcF^\rSrSrSrSU4SjjrSrSrS SjrSr U=r $) IntLayerNormia Quantized version of `torch.nn.LayerNorm`. Adds quantization-specific arguments on top of `torch.nn.LayerNorm`. Args: output_bit (`int`, *optional*, defaults to `8`): Bitwidth for the layer output activation. quant_mode (`bool`, *optional*, defaults to `False`): Whether or not the layer is quantized. force_dequant (`str`, *optional*, defaults to `"none"`): Force dequantize the layer if either "layernorm" or "nonlinear" is given. c>[TU]5 XlX l[R "[ R"U55Ul[R "[ R"U55Ul X@l US;a[RS5 SUl URS[ R"S55 X0lSUlSUl[#URURS9Ulg)N)r layernormzForce dequantize layernormFshiftr rr)rrnormalized_shapeepsrrrrrryrrrrrrdim_sqrtrL activation)r$rrrrrr(s r)rIntLayerNorm.__init__s  0ll5;;/?#@A LL-=!>? $ 6 6 KK4 5#DO Wekk!n5$  "4??tOr+c [R"5 US-n[R"USSS9n[R"[R"USUR -- 55R 5R5nURn[R"URU5Ul[RS[U5S[UR535 SSS5 g!,(df  g=f)NrTaxisrzDynamic shift adjustment: z -> ) rrrdlog2sqrtrceilr3rrrint)r$ry_sq_intvar_intr shift_olds r) set_shiftIntLayerNorm.set_shifts ]]_axHiiq$?GZZ 7Q _+D EFKKMRRTE I4::u5DJ KK4S^4DDTZZHYZ [ __s CC:: DcURU5 [RUSUR-- 5nUS-n[R "USSS9nU$)z This fallback function is called when overflow is detected during training time, and adjusts the `self.shift` to avoid overflow in the subsequent runs. rTr)rrr"rrrd)r$r y_int_shiftedrrs r)overflow_fallbackIntLayerNorm.overflow_fallbacksL u!4:: (=>  !#))H1d;r+cUR(drURSSS9nX- n[R"US-SSS9nU[R"URU-5- nXR -UR -nUS4$URcd[R"URS[RS9n[R"U5RUR5UlX- n[RURSSS95nXx- n [RU SUR -- 5n U S-n [R""U SSS9n UR$(a]U R'5SUR(-:a<UR+U 5n U R'5SUR(-S-:dS5e[R[R"U 55SUR --n [RSU - 5n[RX-S- 5n URS- nUR R,R/5UR R,R/5- n[RX- 5nU U-n X R -nX-nX4$) NrTr)dtypeg?zfError detected in overflow handling: `var_int` exceeds `self.max_bit` (the maximum possible bit width)li@)rmeanrrrrryrtensorrfloattodevice round_ster"rrrdrbr3rrr/r0)r$r5rryvarnrmean_intrrrrstd_intrrybias_ints r)r=IntLayerNorm.forwards:66q$6/DA**QT48CEJJtxx#~..AKK$))+Ad7N ==  QWWQZu{{;A!JJqM,,QXX6DM"??5::1d:#CD !4:: (=>  !#))H1d; =={{}4<</007{{}q$,,'<<X< //%**W"56DJJF1 23.yy~~$$&$++*:*:*A*A*CD??4#89 '++5  "  r+) rryrrrrrrrr)r@Frr) rCrDrErFrGrrrr=rHrIrJs@r)rrs# P&\ .!.!r+rcRURSn[USUS-- -5n[XB-S-5n[R"XS9RnUS:XaUS-nO![R"U*US9R*nU(d UR 5nUR 5nX4$)a Calculate the percentile max and min values in a given tensor Args: input (`torch.Tensor`): The target tensor to calculate percentile max and min. lower_percentile (`float`): If 0.1, means we return the value of the smallest 0.1% value in the tensor as percentile min. upper_percentile (`float`): If 99.9, means we return the value of the largest 0.1% value in the tensor as percentile max. output_tensor (`bool`, *optional*, defaults to `False`): If True, this function returns tensors, otherwise it returns values. Returns: `Tuple(torch.Tensor, torch.Tensor)`: Percentile min and max value of *input* rr g{Gz?)r)rroundrkthvaluevaluesr]) inputlower_percentileupper_percentile output_tensor input_length lower_index upper_index upper_bound lower_bounds r)get_percentile_min_maxr s";;q>L ,?K..6==K1!Ao ~~uf <CCC !&&( !&&(  ##r+c[UR5S:Xa)URSSSS5nURSSSS5nO`[UR5S:Xa%URSS5nURSS5nO"URS5nURS5nU(a3URSU- 5R U5R 5 U$[ R"SU- U-U-5$)a  Quantize single-precision input tensor to integers with the given scaling factor and zeropoint. Args: input (`torch.Tensor`): Single-precision input tensor to be quantized. scale (`torch.Tensor`): Scaling factor for quantization. zero_pint (`torch.Tensor`): Shift for quantization. inplace (`bool`, *optional*, defaults to `False`): Whether to compute inplace or not. Returns: `torch.Tensor`: Linearly quantized value of *input* according to *scale* and *zero_point*. rar rr)lenrrfmul_add_round_rr)rscale zero_pointinplaces r)linear_quantizer5s$ 5;;1 2q!Q'__RAq1 U[[ Q  2q!__R+  2__R(  3;$$Z0779 ;;sU{U*Z7 88r+c[R"5 SUS- -S- nU(aa[R"[R"UR 5UR 5/SS9SS9upV[R "USS9U- nO@[UR 5UR 55n[R "USS9U- nSSS5 U$!,(df  W$=f)a Compute the scaling factor with the given quantization range for symmetric quantization. Args: saturation_min (`torch.Tensor`): Lower bound for quantization range. saturation_max (`torch.Tensor`): Upper bound for quantization range. per_channel (`bool`, *optional*, defaults to `False`): Whether to or not use channel-wise quantization. Returns: `torch.Tensor`: Scaling factor that linearly quantizes the given range between *saturation_min* and *saturation_max*. rr rg:0yE>rN)rrr3stackrr)num_bitssaturation_minsaturation_maxrUrrrs r)r4r4Xs$  (Q, ! # yyn.@.@.BNDVDVDX-Y_`!aghiHEKK4014E**,n.@.@.BCEKK4014E  L  Ls B4C C#c8\rSrSrSr\S5r\S5rSrg)r!ixzo Class to quantize the given floating-point values using symmetric quantization with given range and bitwidth. c[R"SURS9nSUS- -S- n[XUSS9n[R"Xv*US- 5nX@lU$)a Args: x (`torch.Tensor`): Floating point tensor to be quantized. k (`int`): Quantization bitwidth. percentile_mode (`bool`): Whether or not to use percentile calibration. scale (`torch.Tensor`): Pre-calculated scaling factor for *x*. Note that the current implementation of SymmetricQuantFunction requires pre-calculated scaling factor. Returns: `torch.Tensor`: Symmetric-quantized value of *input*. g)rrr F)r)rrrrrr)ctxr5rr rrr new_quant_xs r)r=SymmetricQuantFunction.forward}sY"\\#ell; !a%L1 %a EJ kk+r1q59  r+c URn[UR5S:XaURSSSS5nO=[UR5S:XaURSS5nOURS5nUR 5U- SSSS4$)Nr rar r)rr rrfclone)r grad_outputrs r)backwardSymmetricQuantFunction.backwards  {  !Q &JJr1a+E "" #q (JJr1%EJJrNE  "U*D$dBBr+N rCrDrErFrG staticmethodr=r!rHr#r+r)r!r!xs12 C Cr+r!c8\rSrSrSr\S5r\S5rSrg)riz3 Straight-through Estimator(STE) for torch.floor() c.[R"U5$r)rrrr5s r)r=floor_ste.forward{{1~r+c"UR5$rrrr s r)r!floor_ste.backward  ""r+r#Nr$r#r+r)rr/##r+rc8\rSrSrSr\S5r\S5rSrg)riz3 Straight-through Estimator(STE) for torch.round() c.[R"U5$r)rrr(s r)r=round_ste.forwardr*r+c"UR5$rr,r-s r)r!round_ste.backwardr/r+r#Nr$r#r+r)rrr0r+rcUR5nURS5n[R"UR 5R 55up4/nUHin[ [R"USU--5R[R"S5[RS95nURU5 Mk [R"U5n[U5U- n[R"U5R!UR"5RU5[R"U5R!UR"5RU54$)z Decompose the scaling factor into mantissa and twos exponent. Args: scaling_factor (`torch.Tensor`): Target scaling factor to decompose. Returns: ``Tuple(torch.Tensor, torch.Tensor)`: mantisa and exponent rar1)rounding)sizerfnpfrexpcpunumpyrdecimalDecimalquantize ROUND_HALF_UPappendarrayrr from_numpyrr)inputsrshape_of_inputoutput_moutput_etmp_mm int_m_shifteds r) batch_frexprLs[[]N[[_F&**,"4"4"67H E  OOAG, - 6 6ws7KV]VkVk 6 l   ]#  xxHW~(H "%%fmm499.I "%%fmm499.I r+c@\rSrSrSr\SSj5r\S5rSrg)reia  Function to perform fixed-point arithmetic that can match integer arithmetic on hardware. Args: pre_act (`torch.Tensor`): Input tensor. pre_act_scaling_factor (`torch.Tensor`): Scaling factor of the input tensor *pre_act*. bit_num (`int`): Quantization bitwidth. z_scaling_factor (`torch.Tensor`): Scaling factor of the output tensor. identity (`torch.Tensor`, *optional*): Identity tensor, if exists. identity_scaling_factor (`torch.Tensor`, *optional*): Scaling factor of the identity tensor *identity*, if exists. Returns: `torch.Tensor`: Output tensor(*pre_act* if *identity* is not given, otherwise the addition of *pre_act* and *identity*), whose scale is rescaled to *z_scaling_factor*. Nc[UR5S:XaSnOSnXPlSUS- -S- n[R"5 U"U5nUbU"U5nX@l[R "X- 5n UR[R5n UR[R5R[R5n X- n U"U 5n [U 5upU R[R5U R[R5-n[R "USU-- 5nUb[R "XV- 5nUR[R5n UR[R5R[R5n X- n U"U 5n [U 5unnUR[R5UR[R5-n[R "USU-- 5nUU-n[R"UR[R5U*S- U5sSSS5 $!,(df  g=f)NrcU$rr#r5s r)'FixedPointMul.forward..sr+c(URSSS5$)Nr ra)rfrPs r)rQrRsq!R 0r+rr r?) r rrhrrz_scaling_factorrtypedoublerrLr)rpre_actrgbit_numrTrhrireshaperz_int_A_B new_scalerJeoutputwx_intm1e1output1s r)r=FixedPointMul.forwards %++ , 1!G0G 'A+  " ]]_%,-C%D "#*12I*J'#3 KK @AE',,U\\:B"'' 4::5<<HBI *Iy)DAZZ -u||0DDF[[36!23F#X%GH,11%,,?&++EKK8>>u||LG #I. $Y/B ++ell3bggell6KK++gb&9: 6);;v{{5;;7!aCC__s H)I:: JcSnURbUR5UR- nUR5UR- SSSSUS4$r)rhrrT)rr  identity_grads r)r!FixedPointMul.backward/sU << #'--/#2F2FFM  "S%9%994tTS`bfffr+r#rBr$r#r+r)reres<, $2D2Dhggr+re)F)r )r>r=r:rrtorch.autogradrutilsr get_loggerrCrModulerrLrrrrrr rr4r!rrrLrer#r+r)rls$ #   H %PPRYYPPfgMryygMTM "))M `66bii66rD8D8Nb!299b!J!$H 9F@*CX*CZ # # # #DQgHQgr+