[Th'%SrSSKrSSKrSSKJrJr SSKJr SSKJ r SSK J r J r SSK r SSKJr SSKJrJrJrJrJr /SQr"S S 5rS rS r\"S \405r\ \S '"SS\\R:5r"SS\5r\r "SS\5r!"SS\!5r""SS\5r#"SS\#5r$"SS\5r%"SS\5r&"SS\5r'"S S!\5r("S"S#\5r)"S$S%\5r*SS&K+J,r, SS'K-J.r.J/r/ "S(S)\/5r0"S*S+\/5r1"S,S-\/5r2\,"S.S/9"S0S155r3\,"S.S/9"S2S3\355r4\,"S.S/9"S4S5\355r5\,"S.S/9"S6S7\355r6\,"S.S/9"S8S9\355r7"S:S;\35r8"S<S=\35r9S>\:\;S?4S@\3SA\:\;S?44SBjr<"SCSD\\ RR:5r=SEr>SFr?SGr@SHrASIrB\!RSSJSK9rD\'rE\(rF\!R\ R\ RSL9rI\!R\ R\ RSMSJSNSO9rJ\%RSSJSK9rK\#R\ R\ RSL9rM\#R\ R\ RSMSJSNSO9rN\'R\ RSSPS.SQ9rP\#R\ R\ RSSR9rR\#R\ R\ RSSR9rT\&RSSST\ RSSPSU9rU\&RSVS\ RSSPSU9rV\UrW\VrX\*rYg)Wz This module implements observers which are used to collect statistics about the values observed during calibration (PTQ) or training (QAT). N)ABCMetaabstractmethod) OrderedDict)partial)AnyOptional)calculate_qmin_qmaxcheck_min_max_validis_per_channel is_per_tensorvalidate_qmin_qmax)*%default_affine_fixed_qparams_observerdefault_debug_observerdefault_dynamic_quant_observer)default_fixed_qparams_range_0to1_observer,default_fixed_qparams_range_neg1to1_observerdefault_float_qparams_observer#default_float_qparams_observer_4bitdefault_histogram_observerdefault_observer#default_per_channel_weight_observerdefault_placeholder_observerdefault_reuse_input_observer(default_symmetric_fixed_qparams_observerdefault_weight_observerget_observer_state_dictload_observer_state_dict0per_channel_weight_observer_range_neg_127_to_127$weight_observer_range_neg_127_to_127FixedQParamsObserverHistogramObserverMinMaxObserverMovingAverageMinMaxObserver%MovingAveragePerChannelMinMaxObserver NoopObserver ObserverBasePerChannelMinMaxObserverPlaceholderObserverRecordingObserverReuseInputObserverUniformQuantizationObserverBaseAffineQuantizedObserverBase Granularity MappingTypePerAxisPerBlockPerGroupPerRow PerTensorPerToken TorchAODTypeZeroPointDomainget_block_sizec2\rSrSrSrSrSrSrSrSr g) _PartialWrapperKcXl0UlgN)p callable_args)selfr=s V/var/www/auris/envauris/lib/python3.13/site-packages/torch/ao/quantization/observer.py__init___PartialWrapper.__init__LscURH#nX2;dM 0UEX0RU"50EnM% UR"U0UD6$r<r>r=)r?argskeywordsarg_names r@__call___PartialWrapper.__call__PsM**H'QhQ2D2DX2N2PQ+vvt(x((rCclURR5URR5-$r<)r=__repr__r>r?s r@rL_PartialWrapper.__repr__Xs'vv 4#5#5#>#>#@@@rCc [U40UD6$r<) _with_argsr?kwargss r@ with_args_PartialWrapper.with_args[s$)&))rCc V[URS9n0UREUEUlU$)N)r=)r9r=r>)r?rRresults r@with_callable_args"_PartialWrapper.with_callable_args^s- 466*?$"4"4?? rCrEN) __name__ __module__ __qualname____firstlineno__rArIrLrSrW__static_attributes__rCr@r9r9Ks )A*rCr9c 0[[U40UD65nU$)a Wrapper that allows creation of class factories. This can be useful when there is a need to create classes with the same constructor arguments, but different instances. Can be used in conjunction with _callable_args Example:: >>> # xdoctest: +SKIP("Undefined vars") >>> Foo.with_args = classmethod(_with_args) >>> foo_builder = Foo.with_args(a=3, b=4).with_args(answer=42) >>> foo_instance1 = foo_builder() >>> foo_instance2 = foo_builder() >>> id(foo_instance1) == id(foo_instance2) False )r9r cls_or_selfrRrs r@rPrPds"  6v67A HrCc N[[U55nUR"S0UD6$)aWrapper that allows creation of class factories args that need to be called at construction time. This can be useful when there is a need to create classes with the same constructor arguments, but different instances and those arguments should only be calculated at construction time. Can be used in conjunction with _with_args Example:: >>> # xdoctest: +SKIP("Undefined vars") >>> Foo.with_callable_args = classmethod(_with_callable_args) >>> Foo.with_args = classmethod(_with_args) >>> foo_builder = Foo.with_callable_args(cur_time=get_time_func).with_args(name="dan") >>> foo_instance1 = foo_builder() >>> # wait 50 >>> foo_instance2 = foo_builder() >>> id(foo_instance1.creation_time) == id(foo_instance2.creation_time) False r^)r9rrWr`s r@_with_callable_argsrdys'(  ,-A   )& ))rCABCcx^\rSrSrSrSS\4U4Sjjjr\S5r\S5r \ "\ 5r \ "\ 5rSrU=r$) r&a/Base observer Module. Any observer implementation should derive from this class. Concrete observers should follow the same API. In forward, they will update the statistics of the observed Tensor. And they should provide a `calculate_qparams` function that computes the quantization parameters given the collected statistics. Args: dtype: dtype argument to the `quantize` node needed to implement the reference model spec. is_dynamic: indicator for whether the observer is a placeholder for dynamic quantization or static quantization is_dynamicc:>[TU]5 XlX lgr<)superrAdtyperh)r?rkrh __class__s r@rAObserverBase.__init__s  $rCcgr<r^r?xs r@forwardObserverBase.forward rCc gr<r^rQs r@calculate_qparamsObserverBase.calculate_qparamsrsrCrkrh)F)rYrZr[r\__doc__boolrArrqru classmethodrPrSrdrWr] __classcell__rls@r@r&r&sY %$%%     J'I$%89rCr&c ^\rSrSr%SrSr\R\S'\R\RSSSS\R"\R5RS4SU4SjjjrU4S jr\R R"S \S \SS4S j5r\R R"S \RS\RS\\R\R44Sj5r\R R"S5rSrU=r$)r+aCommon base for all observers using uniform quantization to calculate scale and zero_point. Args: dtype: dtype argument to the `quantize` node needed to implement the reference model spec. qscheme: Quantization scheme to be used. reduce_range: Reduces the range of the quantized data type by 1 bit. This is sometimes required to avoid instruction overflow. quant_min: Minimum quantization value. If unspecified, it will follow the 8-bit setup. quant_max: Maximum quantization value. If unspecified, it will follow the 8-bit setup. eps: Epsilon value for float32, Defaults to `torch.finfo(torch.float32).eps`. .. warning:: :attr:`dtype` can only take ``torch.qint8`` or ``torch.quint8``. or `torch.int8` or `torch.uint8` .. warning:: :attr:`qscheme` can only take one of the following options: - ``torch.per_tensor_affine`` - ``torch.per_tensor_symmetric`` - ``torch.per_channel_affine`` - ``torch.per_channel_symmetric`` epsFNreturnc &>[RRU5n[T U]"SXS.U D6 X lU(a[ R"S5 X0lURS[R"U/40UD65 UR [R[R[R[R[R4;dS5e[R [R"[R$[R&[R([R*[R,[R.[R0[R2[R44 n UR6U ;d SU S35eUSL=(a USLUlUR8(a [;XE5 [=UUUR8UR6UR5uUlUl g)NrwzPlease use quant_min and quant_max to specify the range for observers. reduce_range will be deprecated in a future release of PyTorch.rzDefault Observer only works for per_tensor_affine, per_tensor_symmetric, per_channel_affine, per_channel_symmetric and per_channel_float_qparams quantization schemez Default Observer only works for z data typer^)!torchnnfactory_kwargsrjrAqschemewarningswarn reduce_rangeregister_buffertensorper_tensor_affineper_tensor_symmetricper_channel_affineper_channel_symmetric per_channel_affine_float_qparamsqint8quint8quint4x2qint32int8uint8int16int32 float8_e5m2 float8_e4m3fnuint16rkhas_customized_qranger r quant_min quant_max) r?rkrrrrrrrhrR_ALLOWED_DTYPESrls r@rA(UniformQuantizationObserverBase.__init__s00@ FuFvF  MMU ) UELL#$I.$IJ||  # #  & &  $ $  ' '  2 2    Y  Y  Y  KK LL NN LL JJ KK KK KK       LL   JJ/ ) J -o->j I J )&/t&;%X)SWBW"  % % y 4)<    & & JJ    * &rCc >URSS5nUbUS:XaJ[R"[R"[R5R /5n XUS-'[ T U]UUUUUUU5 g)Nversionr)getrrfinfofloat32rrj_load_from_state_dict) r? state_dictprefixlocal_metadatastrict missing_keysunexpected_keys error_msgsrrrls r@r5UniformQuantizationObserverBase._load_from_state_dict st!$$Y5 ?gl,, EMM : > >?@C),v~ & %        rCrrcVUSs=::a U::dS5e S5eX:dS5eg)a=Validates that the user-specified quantization range is properly initialized and within the given bound supported by the observer dtype. To accommodate lower-bit quantization with respect to the existing torch.qint8 and torch.quint8 datatypes, the user can choose to use dynamic quantization range by passing in a tuple of initial qmin and qmax values. One use case is these customized qmin and qmax values are used to calculate static estimates of the scale and zero point for aggressive lower-bit fake quantization. These estimates are compared against parameters learned through backpropagation. The related literatures for scale and zero point via backpropagation are as follows: Learned Step Size Quantization: https://openreview.net/pdf?id=rkgO66VKDS Trained Quantization Thresholds: https://arxiv.org/pdf/1903.08066.pdf rz1Used-specified quantization range must include 0.zKqmin must be strictly less than qmax for user-specified quantization range.Nr^)r?rrs r@_validate_qmin_qmax3UniformQuantizationObserverBase._validate_qmin_qmax;sK$  'i ' ? > ? ' ? > ? '  ! Y X Y !rCmin_valmax_valc[X5(dT[R"S/URRS9[R"S/URRS94$UR UR pC[R"U[R"U55n[R"U[R"U55nURn[R"UR5[RUS9n[R"UR5[RUS9n UR[R :XdUR[R":XGa[R"U*U5nU[%XC- 5S- - n[R"XR&5nUR([R*[R,4;aZUR.(a'U R1U R5X4-S-5n GOTU R1U R5S5n GO2UR([R24;a U R1U R5S5n OUR[R4:XaSX!- [%XC- 5- n[R6"XR&:U[R8"U55nSU-U- n OXe- [%XC- 5- n[R"XR&5nU[R:"XX- 5R=[R>5- n [R@"XU5n [CURD5S:Xa*[R"[%U5/UR(US9n[CU RD5S:Xar[R"[?U 5/U R(US9n UR[R4:Xa*[R"[%U 5/U R(US9n X4$) aDCalculates the quantization parameters, given min and max value tensors. Works for both per tensor and per channel cases Args: min_val: Minimum values per channel max_val: Maximum values per channel Returns: scales: Scales tensor of shape (#channels,) zero_points: Zero points tensor of shape (#channels,) ?devicer)rkri)#r rrrtyperrmin zeros_likemaxonessizerzerosint64rrrfloatrrkrrrnew_fullrrwhere ones_likeroundtointclamplenshape) r?rrrr min_val_neg max_val_posrscale zero_points r@_calculate_qparams2UniformQuantizationObserverBase._calculate_qparamsSs)(#744<<gnn.A.ABELLGNN//E  $~~t~~9ii)9)9')BC ii)9)9')BC ## ;++-U]]6R[[!1!1!35;;vV  LLE66 6||u:::))[L+>K5)>#?!#CDEIIeXX.EzzellEKK88--!+!4!4")I,Aa+G"J",!4!4Z__5F!LJ ~-'001BEJ \\UCC C&% 0E*FFEKK 0%9OPE g-J .% 8M2NNEIIeXX.E"U[[1D%E%H%H%SSJZIFJ u{{ q LL%,u{{6RE z A %Z!)9)9&J||uEEE"\\:&'z/?/?   rCc[S5e)Nz2Cannot reset min/max values in the given observer.)NotImplementedErrorrMs r@reset_min_max_vals2UniformQuantizationObserverBase.reset_min_max_valss!"VWWrC)rrrrrrN)rYrZr[r\rx_versionrTensor__annotations__rrrrrrArjitexportrrtuplerrr]r{r|s@r@r+r+sTH ll'' KK & * *:  : : x 6 YYYSYSYTYY. YYN!||N!.3llN! u||U\\) *N!N!` YYXXrCr+c ^\rSrSr%Sr\R \S'\R \S'\R\RSSSS\R"\R5RS4S U4Sjjjr Sr\RR S 5r\RR S 5r\RR S 5rS rU=r$)r"iaK Observer module for computing the quantization parameters based on the running min and max values. This observer uses the tensor min/max statistics to compute the quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters. Args: dtype: dtype argument to the `quantize` node needed to implement the reference model spec. qscheme: Quantization scheme to be used reduce_range: Reduces the range of the quantized data type by 1 bit quant_min: Minimum quantization value. If unspecified, it will follow the 8-bit setup. quant_max: Maximum quantization value. If unspecified, it will follow the 8-bit setup. eps: Epsilon value for float32, Defaults to `torch.finfo(torch.float32).eps`. Given running min/max as :math:`x_\text{min}` and :math:`x_\text{max}`, scale :math:`s` and zero point :math:`z` are computed as: The running minimum/maximum :math:`x_\text{min/max}` is computed as: .. math:: \begin{array}{ll} x_\text{min} &= \begin{cases} \min(X) & \text{if~}x_\text{min} = \text{None} \\ \min\left(x_\text{min}, \min(X)\right) & \text{otherwise} \end{cases}\\ x_\text{max} &= \begin{cases} \max(X) & \text{if~}x_\text{max} = \text{None} \\ \max\left(x_\text{max}, \max(X)\right) & \text{otherwise} \end{cases}\\ \end{array} where :math:`X` is the observed tensor. The scale :math:`s` and zero point :math:`z` are then computed as: .. math:: \begin{aligned} \text{if Symmetric:}&\\ &s = 2 \max(|x_\text{min}|, x_\text{max}) / \left( Q_\text{max} - Q_\text{min} \right) \\ &z = \begin{cases} 0 & \text{if dtype is qint8} \\ 128 & \text{otherwise} \end{cases}\\ \text{Otherwise:}&\\ &s = \left( x_\text{max} - x_\text{min} \right ) / \left( Q_\text{max} - Q_\text{min} \right ) \\ &z = Q_\text{min} - \text{round}(x_\text{min} / s) \end{aligned} where :math:`Q_\text{min}` and :math:`Q_\text{max}` are the minimum and maximum of the quantized data type. .. warning:: :attr:`dtype` can only take ``torch.qint8`` or ``torch.quint8``. .. note:: If the running minimum equals to the running maximum, the scale and zero_point are set to 1.0 and 0. rrFNc >[U5(d [S5e[T U] "SUUUUUUUUS.U D6 [R R U5nURS[R"[S540UD65 URS[R"[S540UD65 UR[R:Xa<UR(a*UR[R:Xa [S5eggg) NzqMinMaxObserver's qscheme only support torch.per_tensor_symmetric and torch.per_tensor_affine.rkrrrrrrrhrinfr-infz`Cannot reduce range for symmetric quantization for quint8r^)r rrjrArrrrrrrrrrkr) r?rkrrrrrrrhrRrls r@rAMinMaxObserver.__init__sW%%%2   %)!  00@ Y U5\(T^(TU Y U6](Un(UV LLE66 6!! ell*%@ +" 7rCcUR5S:XaU$UR5nURURR5n[ R "U5up4[ R"X0R5n[ R"X@R5nURRU5 URRU5 U$)z1Records the running minimum and maximum of ``x``.r) numeldetachrrrkraminmaxrrrcopy_)r?x_origrp min_val_cur max_val_currrs r@rqMinMaxObserver.forward$s <<>Q M MMO DD## $#(==#3 ))K6))K6 7# 7# rCcNURURUR5$)z'Calculates the quantization parameters.rrrrMs r@ru MinMaxObserver.calculate_qparams1s&&t||T\\BBrCc:SURSUR3$Nzmin_val=z , max_val=rrrMs r@ extra_reprMinMaxObserver.extra_repr6s$,,z$,,@@rCcURR[R"[ S555 UR R[R"[ S555 g)Resets the min/max values.rrN)rrrrrrrMs r@r!MinMaxObserver.reset_min_max_vals:sB 5<<e 56 5<<f 67rCr^r)rYrZr[r\rxrrrrrrrrrArqrrrurrr]r{r|s@r@r"r"s=|\\ \\ll'' KK & * *1 11f  YYCC YYAA YY88rCr"c ^\rSrSrSrS\R \RSSS\R"\R5RS4S U4Sjjjr Sr Sr U=r$) r#iAa[Observer module for computing the quantization parameters based on the moving average of the min and max values. This observer computes the quantization parameters based on the moving averages of minimums and maximums of the incoming tensors. The module records the average minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters. Args: averaging_constant: Averaging constant for min/max. dtype: dtype argument to the `quantize` node needed to implement the reference model spec. qscheme: Quantization scheme to be used reduce_range: Reduces the range of the quantized data type by 1 bit quant_min: Minimum quantization value. If unspecified, it will follow the 8-bit setup. quant_max: Maximum quantization value. If unspecified, it will follow the 8-bit setup. eps: Epsilon value for float32, Defaults to `torch.finfo(torch.float32).eps`. The moving average min/max is computed as follows .. math:: \begin{array}{ll} x_\text{min} = \begin{cases} \min(X) & \text{if~}x_\text{min} = \text{None} \\ (1 - c) x_\text{min} + c \min(X) & \text{otherwise} \end{cases}\\ x_\text{max} = \begin{cases} \max(X) & \text{if~}x_\text{max} = \text{None} \\ (1 - c) x_\text{max} + c \max(X) & \text{otherwise} \end{cases}\\ \end{array} where :math:`x_\text{min/max}` is the running average min/max, :math:`X` is is the incoming tensor, and :math:`c` is the ``averaging_constant``. The scale and zero point are then computed as in :class:`~torch.ao.quantization.observer.MinMaxObserver`. .. note:: Only works with ``torch.per_tensor_affine`` quantization scheme. .. note:: If the running minimum equals to the running maximum, the scale and zero_point are set to 1.0 and 0. {Gz?FNc >[U5(d[SU35eXlU(a(URS:wa[SUR35e[T U]"SUUUUUUUS.U D6 g)NzMovingAverageMinMaxObserver's qscheme only support torch.per_tensor_symmetric and torch.per_tensor_affine. but got: rz[MovingAverageMinMaxObserver doesn't support dynamic quantization for averaging constant of )rkrrrrrrhr^)r raveraging_constantrjrA) r?rrkrrrrrrhrRrls r@rA$MovingAverageMinMaxObserver.__init__osW%%%!$  #5 $11Q6%))-)@)@(AC   %!  rCc"UR5S:XaU$UR5nURURR5nURnUR nU[ S5:Xa(U[ S5:Xa[R"U5up4O>[R"U5upVX0RXS- --nX@RXd- --nURRU5 UR RU5 U$)Nrrr) rrrrrkrrrrrr)r?rrprrrrs r@rq#MovingAverageMinMaxObserver.forwards <<>Q M MMO DD## $,,,, eEl "w%-'?$}}Q/ GW',}}Q'7 $K 7 7;;P QQG 7 7;;P QQG 7# 7# rCrr)rYrZr[r\rxrrrrrrrArqr]r{r|s@r@r#r#As_+^ ll'' KK & * *!  ! ! FrCr#c>^\rSrSr%Sr\R \S'\R \S'S\R\RSSSS\R"\R5RS4 SU4Sjjjr S rS r\R R"S 5rS rS \\\4S\S\\\R 4S\S\\S\\S\\4U4SjjrS \\\4S\S\\\R 4S\S\\S\\S\\4Sjr\R R"S5rSrU=r$)r'iaObserver module for computing the quantization parameters based on the running per channel min and max values. This observer uses the tensor min/max statistics to compute the per channel quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters. Args: ch_axis: Channel axis dtype: dtype argument to the `quantize` node needed to implement the reference model spec. qscheme: Quantization scheme to be used reduce_range: Reduces the range of the quantized data type by 1 bit quant_min: Minimum quantization value. If unspecified, it will follow the 8-bit setup. quant_max: Maximum quantization value. If unspecified, it will follow the 8-bit setup. eps: Epsilon value for float32, Defaults to `torch.finfo(torch.float32).eps`. The quantization parameters are computed the same way as in :class:`~torch.ao.quantization.observer.MinMaxObserver`, with the difference that the running min/max values are stored per channel. Scales and zero points are thus computed per channel as well. .. note:: If the running minimum equals to the running maximum, the scales and zero_points are set to 1.0 and 0. rrrFNc ,>[U5(d [S5eU (a [S5e[T U] "SUUUUUUUU S.U D6 [R R U5nXlURS[R"/40UD65 URS[R"/40UD65 UR[R:Xa<UR(a*UR[R:Xa [S5eggg)NzPerChannelMinMaxObserver's qscheme only support torch.per_channel_symmetric, torch.per_channel_affine and torch.per_channel_affine_float_qparams.z=PerChannelMinMaxObserver doesn't support dynamic quantizationrrrz9Cannot reduce range for symmetric quantization for quint8r^)r rrjrArrrch_axisrrrrrrkr) r?rrkrrrrrrrhrRrls r@rA!PerChannelMinMaxObserver.__init__sg&&%w  %O   %)!  00@  Y R(J>(JK Y R(J>(JK LLE77 7!! ell*%K +" 8rCc$URU5$r<)_forward)r?rs r@rq PerChannelMinMaxObserver.forwards}}V$$rCcUR5S:XaU$UR5nURnURnUR 5n[ [ U55Vs/sHofPM nnSXpR'URUS'URU5nURURR5n[R"USS9nUR5S:XdUR5S:Xa[R"USS9up4OC[R"USS9up[R"X5n[R"X5nURR!UR"5 URR!UR"5 URR%U5 URR%U5 U$s snfNrr) start_dimdim)rrrrrrangerrpermuterrkrflattenrrrresize_rr r?rrprrx_dimi new_axis_listyrrs r@r!PerChannelMinMaxObserver._forwardsg <<>Q M MMO,,,,$)#e*$56$5q$5 6&' ll#<< a IIm $ DD## $ MM!q ) ==?a 7==?a#7$}}QA6 GW',}}QA'> $Kii 5Gii 5G W]]+ W]]+ 7# 7# %7s% GcNURURUR5$r<rrMs r@ru*PerChannelMinMaxObserver.calculate_qparamss&&t||T\\BBrCc:SURSUR3$rrrMs r@r#PerChannelMinMaxObserver.extra_repr$,,z$,,@@rCrrrrrrrc  >URSS5nUbUS:a SS/n Sn Sn OSS/n Sn Sn U GH!n X,-n X;aXnX:Xa&URRUR5 ODX:Xa&URRUR5 O[ R "SU 35 [RR5(a`X:XaURRU5 MX:XaURRU5 M[ R "SU 35 GMGMU(dGMURU 5 GM$ [RR5(d[TU]5UUUSUUU5 gg) Nrrmin_valsmax_valsrrz2Observer load_from_state_dict got unexpected name F)rrrrrrrrr is_scriptingrappendrjr)r?rrrrrrrr local_stateexpected_min_nameexpected_max_namenamekeyvalrls r@r.PerChannelMinMaxObserver._load_from_state_dictsp!$$Y5  7Q;%z2K *  * $i0K )  ) D-C  o ,LL((3.LL((3MMLTFS 99))++0 **3/2 **3/ PQUPVW ,##C(9 <yy%%'' G ) (rCc 2URUUUUUUU5 gr<)r)r?rrrrrrrs r@_load_from_state_dict_script5PerChannelMinMaxObserver._load_from_state_dict_scriptRs' ""        rCcp[R"S5Ul[R"S5Ulg)rrN)rrandrrrMs r@r+PerChannelMinMaxObserver.reset_min_max_valsfs, zz   zz   rC)rrrr)rYrZr[r\rxrrrrrrrrrArqrrrrurdictstrrrylistrr)rr]r{r|s@r@r'r's4\\ \\ll(( KK & * *, ,,\%8 YYCCA:cN::S%,,./ :  : 3i :c:I:x cN  S%,,./    3i  c I ( YY    rCr'c ^\rSrSrSrSS\R \RSSS\R"\R5RS4 S U4Sjjjr Sr S r U=r$) r$itaObserver module for computing the quantization parameters based on the running per channel min and max values. This observer uses the tensor min/max statistics to compute the per channel quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters. Args: averaging_constant: Averaging constant for min/max. ch_axis: Channel axis dtype: Quantized data type qscheme: Quantization scheme to be used reduce_range: Reduces the range of the quantized data type by 1 bit quant_min: Minimum quantization value. If unspecified, it will follow the 8-bit setup. quant_max: Maximum quantization value. If unspecified, it will follow the 8-bit setup. eps: Epsilon value for float32, Defaults to `torch.finfo(torch.float32).eps`. The quantization parameters are computed the same way as in :class:`~torch.ao.quantization.observer.MovingAverageMinMaxObserver`, with the difference that the running min/max values are stored per channel. Scales and zero points are thus computed per channel as well. .. note:: If the running minimum equals to the running maximum, the scales and zero_points are set to 1.0 and 0. rrFNc >[U5(d [S5eU (a [S5e[T U] "SUUUUUUUU S.U D6 Xlg)NzMovingAveragePerChannelMinMaxObserver's qscheme only support torch.per_channel_symmetric, torch.per_channel_affine and torch.per_channel_affine_float_qparams.zJMovingAveragePerChannelMinMaxObserver doesn't support dynamic quantization)rrkrrrrrrhr^)r rrjrAr) r?rrrkrrrrrrhrRrls r@rA.MovingAveragePerChannelMinMaxObserver.__init__svg&&%w  %\   %!  #5rCcUR5S:XaU$UR5nURURR5nURnUR nUR 5n[[U55Vs/sHofPM nnSXpR'URUS'URU5n[R"USS9nUR5S:XdUR5S:Xa[R"USS9up4O=[R"USS9upX0RX- --nX@RX- --nURRUR 5 UR RUR 5 URR#U5 UR R#U5 U$s snfr )rrrrrkrrr rrrrrrrrrrrs r@rq-MovingAveragePerChannelMinMaxObserver.forwardsu <<>Q M MMO DD## $,,,,$)#e*$56$5q$5 6&' ll#<< a IIm $ MM!q ) ==?a 7==?a#7$}}QA6 GW',}}QA'> $K 7 7;;P QQG 7 7;;P QQG W]]+ W]]+ 7# 7# 7s Grr)rYrZr[r\rxrrrrrrrArqr]r{r|s@r@r$r$tsa: ll(( KK & * *!5 !5!5FrCr$c^\rSrSr%Sr\R \S'\R \S'\R \S'S\R\RSSSS\R"\R5RS4 S \ S \RS S4U4S jjjrS \R S\R S\R S \R 4SjrS\ S\ 4SjrS \\R \R 44SjrS\R S\R S\R S\R S\R 4 SjrS\R S\R S\R S\R S\R S\R S \R 4SjrS\R S\R S\R S S4SjrS\R S \R 4S jr\R0R2S!5rU4S"jrU4S#jrS$rS%rU=r$)&r!ia The module records the running histogram of tensor values along with min/max values. ``calculate_qparams`` will calculate scale and zero_point. Args: bins: Number of bins to use for the histogram dtype: dtype argument to the `quantize` node needed to implement the reference model spec qscheme: Quantization scheme to be used reduce_range: Reduces the range of the quantized data type by 1 bit eps: Epsilon value for float32, Defaults to `torch.finfo(torch.float32).eps`. The scale and zero point are computed as follows: 1. Create the histogram of the incoming inputs. The histogram is computed continuously, and the ranges per bin change with every new tensor observed. 2. Search the distribution in the histogram for optimal min/max values. The search for the min/max values ensures the minimization of the quantization error with respect to the floating point model. 3. Compute the scale and zero point the same way as in the :class:`~torch.ao.quantization.MinMaxObserver` histogramrriFNbinsrkrc p>[U5(d [S5eU (a [S5e[T U] "S UUUUUUUU S.U D6 [R R U5nXlURS[R"UR40UD65 URS[R"[S540UD65 URS[R"[S540UD65 S [R"UR5R-UlS Ulg) NztHistogramObserver's qscheme only support torch.per_tensor_symmetric and torch.per_tensor_affine.z6HistogramObserver doesn't support dynamic quantizationrr7rrrrrr^)r rrjrArrrr8rrrriinforkbits dst_nbins upsample_rate) r?r8rkrrrrrrrhrRrls r@rAHistogramObserver.__init__sW%%%2  %H   %)!  00@  [%++dii*R>*RS Y U5\(T^(TU Y U6](Un(UVekk$**5:::  rC delta_begin delta_enddensityc.X"-U-X-U-- S- nX4-$)z Compute the norm of the values uniformaly distributed between delta_begin and delta_end. Currently only L2 norm is supported. norm = density * (integral_{begin, end} x^2) = density * (end^3 - begin^3) / 3 rr^)r?r@rArBnorms r@ _get_normHistogramObserver._get_norms0  !I - 0IK0W W ~rCnext_start_bin next_end_binc URR5URR5- UR- nX2U- S--UR- nUS:Xag[ R "URURRS9nXQ- U-nXc-n[ R"[ R"XdSS9SURS- 5nUS-U-n [ R"[ R"XtSS9SURS- 5n URU- n [ R"URURRS9n Xi- n US- nXRU [ R"URURRS9U-U 5- n XU- S- UR[ R"U*S- 5[ R"US- 5U 5-- n X-US- -nU*S- n X- nXR[ R"U 5X5- n U R5R5$) zj Compute the quantization error if we use start_bin to end_bin as the min and max to do the quantization. rrfloor) rounding_moder?r)ritemrr8r=raranger7rrdivrrErrsum)r?rGrH bin_width dst_bin_widthsrc_bin src_bin_begin src_bin_enddst_bin_of_begindst_bin_of_begin_centerdst_bin_of_endrBrDr@rAdst_bin_of_end_centers r@_compute_quantization_error-HistogramObserver._compute_quantization_error&s \\&&(4<<+<+<+>>$))K !N%BQ%FG$..X C ,,tyy1F1FG!1Y> #/ !;; IIm' J NNQ   $4c#9]"J IIk H NNQ   ..9,{{499T^^-B-BC#= !A%    JJtyy)>)> ?) K    "22Q6$.. LL-!+ ,ell=1;L.Mw;   !/ >QRAR R$nq( 7  u||K8)MMxxz  rCc(URR5SUR:XdS5eURUR- UR- n[ R "UR5R5n[ R"URSS9nSnSnSnSnURS- n[S5n XV:aXT-n Xd- n Un Un X:a X<X-:aU S-n X:a X<X-:aMX:a X=X-:aU S- n X:a X=X-:aMUnUnX- X- :aU nU nOU nU nX:XaX:XaM~URX5nUU :aO Un UnUnXV:aMURX--nURXS---nUU4$) a2Non-linear parameter search. An approximation for L2 error minimization for selecting min/max. By selecting new min/max, we filter out outliers in input distribution. This follows the implementation of NormMinimization::NonlinearQuantizationParamsSearch in caffe2/quantization/server/norm_minimization.cc rz bins mismatchr gh㈵>rJrrr) r7rr8rrrrQrNcumsumrr[)r?rRtotalcSumstepsizealphabeta start_binend_binnorm_min next_alpha next_betalrbrGrHrDnew_minnew_maxs r@_non_linear_param_search*HistogramObserver._non_linear_param_search\s~~""$Q'4994EoE4\\DLL0DII=  $..)..0||DNN2 ))a-<l)JIAA+$'J,>">E+$'J,>">-DGi.?$?E-DGi.?$?'N"L '+.!"" !  *|/F33NQDhH&I"GGlJ,,!66,,k!::rCorig_minorig_max update_min update_maxcURUR5UR- nX2- URUR-- n[R"UUURUR-S-UR S9SSR UR 5SU--n[R"XEURS-UR S9R UR 5n[R"XxSS9S- n URS- XUR:'SXS:'[R"XURS9n U $) NrrrrMT)rightr)weights minlength) repeat_interleaver>r8rlinspacerr bucketizebincount) r?r7rnrorprqbin_sizemid_points_histogramboundaries_new_histogrambucket_assignmentsupdate_histograms r@_upscale_histogram$HistogramObserver._upscale_histogramsB//0B0BCdFXFXX 'DII8J8J,JK NN D...2   r   2i&&'Hn   $)>> DIIM*:K:K$ "Y    ! OO0RV W  ?Cii!m:;5612 >> TYY  rC orig_hist update_histcXR:Xa Xc:XaX-$X#:Xa<[R"U5n[R"X RXVS9U-nX-$XR::deXc:deUR UUUUU5nXH-$)N)r8rr)rrQhistcr8r) r?rrnrorrprq bin_valuetransformed_orig_hists r@_combine_histograms%HistogramObserver._combine_histogramss  !j&<* *   +.I H99*U ")6 6%%%%%%!% 7 7      ! 22rCrpc<URRUR5 URRU5 URRUR5 URRU5 UR 5S:XaUR 5S:XdS5e[ R"XRX#S9nURR5RUR5 URRU5 g)Nrz(histogram min/max values must be scalar.rr) rrrrrrrrr8r7detach_)r?rprr new_histograms r@reset_histogram!HistogramObserver.reset_histograms W]]+ 7# W]]+ 7# MMOq W]]_%9 6 5 6 9 AyygK   (()<)<= ]+rCrczUR5S:XaU$UR5n[R"U5up4U[R*:XdU[R:Xag[ R "S5 X"R5[R:gnUR5S:XaU$[R"U5up4URnURnUR[S5:H=(d UR[S5:HnU(aURX#U5 U$X4p[R"XX5n [R"Xi5n U R5U R5p[R"X RXS9R!UR"R$5n X:XadX:Xa_UR"U -n UR"R'5R)U R*5 UR"R-U 5 U$UR/UR"UUU U U 5n UR"R'5R)U R*5 UR"R-U 5 URR'5R)U R*5 URR-U 5 URR'5R)U R*5 URR-U 5 U$)Nrz2torch.inf detected in input tensor, ignoring inputrrr)rrrrrrrabsrrrrrrrr8rr7rrrrrr)r?rrpx_minx_max current_min current_maxis_uninitializedrprqrjrkr~combined_histograms r@rqHistogramObserver.forwardsl <<>Q M MMO}}Q'  UYYJ %599"4 MMN O%%'UYY&'AwwyA~  ==+LEll ll <<5<7X4<<5QW=;X   5 1@ =&+ ii 8Gii 8G '~~/1AW${{99' b&&' %'*@%)^^6F%F"&&(001C1I1IJ$$%78" &*%=%=NN$ &"&&(001C1I1IJ$$%78 $$&..w}}= ""7+ $$&..w}}= ""7+ rCcUR[S5:H=(a UR[S5:HnU(a~[R"S5 [ R "S/URRRS9[ R "S/URRRS94$UR[UR5:XdS5eUR5up#URX#5$)Nrrz~must run observer before calling calculate_qparams. Returning default scale and zero point rrrziThe number of bins in histogram should be equal to the number of bins supplied while making this observer)rrrrrrrrrr8rr7rlr)r?rrjrks r@ru#HistogramObserver.calculate_qparams/s<<5<7 DLLE M =   MMM <<dll.A.A.F.FGDLL//44J yyC//  2 /  88:&&w88rCcj>[TU]XU5 URXS-'URXS-'g)Nrr)rj_save_to_state_dictrr)r? destinationr keep_varsrls r@r%HistogramObserver._save_to_state_dictEs4 #KC*.,, Y&'*.,, Y&'rCc 0>URSS5nUbUS:aUS-US-pX;aHXR[R"S/5:Xa![R"[ S55X'X;aHXR[R"S/5:Xa![R"[ S55X'SS/n U H8n X,-n X;aXn[ X U5 MU(dM'URU 5 M: [TU]%UUUUUUU5 g)Nrrrrrrr) rrrSizerrsetattrr rjr)r?rrrrrrrr min_val_name max_val_namer!r$r%r&rls r@r'HistogramObserver._load_from_state_dictJs !$$Y5 ?gk*0));Vi=O,)+11UZZ_D/4||E%L/IJ,)+11UZZ_D/4||E&M/JJ, ), D-C  oC(##C(   %        rCc:SURSUR3$rrrMs r@rHistogramObserver.extra_reprsrrC)r8r=r>) rYrZr[r\rxrrrrrrrrrrkrArEr[rrlrrrrqrrrurrrr]r{r|s@r@r!r!s'.|| \\ \\"\\'' KK & * ** * {{*  * * X <<49LLKP<<  4!#4!S4!l= % ell0J*K= ~' <<' ,,' ,, ' LL ' LL ' R%3<<%3,,%3,, %3 \\ %3 LL %3LL%3 %3N , ,(-  ,?D|| ,  ,4ell4u||4l YY99*7 ' RAArCr!c^\rSrSr%Sr\R \S'\R \S'\R\RSSS4U4Sjjr S r \RRS 5rS rU=r$) r iwa5 Observer that simulates quantize and dequantize with fixed quantization parameters in training time. Only per tensor quantization is supported. Args: `scale` (float): fixed scale for the observer `zero_point` (int): fixed zero point for the observer `dtype`, `qscheme`, `quant_min`, `quant_max` rrrFc N>U(a [S5e[T U]"SX7S.UD6 XPlX`lUR S[ R"U/[ RS95 UR S[ R"U/[ RS95 X0l X@l g)Nz9FixedQParamsObserver doesn't support dynamic quantizationrwr)rkrr^) rrjrArrrrrrrrkr) r?rrrkrrrrhrRrls r@rAFixedQParamsObserver.__init__s %K  FuFvF"" WellE7%++&NO \5<< EII+VW  rCcU$r<r^)r?Xs r@rqFixedQParamsObserver.forwardrCc2URUR4$r<)rrrMs r@ru&FixedQParamsObserver.calculate_qparamsszz4??**rC)rkrrr)rYrZr[r\rxrrrrrrArqrrrur]r{r|s@r@r r wsa  <<  ll''. YY++rCr c^\rSrSrSr\R SSSSSSS4S U4SjjjrSr\RRS5r \RRS 5r S r U=r$) r(ia! Observer that doesn't do anything and just passes its configuration to the quantized module's ``.from_float()``. Can be used for quantization to float16 which doesn't require determining ranges. Args: dtype: dtype argument to the `quantize` node needed to implement the reference model spec. quant_min: minimum value in quantized domain (TODO: align behavior with other observers) quant_max: maximum value in quantized domain custom_op_name: (temporary) specify this observer for an operator that doesn't require any observation (Can be used in Graph Mode Passes for special case ops). compute_dtype (deprecated): if set, marks the future quantize function to use dynamic quantization instead of static quantization. This field is deprecated, use `is_dynamic=True` instead. is_dynamic: if True, the `quantize` function in the reference model representation taking stats from this observer instance will use dynamic quantization. NFc 0>[T U]XS9 Uc[RnUc.[R"[R 5R nXlX`lX@l XPl XplX l U(aSn[R"S5 gg)NrwTzPlease use `is_dynamic` instead of `compute_dtype`. `compute_dtype` will be deprecated in a future release of PyTorch.)rjrArrrrrrkrrr custom_oprr) r?rkcustom_op_name compute_dtyperrrrrhrls r@rAPlaceholderObserver.__init__s u< ?--G ;++emm,00C  ""' J MM!  rCcU$r<r^ros r@rqPlaceholderObserver.forwardrrCc:SURSUR3$)Nzdtype=z , is_dynamic=rwrMs r@rPlaceholderObserver.extra_reprs |=0ABBrCc[S5e)Nz>calculate_qparams should not be called for PlaceholderObserver ExceptionrMs r@ru%PlaceholderObserver.calculate_qparamss L  rC)rrkrrrrr)rYrZr[r\rxrrrArqrrrrur]r{r|s@r@r(r(s~0mm     D YYCC YY  rCr(c^\rSrSrSrS\\\R0r \R4U4Sjjr Sr \RRS5r\RRS5rSrU=r$) r)iz The module is mainly for debug and records the tensor values during runtime. Args: dtype: Quantized data type qscheme: Quantization scheme to be used reduce_range: Reduces the range of the quantized data type by 1 bit tensor_valc0>[TU]USS9 /UlgNFrw)rjrAr)r?rkrls r@rARecordingObserver.__init__s u7rCcXURRUR55 U$r<)rr cloneros r@rqRecordingObserver.forwards qwwy)rCc[S5e)Nz[TU]USS9 XlX lgr)rjrArkr)r?rkrrls r@rANoopObserver.__init__s u7 'rCcU$r<r^ros r@rqNoopObserver.forwardrrCc[S5e)Nz7calculate_qparams should not be called for NoopObserverrrMs r@ruNoopObserver.calculate_qparams s E  rC)rrkr)rYrZr[r\rxrfloat16rArqrrrur]r{r|s@r@r%r% s@ #]]2((  YY  rCr%cn^\rSrSrSrSU4SjjrSr\RRS5r Sr U=r $)r*i'aThis observer is used when we want to reuse the observer from the operator that produces the input Tensor, typically used for operators like reshape, e.g. ``` x0 = ... x1 = x0.reshape() ``` if we configure x0 to be observed by some observer, let's say MinMaxObserver, and reshape is configured with ReuseInputObserver, we'll reuse the observer instance for x0 for x1 (output of reshape). If x0 is not observed, we also won't observe x1. Note: this is only enabled in FX Graph Mode Quantization c>>[TU][RSS9 g)NF)rh)rjrArr)r?rls r@rAReuseInputObserver.__init__5s %8rCcU$r<r^ros r@rqReuseInputObserver.forward8rrCc[S5e)Nz=calculate_qparams should not be called for ReuseInputObserverrrMs r@ru$ReuseInputObserver.calculate_qparams;s K  rCr^r) rYrZr[r\rxrArqrrrrur]r{r|s@r@r*r*'s0 9 YY  rCr*) dataclass)autoEnumcB\rSrSrSr\"5r\"5r\"5rSr g)r.iKaHow floating point number is mapped to integer number symmetric mapping means floating point range is symmetrically mapped to integer range let's say we have floating point range (-3.5, 10.2) and integer range (-8, 7) (int4) we'll use (-10.2, 10.2) as the range for floating point and map that to (-8, 7) e.g. scale = (10.2 - (-10.2)) / (7 - (-8)) SYMMETRIC_NO_CLIPPING_ERR is a variant of symmetric mapping, where the scale is the max of smin and smax, where smin = min_val_neg / quant_min, and smax = max_val_pos / quant_max. By calculating smin and smax individually, there can be less round error on negative values, and no out-of-range of all floating point values. asymmetric mapping means we just directly map the floating point range to integer range, for the above example, we will map (-3.5, 10.2) to (-8, 7) and calculate quantization parameter based on this mapping e.g. scale = (10.2 - (-3.5)) / (7 - (-8)) r^N) rYrZr[r\rxr SYMMETRICSYMMETRIC_NO_CLIPPING_ERR ASYMMETRICr]r^rCr@r.r.Ks$I $JrCr.cB\rSrSrSr\"5r\"5r\"5rSr g)r6ica8Enum that indicate whether zero_point is in integer domain or floating point domain integer domain: quantized_val = (float_val / scale) (integer) + zero_point (integer) float domain: quantized_val = (float_val - (zero_point (float) - scale * mid_point)) / scale none domain: quantized_val = (float_val / scale) r^N) rYrZr[r\rxrINTFLOATNONEr]r^rCr@r6r6cs &C FE 6DrCr6cz\rSrSrSr\"5r\"5r\"5r\"5r \"5r \"5r \"5r Sr g)r5ipz? Placeholder for dtypes that do not exist in PyTorch core yet. r^N)rYrZr[r\rxrINT1INT2INT3INT4INT5INT6INT7r]r^rCr@r5r5ps: 6D 6D 6D 6D 6D 6D 6DrCr5T)frozenc\rSrSrSrSrg)r-iz Base class for representing the granularity of quantization. This class serves as a parent for specific granularity types used in quantization operations, such as per-tensor or per-axis quantization. r^NrYrZr[r\rxr]r^rCr@r-r-srCr-c.\rSrSr%Sr\\S4\S'Srg)r0iz Represents per-block granularity in quantization. See :func:`~torchao.quantization.quant_primitives.quantize_affine` for docs for `block_size` Attributes: block_size (Tuple[int, ...]): The size of each quantization group . block_sizer^N) rYrZr[r\rxrrrr]r^rCr@r0r0sc3hrCr0c\rSrSrSrSrg)r3iz Represents per-tensor granularity in quantization. This granularity type calculates the quantization parameters based off the entire tensor. r^Nrr^rCr@r3r3srCr3c$\rSrSr%Sr\\S'Srg)r/ia Represents per-axis granularity in quantization. This granularity type calculates different quantization parameters along a specified axis of the tensor. For example if the input tensor is shape [8, 16] and axis=0, then the quantization parameters are calculated for each row of the tensor. Giving a total of 8 quantization parameters. Attributes: axis (int): The axis along which reduction is performed. axisr^NrYrZr[r\rxrrr]r^rCr@r/r/s  IrCr/c$\rSrSr%Sr\\S'Srg)r1ia Represents per-channel group granularity in quantization. This granularity type calculates different quantization parameters for each group of elements. For example if the input tensor is shape [8, 16], and the group size is 4, then the input tensor is reshaped to [64, 4] quantization parameters are calculated for each group of 4 elements, giving a total of 64 quantization parameters. Attributes: group_size (int): The size of each quantization group group_sizer^Nrr^rCr@r1r1s OrCr1c\rSrSrSrSrg)r2ia Represents row-wise granularity in quantization. This is a special case of per-axis quantization and is unique to Float8 matmuls where the input is quantized with a block_size of (1, ..., input.shape[-1]). And the weight is quantized with a block_size of (1, weight.shape[1]). r^Nrr^rCr@r2r2srCr2c\rSrSrSrSrg)r4ia Represents per-token granularity in quantization. This granularity type calculates a different set of quantization parameters for each token, which is represented as the last dimension of the tensor. For example, if the input tensor has shape [2, 3, 4], then there are 6 tokens with 4 elements each, and we will calculate 6 sets of quantization parameters, one for each token. If the input tensor has only two dimensions, e.g. [8, 16], then this is equivalent to `PerAxis(axis=0)`, which yields 8 sets of quantization parameters. r^Nrr^rCr@r4r4s rCr4 input_shape. granularityrc.[U[5(dS5e[U[5(aU$[U[5(a$[ U5nSX!R '[ U5$[U[5(aS[U5S- -US4-$[U[5(a'[U5S:Xd SU35eSUR4$[U[5(a[ U5nUSUS'[ U5$[SU35e)zGet the block size based on the input shape and granularity type. Args: input_shape: The input tensor shape possibly more than 2 dimensions granularity: The granularity type of the quantization z=Please provide an instance of Granularity, not subclass of itr)rrrzNExpecting input shape dim to be 2 for per group quantization, gotinput shape: zUnsupported Granularity: ) isinstancer-r3r/r0rrr2rr1rr4 ValueError)rrrs r@r7r7s% [  GFG +y)) K ) )+& '( ##$Z  K ( (s;'!+, B/AAA K * *   ! j [\g[h i j !;))** K * *+& $R 2Z  0 > ??rCc|^\rSrSrSr\"\5rSSSSSS\R4S\ S\ RS\ S\\S \\S \\S \\ RS \\ RS \S\\4U4Sjjjr\S\ R(S\ R(4Sj5r\S\\ R(\ R(44Sj5rSrU=r$)r,iaObserver module for affine quantization (https://github.com/pytorch/ao/tree/main/torchao/quantization#affine-quantization) Args: `granularity` and `block_size`: The granularity of the quantization, must specify at least one, if both are specified `block_size` takes precedence Current supported granularity type are `PerTensor` and `PerAxis` other args: please see `:class:torchao.dtypes.AffineQuantizedTensor` NT mapping_type target_dtyperrrr scale_dtypezero_point_dtype preserve_zerozero_point_domainc  >[T U]5 UcS5eXlX lX0lX@lXPlX`lXplXl Xl Xl SUl SUl g)Nzgranularity is None)rjrArrrrrrrrrrroriginal_dtype) r?rrrrrrrrrrrRrls r@rA$AffineQuantizedObserverBase.__init__sj &=(==&((&""& 0*!2"rCinputrcg)znforward function should take the input tensor and updates internal stats and return the original input Tensor Nr^)r?r s r@rq#AffineQuantizedObserverBase.forward1rCcg)zCalculate quantization parameter based on the stats attached to the observer module and returns a tuple of scale and zero_point Tensor Nr^rMs r@ru-AffineQuantizedObserverBase.calculate_qparams7rrC) rrrrr rrrrrrr)rYrZr[r\rxrzrPrSr6rr.rrkr-rrrryrArrrqrrur]r{r|s@r@r,r,sJ'I$(#'#-126"7F7J7J#!#kk#! # C= # C= #e_#ekk*##5;;/##$O4##@ U\\ ell   5u||)C#D  rCr,c[U[RR5(aEURR R SS5Sn[R"SSU5nX;$g)zCReturns true if given mod is an instance of Observer script module..rz\.___torch_mangle_\d+rF) rrrRecursiveScriptModule_cqualified_namesplitresub)mod obs_type_namesuffixr$s r@_is_observer_script_moduler>sY#uyy6677&&,,S!4Q7vv.F;$$ rCc[U[RRR[RRR [ 45=(d [US5$)Nzquantization.observer)rrao quantizationr&FakeQuantizeBaser,rmodules r@_is_activation_post_processr$KsT  HH ! ! . . HH ! ! 2 2 '   E $F,C DErCc[U[RR5(a[ US5=(d [ US5$g)Nz.quantization.observer.PerChannelMinMaxObserverz;quantization.observer.MovingAveragePerChannelMinMaxObserverF)rrrrrr"s r@#_is_per_channel_script_obs_instancer&VsD&%))99::) D  ' Q  rCcx[5n[U[RR5(a4UR 5R 5Hup#SU;dM X1U'M O3UR 5R 5Hup#SU;dM X1U'M UR 5RUlU$)zv Returns the state dict corresponding to the observer stats. Traverse the model state_dict and extract out the stats. observeractivation_post_process)rrrrrritems _metadata)rodkvs r@rr`s B#uyy6677NN$**,DAQ1- NN$**,DA(A-1->>#--BL IrCc t/n/nUR5HXupEUS-n[U5(dM[U5(aURX0SX#/5 MCUR X0SX#/5 MZ UHnSU;dSU;dM[ SUS35e UHnSU;dSU;dM[ SUS35e g ) z Given input model and a state_dict containing model observer stats, load the stats back into the model. The observer state_dict can be saved using torch.ao.quantization.get_observer_state_dict rTFr(r)zMissing keys for observer z in state_dictzUnexpected keys for observer N) named_modulesr$r&r)rr)robs_dictrrr$r#rr-s r@rrss !L!#O))+  &v . .26::33b$ r,,b%PR, ?71<,QC~>   ?71</s.A rC)rr)rkrig0?)rkrrrrr)rkrrrh)rkrrg?r)rrrkrrgp?)Zrxrrabcrr collectionsr functoolsrtypingrrrtorch.nnrtorch.ao.quantization.utilsr r r r r __all__r9rPrdobjectrerModuler&r+ _ObserverBaser"r#r'r$r!r r(r)r%r* dataclassesrenumrrr.r6r5r-r0r3r/r1r2r4rrr7r,rr$r&rrrSrrrrrrrrrrrrrrrrrrrrrrr^rCr@r?s   '# + \2 **0 56)R (S(:3 :BqXlqXn0 O84O8d`.`FM >M `W,DWtfA7fAR ++<++\D ,D N < < :  6 "$0 d  4   $ $  {     $  $k$ ${([ {  @sCx@/:@ 38_@@6 #uxx6 rE&D"++a3+G 3+)22 ++u993(6'?'? ++  & & (@($/881PS8T'?&H&H ++u::'I'# 4L3U3U ++  ' ' 4V40"5!>!> ,, "?" ":!C!C ,, F FPQ"D"'?&H&H ..%"H"HRS'I'# 0D/M/M #U\\QRU0N0,-A,J,J !5<<1PS-K-),X((Q% 2rC