[Thm SrSSKrSSKrSSKJr SSKJrJr SSKJ r J r J r J r SSK r SSKJr SSKJr SSKJr \ \\\4\\\\\44\ 4rS \l\ r\ \ \\ \ 4\\ \\ \ 44\ 4rS \l"S S 5r\ R4R6\ R4R8\ R4R:\ R4R<\ R4R>\ R4R@\ R4RB\ R4RD\ R4RF\ R4RH\ R4RJ\ R4RL\ R4RN\ R4RP\ R4RR1r*\ R4RVRX\ R4RVRZ\ R4RVR\\ R4RVR^\ R4RVR`\ R4RVRb\ R4RVRd\ R4RVRf\ R4RVRh\ R4RVRj\ R4RVRl\ R4RVRn\ R4RVRp\ R4RVRr\ R4RVRt\ R4RVRv\ R4RVRx\ R4RVRz\ R4RVR|\ R~\ R\ R|\ R\ R\ R\ R\ R\ R1rG\ RS S SSSSSSSSSSSSSSSSSSS S!S"S#1rIS$rJS%rKS&rLS'rMS(\ S)\NS*\ 4S+jrOS,rPS-rQS.rRS/rSS0rTS1rUS2rVS3rWS4rXS5rYS6rZS*\[4S7jr\S8r]S9r^S:\ RS;\ RS*\[4S<jr`S=\aS>\aS?\[S@\ RSA\[S*\\a\a44 SBjrcSCrdSDreSE\ R4RSF\ R4RS*\ \N4SGjrgSH\ SI\h\N\ 4S*\h\N\ 44SJjriSH\ S*SK4SLjrjSM\ SI\h\N\ 4S*SK4SNjrkS=\aS>\aS*S4SOjrl\ R4S:\ RS;\ RS=\aS>\aS@\ RSP\ RS?\[SQ\ RS*\\ R\ R44SRjjroSH\ S*\a4SSjrpST\ R4RSU\\ SV4S*\h\N\\ SV444SWjrqSX\ R4RS*\ 4SYjrr/SZQrsg)[z? Utils shared by different modes of quantization (eager/graph) N) OrderedDict)getfullargspec signature)AnyCallableOptionalUnion) QuantType)Node)is_parametrizedztorch.ao.quantization.utilsc\rSrSrSrSrg) MatchAllNode%zfA node pattern that matches all nodes, used in defining fusion patterns in FX Graph Mode Quantization N)__name__ __module__ __qualname____firstlineno____doc____static_attributes__rS/var/www/auris/envauris/lib/python3.13/site-packages/torch/ao/quantization/utils.pyrr%srrrelurelu_ contiguousdetachdetach_ hardsigmoid hardsigmoid_permuterepeatrepeat_interleavereshaperesize_shapesigmoidsigmoid_sizesqueezesqueeze_tanhtanh_ transpose unsqueeze unsqueeze_viewc*URS:H=(a UR[;nURS:H=(a UR[;nURS:H=(a( [ U[ UR55[ ;nX#U4$)N call_function call_method call_module)optarget func_list method_listtypestrmodule_type_list)nodemodulesis_call_functionis_call_methodis_call_modules r check_noderAxswww/1NdkkY6NWW -L$++2LN = XT'#dkk2B*C%DHX%X ^ ;;rcHUR5nURU5 U$)a{ Combines two dictionaries. This function takes two dictionaries as input and returns a new dictionary that contains all the key-value pairs from both input dictionaries. If there are any duplicate keys in the `additional_dict`, the values from the `additional_dict` will overwrite those in the `default_dict`. Args: default_dict (dict): The main dictionary that will be used as the base additional_dict (dict): The dictionary used to update `default_dict` Returns: dict: The resulting dictionary Example: >>> x = dict(a=1, b=1) >>> y = dict(b=2, c=3) >>> get_combined_dict(x, y) {'a': 1, 'b': 2, 'c': 3} )copyupdate) default_dictadditional_dictds rget_combined_dictrHs#( AHH_ Hrc\U[R:H=(d U[R:H$N)torchper_tensor_affineper_tensor_symmetricqschemes r is_per_tensorrPs# e-- - VEG " ( (E!2G j!7MM%00W))   e11 1..G0883G9=>>!I.@@BEG&L%{335?? %{335??  Nrc[U5nURU05n[U5U;dS[U5SU35eU[U5$)aGet the observed/quantized custom module class that we need to swap `custom_module` to Input: custom_module: input, can be an instance of either a float or observed custom module custom_module_class_mapping: the float to observed or observed to quantized custom module class mapping qconfig: qconfig configured for the custom module Output: corresponding observed/quantized custom module class for input custom module instance z5did not find corresponding observed module class for z in mapping: )get_quant_typegetr9) custom_modulecustom_module_class_mappingqconfig quant_type class_mappings rget_swapped_custom_module_classrsh (J/33JCM  - /  /0 m_ N / m, --rcDUceUR5nUR$rJ) activationrr)rrs ractivation_dtypers(   ##%J   rcDUceUR5nUR$rJ)weightrr)rrs r weight_dtypers$    ^^ F <<rc x[U5[R[R[R[R [R [R[R[R[R[R4 ;=(a [U5(+$)zGiven a qconfig, decide if the activation needs to be quantized or not, this includes quantizing to quint8, qint8 and qint32 and float16 ) rrKrarcrefloat16rbrdrjrfrkrl#activation_is_dynamically_quantizedrs r"activation_is_statically_quantizedrsw G $           )  =37; ; =rc"[U5upnU$)zGiven a qconfig, decide if the activation needs to be dynamically quantized or not, this includes dynamically quantizing to quint8, qint8 and float16 )get_qconfig_dtypes)r_activation_dtype_activation_is_dynamics rrrs 3EW2M//  rc[U5[R[R[R[R 4;$)zzGiven a qconfig, decide if the activation needs to be quantized to int8 or not, this includes quantizing to quint8, qint8 )rrKrarcrbrdrs ractivation_is_int8_quantizedr"s7 G $    ) rcZ[U5[R[R4;$)zPGiven a qconfig, decide if the activation needs to be quantized to int32 or not )rrKrerfrs ractivation_is_int32_quantizedr.s! G $u{{(C CCrc J[U5[R[R[R[R [R [R[R[R[R[R4 ;$)zCGiven a qconfig, decide if the weight needs to be quantized or not ) rrKrarcrrgrbrdrjrfrkrlrs rweight_is_quantizedr5se             %  rc[U5[R[R[R[R 4;$)zNGiven a qconfig, decide if the weight needs to be statically quantized or not )rrKrarcrbrdrs rweight_is_statically_quantizedrGs-  U\\5;; UZZ$X XXrc[U5upnU[R[R4;=(a, U[R[R 4;=(a U$)zLGiven a qconfig, returns True if this op is using int8 dynamic quantization )rrKrarbrcrd)rrrrs r op_is_int8_dynamically_quantizedrNsV=Ow '3G9 = >  rrvrwhas_customized_qrangerr reduce_rangecvU(aU[R[R4;aSupVOSupVXpUbUbUUpeXe- S-n U[R[R4;aSU s=:a S::dS5e S5eOAU[R[R4;aSU s=:a S::dS5e S5eU(a US -US -pX4$U[R[R4;aU(aS upX4$S upX4$U[R [R 4;aU(aS upX4$SupX4$U[R[R4;aS upX4$U[R4;aSupX4$U[R4;aSupX4$SupX4$)zlCalculates actual qmin and qmax based on the quantization range, observer datatype and if range is reduced. )rl)rrzRquantization range should be positive and not exceed the maximum bit range (=256).lzYquantization range should be positive and not exceed the maximum bit range (=4294967296).)i?)i)rr)ii)ri)ii)r) rKrerfrcrdrarbrirj) rvrwrrrrinitial_quant_mininitial_quant_maxcustom_quant_mincustom_quant_max qrange_lens rcalculate_qmin_qmaxrs U\\5;;/ /3? 0 039 0 .7*  ',<,H   1 ':Q> U[[%**- -J%#% dc d% dc d% u||U[[1 1J'%' kj k' kj k' #,>9>y*  % U[[%**- -'.$  (1$ 9  u||U[[1 1'-$   (.$ 9  u||U[[1 1#< I   u||n $#/ I   u{{m ##6 I  $) I  rcfURSS5n[U5S:XaSUS4$USUS4$)z$ Turn 'foo.bar' into ['foo', 'bar'] rZrr)rsplitlen)r6rs r _parent_namers>  c1A 1v{1Q4xtQqTzrc[UR5S:Xag[U5(a.[UR5S:H=(a SUR;$g)z Checks if module._modules is empty or if module is a parametrization, checks that module._modules only has the 'parametrizations' module rTrparametrizationsF)r_modulesr )modules r)has_no_children_ignoring_parametrizationsrsI  6??q   6??#q(R-?6??-RRrroot submodulecFUR5H up#XLdM Us $ g)a.Get the path (fully qualified name) of a submodule Example:: >> class M(torch.nn.Module): def __init__(self) -> None: self.linear = torch.nn.Linear(5, 5) def forward(self, x): return self.linear(x) >> m = M() >> l = m.linear >> _get_path_of_module(m, l) "linear" N) named_modules)rrnps r_get_path_of_modulers($""$ >H% rfloccUR5VVs0sH"up#U[U5R;dM X#_M$ snn$s snnf)zGet local keyword arguments Example:: >> def f(self, a, b=9): pass >> loc = {"a": 6, "c": 7} >> _get_signature_locals(f, loc) {"a": 6} )itemsr parameters)rrkvs r_get_signature_localsrs9!YY[ I[TQA11H1H,HDAD[ II Is AAzOrderedDict[str, Any]cB0n[U5RR5Hlup#URURLaURX'M.UR UR LaSX'MMUR URLdMh0X'Mn [U5$)zGet all default keyword arguments from function signature Example:: >> def f(self, a, b=9): pass >> _get_default_kwargs(f) {"b": 9} r) rrrdefaultemptykindVAR_POSITIONAL VAR_KEYWORDr)rkwargsnameparams r_get_default_kwargsr%sF |..446  == + ==FL ZZ5// /FL ZZ5,, ,FL 7 v rfuncc[U5n[X5nUR5nUR5HupVXT;dM XdU'M U$)aGiven a function and local function arguments, normalize the keyword arguments by filling in default arguments from function signature Example:: >> def f(self, key1=3, key2=3): pass >> loc = {"key2": 6} >> _normalize_kwargs(f, loc) {"key1": 3, "key2": 6} )rrrCr)rrdefault_kwargs local_kwargsnormalized_kwargsattrvals r_normalize_kwargsr:sR).N(3L&++-!'')   $&)d #* rcVUSs=::a U::dS5e S5eX:dS5eg)a=Validates that the user-specified quantization range is properly initialized and within the given bound supported by the observer dtype. To accommodate lower-bit quantization with respect to the existing torch.qint8 and torch.quint8 datatypes, the user can choose to use dynamic quantization range by passing in a tuple of initial qmin and qmax values. One use case is these customized qmin and qmax values are used to calculate static estimates of the scale and zero point for aggressive lower-bit fake quantization. These estimates are compared against parameters learned through backpropagation. The related literatures for scale and zero point via backpropagation are as follows: Learned Step Size Quantization: https://openreview.net/pdf?id=rkgO66VKDS Trained Quantization Thresholds: https://arxiv.org/pdf/1903.08066.pdf rz1Used-specified quantization range must include 0.zKqmin must be strictly less than qmax for user-specified quantization range.Nr)rvrws rvalidate_qmin_qmaxrPsK" Q#)#;:;#;:;# UTUrepsrOc[X5(dT[R"S/URRS9[R"S/URRS94$[R "U[R "U55n[R"U[R "U55n URn [R"UR5[RU S9n [R"UR5[RU S9n URU 5nU[R:XdU[R:Xa[R"U*U 5n U [!X2- 5S- - n [R"X5n U[R"[R$4;aMU(a&U R'U R5X#-S-5n OU R'U R5S5n OU[R(:XaIX- [!X2- 5- n [R*"X:U [R,"U 55n SU-U - n OwX- [!X2- 5- n [R"X5n U[R."X- 5R[R05- n [R2"XU5n [5U R65S:Xa*[R"[!U 5/U R8U S9n [5U R65S:Xah[R"[1U 5/U R8U S9n U[R(:Xa*[R"[!U 5/U R8U S9n U R[R5U R[R54$)aDCalculates the quantization parameters, given min and max value tensors. Works for both per tensor and per channel cases Args: min_val: Minimum values per channel max_val: Maximum values per channel Returns: scales: Scales tensor of shape (#channels,) zero_points: Zero points tensor of shape (#channels,) g?)devicer)rrrr)rrKtensorrr9min zeros_likemaxonesr(doublezerosint64torMrTrrbranew_fullrSwhere ones_likeroundintclamprr%rr) rrrvrwrrrrrO min_val_neg max_val_posrrtrus rdetermine_qparamsr ls* w 0 0||SE'..*=*=> C++A   ))GU%5%5g%>?K))GU%5%5g%>?K   F JJ{'')f ME[--/u{{6RJ &&.C%,,,5;V;V0Vii k: uY%:;a?@ %% U[[%,,/ /$'00OO% (=!'C (001BCH E:: :"eI,A&BB EK0FG '\E) *eI4I.JJ %%[-@!A!D!DUYY!OO [[ B  5;;1 eEl^5;;vN :  !\\ _ Z%5%5f  e<< <z"#:+;+;FJ 88ELL !:==#= ==rc>[[U5R5$)zGet number of positional args for a function Example:: >> def f(self, key1=3, key2=3): pass >> _get_num_pos_args(f) 3 )rrargs)rs r_get_num_pos_argsrs ~a %% &&rmodelexample_inputs.c<^^^Um0mUUU4Sjn[RRRmU[RRlU"U6 T[RRlT$!T[RRlf=f)aGiven a model and its example inputs, return a dictionary from fully qualified name of submodules to example_inputs for that submodule, e.g. {"linear1": (tensor1,), "linear2": (tensor2,), "sub": (tensor3,), "sub.linear1": (tensor4,), ...} Used to make quantizing submodules easier now that FX Graph Mode Quantization requires example inputs. Also works for keyword arguments with default values, we would flatten keyword arguments as positional arguments and fill in the missing keyword args with default values, e.g. if we have a forward function: def forward(self, x, key1=3, key2=3): ... and we call it with self.submodule(x, key2=6) we'll get example_inputs: (x, 3, 6) user can also override `key1` with positional arguments as well: for self.submodule(x, 5, key2=6) we'll get: (x, 5, 6) variable positional arguments and variable positional keyword arguments in forward function are not supported currently, so please make sure no submodules is using them. c>[U5R5n[URU5n[ UR5S- nU[ U5- nU(a+U(a$UR SS9 US-nU(a U(aM$URUR55 [U5n[T U5nUbUT U'T "U/UQ70UD6$)NrF)last) listrCrforwardrrpopitemextendvaluestupler) selfrrsubmodule_example_inputsrnum_args num_to_popsubmodule_example_inputs_tuplerWfqn_to_example_inputsorig_module_callrs r_patched_module_call7get_fqn_to_example_inputs.._patched_module_calls#':??#4 -dllFC$T\\2Q6$< == .  % %5 % 1 !OJ.. !''(9(@(@(BC)./G)H&!$- ?)G !# &6t6v66r)rKnnModule__call__)rrr#r!r"rs @@@rget_fqn_to_example_inputsr(st8 D7 xx//3EHHOO4 ~$4  $4 s A::!BrcUR5Vs1sHoRiM snUR5Vs1sHoRiM sn-n[R"S5[R"S51U:Xa-[R "S5 [R"S51n[ U5S::d SU35e[ U5S:a[[U55nU$SnU$s snfs snf)zz Returns the unique device for a module, or None if no device is found. Throws an error if multiple devices are detected. cpumetazfBoth 'meta' and 'cpu' are present in the list of devices. Module can have one device. We Select 'cpu'.rzKprepare only works with cpu or single-device CUDA modules, but got devices rN) rrbuffersrKrrrnextiter)rrdevicesrs r_assert_and_get_unique_devicer0s "(!2!2!45!4Axx!45 ..*9*Q*9G  UU\\&12g= t <<&' w<1  ") % %(L1$4T$w- F M;?F M#69s C7C<) NodePatternPatternrrArHrPrUr_rrrrrrrrrrrrrrrrr(ror r)trr[r collectionsrinspectrrtypingrrrr rK torch.ao.quantization.quant_typer torch.fxr torch.nn.utils.parametrizer rr1r QuantizerClsr2rr%ReLUReLU6AdaptiveAvgPool1dAdaptiveAvgPool2dAdaptiveAvgPool3d AvgPool1d AvgPool2d AvgPool3d MaxPool1d MaxPool2d MaxPool3dIdentity HardsigmoidSigmoidTanhr; functionaladaptive_avg_pool1dadaptive_avg_pool2dadaptive_avg_pool3delu hardswish instance_norm layer_norm leaky_relusilumishdropout max_pool1d max_pool2d max_pool3drhardtanh hardtanh_rr&r-r"r)stacksumr+r.catr7meanr8rArHrPrUr:r_rorrrrrrrrrrboolrrrTensorrr rrrrrr&rdictrrrrrLrOr rr(r0__all__rrrrbs<#-11 66E$*%uT5t3D-D'EsJK 6   eHh&'xx?Q9R/R)SUXX 3 HHMM HHNN HH HH HH HH HH HH HH HH HH HH HH HH HHMM$ HH++ HH++ HH++ HH HH!! HH%% HH"" HH"" HH HH HH HH"" HH"" HH"" HH HH   HH!! HH## HH OO  MM MM KK II JJ OO II9  > JJ               3 << 2W:#:C:C:!&!H.,  =$! D$Y  <!H :; ; ;  ;  ;; ;  ;  38_ ; |  ((//&+hhoo c]0 JX JDcN JtCH~ J8(?*H4S>>U,U#U#U$UH#44K> \\K> \\K>K> K> ;; K>  K> K>]]K> 5<< %&K>\ ' 'c '6! 88??6!,1#s(O6! #uS#X 6!r%((//c2 r