o Zh7$ @sHddlZddlmZddlmZddlmZddlmZm Z m Z m Z m Z m Z mZddlmZmZddlmZdd lmZdd lmZmZdd lmZdd lmZmZmZdd lm Z gdZ!dede defddZ"dede defddZ#ej$j%j&j'ej$j%j&j(ej$j%j)j'ej$j*j+gZ,dede-fddZ.  ddede-de-defddZ/dS) N) constant_fold)DuplicateDQPass)PortNodeMetaForQDQ)DerivedQuantizationSpecFixedQParamsQuantizationSpecQuantizationAnnotationQuantizationSpecQuantizationSpecBase QuantizerSharedQuantizationSpec) GraphModuleNode) PassManager)prepare)_fold_conv_bn_qat_fuse_conv_bn_qat) reference_representation_rewrite)_disallow_eval_train_fuse_conv_bn__get_node_name_to_scope)#_convert_to_reference_decomposed_fx) prepare_pt2eprepare_qat_pt2e convert_pt2emodel quantizerreturncCsjtjd|j}t|}t|||}||||t ||d|j d}|j |t |}|S)aPrepare a model for post training quantization Args: * `model` (torch.fx.GraphModule): a model captured by `torch.export.export_for_training` API. * `quantizer`: A backend specific quantizer that conveys how user want the model to be quantized. Tutorial for how to write a quantizer can be found here: https://pytorch.org/tutorials/prototype/pt2e_quantizer.html Return: A GraphModule with observer (based on quantizer annotation), ready for calibration Example:: import torch from torch.ao.quantization.quantize_pt2e import prepare_pt2e from torch.ao.quantization.quantizer import ( XNNPACKQuantizer, get_symmetric_quantization_config, ) class M(torch.nn.Module): def __init__(self) -> None: super().__init__() self.linear = torch.nn.Linear(5, 10) def forward(self, x): return self.linear(x) # initialize a floating point model float_model = M().eval() # define calibration function def calibrate(model, data_loader): model.eval() with torch.no_grad(): for image, target in data_loader: model(image) # Step 1. program capture # NOTE: this API will be updated to torch.export API in the future, but the captured # result shoud mostly stay the same m = torch.export.export_for_training(m, *example_inputs).module() # we get a model with aten ops # Step 2. quantization # backend developer will write their own Quantizer and expose methods to allow # users to express how they # want the model to be quantized quantizer = XNNPACKQuantizer().set_global(get_symmetric_quantization_config()) m = prepare_pt2e(m, quantizer) # run calibration # calibrate(m, sample_inference_data) z+quantization_api.quantize_pt2e.prepare_pt2eFZis_qatZobs_or_fq_callback) torch_C_log_api_usage_oncemetarrtransform_for_annotationannotatevalidaterprepare_obs_or_fq_callbackupdaterrroriginal_graph_metaZnode_name_to_scoper*R/var/www/auris/lib/python3.10/site-packages/torch/ao/quantization/quantize_pt2e.pyrs :    rcCsjtjd|j}t|}||}||||t|t ||d|j d}|j |t |}|S)a9Prepare a model for quantization aware training Args: * `model` (torch.fx.GraphModule): see :func:`~torch.ao.quantization.quantize_pt2e.prepare_pt2e` * `quantizer`: see :func:`~torch.ao.quantization.quantize_pt2e.prepare_pt2e` Return: A GraphModule with fake quant modules (based on quantizer annotation), ready for quantization aware training Example:: import torch from torch.ao.quantization.quantize_pt2e import prepare_qat_pt2e from torch.ao.quantization.quantizer import ( XNNPACKQuantizer, get_symmetric_quantization_config, ) class M(torch.nn.Module): def __init__(self) -> None: super().__init__() self.linear = torch.nn.Linear(5, 10) def forward(self, x): return self.linear(x) # initialize a floating point model float_model = M().eval() # define the training loop for quantization aware training def train_loop(model, train_data): model.train() for image, target in data_loader: ... # Step 1. program capture # NOTE: this API will be updated to torch.export API in the future, but the captured # result shoud mostly stay the same m = torch.export.export_for_training(m, *example_inputs).module() # we get a model with aten ops # Step 2. quantization # backend developer will write their own Quantizer and expose methods to allow # users to express how they # want the model to be quantized quantizer = XNNPACKQuantizer().set_global(get_symmetric_quantization_config()) m = prepare_qat_pt2e(m, quantizer) # run quantization aware training train_loop(prepared_model, train_loop) z/quantization_api.quantize_pt2e.prepare_qat_pt2eTr) rr r!r"rr#r$r%rrr&r'rr(r*r*r+rns 8    rncCs|jdko |jtvS)aTIf there is any pure ops between get_attr and quantize op they will be const propagated e.g. get_attr(weight) -> transpose -> quantize -> dequantize* (Note: dequantize op is not going to be constant propagated) This filter is added because we don't want to constant fold the things that are not related to quantization Z call_function)optarget _QUANT_OPS)r,r*r*r+_quant_node_constraintsr0FTuse_reference_representation fold_quantizecCstjdt|tstd|d|j}t|}t|}t t g}||j }t t g}||j }|r;t |t|rAt|}|j|t|}|S)aConvert a calibrated/trained model to a quantized model Args: * `model` (torch.fx.GraphModule): calibrated/trained model * `use_reference_representation` (bool): boolean flag to indicate whether to produce referece representation or not * `fold_quantize` (bool): boolean flag for whether fold the quantize op or not Returns: quantized model, either in q/dq representation or reference representation Example:: # prepared_model: the model produced by `prepare_pt2e`/`prepare_qat_pt2e` and calibration/training # `convert_pt2e` produces a quantized model that represents quantized computation with # quantize dequantize ops and fp32 ops by default. # Please refer to # https://pytorch.org/tutorials/prototype/pt2e_quant_ptq_static.html#convert-the-calibrated-model-to-a-quantized-model # for detailed explanation of output quantized model quantized_model = convert_pt2e(prepared_model) z+quantization_api.quantize_pt2e.convert_pt2ezjUnexpected argument type for `use_reference_representation`, please make sure you intend to pass argument z to convert_pt2e)rr r! isinstancebool ValueErrorr"rrrrZ graph_modulerrr0rr'r)rr1r2r)pmr*r*r+rs*        r)FT)0rZ%torch._export.passes.constant_foldingrZ,torch.ao.quantization.pt2e.duplicate_dq_passrZ-torch.ao.quantization.pt2e.port_metadata_passrZtorch.ao.quantization.quantizerrrrrr r r Ztorch.fxr r Z"torch.fx.passes.infra.pass_managerrZ pt2e.preparerZpt2e.qat_utilsrrZpt2e.representationrZ pt2e.utilsrrrZ quantize_fxr__all__rropsZquantized_decomposedZquantize_per_tensordefaultZtensorZquantize_per_channelZ pt2e_quantZquantize_affiner/r4r0rr*r*r*r+sX   $     O N