o Zh@s>dZddlmZddlmZddlmZ    dddZdS) z9VPTQ (Vector Post-Training Quantization) integration fileN)init_empty_weights) VQuantLinearFcCsf|sdgn|}|D]\}}|durg}||d|}|j}|j} t|tjr||vr|| vs9|d|vr| |dpF||dd} t B|j } |j } t | | | d| d| d| d| d | d | d | d | d dd|j dud|j|<d}|j|dWdn1swYtt|dkrt|||||d\} }|dq ||fS)aw Public method that recursively replaces the Linear layers of the given model with VPTQ quantized layers. `accelerate` is needed to use this method. Returns the converted model and a boolean that indicates if the conversion has been successful or not. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`VptqConfig`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`List[`str`]`, *optional*, defaults to `["lm_head"]`): Names of the modules to not convert in `VQuantLinear`. In practice we keep the `lm_head` in full precision for numerical stability reasons. current_key_name (`list`, *optional*): A list that contains the current key name. This is used for recursion and should not be passed by the user. has_been_replaced (`bool`, *optional*): A boolean that indicates if the conversion has been successful or not. This is used for recursion and should not be passed by the user. Zlm_headN. vector_lens num_centroidsnum_res_centroids group_num group_size outlier_sizeindices_as_float enable_norm enable_permTF) rrrr r r r r rZis_indice_packedZenable_proxy_errorbiasr)quantization_configmodules_to_not_convertcurrent_key_namehas_been_replaced)Znamed_childrenappendjoinshared_layer_configconfig_for_layers isinstancennZLineargetr in_features out_featuresrrZ_modulesZrequires_grad_lenlistchildrenreplace_with_vptq_linearpop)modelrrrrnamemoduleZ layer_namerrZ layer_paramsrr_r&M/var/www/auris/lib/python3.10/site-packages/transformers/integrations/vptq.pyr s^     r )NNNF)__doc__Ztorch.nnrZ acceleraterZvptqrr r&r&r&r's