o Zh@s@dZddlmZmZmZerddlmZ    dddZdS) z7SpQR (Sparse-Quantized Representation) integration file)is_accelerate_availableis_spqr_availableis_torch_availableNFc Cs|durg}trddlm}trddlm}|D]\}}|dur&g}||t|t j rd |d|vr|d |} |j } | } | d| voe| d| voe| d | voe| d | v} | sptd | d | | d} | | d}| | d }| | d }|j}|j}|j|||j|j|j| |||d |j|<d}t||j|_|j|dWdn1swYn tt|dkrt|||||d\}}|dq||fS)af Public method that recursively replaces the Linear layers of the given model with SpQR quantized layers. `accelerate` is needed to use this method. Returns the converted model and a boolean that indicates if the conversion has been successful or not. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`SpQRConfig`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`list[str]`, *optional*): A list of nn.Linear weights to not convert. If a parameter path is in the list (e.g. `lm_head.weight`), the corresponding module will not be converted. current_key_name (`list`, *optional*): A list that contains the current key name. This is used for recursion and should not be passed by the user. has_been_replaced (`bool`, *optional*): A boolean that indicates if the conversion has been successful or not. This is used for recursion and should not be passed by the user. Nr)init_empty_weights)QuantizedLinear.z.weightz.dense_weights.shapez.row_offsets.shapez.col_vals.shapez.in_perm.shapezJThe SpQR quantization config does not contain the shape configuration for zG. This indicates that the configuration is either invalid or corrupted.) rowscolsbitsbeta1beta2dense_weights_shaperow_offsets_shapecol_vals_shape in_perm_shapeTF)quantization_configmodules_to_not_convertcurrent_key_namehas_been_replaced)rZ acceleraterrZ spqr_quantrZnamed_childrenappend isinstancennZLinearjoinshapeskeys ValueError in_features out_featuresZcreate_placehodlerr r r Z_modulestypeZ source_clsZrequires_grad_lenlistchildrenreplace_with_spqr_linearpop)modelrrrrrrnamemoduleZ tensor_namerZ shapes_keysZ shapes_validrrrrrr_r*M/var/www/auris/lib/python3.10/site-packages/transformers/integrations/spqr.pyr$s|          . r$)NNNF)__doc__utilsrrrZtorch.nnrr$r*r*r*r+s