o cZh-@s,dZddlZGdddZGdddZdS)aN Helper classes for working with low precision floating point types that align with the opencompute (OCP) microscaling (MX) specification. * MXFP4Tensor: 4-bit E2M1 floating point data * MXScaleTensor: 8-bit E8M0 floating point data Reference: https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf Nc@s>eZdZdddZddZddZdd Zd d Zd d ZdS) MXFP4TensorNcCj||_|durt|tjsJd|j|_|||_dS|dur1t|tr+||_dS|f|_dStd)at Tensor class for working with four bit E2M1 floating point data as defined by the opencompute microscaling specification. Parameters: - data: A torch tensor of float32 numbers to convert to fp4e2m1 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. N%Parameter data must be a torch tensor.Either parameter data or size must be provided device isinstancetorchZTensor _from_floatdatatuplesize ValueErrorselfr r rr@/var/www/auris/lib/python3.10/site-packages/triton/tools/mxfp.py__init__s  zMXFP4Tensor.__init__cCsptjdd|jtj|jd}tjdd|jtj|jd}tjdd|jtj|jd}|d>|d>B|Btj|_|S)Nrr dtyper)r randintr uint8rtyper )rSEMrrrrandom#s zMXFP4Tensor.randomcCs |tjks Jd|j}|d?d@|}|d?d@|}|d@|}t|}|dk|dk@}|}|rs||} ||} ||} td| } t| dk| | d} t| dk| dd| d}| td| |}|||<|||dk@d9<|tjS) z Convert fp4e2m1 data to float32. Returns: - A torch tensor of type dtype representing the fp4e2m1 data. zCCurrently only float32 is supported for fp4e2m1 to float conversionrrr??r)r float32r rZ zeros_likeanypowwhere)rrr rrrvalueis_zeroZ non_zero_maskZS_nzZE_nzZM_nzsignexponentZmantissaZvalue_nzrrrto+s&   zMXFP4Tensor.tocCsTt|tj}t|}|dk}t|t|B}tjgdtj|jd}tjddgtj|jd}g}g} g} |D]M} | dkrcd} |D]} | d}|d| }| || | | | qFq<| d} |D]} d| d}|d| }| || | | | qkq| d>B| BtjS)a5 Convert float32 numbers to mxf4 e2m1 format. * No encodings are reserved for Inf or NaN in mxf4. * Conversion from float supports roundTiesToEven rounding mode. * If a value exceeds the mxf4 representable range after rounding, clamps to the maximum mxf4 magnitude, preserving the sign. * If a value has magnitude less than the minimum subnormal magnitude in mxf4 after rounding, converts to zero. Parameters: - values: A torch tensor of float32 numbers to convert to fp4 format. r)rrrrrrrr"rr#r!T)dimZkeepdimgư>r.r)r Zsignbitrrabsisnanisinftensorrappenditemr$viewshapeZ unsqueezemaxminsumexpandint32Zargmin)rvaluesrZ abs_valuesr) is_invalidZE_bitsZM_bitsZcandidate_valuesZ candidate_EZ candidate_Mrr+rZ significandr( candidatesZabs_values_flatNZabs_values_expandedZmax_candidate_valueerrorsZ min_errors_Zis_tieZM_bits_expandedZ tie_breakerZ best_indicesZ E_selectedZ M_selectedrrrr Nsd                   zMXFP4Tensor._from_floatc Cs|j}d|kr|jksJdJd||}|dd}|ddkrIdgd|j}|j|ddd}d||<tjjj||ddd}t|j}|||<| |dd|j |}| |dd}| |dd} | d>|B} | S)a Packs two e2m1 elements into a single uint8 along the specified dimension. Parameters: - dim: The dimension along which to pack the elements. Returns: - A torch tensor of dtype uint8 with two e2m1 elements packed into one uint8. rzHThe dimension to pack along is not within the range of tensor dimensionsrrZconstant)moder(r) r ndimr r nnZ functionalpadlistr7insertreshapeselect) rr.r Zsize_along_dimZnew_size_along_dimZ pad_sizesZ pad_index new_shapelowhighpackedrrrto_packed_tensors*       zMXFP4Tensor.to_packed_tensorc Cs|d?d@}|d@}tj||f|dd}t|j}|d|||dg||dd}|j|} ||ddkrStdg| j} td||| |<| t| } | tj S)a Unpacks a tensor where two fp4 elements are packed into a single uint8. Parameters: - packed_tensor: The packed tensor - dim: The dimension along which the tensor was packed. - original_shape: The shape of the original tensor before packing. Returns: - A tensor with the original data unpacked into uint8 elements containing one fp4e2m1 element in the least significant bits. rrr/Nrr) r stackrGr7rIslicerDr rr) rZ packed_tensorr.Zoriginal_shaperMrLZstackedr7rKr indicesrrrunpack_packed_tensors  *   z MXFP4Tensor.unpack_packed_tensorNNN) __name__ __module__ __qualname__rr r,r rOrTrrrrr s #X #rc@s0eZdZd ddZd ddZddZdd ZdS) MXScaleTensorNcCr)a6 Tensor class for working with microscaling E8M0 block scale factors. Parameters: - data: A torch tensor of float32 numbers to convert to fp8e8m0 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. Nrrrrrrrrs  zMXScaleTensor.__init__c Csd}|durdntdttt||}|durdntdtdttt||}||ks7Jdtj||d|jtj|j d}||_ |S)zp Generate random E8M0 data within a specified range. * Excludes the NaN encoding (255). Nrz&Low must be less than or equal to highrr) r8intr log2r3r9rr rrr )rrLrMZbiasZ min_exponentZ max_exponentrrrrr s*0zMXScaleTensor.randomcCs^|tjks Jd|j|}|dk}|}d||<|d}td|}tj||<||S)NzBCurrently only float32 is supported for f8e8m0 to float conversionrrZg@)r r$r rcloner&nan)rrr is_nane_biaseder(rrrr, s    zMXScaleTensor.toc Cstj|tj|jd}t|t|B|dkB}d||<||}tt|}|d}|tj }t |dd}|tj||<|S)aO Convert float32 numbers to E8M0 format. * Values <= 0, NaNs, and Infs are converted to the NaN encoding (255). * Positive values are converted by computing the floor of log2(value) to get the exponent. Parameters: - values: A torch tensor of float32 numbers to convert to E8M0 format. r-rr^rZr[) r Z empty_likerrr1r2floorr]rr<clamp) rr=resultr>Z valid_valuesrcrbZ e_biased_intZe_biased_clampedrrrr s   zMXScaleTensor._from_floatrU)NN)rVrWrXrr r,r rrrrrYs    rY)__doc__r rrYrrrrs ^