a kº”hÙ-ã@s,dZddlZGdd„dƒZGdd„dƒZdS)aN Helper classes for working with low precision floating point types that align with the opencompute (OCP) microscaling (MX) specification. * MXFP4Tensor: 4-bit E2M1 floating point data * MXScaleTensor: 8-bit E8M0 floating point data Reference: https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf éNc@s>eZdZddd„Zdd„Zdd„Zdd „Zd d „Zd d „ZdS)Ú MXFP4TensorNcCsd||_|dur8t|tjƒs"Jdƒ‚|j|_| |¡|_n(|durXt|tƒrN|n|f|_ntdƒ‚dS)at Tensor class for working with four bit E2M1 floating point data as defined by the opencompute microscaling specification. Parameters: - data: A torch tensor of float32 numbers to convert to fp4e2m1 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. Nú%Parameter data must be a torch tensorú.Either parameter data or size must be provided© ÚdeviceÚ isinstanceÚtorchZTensorÚ _from_floatÚdataÚtupleÚsizeÚ ValueError©Úselfr r r©rú?/var/www/auris/lib/python3.9/site-packages/triton/tools/mxfp.pyÚ__init__s zMXFP4Tensor.__init__cCsptjdd|jtj|jd}tjdd|jtj|jd}tjdd|jtj|jd}|d>|d>B|B tj¡|_|S)Nré©r Údtyperééé)rÚrandintr Úuint8rÚtyper )rÚSÚEÚMrrrÚrandom#s zMXFP4Tensor.randomcCs |tjksJdƒ‚|j}|d?d@ |¡}|d?d@ |¡}|d@ |¡}t |¡}|dk|dk@}|}| ¡ræ||} ||} ||} t d| ¡} t | dk| | d¡} t | dk| dd| d¡}| t d| ¡|}|||<|||dk@d9<| tj¡S) zŠ Convert fp4e2m1 data to float32. Returns: - A torch tensor of type dtype representing the fp4e2m1 data. zCCurrently only float32 is supported for fp4e2m1 to float conversionrrréÿÿÿÿçà?çð?r)rÚfloat32r rZ zeros_likeÚanyÚpowÚwhere)rrr rrrÚvalueÚis_zeroZ non_zero_maskZS_nzZE_nzZM_nzÚsignÚexponentZmantissaZvalue_nzrrrÚto+s&  zMXFP4Tensor.tocCsVt |¡ tj¡}t |¡}|dk}t |¡t |¡B}tjgd¢tj|jd}tjddgtj|jd}g}g} g} |D]š} | dkrÆd} |D]6} | d}|d| }|  |¡|   | ¡|   | ¡qŒqx|   ¡d} |D]:} d| d}|d| }|  |¡|   | ¡|   | ¡qÖqxtj|tj |jd}tj| tj|jd} tj| tj|jd} |  d¡}|j d}| d¡}| ¡  ¡}|||  d¡<t || d¡¡}tj|dd d \}}||k}| ¡dkrô|  d¡ |d¡}|dk tj¡}||d }tj|dd }| |}| |}|  |j ¡} |  |j ¡} d| |<d| |<|d >| d>B| B tj¡S)a5 Convert float32 numbers to mxf4 e2m1 format. * No encodings are reserved for Inf or NaN in mxf4. * Conversion from float supports roundTiesToEven rounding mode. * If a value exceeds the mxf4 representable range after rounding, clamps to the maximum mxf4 magnitude, preserving the sign. * If a value has magnitude less than the minimum subnormal magnitude in mxf4 after rounding, converts to zero. Parameters: - values: A torch tensor of float32 numbers to convert to fp4 format. r)rrrr©rrrr!rr"r T)ÚdimZkeepdimgíµ ÷Æ°>©r-r)rZsignbitrrÚabsÚisnanÚisinfÚtensorrÚappendÚitemr#ÚviewÚshapeZ unsqueezeÚmaxÚminÚsumÚexpandÚint32Zargmin)rÚvaluesrZ abs_valuesr(Ú is_invalidZE_bitsZM_bitsZcandidate_valuesZ candidate_EZ candidate_Mrr*rZ significandr'Ú candidatesZabs_values_flatÚNZabs_values_expandedZmax_candidate_valueÚerrorsZ min_errorsÚ_Zis_tieZM_bits_expandedZ tie_breakerZ best_indicesZ E_selectedZ M_selectedrrrr Ns`                 zMXFP4Tensor._from_floatc Csè|j}d|kr|jks&nJdƒ‚| |¡}|dd}|ddkrŒdgd|j}|j|ddd}d||<tjjj||ddd}t|jƒ}|||<|  |dd¡|j |Ž}|  |dd¡}|  |dd¡} | d>|B} | S)a Packs two e2m1 elements into a single uint8 along the specified dimension. Parameters: - dim: The dimension along which to pack the elements. Returns: - A torch tensor of dtype uint8 with two e2m1 elements packed into one uint8. rzHThe dimension to pack along is not within the range of tensor dimensionsrrZconstant)Úmoder'r) r Úndimr rÚnnZ functionalÚpadÚlistr6ÚinsertÚreshapeÚselect) rr-r Zsize_along_dimZnew_size_along_dimZ pad_sizesZ pad_indexÚ new_shapeÚlowÚhighÚpackedrrrÚto_packed_tensor¦s& ÿ      zMXFP4Tensor.to_packed_tensorc Cs²|d?d@}|d@}tj||f|dd}t|jƒ}|d|…||dg||dd…}|j|Ž} ||ddkr¦tdƒg| j} td||ƒ| |<| t| ƒ} |  tj ¡S)aÅ Unpacks a tensor where two fp4 elements are packed into a single uint8. Parameters: - packed_tensor: The packed tensor - dim: The dimension along which the tensor was packed. - original_shape: The shape of the original tensor before packing. Returns: - A tensor with the original data unpacked into uint8 elements containing one fp4e2m1 element in the least significant bits. rérr.Nrr) rÚstackrFr6rHÚslicerCr rr) rZ packed_tensorr-Zoriginal_shaperLrKZstackedr6rJr ÚindicesrrrÚunpack_packed_tensorÉs  *  z MXFP4Tensor.unpack_packed_tensor)NNN) Ú__name__Ú __module__Ú __qualname__rrr+r rNrSrrrrr s  #X#rc@s0eZdZd dd„Zd dd„Zdd„Zdd „ZdS) Ú MXScaleTensorNcCsd||_|dur8t|tjƒs"Jdƒ‚|j|_| |¡|_n(|durXt|tƒrN|n|f|_ntdƒ‚dS)a6 Tensor class for working with microscaling E8M0 block scale factors. Parameters: - data: A torch tensor of float32 numbers to convert to fp8e8m0 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. Nrrrrrrrrës zMXScaleTensor.__init__c Cs–d}|durdntdtt t |¡¡ƒ|ƒ}|dur:dn"tdtdtt t |¡¡ƒ|ƒƒ}||ksnJdƒ‚tj||d|jtj|j d}||_ |S)zp Generate random E8M0 data within a specified range. * Excludes the NaN encoding (255). éNréþz&Low must be less than or equal to highrr) r7ÚintrÚlog2r2r8rr rrr )rrKrLZbiasZ min_exponentZ max_exponentrrrrrþs*0zMXScaleTensor.randomcCs^|tjksJdƒ‚|j |¡}|dk}| ¡}d||<|d}t d|¡}tj||<| |¡S)NzBCurrently only float32 is supported for f8e8m0 to float conversionéÿrrXg@)rr#r rÚcloner%Únan)rrr Úis_nanÚe_biasedÚer'rrrr+ s   zMXScaleTensor.toc CsŠtj|tj|jd}t |¡t |¡B|dkB}d||<||}t t |¡¡}|d}| tj ¡}t  |dd¡}| tj¡||<|S)aO Convert float32 numbers to E8M0 format. * Values <= 0, NaNs, and Infs are converted to the NaN encoding (255). * Positive values are converted by computing the floor of log2(value) to get the exponent. Parameters: - values: A torch tensor of float32 numbers to convert to E8M0 format. r,rr\rXrY) rZ empty_likerrr0r1Úfloorr[rr;Úclamp) rr<Úresultr=Z valid_valuesrar`Z e_biased_intZe_biased_clampedrrrr s   zMXScaleTensor._from_float)NNN)NN)rTrUrVrrr+r rrrrrWés   rW)Ú__doc__rrrWrrrrÚs^