o wZh @sUdZddlZddlmZmZddlZgZeee d<dej de fddZ d ej d e dej fd d Zd e deedefddZgdZdee ejfde fddZ    ddej dej dej deej fddZdS)zCDefines utilities for interacting with scaled_dot_product_attentionN)OptionalUnion__all__tensorsreturncGstdd|DS)z0Returns True if any of the tensors requires gradcss|]}|jVqdS)N)Z requires_grad).0tr H/var/www/auris/lib/python3.10/site-packages/torch/nn/attention/_utils.py sz'_input_requires_grad..)any)rr r r _input_requires_grad sr inpt_tensorog_sizecCs"|d|kr|dd|fS|S)z'Handles the unpad of the last dimension.N)size)rrr r r _postprocess_flash_outputsr head_dim_sizescalecCs|dur|Sdt|S)z For FlashAttention we pad the head dimension to be a multiple of 8 so we need to scale the output by the original head size and not the padded. Ng?)mathsqrt)rrr r r _calculate_scalesr)  @iincCs|tvS)z:Returns true if the head dim is supported by FlexAttention)_SUPPORTED_HEAD_DIMS)r r r r _supported_head_dim%sr"Fquerykeyvalue attn_maskcCs|j|jks |j|jkrtd|jd|jd|jd|j|jks)|j|jkr:td|jd|jd|jd|dksL|dksL|dkr`td |d |d |ddS) NzLExpected query, key, and value to have the same dtype, but got query.dtype: z , key.dtype: z, and value.dtype: z instead.zSExpected query, key, and value to have the same device type, but got query.device: z, key.device: z, and value.device: rzUExpected query, key, and value to all be at least 2 dimensional, but got query.dim: z , key.dim: z and value.dim: )Zdtype ValueErrorZdevicedim)r$r%r&r'Z dropout_pZ is_causalrr r r _validate_sdpa_input*s> $r*)Nr#FN)__doc__rtypingrrZtorchrliststr__annotations__ZTensorboolr intrfloatrr!ZSymIntr"r*r r r r s.