a h@sUdZddlZddlmZddlZgZeeed<ej e dddZ ej e ej dd d Z e eeed d d Zdej ej ej eej dddZdS)zCDefines utilities for interacting with scaled_dot_product_attentionN)Optional__all__)tensorsreturncGstdd|DS)z0Returns True if any of the tensors requires gradcss|] }|jVqdS)N)Z requires_grad).0trG/var/www/auris/lib/python3.9/site-packages/torch/nn/attention/_utils.py z'_input_requires_grad..)any)rrrr _input_requires_grad sr ) inpt_tensorog_sizercCs"|d|kr|dd|fS|S)z'Handles the unpad of the last dimension.N)size)rrrrr _postprocess_flash_outputsr) head_dim_sizescalercCs|dur |Sdt|S)z For FlashAttention we pad the head dimension to be a multiple of 8 so we need to scale the output by the original head size and not the padded. Ng?)mathsqrt)rrrrr _calculate_scalesrF)querykeyvalue attn_maskcCs|j|jks|j|jkr:td|jd|jd|jd|j|jksR|j|jkrttd|jd|jd|jd|dks|dks|dkrtd |d |d |ddS) NzLExpected query, key, and value to have the same dtype, but got query.dtype: z , key.dtype: z, and value.dtype: z instead.zSExpected query, key, and value to have the same device type, but got query.device: z, key.device: z, and value.device: zUExpected query, key, and value to all be at least 2 dimensional, but got query.dim: z , key.dim: z and value.dim: )Zdtype ValueErrorZdeviceZdim)rrrrZ dropout_pZ is_causalrrrr _validate_sdpa_input#s< $r)NrFN)__doc__rtypingrZtorchrliststr__annotations__ZTensorboolr intrfloatrrrrrr s"