o Zh@s ddlmZddZddZdS))ceilcsttd|ddt|Dg}D]}|dkr(||vr(||qfddD}fddD}t|dkrItdt|t|dkrWtdt|t|dkretd t|dS) NrcSsg|] }|D]}|qqSr).0ZsublistitemrrV/var/www/auris/lib/python3.10/site-packages/transformers/utils/model_parallel_utils.py sz%assert_device_map..cg|]}|vr|qSrrri)device_map_blocksrrrcr rrr )blocksrrrr zDuplicate attention blocks specified in device_map. Attention blocks must be specified to one device. These attention blocks were specified more than once: zThere are attention blocks for this model that are not specified in the device_map. Add these attention blocks to a device on the device_map: zdThe device_map contains more attention blocks than this model has. Remove these from the device_map:)listrangevaluescountappendlen ValueErrorstr)Z device_mapZ num_blocksZduplicate_blocksr Zmissing_blocksZ extra_blocksr)rr rassert_device_maps8    rcsJtt|tt|t|fddtd|D}tt||S)zEReturns a dictionary of layers distributed evenly across all devices.csg|] }||qSrrr ZlayersZn_blocksrrr5sz"get_device_map..r)rrintrrdictzip)Zn_layersZdevicesZ layers_listrrrget_device_map1s rN)mathrrrrrrrs