a h4@sUdZddlZddlZddlmZmZmZmZddlZddlm Z m Z ddl m Z mZmZddlmZddlmZee eefZejZejZd agaeeegdfeefed <eZe Z!eZ"e#ej$d rej$j%Z&need d dZ&e#ej$drej$j'Z(need ddZ(ddZ)ddZ*e+dddZ,ddddZ-Gddde.Z/e+dddZ0e+ddd Z1dPeedd d!d"Z2edd#d$Z3edd%d&Z4dQeeed d'd(Z5dReeed d)d*Z6dSeeeedd-d.d/Z7e8eefdd0d1Z9eeeeegdfdd2d3d4Z:dTeeeeefd d5d6Z;ddd7d8ZdUeee8eefd d>d?Z?Gd@dAdAZ GdBdCdCZ@edDe@dEdFdGZAdVeeeej fe d dIdJZBdWe eeeej fddKdLdMZCddNlDTgdOZEdS)XzH This package enables an interface for accessing MTIA backend in python N)AnyCallableOptionalUnion)deviceTensor) _dummy_type_LazySeedTracker classproperty)Device)_get_device_indexF _queued_calls_mtia_exchangeDevice)rreturncCs|dkr dStddSNrz)PyTorch was compiled without MTIA support RuntimeErrorrrA/var/www/auris/lib/python3.9/site-packages/torch/mtia/__init__.py_exchange_device%sr_mtia_maybeExchangeDevicecCs|dkr dStddSrrrrrr_maybe_exchange_device/srcCs tdSN) _lazy_initrrrrinit5srcCs to t S)z9Return whether PyTorch's MTIA state has been initialized.) _initialized_is_in_bad_forkrrrris_initialized9sr )rcCs tjSr)torch_CZ_mtia_isInBadForkrrrrr>src CststtdrdSttr0WddStr>tdtsLtdtj dt_ t ddtDzpt D]\\}}z |Wqzty}z0dt|dd |}t||WYd}~qzd}~00qzWttdn ttd0daWdn1s 0YdS) Nis_initializingzwCannot re-initialize MTIA in forked subprocess. To use MTIA with multiprocessing, you must use the 'spawn' start methodzTorch not compiled with MTIA enabled. Ensure you have `import mtia.host_runtime.torch_mtia.dynamic_library` in your python src file and include `//mtia/host_runtime/torch_mtia:torch_mtia` as your target dependency!Tcss|]}|r|VqdSrr).0Zcallsrrr dz_lazy_init..z6MTIA call failed lazily at initialization with error: z( MTIA call was originally invoked at: )r hasattr_tls_initialization_lockrr _is_compiledAssertionErrorr!r"Z _mtia_initr#rextend_lazy_seed_trackerZ get_calls ExceptionstrjoinDeferredMtiaCallErrordelattrr)Z queued_callZorig_tracebackemsgrrrrBs8    $rc@s eZdZdS)r2N)__name__ __module__ __qualname__rrrrr2usr2cCs tjS)z*Return true if compiled with MTIA support.)r!r"Z _mtia_isBuiltrrrrr+ysr+cCsts dStdkS)z'Return true if MTIA device is availableFr)r+ device_countrrrr is_available~sr:cCs:tj|tjWdS1s,0YdS)z?Waits for all jobs in all streams on a MTIA device to complete.N)r!mtiarr"Z_mtia_deviceSynchronizerrrr synchronizesr<cCs tjS)z,Return the number of MTIA devices available.)r!r"Z_mtia_getDeviceCountrrrrr9sr9cCs tjS)z0Return the index of a currently selected device.)r!r"Z%_accelerator_hooks_get_current_devicerrrrcurrent_devicesr=cCstjt|ddS)aSReturn the currently selected :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the currently selected :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Toptional)r!r"Z_mtia_getCurrentStreamr rrrrcurrent_streams r@cCstjt|ddS)a=Return the default :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the default :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Tr>)r!r"Z_mtia_getDefaultStreamr rrrrdefault_streams rAallpython)enabledstacks max_entriesrcCsts dStj|||dS)aEnable/Disable the memory profiler on MTIA allocator Args: enabled (all or state, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). stacks ("python" or "cpp", optional). Select the stack trace to record. max_entries (int, optional). Maximum number of entries to record. N)r r!r"Z_mtia_recordMemoryHistory)rDrErFrrrrecord_memory_historysrGcCs tjS)z4Return a dictionary of MTIA memory allocator history)r!r"Z_mtia_memorySnapshotrrrrsnapshotsrH)observerrcCstj|dS)z9Attach an out-of-memory observer to MTIA memory allocatorN)r!r"Z_mtia_attachOutOfMemoryObserver)rIrrrattach_out_of_memory_observersrJcCstjt|ddS)aReturn capability of a given device as a tuple of (major version, minor version). Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). Tr>)r!r"Z_mtia_getDeviceCapabilityr rrrrget_device_capabilitysrKcCs tjS)zEmpty the MTIA device cache.)r!r"Z_mtia_emptyCacherrrr empty_cachesrLstreamcCs|dur dStj|dS)aSet the current stream.This is a wrapper API to set the stream. Usage of this function is discouraged in favor of the ``stream`` context manager. Args: stream (Stream): selected stream. This function is a no-op if this argument is ``None``. N)r!r"Z_mtia_setCurrentStreamrMrrr set_streams rOcCs t|}|dkrtj|dS)zSet the current device. Args: device (torch.device or int): selected device. This function is a no-op if this argument is negative. rN)r r!r"Z%_accelerator_hooks_set_current_devicerrrr set_devicesrPcCstjt|ddS)zReturn a dictionary of MTIA device properties Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). Tr>)r!r"Z_mtia_getDevicePropertiesr rrrrget_device_propertiessrQc@s8eZdZdZedddZddZeeeddd Zd S) rzContext-manager that changes the selected device. Args: device (torch.device or int): device index to select. It's a no-op if this argument is a negative integer or ``None``. rcCst|dd|_d|_dS)NTr>r)r idxprev_idx)selfrrrr__init__ szdevice.__init__cCstj|j|_dSr)r!r"(_accelerator_hooks_maybe_exchange_devicerRrS)rTrrr __enter__szdevice.__enter__typevalue tracebackcCstj|j|_dS)NF)r!r"rVrSrR)rTrYrZr[rrr__exit__szdevice.__exit__N)r6r7r8__doc__rrUrWr\rrrrrsrc@sJeZdZUdZeded<eddddZddZeeed d d Z d S) StreamContextaContext-manager that selects a given stream. All MTIA kernels queued within its context will be enqueued on a selected stream. Args: Stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: Streams are per-device. torch.mtia.Stream cur_streamrMcCsnd|_||_tdd|_tjs2|jdur2d|_tjs@dn tjd|_ tjs\dn tjd|_ dS)NTr) r`rNr rRr!ZjitZ is_scriptingr;rAsrc_prev_streamdst_prev_stream)rTrNrrrrU&s   zStreamContext.__init__cCs|j}|dus|jdkrdStjd|_|jj|jkrrt|j tj|j|_Wdn1sh0Ytj|dSNr) rNrRr!r;r@rarrbrO)rTr`rrrrW5s .zStreamContext.__enter__rXcCsJ|j}|dus|jdkrdS|jj|jkr8tj|jtj|jdSrc)rNrRrarr!r;rOrb)rTrYrZr[r`rrrr\Ds zStreamContext.__exit__N) r6r7r8r]r__annotations__rUrWrr\rrrrr^s  r^r_)rNrcCst|S)a,Wrap around the Context-manager StreamContext that selects a given stream. Arguments: stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: In eager mode stream is of type Stream class while in JIT it doesn't support torch.mtia.stream )r^rMrrrrNRsrNr;cCs$tjdtddtjdgtj|dS)zReturns the random number generator state as a ByteTensor. Args: device (torch.device or int, optional): The device to return the RNG state of. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). z.get_rng_state is not implemented in torch.mtia stacklevelr )Zdtyper)warningswarn UserWarningr!zerosZuint8rrrr get_rng_state]s rl) new_staterrcCstjdtdddS)aSets the random number generator state. Args: new_state (torch.ByteTensor): The desired state device (torch.device or int, optional): The device to set the RNG state. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). z.set_rng_state is not implemented in torch.mtiarerfN)rhrirj)rmrrrr set_rng_statels rn)*)rr:r r<r9r=r@rAZ memory_statsZmax_memory_allocatedZreset_peak_memory_statsrKrQrGrHrJrLrPrOrNrrnrl)N)N)N)rBrCr)N)N)r;)r;)Fr] threadingrhtypingrrrrr!rZ_devicerZ torch._utilsrr r Z torch.typesr _utilsr r0intZ _device_tEventZStreamrrlisttuplerdlocalr)Lockr*r.r(r"rrrrrr boolrrr/r2r+r:r<r9r=r@rArGdictrHrJrKrLrOrPrQr^rNrlrnZmemory__all__rrrrs|     3        :