o Zh%@sUddlZddlZddlZddlZddlmZddlmZddlm Z ddl Z ddl m Z dgZeeZdaeZdefddZerNe jsNeder ddlZddlm mZdd lmZdd l m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAd d lBmCZCmDZDmEZEd d lCTd dlDmFZFd dlGmHZHd dlImJZJeeKeeLeLfddfeMd<egd7ZeeCjeDjZ    dddZNddZOeFjPdddddfddZQeCjRddZSdSdS)N) Generator) timedelta)urlparse is_availablereturncCs ttjdS)N _rpc_init)hasattrtorch_Cr r M/var/www/auris/lib/python3.10/site-packages/torch/distributed/rpc/__init__.pyrs z*Failed to initialize torch.distributed.rpc)Store)!_cleanup_python_rpc_handler_DEFAULT_INIT_METHOD_DEFAULT_NUM_WORKER_THREADS_DEFAULT_RPC_TIMEOUT_SEC)_delete_all_user_and_unforked_owner_rrefs_destroy_rref_context_disable_jit_rref_pickle'_disable_server_process_global_profiler_enable_jit_rref_pickle&_enable_server_process_global_profiler_get_current_rpc_agent_invoke_remote_builtin_invoke_remote_python_udf_invoke_remote_torchscript_invoke_rpc_builtin_invoke_rpc_python_udf_invoke_rpc_torchscript_is_current_rpc_agent_set_reset_current_rpc_agent_rref_context_get_debug_info_set_and_start_rpc_agent_set_profiler_node_id_set_rpc_timeout _TensorPipeRpcBackendOptionsBase_UNSET_RPC_TIMEOUTenable_gil_profilingget_rpc_timeoutPyRRefRemoteProfilerManagerRpcAgentRpcBackendOptionsTensorPipeAgent WorkerInfo)apibackend_registry functions)*) BackendType)TensorPipeRpcBackendOptions)_server_process_global_profilerendezvous_iterator)init_rpcr4r5cCsntjd|durt|tjstd|dur!t|ts!td|durP|durPtD]}t|tt |r;|}nq+td||tj krPt dd|i|durWtj }|dur`t |}|sit ||}nt j|j||datt\}}}|t|jd tt td t|}td 7aWdn1swYt|t|t||||||dS) ak Initializes RPC primitives such as the local RPC agent and distributed autograd, which immediately makes the current process ready to send and receive RPCs. Args: name (str): a globally unique name of this node. (e.g., ``Trainer3``, ``ParameterServer2``, ``Master``, ``Worker1``) Name can only contain number, alphabet, underscore, colon, and/or dash, and must be shorter than 128 characters. backend (BackendType, optional): The type of RPC backend implementation. Supported values is ``BackendType.TENSORPIPE`` (the default). See :ref:`rpc-backends` for more information. rank (int): a globally unique id/rank of this node. world_size (int): The number of workers in the group. rpc_backend_options (RpcBackendOptions, optional): The options passed to the RpcAgent constructor. It must be an agent-specific subclass of :class:`~torch.distributed.rpc.RpcBackendOptions` and contains agent-specific initialization configurations. By default, for all agents, it sets the default timeout to 60 seconds and performs the rendezvous with an underlying process group initialized using ``init_method = "env://"``, meaning that environment variables ``MASTER_ADDR`` and ``MASTER_PORT`` need to be set properly. See :ref:`rpc-backends` for more information and find which options are available. ztorch.distributed.init_rpcNz0Argument backend must be a member of BackendTypezEArgument rpc_backend_options must be an instance of RpcBackendOptionsz$Could not infer backend for options zRPC was initialized with no explicit backend but with options corresponding to %(backend)s, hence that backend will be used instead of the default BackendType.TENSORPIPE. To silence this warning pass `backend=%(backend)s` explicitly.backend)rank world_size)secondsZ rpc_prefix_r/)r r Z_log_api_usage_once isinstancer1r4 TypeErrorr,typeZconstruct_rpc_backend_options TENSORPIPEloggerwarningdistZ_create_store_from_optionsZ rendezvousZ init_methodr7nextZ set_timeoutrZ rpc_timeout_init_counter_lockZ PrefixStorestr _init_counter dist_autograd_initr#_init_rpc_backend)namer:r;r<rpc_backend_optionsZcandidate_backendstore_r r r r8Tsh #      r8c Csj|tj|tj|t|tj|tjtdf|ti}| D]\}}t ||s2t d|d|dt|qdS)Nz Argument z must be of type z but got type ) r1r4rDr rGnumbersIntegralr@r,itemsr> RuntimeError) r:rNrLr;r<rMZ type_mappingargZarg_typer r r _validate_rpc_argss rUcCsDt||||||trtdtj||||||d}t|dS)NzRPC is already initialized)rNrLr;r<rM)rUrrSr1Z init_backendr0Z_init_rpc_states)r:rNrLr;r<rMZ rpc_agentr r r rKs rKcCs*t}|t|t|S)N)r!updater0rZget_debug_inforI_get_debug_info)infor r r rWsrW)Nr9NN)Tloggingos threadingwarningscollections.abcrdatetimer urllib.parserr Ztorch.distributed distributedrD__all__ getLogger__name__rBrHLockrFboolrr rrSrPZtorch.distributed.autogradZautogradrIZtorch._C._distributed_c10dr Ztorch._C._distributed_rpcrrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r0r1r2r4optionsr5Zserver_process_global_profilerr6tupleint__annotations__r8rUrArKZ_require_initializedrWr r r r s\       $     w