o vZh8 @sUdZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZddlmZddlmZmZmZm Z m!Z!m"Z"d d l#m$Z$m%Z%m&Z&m'Z'd d l(m)Z)m*Z*e rdd l+m,Z,m-Z-ddl.m/Z/e0Z1ee0e fZ2ee3efZ4ee4e2e1fZ5ddhZ6e7e8Z9eGdddZ:eGdddZ;ddZede5dee4fddZ?ededdeedfddZ?edee5deee4ddffd dZ?de5de0fd!d"Z@d#e0dd$fd%d&ZAde3defd'd(ZBde3defd)d*ZCde3dd$fd+d,ZDd-ee3efdefd.d/ZEd0ee3d1eFdeee0ee*ffd2d3ZGd0ee3d1eFdeee0ee*ffd4d5ZHd6e3d1eFdeee0e*ffd7d8ZId9ee3dee)fd:d;ZJd9ee3dee)fdd?ZLd@dAd-dBdee3fdCdDZMiZNeee0ee0feOdE<dFee0dGee0ddfdHdIZPdFee0dee0fdJdKZQdLedefdMdNZRdOee0dPee0de fdQdRZSdS)SzEContains utilities used by both the sync and async inference clients.N)contextmanager) dataclass)Path) TYPE_CHECKINGAny AsyncIterableBinaryIOContextManagerDict GeneratorIterableListLiteralNoReturnOptionalUnionoverload) HTTPError)GenerationErrorIncompleteGenerationErrorOverloadedErrorTextGenerationError UnknownErrorValidationError) get_sessionis_aiohttp_availableis_numpy_availableis_pillow_available)ChatCompletionStreamOutputTextGenerationStreamOutput)ClientResponse ClientSessionImagez text-to-imagezimage-to-imagec@s\eZdZUeed<eed<eeed<eeeeefed<ee ed<eee fed<dS)RequestParametersurltaskmodeljsondataheadersN) __name__ __module__ __qualname__str__annotations__rrr r ContentTrr3r3P/var/www/auris/lib/python3.10/site-packages/huggingface_hub/inference/_common.pyr&Hs   r&c@s2eZdZUdZeed<eed<eed<eed<dS) ModelStatusas This Dataclass represents the model status in the HF Inference API. Args: loaded (`bool`): If the model is currently loaded into HF's Inference API. Models are loaded on-demand, leading to the user's first request taking longer. If a model is loaded, you can be assured that it is in a healthy state. state (`str`): The current state of the model. This can be 'Loaded', 'Loadable', 'TooBig'. If a model's state is 'Loadable', it's not too big and has a supported backend. Loadable models are automatically loaded when the user first requests inference on the endpoint. This means it is transparent for the user to load a model, except that the first call takes longer to complete. compute_type (`Dict`): Information about the compute resource the model is using or will use, such as 'gpu' type and number of replicas. framework (`str`): The name of the framework that the model was built with, such as 'transformers' or 'text-generation-inference'. ZloadedstateZ compute_typeZ frameworkN)r-r.r/__doc__boolr1r0r r3r3r3r4r5Ss  r5cCststdddl}|S)NzMPlease install aiohttp to use `AsyncInferenceClient` (`pip install aiohttp`).r)r ImportErroraiohttp)r:r3r3r4_import_aiohttptr;cCststdddl}|S)z.Make sure `numpy` is installed on the machine.zGPlease install numpy to use deal with embeddings (`pip install numpy`).rN)rr9numpy)r=r3r3r4 _import_numpy}r<r>cCststdddlm}|S)z,Make sure `PIL` is installed on the machine.zPlease install Pillow to use deal with images (`pip install Pillow`). If you don't want the image to be post-processed, use `client.post(...)` and get the raw response from the server.rr$)rr9ZPILr%r$r3r3r4_import_pil_images  r?contentreturncCdSNr3r@r3r3r4_open_as_binaryrEcCrBrCr3rDr3r3r4rErFccst|tr2|ds|dr"td|t|jVdSt|}| s2t d|dt|trZtd|| d }|VWddS1sSwYdS|VdS) zOpen `content` as a binary file, either from a URL, a local path, or raw bytes. Do nothing if `content` is None, TODO: handle a PIL.Image as input TODO: handle base64 as input zhttps://zhttp://zDownloading content from NzFile not found at z. If `data` is a string, it must either be a URL or a path to a local file. To pass raw content, please encode it as bytes first.zOpening content from rb) isinstancer0 startswithloggerdebugrgetr@rexistsFileNotFoundErroropen)r@fr3r3r4rEs"    " cCsPt|}t|tr |n|}t|WdS1s!wYdS)z\Encode a raw file (image, audio) into base64. Can be bytes, an opened file, a path or a URL.N)rErHbytesreadbase64 b64encodedecode)r@r+Z data_as_bytesr3r3r4 _b64_encodes  $rV encoded_imager%cCst}|tt|S)z/Parse a base64-encoded string into a PIL Image.)r?rOioBytesIOrS b64decode)rWr%r3r3r4 _b64_to_imagesr[cCt|S)acParse bytes from a Response object into a Python list. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_dict` and will not complain if the returned data is a dictionary. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. r*loadsrUrDr3r3r4_bytes_to_listr_cCr\)acParse bytes from a Response object into a Python dictionary. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_list` and will not complain if the returned data is a list. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. r]rDr3r3r4_bytes_to_dictr`racCst}|t|S)zParse bytes from a Response object into a PIL Image. Expects the response body to be raw bytes. To deal with b64 encoded images, use `_b64_to_image` instead. )r?rOrXrY)r@r%r3r3r4_bytes_to_imagesrbresponsecCst|tr t|S|SrC)rHrQr*r^)rcr3r3r4_as_dictsrdbytes_output_as_linesdetailsc csD|D]}zt||}Wn tyYdSw|dur|VqdS)z*Used in `InferenceClient.text_generation`.N%_format_text_generation_stream_output StopIterationrerf byte_payloadoutputr3r3r4 _stream_text_generation_responses rmc CsN|2z 3dHW}zt||}Wn tyYdSw|dur#|Vq6dS)z/Used in `AsyncInferenceClient.text_generation`.Nrgrjr3r3r4&_async_stream_text_generation_responses rnrkcCs~|dsdS|dkrtd|d}t|dd}|ddur2t |d|dt |}|s=|j j S|S Nsdata:s data: [DONE]z[DONE] signal received.zutf-8zdata:z/nerror error_type)rIstriprirUr*r^lstriprstriprL_parse_text_generation_errorr!parse_obj_as_instancetokentext)rkrfpayload json_payloadrlr3r3r4rhs    rh bytes_linesc csB|D]}zt|}Wn tyYdSw|dur|VqdS)zFUsed in `InferenceClient.chat_completion` if model is served with TGI.N%_format_chat_completion_stream_outputrir{itemrlr3r3r4 _stream_chat_completion_response)s  rc CsL|2z3dHW}zt|}Wn tyYdSw|dur"|Vq6dS)z/Used in `AsyncInferenceClient.chat_completion`.Nr|r~r3r3r4&_async_stream_chat_completion_response6s  rcCsn|dsdS|dkrtd|d}t|dd}|ddur2t |d|dt |Sro) rIrrrirUr*r^rsrtrLrur rv)rkryrzr3r3r4r}Cs    r}clientr#r"cCs4|j2z 3dHW}|Vq6|IdHdSrC)r@rrclose)rrcrkr3r3r4_async_yield_fromXs  r#_UNSUPPORTED_TEXT_GENERATION_KWARGSr)unsupported_kwargscCst|g|dSrC)r setdefaultextend)r)rr3r3r4'_set_unsupported_text_generation_kwargsssrcCs t|gSrC)rrL)r)r3r3r4'_get_unsupported_text_generation_kwargsws r http_errorcCs^zt|ddp |j}|d}|d}Wn ty |w|dur-t||}|||)z Try to parse text-generation-inference error message and raise HTTPError in any case. Args: error (`HTTPError`): The HTTPError that have been raised. Zresponse_error_payloadNrprq)getattrrcr*rL Exceptionru)rryrprq exceptionr3r3r4raise_text_generation_errors    rrprqcCsH|dkrt|S|dkrt|S|dkrt|S|dkr t|St|S)NZ generationZincomplete_generationZ overloadedZ validation)rrrrr)rprqr3r3r4rusru)Tr7rSrXr*logging contextlibr dataclassesrpathlibrtypingrrrrr r r r r rrrrrrequestsrZhuggingface_hub.errorsrrrrrrutilsrrrrZ_generated.typesr r!r:r"r#Z PIL.Imager%r0ZUrlTZPathTrQZBinaryTr2ZTASKS_EXPECTING_IMAGES getLoggerr-rJr&r5r;r>r?rErVr[r_rarbrdr8rmrnrhrrr}rrr1rrrrur3r3r3r4s   @              &             "