o ZhL@s<dZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z m Z ddl mZddlmZdd lmZdd lmZdd lmZmZmZmZmZdd lmZd dlmZd dlm Z m!Z!m"Z"m#Z#d dl$m%Z%d dl&m'Z'd dl(m)Z)e*e+Z,egdZ-ee e-Z.de/fddZ0GdddZ1ddgZ2dS)zAutoProcessor class.N) OrderedDict)PretrainedConfig)get_class_from_dynamic_moduleresolve_trust_remote_code)FeatureExtractionMixin)ImageProcessingMixin)ProcessorMixin)TOKENIZER_CONFIG_FILE)FEATURE_EXTRACTOR_NAMEPROCESSOR_NAMEVIDEO_PROCESSOR_NAME cached_filelogging)BaseVideoProcessor)_LazyAutoMapping)CONFIG_MAPPING_NAMES AutoConfigmodel_type_to_module_name!replace_list_option_in_docstrings)AutoFeatureExtractor)AutoImageProcessor) AutoTokenizer)U)alignZAlignProcessor)ZaltclipZAltCLIPProcessor)ZariaZ AriaProcessor)Z aya_visionZAyaVisionProcessor)ZbarkZ BarkProcessor)ZblipZ BlipProcessor)zblip-2ZBlip2Processor)Z bridgetowerZBridgeTowerProcessor)Z chameleonZChameleonProcessor)Z chinese_clipZChineseCLIPProcessor)clapZ ClapProcessor)Zclip CLIPProcessor)ZclipsegZCLIPSegProcessor)ZclvpZ ClvpProcessor)ZcolpaliZColPaliProcessor)Zemu3Z Emu3Processor)ZflavaZFlavaProcessor)ZfuyuZ FuyuProcessor)Zgemma3ZGemma3Processor)gitZ GitProcessor)Zgot_ocr2ZGotOcr2Processor)Zgranite_speechZGraniteSpeechProcessor)zgrounding-dinoZGroundingDinoProcessor)Zgroupvitr)ZhubertWav2Vec2Processor)ZideficsZIdeficsProcessor)Zidefics2ZIdefics2Processor)Zidefics3ZIdefics3Processor)Z instructblipZInstructBlipProcessor)ZinstructblipvideoZInstructBlipVideoProcessor)ZinternvlZInternVLProcessor)ZjanusZJanusProcessor)zkosmos-2ZKosmos2Processor)Z layoutlmv2ZLayoutLMv2Processor)Z layoutlmv3ZLayoutLMv3Processor)Zllama4ZLlama4Processor)ZllavaLlavaProcessor)Z llava_nextZLlavaNextProcessor)Zllava_next_videoZLlavaNextVideoProcessor)Zllava_onevisionZLlavaOnevisionProcessor)ZmarkuplmZMarkupLMProcessor)ZmctctZMCTCTProcessor)zmgp-strZMgpstrProcessor)Zmistral3PixtralProcessor)ZmllamaZMllamaProcessor)Z moonshiner)Z oneformerZOneFormerProcessor)Zowlv2ZOwlv2Processor)ZowlvitZOwlViTProcessor)Z paligemmaZPaliGemmaProcessor)Zphi4_multimodalZPhi4MultimodalProcessor)Z pix2structZPix2StructProcessor)Zpixtralr )Z pop2pianoZPop2PianoProcessor)Z qwen2_5_omniZQwen2_5OmniProcessor)Z qwen2_5_vlZQwen2_5_VLProcessor)Z qwen2_audioZQwen2AudioProcessor)Zqwen2_vlZQwen2VLProcessor)ZsamZ SamProcessor)Zsam_hqZSamHQProcessor)Z seamless_m4tZSeamlessM4TProcessor)Zsewr)zsew-dr)Z shieldgemma2ZShieldGemma2Processor)ZsiglipZSiglipProcessor)Zsiglip2ZSiglip2Processor)Zspeech_to_textZSpeech2TextProcessor)Zspeech_to_text_2ZSpeech2Text2Processor)Zspeecht5ZSpeechT5Processor)ZtrocrZTrOCRProcessor)ZtvltZ TvltProcessor)ZtvpZ TvpProcessor)ZudopZ UdopProcessor)Z unispeechr)z unispeech-satr)Z video_llavaZVideoLlavaProcessor)ZviltZ ViltProcessor)Zvipllavar)zvision-text-dual-encoderZVisionTextDualEncoderProcessor)Zwav2vec2r)z wav2vec2-bertr)zwav2vec2-conformerr)Zwavlmr)ZwhisperZWhisperProcessor)ZxclipZXCLIPProcessor class_namec CstD]'\}}||vr+t|}td|d}zt||WSty*Yqwqtj D]}t|dd|kr?|Sq1td}t ||rOt||SdS)N.ztransformers.models__name__Z transformers) PROCESSOR_MAPPING_NAMESitemsr importlib import_modulegetattrAttributeErrorPROCESSOR_MAPPINGZ_extra_contentvalueshasattr)r! module_nameZ processorsmodule processorZ main_moduler0W/var/www/auris/lib/python3.10/site-packages/transformers/models/auto/processing_auto.pyprocessor_class_from_names$     r2c@s:eZdZdZddZeeeddZe d ddZ d S) AutoProcessora This is a generic processor class that will be instantiated as one of the processor classes of the library when created with the [`AutoProcessor.from_pretrained`] class method. This class cannot be instantiated directly using `__init__()` (throws an error). cCstd)Nz}AutoProcessor is designed to be instantiated using the `AutoProcessor.from_pretrained(pretrained_model_name_or_path)` method.)EnvironmentError)selfr0r0r1__init__szAutoProcessor.__init__c sdd}|dur tdtdddurtd|d<dd}dd}dd <d}d}fd d ttj D}| d d d d t|t fi|} | durxt j|fi\} } | dd}d| divrx| dd}|durt|tfi|} | durtj|fi\} } | dd}d| divr| dd}| durt|tfi|} | durtj|fi\} } | dd}d| divr| dd}| durt|tfi|} | dur|durtj|fi\} } | dd}d| divr| dd}|durUt|tfi|} | durUt| dd }t|} Wdn 1s;wY| dd}d| divrU| dd}|durt|tsktj|fd|i}t|dd}t |drd|j!vr|j!d}|durt"|}|du}|dupt#|t$v}t%||||}|r|rt&||fi}dd} t'j()|r|*|j|fd|iS|dur|j|fd|iSt#|t$vrt$t#|j|fiSz t+j|fd|iWSt,y8zt-j|fd|iWYSt,yYnwzt.j|fd|iWYSt,y5YnwYnwtd|d)a Instantiate one of the processor classes of the library from a pretrained model vocabulary. The processor class to instantiate is selected based on the `model_type` property of the config object (either passed as an argument or loaded from `pretrained_model_name_or_path` if possible): List options Params: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained feature_extractor hosted inside a model repo on huggingface.co. - a path to a *directory* containing a processor files saved using the `save_pretrained()` method, e.g., `./my_model_directory/`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model feature extractor should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the feature extractor files and override the cached versions if they exist. resume_download: Deprecated and ignored. All downloads are now resumed by default when possible. Will be removed in v5 of Transformers. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. return_unused_kwargs (`bool`, *optional*, defaults to `False`): If `False`, then this function returns just the final feature extractor object. If `True`, then this functions returns a `Tuple(feature_extractor, unused_kwargs)` where *unused_kwargs* is a dictionary consisting of the key/value pairs whose keys are not feature extractor attributes: i.e., the part of `kwargs` which has not been used to update `feature_extractor` and is otherwise ignored. trust_remote_code (`bool`, *optional*, defaults to `False`): Whether or not to allow for custom models defined on the Hub in their own modeling files. This option should only be set to `True` for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine. kwargs (`Dict[str, Any]`, *optional*): The values in kwargs of any keys which are feature extractor attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are *not* feature extractor attributes is controlled by the `return_unused_kwargs` keyword parameter. Passing `token=True` is required when you want to use a private model. Examples: ```python >>> from transformers import AutoProcessor >>> # Download processor from huggingface.co and cache. >>> processor = AutoProcessor.from_pretrained("facebook/wav2vec2-base-960h") >>> # If processor files are in a directory (e.g. processor was saved using *save_pretrained('./test/saved_model/')*) >>> # processor = AutoProcessor.from_pretrained("./test/saved_model/") ```use_auth_tokenNzrThe `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.tokenzV`token` and `use_auth_token` are both specified. Please set only the argument `token`.configtrust_remote_codeTZ _from_autocsi|] }|vr||qSr0r0).0keykwargsr0r1 sz1AutoProcessor.from_pretrained..F)Z _raise_exceptions_for_gated_repoZ%_raise_exceptions_for_missing_entriesZ'_raise_exceptions_for_connection_errorsprocessor_classr3auto_mapzutf-8)encodingZ code_revisionz!Unrecognized processing class in z. Can't instantiate a processor, a tokenizer, an image processor or a feature extractor for this model. Make sure the repository contains the files of at least one of those processing classes.)/popwarningswarn FutureWarningget ValueErrorinspect signaturer parameterskeysupdater r Zget_processor_dictr rZget_image_processor_dictr rZget_video_processor_dictrZget_feature_extractor_dictr openjsonload isinstancerrfrom_pretrainedr(r,rAr2typer*rrospathisdirZregister_for_auto_classr Exceptionrr)clsZpretrained_model_name_or_pathr>r7r9r:r@Zprocessor_auto_mapZcached_file_kwargsZprocessor_config_fileZ config_dict_Zpreprocessor_config_fileZtokenizer_config_filereaderZhas_remote_codeZhas_local_coder0r=r1rRsL E                              zAutoProcessor.from_pretrainedFcCstj|||ddS)a Register a new processor for this class. Args: config_class ([`PretrainedConfig`]): The configuration corresponding to the model to register. processor_class ([`ProcessorMixin`]): The processor to register. )exist_okN)r*register)Z config_classr@r[r0r0r1r\s zAutoProcessor.registerN)F) r# __module__ __qualname____doc__r6 classmethodrr$rR staticmethodr\r0r0r0r1r3s ar3r*)3r_r&rIrOrTrD collectionsrZconfiguration_utilsrZdynamic_module_utilsrrZfeature_extraction_utilsrZimage_processing_utilsrZprocessing_utilsr Ztokenization_utilsr utilsr r r rrZvideo_processing_utilsrZ auto_factoryrZconfiguration_autorrrrZfeature_extraction_autorZimage_processing_autorZtokenization_autorZ get_loggerr#loggerr$r*strr2r3__all__r0r0r0r1s:             Z ~