o
ZhI @ s U d Z ddlZddlZddlZddlZddlmZ ddlmZm Z m
Z
m Z m Z ddl
mZ ddlmZmZ ddlmZmZmZmZmZ dd lmZ dd
lmZ d d l m Z d d
l m Z m Z m!Z!m"Z" e#e$Z%er|e Z&ee'e e
e' e
e' f f e(d< neg dZ&e&) D ]\Z*Z+e+Z,e sdZ,e,e&e*< qe e e&Z-de'fddZ. d#de e'ej/f de
e e'ej/f de0de
e0 de
e e'e'f de
e e0e'f de
e' de0fd d Z1ed d G d d! d!Z2d"d!gZ3dS )$zAutoVideoProcessor class. N) OrderedDict)
TYPE_CHECKINGDictOptionalTupleUnion )PretrainedConfig) get_class_from_dynamic_moduleresolve_trust_remote_code) CONFIG_NAMEVIDEO_PROCESSOR_NAME cached_fileis_torchvision_availablelogging)requires)BaseVideoProcessor )_LazyAutoMapping)CONFIG_MAPPING_NAMES
AutoConfigmodel_type_to_module_name!replace_list_option_in_docstrings VIDEO_PROCESSOR_MAPPING_NAMES)
)Z instructblip InstructBlipVideoVideoProcessor)Zinstructblipvideor )ZinternvlZInternVLVideoProcessor)Zllava_next_videoZ LlavaNextVideoVideoProcessor)Zllava_onevisionZ LlavaOnevisionVideoProcessor)Z qwen2_5_omniQwen2VLVideoProcessor)Z
qwen2_5_vlr )Zqwen2_vlr )ZsmolvlmZSmolVLMVideoProcessor)Z video_llavaZVideoLlavaVideoProcessor
class_namec C s t D ]'\}}| |v r+t|}td| d}zt|| W S ty* Y qw qtj D ]\}}t|dd | krA| S q1td}t || rQt|| S d S )N.ztransformers.models__name__Z transformers)
r itemsr importlib
import_modulegetattrAttributeErrorVIDEO_PROCESSOR_MAPPINGZ_extra_contenthasattr)r module_nameZ
extractorsmodule_Z extractorZ main_module r) ]/var/www/auris/lib/python3.10/site-packages/transformers/models/auto/video_processing_auto.py video_processor_class_from_nameJ s$
r+ F pretrained_model_name_or_path cache_dirforce_downloadresume_downloadproxiestokenrevisionlocal_files_onlyc K s | dd} | durtdt |durtd| }t| t|||||||d }
|
du r2td i S t |
dd
} t
| W d S 1 sHw Y dS ) a
Loads the video processor configuration from a pretrained model video processor configuration.
Args:
pretrained_model_name_or_path (`str` or `os.PathLike`):
This can be either:
- a string, the *model id* of a pretrained model configuration hosted inside a model repo on
huggingface.co.
- a path to a *directory* containing a configuration file saved using the
[`~PreTrainedTokenizer.save_pretrained`] method, e.g., `./my_model_directory/`.
cache_dir (`str` or `os.PathLike`, *optional*):
Path to a directory in which a downloaded pretrained model configuration should be cached if the standard
cache should not be used.
force_download (`bool`, *optional*, defaults to `False`):
Whether or not to force to (re-)download the configuration files and override the cached versions if they
exist.
resume_download:
Deprecated and ignored. All downloads are now resumed by default when possible.
Will be removed in v5 of Transformers.
proxies (`Dict[str, str]`, *optional*):
A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',
'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.
token (`str` or *bool*, *optional*):
The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
when running `huggingface-cli login` (stored in `~/.huggingface`).
revision (`str`, *optional*, defaults to `"main"`):
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
identifier allowed by git.
local_files_only (`bool`, *optional*, defaults to `False`):
If `True`, will only try to load the video processor configuration from local files.
Passing `token=True` is required when you want to use a private model.
Returns:
`Dict`: The configuration of the video processor.
Examples:
```python
# Download configuration from huggingface.co and cache.
video_processor_config = get_video_processor_config("llava-hf/llava-onevision-qwen2-0.5b-ov-hf")
# This model does not have a video processor config so the result will be an empty dict.
video_processor_config = get_video_processor_config("FacebookAI/xlm-roberta-base")
# Save a pretrained video processor locally and you can reload its config
from transformers import AutoVideoProcessor
video_processor = AutoVideoProcessor.from_pretrained("llava-hf/llava-onevision-qwen2-0.5b-ov-hf")
video_processor.save_pretrained("video-processor-test")
video_processor = get_video_processor_config("video-processor-test")
```use_auth_tokenNrThe `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.V`token` and `use_auth_token` are both specified. Please set only the argument `token`.)r- r. r/ r0 r1 r2 r3 zbCould not locate the video processor configuration file, will try to use the model config instead.zutf-8)encoding) popwarningswarn
FutureWarning
ValueErrorr r
loggerinfoopenjsonload) r, r- r. r/ r0 r1 r2 r3 kwargsr4 Zresolved_config_filereaderr) r) r* get_video_processor_configb s8 E $rD )ZvisionZ torchvision)backendsc @ s< e Zd ZdZdd Zeeedd Ze d
ddZ
d S ) AutoVideoProcessora%
This is a generic video processor class that will be instantiated as one of the video processor classes of the
library when created with the [`AutoVideoProcessor.from_pretrained`] class method.
This class cannot be instantiated directly using `__init__()` (throws an error).
c C s t d)NzAutoVideoProcessor is designed to be instantiated using the `AutoVideoProcessor.from_pretrained(pretrained_model_name_or_path)` method.)EnvironmentError)selfr) r) r* __init__ s zAutoVideoProcessor.__init__c O sv | dd}|dur tdt |dddur td||d< | dd}| dd}d|d < tj|fi |\}}|d
d} d}
d |d i v rQ|d d }
| du r}|
du r}| d
d} | duri| dd} d|d i v r}|d d } | dd}
| du r|
du rt |t
st j |fd|i|}t
|d
d} t|d rd |jv r|jd }
| durt| } |
du}
| dupt|tv }t||||
}|
r|r|
}t||fi |} | dd}tj|r| | j|fi |S | dur| j|fi |S t|tv r tt| } | dur| j |g|R i |S tdtd| dt dt dt dd dd t D
)ac
Instantiate one of the video processor classes of the library from a pretrained model vocabulary.
The video processor class to instantiate is selected based on the `model_type` property of the config object
(either passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's
missing, by falling back to using pattern matching on `pretrained_model_name_or_path`:
List options
Params:
pretrained_model_name_or_path (`str` or `os.PathLike`):
This can be either:
- a string, the *model id* of a pretrained video_processor hosted inside a model repo on
huggingface.co.
- a path to a *directory* containing a video processor file saved using the
[`~video_processing_utils.BaseVideoProcessor.save_pretrained`] method, e.g.,
`./my_model_directory/`.
- a path or url to a saved video processor JSON *file*, e.g.,
`./my_model_directory/preprocessor_config.json`.
cache_dir (`str` or `os.PathLike`, *optional*):
Path to a directory in which a downloaded pretrained model video processor should be cached if the
standard cache should not be used.
force_download (`bool`, *optional*, defaults to `False`):
Whether or not to force to (re-)download the video processor files and override the cached versions if
they exist.
resume_download:
Deprecated and ignored. All downloads are now resumed by default when possible.
Will be removed in v5 of Transformers.
proxies (`Dict[str, str]`, *optional*):
A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',
'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.
token (`str` or *bool*, *optional*):
The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
when running `huggingface-cli login` (stored in `~/.huggingface`).
revision (`str`, *optional*, defaults to `"main"`):
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
identifier allowed by git.
return_unused_kwargs (`bool`, *optional*, defaults to `False`):
If `False`, then this function returns just the final video processor object. If `True`, then this
functions returns a `Tuple(video_processor, unused_kwargs)` where *unused_kwargs* is a dictionary
consisting of the key/value pairs whose keys are not video processor attributes: i.e., the part of
`kwargs` which has not been used to update `video_processor` and is otherwise ignored.
trust_remote_code (`bool`, *optional*, defaults to `False`):
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to `True` for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.
kwargs (`Dict[str, Any]`, *optional*):
The values in kwargs of any keys which are video processor attributes will be used to override the
loaded values. Behavior concerning key/value pairs whose keys are *not* video processor attributes is
controlled by the `return_unused_kwargs` keyword parameter.
Passing `token=True` is required when you want to use a private model.
Examples:
```python
>>> from transformers import AutoVideoProcessor
>>> # Download video processor from huggingface.co and cache.
>>> video_processor = AutoVideoProcessor.from_pretrained("llava-hf/llava-onevision-qwen2-0.5b-ov-hf")
>>> # If video processor files are in a directory (e.g. video processor was saved using *save_pretrained('./test/saved_model/')*)
>>> # video_processor = AutoVideoProcessor.from_pretrained("./test/saved_model/")
```r4 Nr5 r1 r6 configtrust_remote_codeTZ
_from_autoZvideo_processor_typerF auto_mapZfeature_extractor_typeZFeatureExtractorZVideoProcessorZAutoFeatureExtractorZ
code_revisionz_This video processor cannot be instantiated. Please make sure you have `torchvision` installed.z Unrecognized video processor in z2. Should have a `video_processor_type` key in its z of z3, or one of the following `model_type` keys in its z: z, c s s | ]}|V qd S )Nr) ).0cr) r) r* i s z5AutoVideoProcessor.from_pretrained..) r8 r9 r: r; getr< r Zget_video_processor_dictreplace
isinstancer r from_pretrainedr" r% rL r+ typer$ r r
ospathisdirZregister_for_auto_class from_dictr
r joinr keys)clsr, ZinputsrB r4 rJ rK Z config_dictr( video_processor_classZvideo_processor_auto_mapZfeature_extractor_classZfeature_extractor_auto_mapZhas_remote_codeZhas_local_codeZ class_refr) r) r* rS s I
z"AutoVideoProcessor.from_pretrainedFc C s t j| ||d dS )a7
Register a new video processor for this class.
Args:
config_class ([`PretrainedConfig`]):
The configuration corresponding to the model to register.
video_processor_class ([`BaseVideoProcessor`]):
The video processor to register.
)exist_okN)r$ register)Z config_classr\ r] r) r) r* r^ l s zAutoVideoProcessor.registerN)F) r
__module__ __qualname____doc__rI classmethodr r rS staticmethodr^ r) r) r) r* rF s rF r$ )NFNNNNF)4ra r r@ rU r9 collectionsr typingr r r r r Zconfiguration_utilsr Zdynamic_module_utilsr
r utilsr r
r r r Zutils.import_utilsr Zvideo_processing_utilsr Z auto_factoryr Zconfiguration_autor r r r Z
get_loggerr r= r str__annotations__r Z
model_typeZvideo_processorsZfast_video_processor_classr$ r+ PathLikeboolrD rF __all__r) r) r) r* sn
(
d 8