o Zh@sPdZddlmZddlmZddlmZmZee Z GdddeZ dgZ dS) zLlava model configuration)PretrainedConfig)logging)CONFIG_MAPPING AutoConfigcsHeZdZdZdZddiZeedZ     dfd d ZZ S) LlavaConfigaI This is the configuration class to store the configuration of a [`LlavaForConditionalGeneration`]. It is used to instantiate an Llava model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the Llava-9B. e.g. [llava-hf/llava-9b](https://huggingface.co/llava-hf/llava-9b) Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vision_config (`Union[AutoConfig, dict]`, *optional*, defaults to `CLIPVisionConfig`): The config object or dictionary of the vision backbone. text_config (`Union[AutoConfig, dict]`, *optional*, defaults to `LlamaConfig`): The config object or dictionary of the text backbone. image_token_index (`int`, *optional*, defaults to 32000): The image token index to encode the image prompt. projector_hidden_act (`str`, *optional*, defaults to `"gelu"`): The activation function used by the multimodal projector. vision_feature_select_strategy (`str`, *optional*, defaults to `"default"`): The feature selection strategy used to select the vision feature from the vision backbone. Can be one of `"default"` or `"full"`. vision_feature_layer (`Union[int, List[int]]`, *optional*, defaults to -2): The index of the layer to select the vision feature. If multiple indices are provided, the vision feature of the corresponding indices will be concatenated to form the vision features. image_seq_length (`int`, *optional*, defaults to 576): Sequence length of one image embedding. multimodal_projector_bias (`bool`, *optional*, defaults to `True`): Whether to use bias in the multimodal projector. Example: ```python >>> from transformers import LlavaForConditionalGeneration, LlavaConfig, CLIPVisionConfig, LlamaConfig >>> # Initializing a CLIP-vision config >>> vision_config = CLIPVisionConfig() >>> # Initializing a Llama config >>> text_config = LlamaConfig() >>> # Initializing a Llava llava-1.5-7b style configuration >>> configuration = LlavaConfig(vision_config, text_config) >>> # Initializing a model from the llava-1.5-7b style configuration >>> model = LlavaForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```ZllavaZimage_token_idimage_token_index) text_config vision_configN}geludefault@Tc s||_||_||_|dvrtd|||_||_t|tr7d|vr'|dnd|d<t|ddi|}n|durItdddddd d d d d }||_ t|trid|vrY|dnd|d<t|ddi|}n |durrtd}||_ ||_ t j di| dS)N)r fullzGvision_feature_select_strategy should be one of 'default', 'full'.Got: model_typeZclip_vision_modeliiiPr i)Zintermediate_sizeZ hidden_sizeZ patch_sizeZ image_sizeZnum_hidden_layersZnum_attention_headsZ vocab_sizeZprojection_dimllama)rprojector_hidden_actimage_seq_length ValueErrorvision_feature_select_strategyvision_feature_layer isinstancedictrr r multimodal_projector_biassuper__init__) selfr r rrrrrrkwargs __class__r\/var/www/auris/lib/python3.10/site-packages/transformers/models/llava/configuration_llava.pyr SsF    zLlavaConfig.__init__)NNr r r rrT) __name__ __module__ __qualname____doc__rZ attribute_maprZ sub_configsr __classcell__rrr#r%rs4 rN) r)Zconfiguration_utilsrutilsrautorrZ get_loggerr&loggerr__all__rrrr%s    s