fTh^G >SrSSKJrJrJr SSKrSSKJr SSK J r J r SSK J r JrJrJr SSKJrJr "S S \ S S 9r"S S\S S 9rS\\S\S\\\4SjrS\\\\S\\\S\S\S\R04 SjrS\S\S\S\4Sjr"SS\5rS/rg)zProcessor class for Mllama.)ListOptionalUnionN) BatchFeature) ImageInputmake_nested_list_of_images) ImagesKwargsProcessingKwargsProcessorMixinUnpack)PreTokenizedInput TextInputc&\rSrSr%\\\S'Srg)MllamaImagesKwargsmax_image_tilesN)__name__ __module__ __qualname____firstlineno__rint__annotations____static_attributes__rd/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/mllama/processing_mllama.pyrrs c]"rrF)totalc,\rSrSr%\\S'SSS00rSrg)MllamaProcessorKwargs# images_kwargs image_kwargsrrN)rrrrrr _defaultsrrrrr r #s%%  q Irr input_idsimage_token_idreturnc[U5VVs/sHup#X1:XdM UPM nnn[U5S:Xa/$[U5S:XaUSS//$[USSUSS5VVs/sHupVXV/PM nnnURUS[U5/5 USSnUSSS2Hn U SU SS- :XaXS'U SnM U$s snnfs snnf)a Generate a cross-attention token mask for image tokens in the input sequence. This function identifies the positions of image tokens in the input sequence and creates a mask that defines which subsequent tokens each image token should attend to. Args: input_ids (List[int]): A list of token ids representing the input sequence. image_token_id (int): The id of the token used to represent images in the sequence. Returns: List[List[int]]: A list of [start, end] pairs, where each pair represents the range of tokens an image token should attend to. Notes: - If no image tokens are present, an empty list is returned. - For a single image token, it attends to all subsequent tokens until the end of the sequence. - For multiple image tokens, each attends to tokens up to the next image token or the end of the sequence. - Consecutive image tokens are treated as a group and attend to all subsequent tokens together. rN) enumeratelenzipappend) r&r'itokenimage_token_locationsloc1loc2 vision_masks last_mask_end vision_masks rget_cross_attention_token_maskr8-s ,09/C_/C81uG^Q/C_ !Q&  !Q&&q)2.//367LSb7QShijikSl3mn3mZTTL3mLn.r2C NCD !$Q'M#DbD) q>[^a/ /*N#A * /`os CC$C cross_attention_token_mask num_tiles max_num_tileslengthc [U5n[UVs/sHn[U5PM sn5n[R"XCXb4[RS9n[ [ X55H[unup[ [ X55H;un up[U 5S:XdMU up[X5nUS:XaUnSXxX2U SU 24'M= M] U$s snf)a Convert the cross attention mask indices to a cross attention mask 4D array. This function takes a sparse representation of cross attention masks and converts it to a dense 4D numpy array. The sparse representation is a nested list structure that defines attention ranges for each image in each batch item. Args: cross_attention_token_mask (List[List[List[int]]]): A nested list structure where: - The outer list represents the batch dimension. - The middle list represents different images within each batch item. - The inner list contains pairs of integers [start, end] representing token ranges for each image. num_tiles (List[List[int]]): A nested list structure specifying the number of tiles for each image in each batch item. max_num_tiles (int): The maximum possible number of tiles. length (int): The total sequence length of the input. Returns: np.ndarray: A 4D numpy array of shape (batch_size, length, max_num_images, max_num_tiles) The array contains `1` where attention is allowed and `0` where it is not. Note: - Special handling is done for cases where the end token is -1, which is interpreted as attending to the end of the sequence. )shapedtyper+r*N)r-maxnpzerosint64r,r.min)r9r:r;r< batch_sizemasksmax_num_imagescross_attention_mask sample_idx sample_maskssample_num_tilesmask_idx locationsmask_num_tilesstartends r,convert_sparse_cross_attention_mask_to_denserR]s:/0J2LM2L#e*2LMNN88>Ahh 9B#F`Bl8m4 4\5>s>> build_string_from_input("Hello world", "", "<|image|>") 'Hello world' >>> build_string_from_input("<|image|>Hello world", "", "<|image|>") '<|image|>Hello world' >>> build_string_from_input("Hello world", "", "<|image|>") 'Hello world' rNr*) startswithr-)rSrTrUnum_image_tokens_on_starts rbuild_string_from_inputrYso4 !   K ( (K(*+!Q&!   K ( (5 6yk& JJrc ^\rSrSrSrSS/rS/rSrSrSU4Sjjr SS \ \ S \ \ \ \\\ \\4S \\S \4S jjrSrSrSSjr\S5rSrU=r$)MllamaProcessora Constructs a Mllama processor which wraps [`MllamaImageProcessor`] and [`PretrainedTokenizerFast`] into a single processor that inherits both the image processor and tokenizer functionalities. See the [`~MllamaProcessor.__call__`] and [`~OwlViTProcessor.decode`] for more information. The preferred way of passing kwargs is as a dictionary per modality, see usage example below. ```python from transformers import MllamaProcessor from PIL import Image processor = MllamaProcessor.from_pretrained("meta-llama/Llama-3.2-11B-Vision") processor( images=your_pil_image, text=["<|image|>If I had to write a haiku for this one"], images_kwargs = {"size": {"height": 448, "width": 448}}, text_kwargs = {"padding": "right"}, common_kwargs = {"return_tensors": "pt"}, ) ``` Args: image_processor ([`MllamaImageProcessor`]): The image processor is a required input. tokenizer ([`PreTrainedTokenizer`, `PreTrainedTokenizerFast`]): The tokenizer is a required input. chat_template (`str`, *optional*): A Jinja template which will be used to convert lists of messages in a chat into a tokenizable string. image_processor tokenizer chat_templateMllamaImageProcessorPreTrainedTokenizerFastcH>[US5(d(SUlURUR5UlO"URUlURUlSUlURUR5UlUR Ul[TU]!XUS9 g)NrUz <|image|>z<|python_tag|>)r_) hasattrrUconvert_tokens_to_idsr' python_tokenpython_token_idrTsuper__init__)selfr]r^r_ __class__s rrhMllamaProcessor.__init__sy-00*D "+"A"A$BRBR"SD (44D "+":":D ,(>>t?P?PQ",, =Qrimagestextkwargsr(c HUcUc [S5eUR"[4SURR0UD6nUSnSUS'USnUSn 0n UGb%[ U[ 5(aU/nO=[ U[[45(a[SU55(d [S 5eUV s/sHoRUR5PM n n UV s/sH#n [XRUR5PM% nn URS S5nUR"U40UD6nURX/S /S 9 US Vs/sHnURUR 5PM nnU R#U5 S/nUb&[%U5nUVs/sHn['U5PM nnUb[)SW 55(a"[SU 55(d [S5e[+U 5S:aYUU :wdWU:waMUc [S5eSn[+U5[+U 5:Xa UU :waSnOWU:waSn[SU SUSU35eUb5UR,"U40UD6nURS5nU R#U5 UbdUbaWS Vs/sHn[/UUR 5PM nn[1UWUR,R2[5SUS 55S9nUU S'U RSS5n[7U US9nU$s sn fs sn fs snfs snfs snf)aF Main method to prepare text(s) and image(s) to be fed as input to the model. This method forwards the `text` arguments to PreTrainedTokenizerFast's [`~PreTrainedTokenizerFast.__call__`] if `text` is not `None` to encode the text. To prepare the image(s), this method forwards the `images` arguments to MllamaImageProcessor's [`~MllamaImageProcessor.__call__`] if `images` is not `None`. Please refer to the docstring of the above two methods for more information. Args: images (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `List[PIL.Image.Image]`, `List[np.ndarray]`, `List[torch.Tensor]`): The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch tensor. Both channels-first and channels-last formats are supported. text (`str`, `List[str]`, `List[List[str]]`): The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). return_tensors (`str` or [`~utils.TensorType`], *optional*): If set, will return tensors of a particular framework. Acceptable values are: - `'tf'`: Return TensorFlow `tf.constant` objects. - `'pt'`: Return PyTorch `torch.Tensor` objects. - `'np'`: Return NumPy `np.ndarray` objects. - `'jax'`: Return JAX `jnp.ndarray` objects. Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. TODO: add aspect_ratio_ids and aspect_ratio_mask and cross_attention_mask Nz'You must specify either text or images.tokenizer_init_kwargs text_kwargsreturn_tensorsr" common_kwargsc3B# UHn[U[5v M g7fN) isinstancestr).0ts r +MllamaProcessor.__call__..s=_Z^UVjC>P>PZ^szAInvalid input text. Please provide a string, or a list of strings padding_sideimage) modalitiesr&rc3*# UH oS:Hv M g7frNrrx batch_imgs rrzr{-sD3Ci>3Cc3*# UH oS:Hv M g7frrrs rrzr{-sQ0@9Q0@rzaIf a batch of text is provided, there should be either no images or at least one image per samplez@No image were provided, but there are image tokens in the promptzZMake sure to pass your images as a nested list, where each sub-list holds images per batchzhIf you activated truncation with `max_length`, increase the `max_length` so image tokens aren't cropped.z)The number of image tokens in each text (zA) should be the same as the number of provided images per batch (z). r:c38# UHn[U5v M g7fru)r-)rxr&s rrzr{RsQ;Pi3y>>;Ps)r:r;r<rI)data tensor_type) ValueError _merge_kwargsr r^ init_kwargsrvrwlisttupleallcountrUrYrTpop_check_special_mm_tokensr'updater r-anysumr]r8rRrrAr)rirlrmaudiovideosrn output_kwargsrqr"rsrryn_images_in_text text_item_encoding token_idsn_images_in_idsn_images_in_imagessample add_messageimage_featuresr:r9rIrr batch_features r__call__MllamaProcessor.__call__sN $%ghh"$K-.#6F2GGL^brLr'C (,>>'Q $CDTCUV@@R?SSVWbVce  !11&JMJN&**;7I KK '  $"2`hit`u*`uS\.y$:M:MN`u '*$P*#"22BBQ8K;PQQ $ ,@D' (&**+;TB$$NK u Iol "DB*s3$L *L%LL Lc:URR"U0UD6$)z This method forwards all its arguments to PreTrainedTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of this method for more information. r^ batch_decoderiargsrns rrMllamaProcessor.batch_decode[s ~~**D;F;;rc:URR"U0UD6$)z This method forwards all its arguments to PreTrainedTokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this method for more information. )r^decoders rrMllamaProcessor.decodebs ~~$$d5f55rc BURR"U4UUS.UD6$)a* Post-process the output of the model to decode the text. Args: generated_outputs (`torch.Tensor` or `np.ndarray`): The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`. skip_special_tokens (`bool`, *optional*, defaults to `True`): Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `batch_decode` method. clean_up_tokenization_spaces (`bool`, *optional*, defaults to `False`): Whether or not to clean up the tokenization spaces. Argument passed to the tokenizer's `batch_decode` method. **kwargs: Additional arguments to be passed to the tokenizer's `batch_decode method`. Returns: `List[str]`: The decoded text. )skip_special_tokensclean_up_tokenization_spacesr)rigenerated_outputsrrrns rpost_process_image_text_to_text/MllamaProcessor.post_process_image_text_to_textis3(~~**  3)E    rcURRnURRnUVs/sH o3S:wdM UPM nn[X-S/-5$s snf)Nr:rI)r^model_input_namesr]r)ritokenizer_input_namesimage_processor_input_namesnames rr!MllamaProcessor.model_input_namessb $ @ @&*&:&:&L&L#9T&k8S_jWjt8S#&k)GKaJbbcc'ls AA)rTrUr'rerfru)NNNN)TF)rrrr__doc__ attributes valid_kwargsimage_processor_classtokenizer_classrhrrrrrrr r rrrrrpropertyrr __classcell__)rjs@rr[r[s>$[1J#$L2/O R(,hl u$uuY(94 ?DQbLccdeu ./ u un<6Y^ 6ddrr[)rtypingrrrnumpyrBfeature_extraction_utilsr image_utilsrr processing_utilsr r r r tokenization_utils_baserrrr rr8ndarrayrRrwrYr[__all__rrrrs "((4AVV #U#,E-d3i--QUVZ[^V_Q`-`- $T$s)_ 5- DI- -   - ZZ - `"KC"KC"Kc"Kc"KJZdnZdz  r