fTh`SrSSKJrJr SSKJr SSKJr SSKJ r J r J r "SS\5r S/r g ) z Processor class for MarkupLM. )OptionalUnion) TensorType)ProcessorMixin) BatchEncodingPaddingStrategyTruncationStrategyc \rSrSrSrSrSrSrSS\S\ \\ \ 4S \ \\ \ 4S \ \S \S \ \S \ \S\ \S\S\S\S\S\S\ \ \ \4S\4SjjrSrSr\S5rSrg)MarkupLMProcessora Constructs a MarkupLM processor which combines a MarkupLM feature extractor and a MarkupLM tokenizer into a single processor. [`MarkupLMProcessor`] offers all the functionalities you need to prepare data for the model. It first uses [`MarkupLMFeatureExtractor`] to extract nodes and corresponding xpaths from one or more HTML strings. Next, these are provided to [`MarkupLMTokenizer`] or [`MarkupLMTokenizerFast`], which turns them into token-level `input_ids`, `attention_mask`, `token_type_ids`, `xpath_tags_seq` and `xpath_subs_seq`. Args: feature_extractor (`MarkupLMFeatureExtractor`): An instance of [`MarkupLMFeatureExtractor`]. The feature extractor is a required input. tokenizer (`MarkupLMTokenizer` or `MarkupLMTokenizerFast`): An instance of [`MarkupLMTokenizer`] or [`MarkupLMTokenizerFast`]. The tokenizer is a required input. parse_html (`bool`, *optional*, defaults to `True`): Whether or not to use `MarkupLMFeatureExtractor` to parse HTML strings into nodes and corresponding xpaths. MarkupLMFeatureExtractor)MarkupLMTokenizerMarkupLMTokenizerFastTNadd_special_tokenspadding truncation max_lengthstridepad_to_multiple_ofreturn_token_type_idsreturn_attention_maskreturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mapping return_lengthverbosereturn_tensorsreturnc UR(a>Uc [S5eUcUcUb [S5eURU5nUSnUSnOUb [S5eUbUc [S5eUb)UR(a[U[5(aU/nUR "S0SUbUOU_S UbUOS_SU_S U_S U_S U_S U_SU _SU _SU _SU _SU _SU_SU_SU_SU_SU_SU_UD6nU$)a This method first forwards the `html_strings` argument to [`~MarkupLMFeatureExtractor.__call__`]. Next, it passes the `nodes` and `xpaths` along with the additional arguments to [`~MarkupLMTokenizer.__call__`] and returns the output. Optionally, one can also provide a `text` argument which is passed along as first sequence. Please refer to the docstring of the above two methods for more information. NzDMake sure to pass HTML strings in case `parse_html` is set to `True`zUPlease don't pass nodes, xpaths nor node labels in case `parse_html` is set to `True`nodesxpathsz@You have passed HTML strings but `parse_html` is set to `False`.zIMake sure to pass nodes and xpaths in case `parse_html` is set to `False`text text_pair node_labelsrrrrrrrrrrrrrr) parse_html ValueErrorfeature_extractor isinstancestr tokenizer)self html_stringsr!r"r% questionsrrrrrrrrrrrrrrkwargsfeaturesencoded_inputss h/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/markuplm/processing_markuplm.py__call__MarkupLMProcessor.__call__2sB ??# !ghh F$6+:Q k--l;HW%Eh'F' !cdd} !lmm  T__)S))&K  '3 (4e$  $  2    " "   2 #8 #8 '@ (B $: (! "# $*' ,c:URR"U0UD6$)z This method forwards all its arguments to TrOCRTokenizer's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of this method for more information. )r, batch_decoder-argsr0s r3r8MarkupLMProcessor.batch_decodes ~~**D;F;;r6c:URR"U0UD6$)z This method forwards all its arguments to TrOCRTokenizer's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this method for more information. )r,decoder9s r3r=MarkupLMProcessor.decodes ~~$$d5f55r6c2URRnU$)N)r,model_input_names)r-tokenizer_input_namess r3r@#MarkupLMProcessor.model_input_namess $ @ @$$r6r&)NNNNNTFNNrNNNFFFFTN)__name__ __module__ __qualname____firstlineno____doc__feature_extractor_classtokenizer_classr'boolrr+r r rintrrr4r8r=propertyr@__static_attributes__r&r6r3r r sU&9DOJ#'5:;?$(,00404*/+0',#;?)N!NtS/12N$%778NSMNN%SMN (~N (~N$(N %)!N"!%#N$%N&'N(!sJ!78)N, -N`<6%%r6r N)rGtypingrr file_utilsrprocessing_utilsrtokenization_utils_baserr r r __all__r&r6r3rSs4#$.YYy%y%x  r6