fTh5$tSrSSKrSSKJrJrJr SSKJr SSKJ r J r J r J r J r SSKJr "SS \5rS /rg) z Processor class for LayoutXLM. N)ListOptionalUnion)ProcessorMixin) BatchEncodingPaddingStrategyPreTokenizedInput TextInputTruncationStrategy) TensorTypec(^\rSrSrSrSS/rSrSrS#U4SjjrS$S\ \ \ \ \ \ \ 4S \ \ \ \ \ 4S \ \ \ \ \\ \ \ \4S \ \ \ \\ \ \4S \S \ \\\4S\ \\\4S\ \S\S\ \S\ \S\ \S\S\S\S\S\S\ \ \\4S\4&SjjrSrSrSr\S5r\S 5r\S!5rS"rU=r$)%LayoutXLMProcessora6 Constructs a LayoutXLM processor which combines a LayoutXLM image processor and a LayoutXLM tokenizer into a single processor. [`LayoutXLMProcessor`] offers all the functionalities you need to prepare data for the model. It first uses [`LayoutLMv2ImageProcessor`] to resize document images to a fixed size, and optionally applies OCR to get words and normalized bounding boxes. These are then provided to [`LayoutXLMTokenizer`] or [`LayoutXLMTokenizerFast`], which turns the words and bounding boxes into token-level `input_ids`, `attention_mask`, `token_type_ids`, `bbox`. Optionally, one can provide integer `word_labels`, which are turned into token-level `labels` for token classification tasks (such as FUNSD, CORD). Args: image_processor (`LayoutLMv2ImageProcessor`, *optional*): An instance of [`LayoutLMv2ImageProcessor`]. The image processor is a required input. tokenizer (`LayoutXLMTokenizer` or `LayoutXLMTokenizerFast`, *optional*): An instance of [`LayoutXLMTokenizer`] or [`LayoutXLMTokenizerFast`]. The tokenizer is a required input. image_processor tokenizerLayoutLMv2ImageProcessor)LayoutXLMTokenizerLayoutXLMTokenizerFastc >SU;a,[R"S[5 URS5nUbUOWnUc [ S5eUc [ S5e[ TU]X5 g)Nfeature_extractorzhThe `feature_extractor` argument is deprecated and will be removed in v5, use `image_processor` instead.z)You need to specify an `image_processor`.z"You need to specify a `tokenizer`.)warningswarn FutureWarningpop ValueErrorsuper__init__)selfrrkwargsr __class__s j/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/layoutxlm/processing_layoutxlm.pyrLayoutXLMProcessor.__init__3so & ( MM  !' +> ? -<-H/N_  "HI I  AB B 4text text_pairboxes word_labelsadd_special_tokenspadding truncation max_lengthstridepad_to_multiple_ofreturn_token_type_idsreturn_attention_maskreturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mapping return_lengthverbosereturn_tensorsreturnc URR(aUb [S5eURR(aUb [S5eUSLaUSLa [S5eURUUS9nUb;URR(a Uc[U[5(aU/nUSnUR "S0S UbUOUS_S UbUOS_S UbUOUS _S U_S U_SU_SU_SU _SU _SU _SU _SU _SU_SU_SU_SU_SU_SU_UD6nUR S5nUSLaURUUS5nUUS'U$)a This method first forwards the `images` argument to [`~LayoutLMv2ImagePrpcessor.__call__`]. In case [`LayoutLMv2ImagePrpcessor`] was initialized with `apply_ocr` set to `True`, it passes the obtained words and bounding boxes along with the additional arguments to [`~LayoutXLMTokenizer.__call__`] and returns the output, together with resized `images`. In case [`LayoutLMv2ImagePrpcessor`] was initialized with `apply_ocr` set to `False`, it passes the words (`text`/``text_pair`) and `boxes` specified by the user along with the additional arguments to [`~LayoutXLMTokenizer.__call__`] and returns the output, together with resized `images``. Please refer to the docstring of the above two methods for more information. NzdYou cannot provide bounding boxes if you initialized the image processor with apply_ocr set to True.zaYou cannot provide word labels if you initialized the image processor with apply_ocr set to True.TFzKYou cannot return overflowing tokens without returning the offsets mapping.)imagesr6wordsr%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6 pixel_valuesoverflow_to_sample_mappingimage)r apply_ocrr isinstancestrrrget_overflowing_images)rr9r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6r featuresencoded_inputss r"__call__LayoutXLMProcessor.__call__DsD    ) )u/@v     ) ){/Fs  % ,1G51Pjk k''vn'U   4 4 > >9CT$$$v )I )x/@ #,#8id !,%(72C $  2    " "   2 #8 #8 '@ (B $: (! "# $*' .n- $ ,00Hd9efF"(wr$c/nUHnURX5 M [U5[U5:wa#[S[U5S[U535eU$)Nz`Expected length of images to be the same as the length of `overflow_to_sample_mapping`, but got z and )appendlenr)rr9r<images_with_overflow sample_idxs r"rB)LayoutXLMProcessor.get_overflowing_imagessr!4J ' '(: ;5 # $,F(G G,-.eC8R4S3TV  $#r$c:URR"U0UD6$)z This method forwards all its arguments to PreTrainedTokenizer's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of this method for more information. )r batch_decoderargsr s r"rNLayoutXLMProcessor.batch_decodes ~~**D;F;;r$c:URR"U0UD6$)z This method forwards all its arguments to PreTrainedTokenizer's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this method for more information. )rdecoderOs r"rSLayoutXLMProcessor.decodes ~~$$d5f55r$c /SQ$)N) input_idsbboxattention_maskr=r>rs r"model_input_names$LayoutXLMProcessor.model_input_namess??r$cP[R"S[5 UR$)Nzg`feature_extractor_class` is deprecated and will be removed in v5. Use `image_processor_class` instead.)rrrimage_processor_classrYs r"feature_extractor_class*LayoutXLMProcessor.feature_extractor_classs" u  )))r$cP[R"S[5 UR$)Nz[`feature_extractor` is deprecated and will be removed in v5. Use `image_processor` instead.)rrrrrYs r"r$LayoutXLMProcessor.feature_extractors" i  ###r$r>)NN)NNNNTFNNrNNNFFFFTN)__name__ __module__ __qualname____firstlineno____doc__ attributesr]tokenizer_classrrr r rrintboolrAr r r rrErBrNrSpropertyrZr^r__static_attributes__ __classcell__)r!s@r"rrs(&$[1J6FO5(_cQUIMCG#'5:;?$(,00404*/+0',#;?)TI0$y/4HYCZZ[TE"3T:K5L"LMN T d49otDcO/DDEF T eDItDI$>?@ T!TtS/12T$%778TSMTT%SMT (~T (~T$(T %)!T"!%#T$%T&'T(!sJ!78)T, -Tl $<6@@**$$r$r)rfrtypingrrrprocessing_utilsrtokenization_utils_baserr r r r utilsr r__all__r>r$r"rss:((.wwm$m$` r$