fThSSrSSKrSSKJrJrJr SSKrSSKrSSKJr SSK J r J r J r SSK Jr SSKJrJrJrJrJr SS KJr SS KJr SS KJrJrJrJr S S KJr \"5(aMSSK r SSK!J"r" \RRFRH\ RJRLRNl$\RP"\)5r*"SS\RF5r+"SS\RF5r,"SS\RF5r-"SS\RF5r."SS\RF5r/"SS\RF5r0"SS\RF5r1S3Sjr2"SS\RF5r3\"S S!\55r4S4S"jr5"S#S$\RF5r6"S%S&\RF5r7\"S'S(\455r8\"S)S*9"S+S,\455r9\"S-S*9"S.S/\455r:\"S0S1\455r;/S2Qr[[U] 5 [R"UR UR URS9Ul[R"URUR 5Ul [R"URUR5Ul [R"URUR5Ul[R"URUR5Ul[R"URUR5Ul[R"UR$UR 5Ul[R("UR UR*S9Ul[R,"UR.5UlUR3S[4R6"UR5R9S5SS9 g)N) padding_idxeps position_ids)rF persistent)superr__init__r Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingsmax_2d_position_embeddingscoordinate_sizex_position_embeddingsy_position_embeddings shape_sizeh_position_embeddingsw_position_embeddingstype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsDropouthidden_dropout_probdropoutregister_buffertorcharangeexpandselfconfig __class__s j/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/layoutlmv2/modeling_layoutlmv2.pyr&LayoutLMv2Embeddings.__init__6sb "D24!||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c %'\\&2S2SU[UkUk%l"%'\\&2S2SU[UkUk%l"%'\\&2S2SU[UfUf%g"%'\\&2S2SU[UfUf%g"%'\\&2H2H&J\J\%]"f&8&8f>S>STzz&"<"<=   ELL)G)GHOOPWXej  cURUSS2SS2S45nURUSS2SS2S45nURUSS2SS2S45nURUSS2SS2S45nURUSS2SS2S4USS2SS2S4- 5nUR USS2SS2S4USS2SS2S4- 5n[ R "UUUUUU/SS9n U $![an[S5UeSnAff=f)Nrrr z;The `bbox` coordinate values should be within 0-1000 range.r"dim)r0r1 IndexErrorr3r4r=cat) rAbboxleft_position_embeddingsupper_position_embeddingsright_position_embeddingslower_position_embeddingser3r4spatial_position_embeddingss rD!_calc_spatial_position_embeddings6LayoutLMv2Embeddings._calc_spatial_position_embeddingsHs( c'+'A'A$q!Qw-'P $(,(B(B41a=(Q %(,(B(B41a=(Q %(,(B(B41a=(Q %!% : :41a=4PQSTVWPW=;X Y $ : :41a=4PQSTVWPW=;X Y&+ii()))%%   ' #+*# cZ[ab b csA,C C6% C11C6) r7r;r3r-r6r4r+r0r1) __name__ __module__ __qualname____firstlineno____doc__r&rT__static_attributes__ __classcell__rCs@rDrr3sQ $++rFrcH^\rSrSrU4SjrSrSrSSjrSrU=r $)LayoutLMv2SelfAttentionbc>[TU]5 URUR-S:wa7[ US5(d&[ SURSURS35eUR UlURUl[URUR- 5UlURUR-Ul URUl URUl UR (a[R"URSUR-SS9Ul[R"[ R""S S UR55Ul[R"[ R""S S UR55UlO[R"URUR5Ul[R"URUR5Ul[R"URUR5Ul[R."UR05Ulg) Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()r Fbiasr)r%r&r)num_attention_headshasattr ValueErrorfast_qkvintattention_head_size all_head_sizehas_relative_attention_biashas_spatial_attention_biasrLinear qkv_linear Parameterr=zerosq_biasv_biasquerykeyvaluer9attention_probs_dropout_probr;r@s rDr& LayoutLMv2SelfAttention.__init__cs     : : :a ?PVXhHiHi#F$6$6#78 445Q8  #)#=#= #&v'9'9FS"S%**,CD 6G=2O\M]rF) rlrkr;rirmrnrvrfrsrprurtrwNNFNN) rVrWrXrYr&rrrr[r\r]s@rDr_r_bs.G4% ())rFr_c<^\rSrSrU4SjrSSjrSrU=r$)LayoutLMv2Attentioncb>[TU]5 [U5Ul[ U5UlgN)r%r&r_rALayoutLMv2SelfOutputoutputr@s rDr&LayoutLMv2Attention.__init__s& +F3 *62 rFc jURUUUUUUS9nURUSU5nU4USS-n U $)Nrrrr)rAr) rArrrrrr self_outputsattention_outputrs rDrLayoutLMv2Attention.forwardsYyy    ! !  ;;|A F#% QR(88rF)rrArrVrWrXrYr&rr[r\r]s@rDrrs#3rFrc.^\rSrSrU4SjrSrSrU=r$)rc(>[TU]5 [R"URUR5Ul[R "URURS9Ul[R"UR5Ul gNr) r%r&rror)denser7r8r9r:r;r@s rDr&LayoutLMv2SelfOutput.__init__s` YYv1163E3EF f&8&8f>S>STzz&"<"<= rFcpURU5nURU5nURX-5nU$rrr;r7rAr input_tensors rDrLayoutLMv2SelfOutput.forward5 =1  ]3 }'CD rFr7rr;rr]s@rDrrs> rFrcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)LayoutLMv2Intermediatec>[TU]5 [R"URUR 5Ul[UR[5(a[URUl gURUl gr) r%r&rror)intermediate_sizer isinstance hidden_actstrr intermediate_act_fnr@s rDr&LayoutLMv2Intermediate.__init__s` YYv1163K3KL f'' - -'-f.?.?'@D $'-'8'8D $rFrreturncJURU5nURU5nU$rrr)rArs rDrLayoutLMv2Intermediate.forwards& =1 00? rFr rVrWrXrYr&r=Tensorrr[r\r]s@rDrrs(9U\\ellrFrcz^\rSrSrU4SjrS\R S\R S\R 4SjrSrU=r $)LayoutLMv2Outputc(>[TU]5 [R"URUR 5Ul[R"UR URS9Ul[R"UR5Ul gr) r%r&rrorr)rr7r8r9r:r;r@s rDr&LayoutLMv2Output.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= rFrrrcpURU5nURU5nURX-5nU$rrrs rDrLayoutLMv2Output.forwardrrFrrr]s@rDrrs6> U\\RWR^R^rFrcB^\rSrSrU4SjrSSjrSrSrU=r$)LayoutLMv2Layeric>[TU]5 URUlSUl[ U5Ul[ U5Ul[U5Ul g)Nr) r%r&chunk_size_feed_forward seq_len_dimr attentionr intermediaterrr@s rDr&LayoutLMv2Layer.__init__ sI '-'E'E$,V426:&v. rFc URUUUUUUS9nUSnUSSn [URURURU5n U 4U -n U $)N)rrrrr)rrfeed_forward_chunkrr) rArrrrrrself_attention_outputsrr layer_outputs rDrLayoutLMv2Layer.forwards|"&   /! "0" 2!4(,0  # #T%A%A4CSCSUe  /G+rFcJURU5nURX!5nU$r)rr)rArintermediate_outputrs rDr"LayoutLMv2Layer.feed_forward_chunk-s)"//0@A{{#6I rF)rrrrrr) rVrWrXrYr&rrr[r\r]s@rDrrs(/8rFrcDSnU(a4US-nX@S:R5U-- n[R"U5nO,[R"U*[R"U55nUS-nXV:nU[R "UR 5U- 5[R "X6- 5- X&- -R[R5-n[R"U[R"XS- 55nU[R"XuU5- nU$)aR Adapted from Mesh Tensorflow: https://github.com/tensorflow/mesh/blob/0cb87fe07da627bf0b7e60475d59f95ed6b5be3d/mesh_tensorflow/transformer/transformer_layers.py#L593 Translate relative position to a bucket number for relative attention. The relative position is defined as memory_position - query_position, i.e. the distance in tokens from the attending position to the attended-to position. If bidirectional=False, then positive relative positions are invalid. We use smaller buckets for small absolute relative_position and larger buckets for larger absolute relative_positions. All relative positions >=max_distance map to the same bucket. All relative positions <=-max_distance map to the same bucket. This should allow for more graceful generalization to longer sequences than the model has been trained on. Args: relative_position: an int32 Tensor bidirectional: a boolean - whether the attention is bidirectional num_buckets: an integer max_distance: an integer Returns: a Tensor with the same shape as relative_position, containing int32 values in the range [0, num_buckets) rrHr) longr=absmax zeros_likelogrrrr full_likewhere) relative_position bidirectional num_buckets max_distanceretn max_exactis_small val_if_larges rDrelative_position_bucketr3s* C  A%++- ;; II' ( II((%*:*:;L*M Nq I}H !'')i'(488L4L+MMQ\QhibnL99\5??[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlURUl URUl UR(aQURUl URUl [R"URURSS9UlUR(aUR UlUR"Ul[R"UR"URSS9Ul[R"UR"URSS9UlSUlgs snf)NFrd)r%r&rBr ModuleListrangenum_hidden_layersrlayerrmrn rel_pos_bins max_rel_posrorf rel_pos_biasmax_rel_2d_posrel_2d_pos_binsrel_pos_x_biasrel_pos_y_biasgradient_checkpointing)rArB_rCs rDr&LayoutLMv2Encoder.__init__`s  ]]U6KcKcEd#eEdOF$;Ed#ef +1+M+M(*0*K*K'  + + & 3 3D %11D  " $*;*;V=W=W^c dD   * *"("7"7D #)#9#9D "$))D,@,@&B\B\ch"iD "$))D,@,@&B\B\ch"iD &+#!$fsE;czURS5URS5- n[UURURS9n[R "5 UR RR5URSSSS5nSSS5 UR5nU$!,(df  N =f)Nrr"rrrr rrH) unsqueezerr r r=no_gradr weighttr}r)rAr! rel_pos_matrs rD!_calculate_1d_position_embeddings3LayoutLMv2Encoder._calculate_1d_position_embeddingsus",,R0<3I3I"3MM * )))) ]]_''..0027;CCAq!QOG$$&_s :B,, B:cUSS2SS2S4nUSS2SS2S4nURS5URS5- nURS5URS5- n[UURURS9n[UURURS9n[R "5 UR RR5URSSSS5nURRR5URSSSS5nSSS5 UR5nUR5nXg-nU$!,(df  N4=f)Nrr rr"rrrH) rrrr r=rrrrr}rr) rArMposition_coord_xposition_coord_yrel_pos_x_2d_matrel_pos_y_2d_mat rel_pos_x rel_pos_yrs rD!_calculate_2d_position_embeddings3LayoutLMv2Encoder._calculate_2d_position_embeddingssP1a=1a=+55b9,LayoutLMv2Encoder.forward..s"A  s )last_hidden_stater attentions) rmrrnr$ enumerater rtraining_gradient_checkpointing_func__call__tupler )rArrrroutput_hidden_states return_dictrMr!all_hidden_statesall_self_attentionsrri layer_modulelayer_head_mask layer_outputss rDrLayoutLMv2Encoder.forwardsV#7BD$5b4JNJjJj$88FptEIEdEdT;;DAjn (4OA#$58H$H!.7.CilO**t}} $ A A ))!"#%#)!B! !-!"#%#) ! *!,M  &9]1=M.  - - KK   " " $ MM   $ $S )  7 8 8{{## ""((* ""((*$ 0 0v9:://44<<#4;;KhKh<i;1rFr'N) rVrWrXrYr config_classbase_model_prefixrKr[r'rFrDr=r=s#L$jrFr=cr[U[RRRR 5(a)[RR RX5$Un[U[RR5(a[RR URURSSUS9n[RRUR5Ul[RRUR5UlUR UlUR"Ul[R$"S[R&UR R(S9UlUR-5H up4UR/U[1XA55 M" AU$)NT) num_featuresr affinetrack_running_stats process_grouprrdevice)rr=rmodules batchnorm _BatchNorm SyncBatchNormconvert_sync_batchnorm detectron2layersFrozenBatchNorm2drPr rqrre running_mean running_vartensorrrUnum_batches_trackednamed_children add_modulemy_convert_sync_batchnorm)rJrS module_outputnamechilds rDrdrds>&%((**44??@@zz''>>vUUM&*++==>>..,,  $' /  %xx11&--@ "XX// < %+%8%8 "$*$6$6 !,1LL%**U[UhUhUoUo,p ),,.   '@'VW/ rFc4^\rSrSrU4SjrSrSrSrU=r$)LayoutLMv2VisualBackboneic j>[TU]5 UR5UlURRR n[ R"U5"UR5n[UR[RRR5(deURUl [URRR5[URRR5:Xde[URRR5nUR!S["R$"URRR5R'USS5SS9 UR!S["R$"URRR5R'USS5SS9 SUl["R*"5(a[,R/S5 SnURR15UR(R2n[4R6"[8R:"[8R:"US U- 5UR<S - 5[8R:"[8R:"USU- 5UR<S- 545UlO([4R@"UR<SS 5Ul[UR<5S :XaJUR<RCURR15UR(RD5 URR15UR(RDUR<S :Xdeg) N pixel_meanrFr# pixel_stdp2z0using `AvgPool2d` instead of `AdaptiveAvgPool2d`)rnrrH)#r%r&get_detectron2_configcfgMODELMETA_ARCHITECTURErgetrbackboner[modelingFPNlen PIXEL_MEAN PIXEL_STDr<r=rr|out_feature_key$are_deterministic_algorithms_enabledloggerwarning output_shapestrider AvgPool2drceilimage_feature_pool_shapepoolAdaptiveAvgPool2dappendchannels)rArB meta_archmodel num_channels input_shapebackbone_striderCs rDr&!LayoutLMv2VisualBackbone.__init__s //1HHNN44 "&&y1$((;%..**=*=*F*F*J*JKKKK 488>>,,-TXX^^5M5M1NNNN488>>445    LL22 3 8 8q! L   dhhnn&>&>?DD\STVWXej   $  5 5 7 7 NNM N$K"mm88:4;O;OPWWO IIdii A(HIFLkLklmLnnoIIdii A(HIFLkLklmLnnoDI,,V-L-LRa-PQDI v.. /1 4  + + 2 24==3M3M3OPTPdPd3e3n3n o}}))+D,@,@AJJfNmNmnoNpppprFcB[R"U5(aUO URUR- UR- nUR U5nX0R nURU5RSS9RSS5R5nU$)NrH) start_dimr) r= is_tensorr`rkrlrtrzrflattenrr)rAimages images_inputfeaturess rDr LayoutLMv2VisualBackbone.forward<s#(??6#:#: QUQ`Q``dhdrdrr ==.00199X&...;EEaKVVXrFc [RR5(aE[RR5(a"[RR 5S:d [ S5e[RR 5n[R R5n[RR5nX2-S:Xd [ S5e[X2-5Vs/sH n[[XB-US-U-55PM" nn[X2-5Vs/sH"n[RRXTS9PM$ nnX-n[URXgS9Ul gs snfs snf)Nr"z/Make sure torch.distributed is set up properly.rzGMake sure the number of processes can be divided by the number of nodesr)ranks)rS)r= distributed is_availableis_initializedget_rank RuntimeErrorcuda device_countget_world_sizerlist new_grouprdrt)rA self_rank node_size world_sizer7node_global_rankssync_bn_groups node_ranks rDsynchronize_batch_norm/LayoutLMv2VisualBackbone.synchronize_batch_normCs@    * * , ,!!0022!!**,r1PQ Q%%..0 JJ++- &&557 &!+hi iV[\f\sVtuVtQRT% A7J"KLVtuMRS]SjMk MkE   ' '.?.B ' CMk  * 1$--~Ohi v s 0'E3()E8)rtrprzr) rVrWrXrYr&rrr[r\r]s@rDriris!qFjjrFric.^\rSrSrU4SjrSrSrU=r$)LayoutLMv2PooleriZc>[TU]5 [R"URUR5Ul[R "5Ulgr)r%r&rror)rTanh activationr@s rDr&LayoutLMv2Pooler.__init__[s9 YYv1163E3EF '')rFc\USS2S4nURU5nURU5nU$)Nr)rr)rArfirst_token_tensor pooled_outputs rDrLayoutLMv2Pooler.forward`s6+1a40 #56  6 rF)rrrr]s@rDrrZs$ rFrc^\rSrSrU4SjrSrSrSSjrSrSr SSjr \ SS \ \ RS \ \ RS \ \ RS \ \ RS \ \ RS\ \ RS\ \ RS\ \ RS\ \S\ \S\ \S\\\44Sjj5rSrU=r$)rIiic>[US5 [TU] U5 XlURUl[ U5Ul[U5Ul[R"URSUR5Ul UR(aG[R"[R"SUR5R S5Ul[R$"URUR&S9Ul[R*"UR,5Ul[1U5Ul[5U5UlUR95 g)Nr[r"rrr)rr%r&rBhas_visual_segment_embeddingr embeddingsrivisualrrorr) visual_projrqr'rrCr7r8visual_LayerNormr9r:visual_dropoutrencoderrpooler post_initr@s rDr&LayoutLMv2Model.__init__ks$ -   ,2,O,O).v6.v6 99V%D%DR%H&J\J\]  , ,,.LLaI[I[9\9c9cde9f,gD ) " V-?-?VEZEZ [ jj)C)CD(0 &v.  rFc.URR$rrr+rAs rDget_input_embeddings$LayoutLMv2Model.get_input_embeddingss...rFc$XRlgrr)rArws rDset_input_embeddings$LayoutLMv2Model.set_input_embeddingss */'rFcUbUR5nOUR5SSnUSnUcN[R"U[RURS9nUR S5R U5nUc[R"U5nUcURRU5nURRU5nURRU5n URRU5n XX-U -U -n URRU 5n URRU 5n U $)Nr"rrTr)r{r=r>rrUr expand_asrrr+r-rTr6r7r;) rA input_idsrMr!token_type_ids inputs_embedsr seq_lengthr-rSr6rs rD_calc_text_embeddings%LayoutLMv2Model._calc_text_embeddingss!  #..*K',,.s3K ^   << %**YM]M]^L'11!4>>yIL  !"--i8N   OO;;IFM"ooAA,O&*oo&W&WX\&]# $ E En U"8;VVYnn __..z: __,,Z8 rFcBURURU55nURRU5nURR U5nXE-U-nUR (aXpR - nURU5nURU5nU$r) rrrr-rTrrCrr)rAimagerMr!visual_embeddingsr-rSrs rD_calc_img_embeddings$LayoutLMv2Model._calc_img_embeddingss ,,T[[-?@"ooAA,O&*oo&W&WX\&]#&rrBrstackrepeatrr|r{)rArrMrU final_shape visual_bbox_x visual_bbox_y visual_bboxs rD_calc_visual_bbox!LayoutLMv2Model._calc_visual_bboxs LL03a78jj   KK 0 0 3!   LL < rFcUbUb [S5eUbUR5$UbUR5SS$[S5e)NDYou cannot specify both input_ids and inputs_embeds at the same timer"5You have to specify either input_ids or inputs_embeds)rhr{)rArrs rD_get_input_shape LayoutLMv2Model._get_input_shapesT  ]%>cd d  ">># #  & %%', ,TU UrFrrMrrrr!rrrr3r4rc " U bU OURRn U bU OURRn U bU OURRn UR X5n Ub UR O UR n [ U 5nURRSURRS-US'[R"U5n[ UR X55nUS==US- ss'[R"U5nURURRX-U5n[R"UU/SS9nUc[R"XS9n[R"XS9n[R"UU/SS9nUc$[R"U [RU S9nUc5U SnURR SS2SU24nUR#U 5n[R$"SUS[RU S9R'U SS5n[R"UU/SS9nUc:[R"[)[ U 5S/-5[RU S9nUR+UUUUUS9nUR-UUUS 9n[R"UU/SS9nUR/S5R/S 5nUR1UR2S 9nS U- [R4"UR25R6-nUbUR95S:XahUR/S5R/S5R/S 5R/S 5nUR#URR:S S S S 5nOCUR95S :Xa/UR/S5R/S 5R/S 5nUR1[=UR?55R2S 9nOS/URR:-nURAUUUUUU U U S9nUSnURCU5nU (d UU4USS-$[EUUURFURHS9$)a{ bbox (`torch.LongTensor` of shape `((batch_size, sequence_length), 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. image (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)` or `detectron.structures.ImageList` whose `tensors` is of shape `(batch_size, num_channels, height, width)`): Batch of document images. Examples: ```python >>> from transformers import AutoProcessor, LayoutLMv2Model, set_seed >>> from PIL import Image >>> import torch >>> from datasets import load_dataset >>> set_seed(0) >>> processor = AutoProcessor.from_pretrained("microsoft/layoutlmv2-base-uncased") >>> model = LayoutLMv2Model.from_pretrained("microsoft/layoutlmv2-base-uncased") >>> dataset = load_dataset("hf-internal-testing/fixtures_docvqa", trust_remote_code=True) >>> image_path = dataset["test"][0]["file"] >>> image = Image.open(image_path).convert("RGB") >>> encoding = processor(image, return_tensors="pt") >>> outputs = model(**encoding) >>> last_hidden_states = outputs.last_hidden_state >>> last_hidden_states.shape torch.Size([1, 342, 768]) ``` NrrrI)rUrT)rrMrr!rrrMr!rH)rrBr")rMr!rrr3r4)r, pooler_outputrr-)%rBrr3use_return_dictrrUrrr=SizerrLonesrrrrr!r?r>rr2rrrrrrrrJrnext parametersrrr rr-)rArrMrrrr!rrrr3r4rrU visual_shaperr final_bboxvisual_attention_maskfinal_attention_maskrvisual_position_idsfinal_position_idstext_layout_emb visual_emb final_embextended_attention_maskencoder_outputssequence_outputrs rDrLayoutLMv2Model.forwardsFf2C1N-TXT_T_TqTq$8$D $++JjJj &1%>qADKKDhDhijDkk Qzz,/ 400JK A,q/)jj- ,,T[[-Q-QSWalm YYk2:  !"ZZ CN % < G$yy.:O)PVWX  !"[[EJJvVN  $QJ??77;J;GL'..{;L#ll1l1oUZZX^_ff NA #YY 6I'JPQR <;;uT+%6!%<=UZZX^_D44)%' 5 ..,/ II ;C "6"@"@"C"M"Ma"P"9"<"<4::"<"N#&)@#@EKKPTPZPZD[D_D_"_  }}!#%//2<!>>I,,  #+/!5#'  *!, O4 #]3oab6II I)-')77&11   rF) rBrrrrrrrrrCr)NN) NNNNNNNNNNN)rVrWrXrYr&rrrrrrrrr= LongTensor FloatTensorrrrr rr[r\r]s@rDrIrIisR(/02 #JV15+/-16:59371559,0/3&*O E,,-O u''(O ))* O !!2!23 O !!1!12 O u//0O E--.O   1 12O $D>O 'tnO d^O  u00 1O O rFrIax LayoutLMv2 Model with a sequence classification head on top (a linear layer on top of the concatenation of the final hidden state of the [CLS] token, average-pooled initial visual embeddings and average-pooled final visual embeddings, e.g. for document image classification tasks such as the [RVL-CDIP](https://www.cs.cmu.edu/~aharley/rvl-cdip/) dataset. ) custom_introc^\rSrSrU4SjrSr\SS\\RS\\RS\\RS\\RS\\RS \\RS \\RS \\RS \\RS \\ S\\ S\\ S\ \ \44Sjj5rSrU=r$)#LayoutLMv2ForSequenceClassificationikc6>[TU]U5 URUl[U5Ul[ R "UR5Ul[ R"URS-UR5Ul UR5 g)Nr r%r& num_labelsrIr>rr9r:r;ror) classifierrr@s rDr&,LayoutLMv2ForSequenceClassification.__init__tsn   ++)&1zz&"<"<= ))F$6$6$:Frr+rs rDr8LayoutLMv2ForSequenceClassification.get_input_embeddings~))999rFrrMrrrr!rrlabelsrr3r4rc hU bU OURRn UbUb [S5eUb"URX5 UR 5n O"UbUR 5SSn O [S5eUb UR O UR n[ U 5nURRSURRS-US'[R"U5n[ U 5nUS==US- ss'[R"U5nURRURRX.U5n[R"SUS[RUS9RU SS5nURRUUUS9nURUUUUUUUUU U U S 9 nUbUR 5n OUR 5SSn U SnUSSS2SU24USSS2US24nnUSS2SSS24nUR!SS 9nUR!SS 9n[R""UUU/SS 9nUR%U5nUR'U5nSnU GbURR(cUR*S:XaS URlOoUR*S:aNU R,[R:XdU R,[R.:XaS URlOS URlURR(S :XaJ[15nUR*S:Xa&U"UR35U R355nOU"UU 5nOURR(S :Xa=[55nU"UR7SUR*5U R7S55nO-URR(S :Xa[95nU"UU 5nU (dU4USS-nUbU4U-$U$[;UUUR<UR>S9$)a input_ids (`torch.LongTensor` of shape `batch_size, sequence_length`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. image (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)` or `detectron.structures.ImageList` whose `tensors` is of shape `(batch_size, num_channels, height, width)`): Batch of document images. token_type_ids (`torch.LongTensor` of shape `batch_size, sequence_length`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. [What are token type IDs?](../glossary#token-type-ids) position_ids (`torch.LongTensor` of shape `batch_size, sequence_length`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Example: ```python >>> from transformers import AutoProcessor, LayoutLMv2ForSequenceClassification, set_seed >>> from PIL import Image >>> import torch >>> from datasets import load_dataset >>> set_seed(0) >>> dataset = load_dataset("aharley/rvl_cdip", split="train", streaming=True, trust_remote_code=True) >>> data = next(iter(dataset)) >>> image = data["image"].convert("RGB") >>> processor = AutoProcessor.from_pretrained("microsoft/layoutlmv2-base-uncased") >>> model = LayoutLMv2ForSequenceClassification.from_pretrained( ... "microsoft/layoutlmv2-base-uncased", num_labels=dataset.info.features["label"].num_classes ... ) >>> encoding = processor(image, return_tensors="pt") >>> sequence_label = torch.tensor([data["label"]]) >>> outputs = model(**encoding, labels=sequence_label) >>> loss, logits = outputs.loss, outputs.logits >>> predicted_idx = logits.argmax(dim=-1).item() >>> predicted_answer = dataset.info.features["label"].names[4] >>> predicted_idx, predicted_answer # results are not good without further fine-tuning (7, 'advertisement') ``` Nrr"rrrrTr rrMrrrr!rrrr3r4rI regressionsingle_label_classificationmulti_label_classificationrHlosslogitsrr-) rBrrh%warn_if_padding_and_no_attention_maskr{rUrrr=rr>rr>rrrr@rLr;r problem_typerrrjr squeezerr|rrrr-)rArrMrrrr!rrr rr3r4rrUrrrrinitial_image_embeddingsrrrfinal_image_embeddingscls_final_outputpooled_initial_image_embeddingspooled_final_image_embeddingsrrloss_fctrs rDr+LayoutLMv2ForSequenceClassification.forwardsb&1%cd d  "  6 6y Q#..*K  &',,.s3KTU U%.%:!! @T@TK( ++>>qADKKDhDhijDkk Qzz,/ ;' A,q/)jj- oo77 KK 0 0$  $ll1l1oUZZX^_ff NA $(??#G#G,$H$  //))%'/!5#"    #..*K',,.s3K ^ 29!*Q  ^2LgVWjYZ\f\gYgNh/*1a73+C*G*GA*G*N'(>(C(C(C(J%)) >@] ^de ,,71  {{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#FF3D))-JJ+- B @&++b/R))-II,./Y,F)-)9TGf$ Ev E'!//))   rFrr;r>r NNNNNNNNNNNN)rVrWrXrYr&rrrr=rrrrrrrr[r\r]s@rDrrksP:15+/-16:59371559-1,0/3&*s E,,-s u''(s ))* s !!2!23 s !!1!12 s u//0s E--.s   1 12s ))*s $D>s 'tns d^s  u.. /s s rFra LayoutLMv2 Model with a token classification head on top (a linear layer on top of the text part of the hidden states) e.g. for sequence labeling (information extraction) tasks such as [FUNSD](https://guillaumejaume.github.io/FUNSD/), [SROIE](https://rrc.cvc.uab.es/?ch=13), [CORD](https://github.com/clovaai/cord) and [Kleister-NDA](https://github.com/applicaai/kleister-nda). c^\rSrSrU4SjrSr\SS\\RS\\RS\\RS\\RS\\RS \\RS \\RS \\RS \\RS \\ S\\ S\\ S\ \ \44Sjj5rSrU=r$) LayoutLMv2ForTokenClassificationi8c0>[TU]U5 URUl[U5Ul[ R "UR5Ul[ R"URUR5Ul UR5 grrr@s rDr&)LayoutLMv2ForTokenClassification.__init__Asi   ++)&1zz&"<"<= ))F$6$68I8IJ rFcBURRR$rrrs rDr5LayoutLMv2ForTokenClassification.get_input_embeddingsKrrFrrMrrrr!rrr rr3r4rc U bU OURRn URUUUUUUUUU U U S9 n UbUR5nOUR5SSnUSnU SSS2SU24nUR U5nUR U5nSnU b<[ 5nU"URSUR5U RS55nU (dU4U SS-nUbU4U-$U$[UUU RU RS9$)a input_ids (`torch.LongTensor` of shape `batch_size, sequence_length`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. image (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)` or `detectron.structures.ImageList` whose `tensors` is of shape `(batch_size, num_channels, height, width)`): Batch of document images. token_type_ids (`torch.LongTensor` of shape `batch_size, sequence_length`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. [What are token type IDs?](../glossary#token-type-ids) position_ids (`torch.LongTensor` of shape `batch_size, sequence_length`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. Example: ```python >>> from transformers import AutoProcessor, LayoutLMv2ForTokenClassification, set_seed >>> from PIL import Image >>> from datasets import load_dataset >>> set_seed(0) >>> datasets = load_dataset("nielsr/funsd", split="test", trust_remote_code=True) >>> labels = datasets.features["ner_tags"].feature.names >>> id2label = {v: k for v, k in enumerate(labels)} >>> processor = AutoProcessor.from_pretrained("microsoft/layoutlmv2-base-uncased", revision="no_ocr") >>> model = LayoutLMv2ForTokenClassification.from_pretrained( ... "microsoft/layoutlmv2-base-uncased", num_labels=len(labels) ... ) >>> data = datasets[0] >>> image = Image.open(data["image_path"]).convert("RGB") >>> words = data["words"] >>> boxes = data["bboxes"] # make sure to normalize your bounding boxes >>> word_labels = data["ner_tags"] >>> encoding = processor( ... image, ... words, ... boxes=boxes, ... word_labels=word_labels, ... padding="max_length", ... truncation=True, ... return_tensors="pt", ... ) >>> outputs = model(**encoding) >>> logits, loss = outputs.logits, outputs.loss >>> predicted_token_class_ids = logits.argmax(-1) >>> predicted_tokens_classes = [id2label[t.item()] for t in predicted_token_class_ids[0]] >>> predicted_tokens_classes[:5] # results are not good without further fine-tuning ['I-HEADER', 'I-HEADER', 'I-QUESTION', 'I-HEADER', 'I-QUESTION'] ``` Nr r"rrrHr) rBrr>r{r;rrr|rrrr-)rArrMrrrr!rrr rr3r4rrrrrrrrs rDr(LayoutLMv2ForTokenClassification.forwardNs:t&1%A 'tnA d^A  u++ ,A A rFrc ^\rSrSrSU4SjjrSr\SS\\RS\\RS\\RS\\RS\\RS \\RS \\RS \\RS \\RS \\RS\\ S\\ S\\ S\ \ \44Sjj5rSrU=r$)LayoutLMv2ForQuestionAnsweringic>[TU]U5 URUlX!l[ U5Ul[ R"URUR5Ul UR5 g)z} has_visual_segment_embedding (`bool`, *optional*, defaults to `True`): Whether or not to add visual segment embeddings. N) r%r&rrrIr>rror) qa_outputsr)rArBrrCs rDr&'LayoutLMv2ForQuestionAnswering.__init__s[   ++.J+)&1))F$6$68I8IJ rFcBURRR$rrrs rDr3LayoutLMv2ForQuestionAnswering.get_input_embeddingsrrFrrMrrrr!rrstart_positions end_positionsrr3r4rcU bU OURRn URUUUUUUUUU U U S9 nUbUR5nOUR5SSnUSnUSSS2SU24nUR U5nUR SSS9unnUR S5R5nUR S5R5nSnU bU b[U R55S:aU R S5n [U R55S:aU R S5n URS5nU RSU5n U RSU5n [US9nU"UU 5nU"UU 5nUU-S- nU (dUU4USS-nUbU4U-$U$[UUUURURS 9$) u input_ids (`torch.LongTensor` of shape `batch_size, sequence_length`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. image (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)` or `detectron.structures.ImageList` whose `tensors` is of shape `(batch_size, num_channels, height, width)`): Batch of document images. token_type_ids (`torch.LongTensor` of shape `batch_size, sequence_length`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. [What are token type IDs?](../glossary#token-type-ids) position_ids (`torch.LongTensor` of shape `batch_size, sequence_length`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) Example: In this example below, we give the LayoutLMv2 model an image (of texts) and ask it a question. It will give us a prediction of what it thinks the answer is (the span of the answer within the texts parsed from the image). ```python >>> from transformers import AutoProcessor, LayoutLMv2ForQuestionAnswering, set_seed >>> import torch >>> from PIL import Image >>> from datasets import load_dataset >>> set_seed(0) >>> processor = AutoProcessor.from_pretrained("microsoft/layoutlmv2-base-uncased") >>> model = LayoutLMv2ForQuestionAnswering.from_pretrained("microsoft/layoutlmv2-base-uncased") >>> dataset = load_dataset("hf-internal-testing/fixtures_docvqa", trust_remote_code=True) >>> image_path = dataset["test"][0]["file"] >>> image = Image.open(image_path).convert("RGB") >>> question = "When is coffee break?" >>> encoding = processor(image, question, return_tensors="pt") >>> outputs = model(**encoding) >>> predicted_start_idx = outputs.start_logits.argmax(-1).item() >>> predicted_end_idx = outputs.end_logits.argmax(-1).item() >>> predicted_start_idx, predicted_end_idx (30, 191) >>> predicted_answer_tokens = encoding.input_ids.squeeze()[predicted_start_idx : predicted_end_idx + 1] >>> predicted_answer = processor.tokenizer.decode(predicted_answer_tokens) >>> predicted_answer # results are not good without further fine-tuning '44 a. m. to 12 : 25 p. m. 12 : 25 to 12 : 58 p. m. 12 : 58 to 4 : 00 p. m. 2 : 00 to 5 : 00 p. m. coffee break coffee will be served for men and women in the lobby adjacent to exhibit area. please move into exhibit area. ( exhibits open ) trrf general session ( part | ) presiding : lee a. waller trrf vice president “ introductory remarks ” lee a. waller, trrf vice presi - dent individual interviews with trrf public board members and sci - entific advisory council mem - bers conducted by trrf treasurer philip g. kuehn to get answers which the public refrigerated warehousing industry is looking for. plus questions from' ``` ```python >>> target_start_index = torch.tensor([7]) >>> target_end_index = torch.tensor([14]) >>> outputs = model(**encoding, start_positions=target_start_index, end_positions=target_end_index) >>> predicted_answer_span_start = outputs.start_logits.argmax(-1).item() >>> predicted_answer_span_end = outputs.end_logits.argmax(-1).item() >>> predicted_answer_span_start, predicted_answer_span_end (30, 191) ``` Nr r"rrrI) ignore_indexrH)r start_logits end_logitsrr-)rBrr>r{r)splitrrrwclamprrrr-)rArrMrrrr!rrr-r.rr3r4rrrrrr1r2 total_loss ignored_indexr start_lossend_lossrs rDr&LayoutLMv2ForQuestionAnswering.forwardst&1%rr))T) NNNNNNNNNNNNN)rVrWrXrYr&rrrr=rrrrrrrr[r\r]s@rDr'r'si :15+/-16:593715596:48,0/3&*R E,,-R u''(R ))* R !!2!23 R !!1!12 R u//0R E--.R   1 12R "%"2"23R   0 01R $D>R 'tnR d^R  u22 3R R rFr')r'rrrrIr=)T r)=rZrtypingrrrr=torch.utils.checkpointrtorch.nnrrr activationsr modeling_outputsr r rrrmodeling_utilsr pytorch_utilsrutilsrrrrconfiguration_layoutlmv2rr[detectron2.modelingrModule_load_from_state_dictr\ batch_normr] get_loggerrVr|rr_rrrrrrrr=rdrirrIrrr'__all__r'rFrDrKs )) AA!.6XX66LQ88??KpKpJ  22H   H %,+299,+^ZbiiZz"))8 299  RYY  ryy (bii(V)XA  A Hjjj80?jryy?jD ryy ~ /~ ~ BB *CB B JP '@P P fe %>e e P rF