fThSrSSKrSSKrSSKJrJrJr SSKrSSKrSSKJ r SSK J r J r J r SSKJr SSKJrJrJrJrJrJr SS KJrJrJrJr SS KJrJr S S KJ r \RB"\"5r#"S S\ RH5r%S7Sjr&"SS\ RH5r'"SS\ RH5r("SS\ RH5r)"SS\ RH5r*"SS\ RH5r+"SS\ RH5r,"SS\ RH5r-"SS\ RH5r."S S!\ RH5r/S"\/0r0"S#S$\ RH5r1"S%S&\ RH5r2"S'S(\ RH5r3\"S)S*\55r4\"S+S,\455r5\"S-S.\455r6\"S/S09"S1S2\455r7\"S3S09"S4S5\455r8/S6Qr9g)8zPyTorch MarkupLM model.N)OptionalTupleUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN))BaseModelOutputWithPastAndCrossAttentions,BaseModelOutputWithPoolingAndCrossAttentionsMaskedLMOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)PreTrainedModelapply_chunking_to_forward find_pruneable_heads_and_indicesprune_linear_layer)auto_docstringlogging)MarkupLMConfigc6^\rSrSrSrU4SjrSSjrSrU=r$)XPathEmbeddings0zConstruct the embeddings from xpath tags and subscripts. We drop tree-id in this version, as its info can be covered by xpath. c >[[U] 5 URUl[R "UR UR-UR5Ul[R"UR5Ul [R"5Ul [R "UR UR-SUR-5Ul[R "SUR-UR5Ul[R "[#UR5Vs/sH.n[R$"UR&UR 5PM0 sn5Ul[R "[#UR5Vs/sH.n[R$"UR*UR 5PM0 sn5Ulgs snfs snf)N)superr__init__ max_depthrLinearxpath_unit_hidden_size hidden_sizexpath_unitseq2_embeddingsDropouthidden_dropout_probdropoutReLU activationxpath_unitseq2_inner inner2emb ModuleListrange Embeddingmax_xpath_tag_unit_embeddingsxpath_tag_sub_embeddingsmax_xpath_subs_unit_embeddingsxpath_subs_sub_embeddingsselfconfig_ __class__s f/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/markuplm/modeling_markuplm.pyr XPathEmbeddings.__init__6sf ot-/)))+63P3PSWSaSa3acicucu)v&zz&"<"<= '')$&IIf.K.Kdnn.\^_bhbtbt^t$u!1v'9'9#96;M;MN(* t~~. .A VAA6C`C`a. ) %*,t~~. .A VBBFDaDab. * &   s ;5G%#5G*c /n/n[UR5H_nURURU"USS2SS2U455 URURU"USS2SS2U455 Ma [ R "USS9n[ R "USS9nX4-nURURURURU5555nU$)Ndim) r.r!appendr1r3torchcatr,r(r*r+)r5xpath_tags_seqxpath_subs_seqxpath_tags_embeddingsxpath_subs_embeddingsixpath_embeddingss r9forwardXPathEmbeddings.forwardPs " "t~~&A ! ( ()F)Fq)I.YZ\]_`Y`Ja)b c ! ( ()G)G)J>Z[]^`aZaKb)c d'!& *?R H % *?R H0H>>$,,ttG`G`aqGr7s*tu)r*r(r,r!r3r1r%r+)NN) __name__ __module__ __qualname____firstlineno____doc__r rH__static_attributes__ __classcell__r8s@r9rr0s  4  rJrcURU5R5n[R"USS9R U5U-U-nUR 5U-$)z Replace non-padding symbols with their position numbers. Position numbers begin at padding_idx+1. Padding symbols are ignored. This is modified from fairseq's `utils.make_positions`. Args: x: torch.Tensor x: Returns: torch.Tensor rr=)neintr@cumsumtype_aslong) input_ids padding_idxpast_key_values_lengthmaskincremental_indicess r9"create_position_ids_from_input_idsr^csW << $ ( ( *D <<!4<[[U] 5 Xl[R "UR URURS9Ul [R "URUR5Ul URUl [U5Ul[R "URUR5Ul[R""URUR$S9Ul[R&"UR(5UlUR-S[.R0"UR5R3S5SS9 URUl[R "URURUR4S9Ul g)N)rZeps position_ids)rr<F) persistent)rr`r r6rr/ vocab_sizer$ pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingsr!rrGtype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsr&r'r(register_bufferr@arangeexpandrZr5r6r8s r9r MarkupLMEmbeddings.__init__vs;  $02 !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c )) / 7%'\\&2H2H&J\J\%]"f&8&8f>S>STzz&"<"<=   ELL)G)GHOOPWXej  "..#%<<  * *F,>,>DL\L\$  rJcUR5SSnUSn[R"URS-X0R-S-[RUR S9nUR S5RU5$)z We are provided embeddings directly. We cannot infer which are padded so just generate sequential position ids. Args: inputs_embeds: torch.Tensor Returns: torch.Tensor Nr<rdtypedevicer)sizer@rqrZrXrx unsqueezerr)r5 inputs_embeds input_shapesequence_lengthres r9&create_position_ids_from_inputs_embeds9MarkupLMEmbeddings.create_position_ids_from_inputs_embedss~$((*3B/ %a.||   q /4D4D"Dq"HPUPZPZcpcwcw %%a(// <>*::;5::V\7N  !![[44uzzd;'4>>*::;5::V\8N)"66|D $ : :> J00P%;>SSVff ^^J/ \\*- rJ) rnr6r(r!rZrkrmrirG)NNNNNNr) rKrLrMrNrOr r~rHrPrQrRs@r9r`r`ss1Q 2=& 22rJr`cz^\rSrSrU4SjrS\R S\R S\R 4SjrSrU=r $)MarkupLMSelfOutputc(>[TU]5 [R"URUR5Ul[R "URURS9Ul[R"UR5Ul gNrc) rr rr"r$densernror&r'r(rss r9r MarkupLMSelfOutput.__init__s` YYv1163E3EF f&8&8f>S>STzz&"<"<= rJ hidden_states input_tensorreturncpURU5nURU5nURX-5nU$Nrr(rnr5rrs r9rHMarkupLMSelfOutput.forward5 =1  ]3 }'CD rJrnrr( rKrLrMrNr r@TensorrHrPrQrRs@r9rr6> U\\RWR^R^rJrcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)MarkupLMIntermediatec>[TU]5 [R"URUR 5Ul[UR[5(a[URUl gURUl gr) rr rr"r$intermediate_sizer isinstance hidden_actstrr intermediate_act_fnrss r9r MarkupLMIntermediate.__init__s` YYv1163K3KL f'' - -'-f.?.?'@D $'-'8'8D $rJrrcJURU5nURU5nU$rrrr5rs r9rHMarkupLMIntermediate.forwards& =1 00? rJrrrRs@r9rrs(9U\\ellrJrcz^\rSrSrU4SjrS\R S\R S\R 4SjrSrU=r $)MarkupLMOutputc(>[TU]5 [R"URUR 5Ul[R"UR URS9Ul[R"UR5Ul gr) rr rr"rr$rrnror&r'r(rss r9r MarkupLMOutput.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= rJrrrcpURU5nURU5nURX-5nU$rrrs r9rHMarkupLMOutput.forwardrrJrrrRs@r9rrrrJrcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)MarkupLMPooleric>[TU]5 [R"URUR5Ul[R "5Ulgr)rr rr"r$rTanhr*rss r9r MarkupLMPooler.__init__s9 YYv1163E3EF '')rJrrc\USS2S4nURU5nURU5nU$)Nr)rr*)r5rfirst_token_tensor pooled_outputs r9rHMarkupLMPooler.forward s6+1a40 #56  6 rJ)r*rrrRs@r9rrs($ U\\ellrJrcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)MarkupLMPredictionHeadTransformicp>[TU]5 [R"URUR5Ul[ UR[5(a[URUl OURUl [R"URURS9Ul gr) rr rr"r$rrrrr transform_act_fnrnrorss r9r (MarkupLMPredictionHeadTransform.__init__s~ YYv1163E3EF f'' - -$*6+<+<$=D !$*$5$5D !f&8&8f>S>STrJrrclURU5nURU5nURU5nU$r)rrrnrs r9rH'MarkupLMPredictionHeadTransform.forwards4 =1 --m< }5 rJ)rnrrrrRs@r9rrs)UU\\ellrJrc4^\rSrSrU4SjrSrSrSrU=r$)MarkupLMLMPredictionHeadi&cH>[TU]5 [U5Ul[R "UR URSS9Ul[R"[R"UR55Ul URURl g)NF)bias) rr r transformrr"r$rgdecoder Parameterr@rrrss r9r !MarkupLMLMPredictionHead.__init__'sm 8@yy!3!3V5F5FUS LLV->->!?@ !II rJc:URURlgr)rrr5s r9 _tie_weights%MarkupLMLMPredictionHead._tie_weights4s II rJcJURU5nURU5nU$r)rrrs r9rH MarkupLMLMPredictionHead.forward7s$}5  ]3 rJ)rrr) rKrLrMrNr rrHrPrQrRs@r9rr&s &&rJrcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)MarkupLMOnlyMLMHeadi>cB>[TU]5 [U5Ulgr)rr r predictionsrss r9r MarkupLMOnlyMLMHead.__init__?s 3F;rJsequence_outputrc(URU5nU$rr)r5rprediction_scoress r9rHMarkupLMOnlyMLMHead.forwardCs ,,_=  rJrrrRs@r9rr>s(<!u||! !!rJrcb^\rSrSrSU4SjjrS\R S\R 4SjrSS\R S\\RS\\RS \\RS \\RS \\ \ \RS \\ S\ \R 4S jjr Sr U=r$)MarkupLMSelfAttentioniIc>[TU]5 URUR-S:wa7[ US5(d&[ SURSURS35eURUl[ URUR- 5UlURUR-Ul[R"URUR5Ul [R"URUR5Ul [R"URUR5Ul [R"UR5UlU=(d [#USS5UlUR$S:XdUR$S :XaGUR&Ul[R("S UR&-S - UR5UlUR,Ulg) Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()position_embedding_typeabsolute relative_keyrelative_key_queryr)rr r$num_attention_headshasattr ValueErrorrUattention_head_size all_head_sizerr"querykeyvaluer&attention_probs_dropout_probr(getattrrrjr/distance_embedding is_decoderr5r6rr8s r9r MarkupLMSelfAttention.__init__Js     : : :a ?PVXhHiHi#F$6$6#78 445Q8  $*#=#= #&v'9'9F( ' -zC $  ' '> 9T=Y=Y]q=q+1+I+ID (&(ll1v7U7U3UXY3Y[_[s[s&tD # ++rJxrcUR5SSURUR4-nURU5nUR SSSS5$)Nr<rrrr )ryrrviewpermute)r5r new_x_shapes r9transpose_for_scores*MarkupLMSelfAttention.transpose_for_scoresdsLffhsmt'?'?AYAY&ZZ FF; yyAq!$$rJrattention_mask head_maskencoder_hidden_statesencoder_attention_maskpast_key_valueoutput_attentionscVURU5nUSLn U (aUbUSn USn UnGOU (aCURURU55n URURU55n UnOUbuURURU55n URURU55n [R "USU /SS9n [R "USU /SS9n O@URURU55n URURU55n URU5n USLn UR (aX4n[R"XRSS55nURS:XdURS:XGaU RSU RSnnU (aB[R"US- [RURS 9RSS5nO>[R"U[RURS 9RSS5n[R"U[RURS 9RSS5nUU- nUR!UUR"-S- 5nUR%U R&S 9nURS:Xa[R("S U U5nUU-nOHURS:Xa8[R("S U U5n[R("S U U5nUU-U-nU[*R,"UR.5- nUbX-n[0R2R5USS9nUR7U5nUbUU-n[R"UU 5nUR9SSSS 5R;5nUR=5SSUR>4-nURU5nU(aUU4OU4nUR (aUU4-nU$)Nrrrr=r<rrrvrwzbhld,lrd->bhlrzbhrd,lrd->bhlrr ) rrrrr@rArmatmul transposershapetensorrXrxrrqrrjtorweinsummathsqrtrr functionalsoftmaxr(r contiguousryr)r5rrrrrrrmixed_query_layeris_cross_attention key_layer value_layer query_layer use_cacheattention_scores query_length key_lengthposition_ids_lposition_ids_rdistancepositional_embeddingrelative_position_scoresrelative_position_scores_queryrelative_position_scores_keyattention_probs context_layernew_context_layer_shapeoutputss r9rHMarkupLMSelfAttention.forwardis!JJ}5 3$> ."<&q)I(+K3N 11$((;P2QRI33DJJ?T4UVK3N  '11$((=2IJI33DJJ}4MNK >!#4i"@aHI))^A%6 $D!LK11$((=2IJI33DJJ}4MNK//0AB "$. ??(5N!<< 5H5HR5PQ  ' '> 9T=Y=Y]q=q'2'8'8';Y__Q=O*L!&j1nEJJWdWkWk!l!q!q""'l%**UbUiUi!j!o!oprtu!v"\\*EJJ}OcOcdiijkmopN%6H#'#:#:8dFbFb;bef;f#g #7#:#:ARAR#:#S ++~=+0<<8H+Wk+l(#36N#N --1EE16>NP[]q1r./4||S"S%**+BC 6G=/2mM] ?? 11GrJ) rrrr(rrrjrrrrrNNNNNF)rKrLrMrNr r@rrr FloatTensorrboolrHrPrQrRs@r9rrIs,4%ell%u||%7;15=A>BDH,1c||c!!2!23cE--. c ((9(9: c !)):): ; c!uU->->'?!@Ac$D>c u|| ccrJreagerc.^\rSrSrSU4SjjrSrSS\RS\\RS\\RS\\RS\\RS \\ \ \RS \\ S \ \R4S jjr S r U=r$)MarkupLMAttentionic>[TU]5 [UR"XS9Ul[ U5Ul[5Ulg)Nr) rr MARKUPLM_SELF_ATTENTION_CLASSES_attn_implementationr5routputset pruned_headsrs r9r MarkupLMAttention.__init__s@ 3F4O4OP   )0 ErJc6[U5S:Xag[XRRURRUR 5up[ URRU5URl[ URRU5URl[ URRU5URl [ URRUSS9URl URR[U5- URlURRURR-URl UR RU5Ulg)Nrrr=)lenrr5rrr0rrrrr.rrunion)r5headsindexs r9 prune_headsMarkupLMAttention.prune_headss  u:? 7 9900$))2O2OQUQbQb  -TYY__eD *499==%@ ,TYY__eD .t{{/@/@%QO )- (E(EE (R %"&))"?"?$))B_B_"_  --33E:rJrrrrrrrrc pURUUUUUUU5nURUSU5n U 4USS-n U $)Nrr)r5r.) r5rrrrrrr self_outputsattention_outputr"s r9rHMarkupLMAttention.forwardsWyy    ! "    ;;|A F#% QR(88rJ)r.r0r5rr$)rKrLrMrNr r7r@rrr%rr&rHrPrQrRs@r9r)r)s";*7;15=A>BDH,1||!!2!23E--.  ((9(9:  !)):): ; !uU->->'?!@A$D> u|| rJr)c*^\rSrSrU4SjrSS\R S\\RS\\RS\\RS\\RS\\ \ \RS \\ S \ \R 4S jjr S r S r U=r$) MarkupLMLayeri ct>[TU]5 URUlSUl[ U5UlUR UlURUlUR(a.UR (d[US35e[ USS9Ul [U5Ul [U5Ul g)Nrz> should be used as a decoder model if cross attention is addedrr+)rr chunk_size_feed_forward seq_len_dimr) attentionradd_cross_attentionrcrossattentionr intermediaterr.rss r9r MarkupLMLayer.__init__ s '-'E'E$*62 ++#)#=#=  # #?? D6)g!hii"3FT^"_D 08$V, rJrrrrrrrrc UbUSSOSnURUUUUUS9n U Sn UR(a U SSn U Sn OU SSn Sn UR(aZUbW[US5(d[SUS35eUbUS SOSnUR U UUUUUU5nUSn XSS-n USn W U -n [ UR URURU 5nU4U -n UR(aU W 4-n U $) Nr)rrrrr<rDz'If `encoder_hidden_states` are passed, z` has to be instantiated with cross-attention layers by setting `config.add_cross_attention=True`r) rBrrrrDrfeed_forward_chunkr@rA)r5rrrrrrrself_attn_past_key_valueself_attention_outputsr;r"present_key_valuecross_attn_present_key_valuecross_attn_past_key_valuecross_attention_outputs layer_outputs r9rHMarkupLMLayer.forwards}:H9S>"1#5Y] !%   /3 "0" 2!4 ??,Qr2G 6r : ,QR0G'+$ ??4@4!122 =dVDDD @N?Yrs(;_c %&*&9&9 %&)!' # 7q9 " ==G,C2+F ( 14P P 0  # #T%A%A4CSCSUe  /G+ ??!2 44GrJcJURU5nURX!5nU$r)rEr.)r5r;intermediate_outputrOs r9rH MarkupLMLayer.feed_forward_chunkYs)"//0@A{{#6I rJ)rCrBr@rDrErr.rAr$)rKrLrMrNr r@rrr%rr&rHrHrPrQrRs@r9r>r> s -"7;15=A>BDH,1?||?!!2!23?E--. ? ((9(9: ? !)):): ; ?!uU->->'?!@A?$D>? u|| ?BrJr>cR^\rSrSrU4SjrSS\R S\\RS\\RS\\RS\\RS\\ \ \RS \\ S \\ S \\ S \\ S \ \ \R \ 44Sjjr SrU=r$)MarkupLMEncoderi`c>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snf)NF) rr r6rr-r.num_hidden_layersr>layergradient_checkpointingr4s r9r MarkupLMEncoder.__init__asR  ]]5IaIaCb#cCbaM&$9Cb#cd &+#$dsA&rrrrrpast_key_valuesrroutput_hidden_states return_dictrc 8U (aSOSn U(aSOSn U(aURR(aSOSn UR(a/UR(aU(a[R S5 SnU(aSOSn[ UR5HunnU (aX4-n UbX?OSnUbXoOSnUR(a4UR(a#URURUUUUUUU5nOU"UUUUUUU5nUSnU(a UUS4- nU(dMU US4-n URR(dMU US4-n M U (aX4-n U (d[SUUU U U 455$[UUU U U S 9$) NzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...Frr<rrc30# UH nUcMUv M g7frr_).0vs r9 *MarkupLMEncoder.forward..s" As )last_hidden_stater[r attentionscross_attentions) r6rCrYtraininglogger warning_once enumeraterX_gradient_checkpointing_func__call__rr )r5rrrrrr[rrr\r]all_hidden_statesall_self_attentionsall_cross_attentionsnext_decoder_cacherF layer_modulelayer_head_maskr layer_outputss r9rHMarkupLMEncoder.forwardgs#7BD$5b4%64;;;Z;Zr`d  & &4==##p" #,R$(4OA|#$58H$H!.7.CilO3B3N_/TXN**t}} $ A A ))!"#)*"% ! !-!"#)*"%! *!,M"}R'8&::"  &9]1=MBEI$(,1/4&*S ||S !!2!23S E--. S ((9(9: S !)):): ; S "%e.?.?(@"ABS D>S $D>S 'tnS d^S  uU\\"$MM NS S rJrUcl^\rSrSr\rSrSr\S\ \ \ \ R44U4Sjj5rSrU=r$)MarkupLMPreTrainedModelimarkuplmc[U[R5(akURRR SUR RS9 URb%URRR5 gg[U[R5(axURRR SUR RS9 URb2URRURR5 gg[U[R5(aJURRR5 URRRS5 g[U[5(a%URRR5 gg)zInitialize the weightsg)meanstdN?)rrr"weightdatanormal_r6initializer_rangerzero_r/rZrnfill_r)r5modules r9 _init_weights%MarkupLMPreTrainedModel._init_weightss3 fbii ( ( MM   & &CT[[5R5R & S{{&   &&('  - - MM   & &CT[[5R5R & S!!- ""6#5#56<<>.  - - KK   " " $ MM   $ $S )  8 9 9 KK   " " $:rJpretrained_model_name_or_pathc4>[[U] "U/UQ70UD6$r)rrwfrom_pretrained)clsr model_argskwargsr8s r9r'MarkupLMPreTrainedModel.from_pretraineds+,cB ) ,6 :@  rJr_)rKrLrMrNr config_classbase_model_prefixr classmethodrrrosPathLikerrPrQrRs@r9rwrwsD!L"%$ HU3PRP[P[K[E\<]  rJrwc^\rSrSrSU4SjjrSrSrSr\SS\ \ RS\ \ RS\ \ RS \ \ RS \ \ RS \ \ RS \ \ RS \ \ RS\ \ S\ \ S\ \ S\\\44Sjj5rSrSrU=r$) MarkupLMModelic>[TU]U5 Xl[U5Ul[ U5UlU(a [U5OSUlUR5 g)z^ add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer N) rr r6r`rrUencoderrpooler post_init)r5r6add_pooling_layerr8s r9r MarkupLMModel.__init__sK   ,V4&v. 0AnV,t  rJc.URR$rrrirs r9get_input_embeddings"MarkupLMModel.get_input_embeddingss...rJc$XRlgrr)r5rs r9set_input_embeddings"MarkupLMModel.set_input_embeddingss */'rJcUR5H7up#URRURR U5 M9 g)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsrrXrBr7)r5heads_to_prunerXr5s r9 _prune_headsMarkupLMModel._prune_headss< +002LE LL  u % / / ; ;E B3rJrYrBrCrrrerr{rr\r]rc U bU OURRn U bU OURRn U bU OURRn UbUb [ S5eUb"UR X5 UR 5n O"UbUR 5SSn O [ S5eUb URO URn Uc[R"XS9nUc$[R"U [RU S9nURS5RS5nURURS 9nS U- S -nUbUR5S:XahURS 5RS 5RS5RS5nUR!URR"SSSS5nOCUR5S:Xa/URS5RS5RS5nUR[%UR'55RS 9nOS/URR"-nUR)UUUUUUS 9nUR+UUUU U U S9nUS nUR,bUR-U5OSnU (d UU4USS-$[/UUUR0UR2UR4S9$)ao xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. Examples: ```python >>> from transformers import AutoProcessor, MarkupLMModel >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> model = MarkupLMModel.from_pretrained("microsoft/markuplm-base") >>> html_string = " Page Title " >>> encoding = processor(html_string, return_tensors="pt") >>> outputs = model(**encoding) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 4, 768] ```NzDYou cannot specify both input_ids and inputs_embeds at the same timer<z5You have to specify either input_ids or inputs_embeds)rxrvrrrr|gr)rYrBrCrerr{)rrr\r])re pooler_outputrrfrg)r6rr\use_return_dictr%warn_if_padding_and_no_attention_maskryrxr@rrrXrzrrwr>rrrWnext parametersrrrr rrfrg)r5rYrBrCrrrerr{rr\r]r|rxextended_attention_maskembedding_outputencoder_outputsrrs r9rHMarkupLMModel.forwardsL2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ CN  !"[[EJJvVN"0":":1"="G"G"J"9"<"<4::"<"N#&)@#@H"L  }}!#%//2<!>>I??))%)' + ,,  #/!5# ' *!,8< 8O O4UY #]3oab6II I;-')77&11,==   rJcP^SnUHnU[U4SjU554- nM U$)Nr_c3x># UH/oRSTRUR55v M1 g7f)rN) index_selectrrx)ra past_statebeam_idxs r9rc/MarkupLMModel._reorder_cache..ms1ncmU_--aZ=N=N1OPPcms7:)r)r5r[rreordered_past layer_pasts ` r9_reorder_cacheMarkupLMModel._reorder_cacheis8)J ncmnn N*rJ)r6rrr)T) NNNNNNNNNNN)rKrLrMrNr rrrrrr@ LongTensorr%r&rrr rHrrPrQrRs@r9rrsM /0C1559596:59371559,0/3&*h E,,-h !!1!12h !!1!12 h !!2!23 h !!1!12 h u//0h E--.h   1 12h $D>h 'tnh d^h  uBB Ch h VrJrc ^\rSrSrU4Sjr\SS\\RS\\RS\\RS\\RS\\RS\\RS \\RS \\RS \\RS \\RS \\ S\\ S\\ S\ \ \R\ 44Sjj5r SrU=r$)MarkupLMForQuestionAnsweringirc>[TU]U5 URUl[USS9Ul[ R "URUR5UlUR5 gNF)r) rr num_labelsrrxrr"r$ qa_outputsrrss r9r %MarkupLMForQuestionAnswering.__init__usU   ++%fF ))F$6$68I8IJ rJrYrBrCrrrerr{start_positions end_positionsrr\r]rc*U bU OURRn URUUUUUUUUU U U S9 nUSnURU5nUR SSS9unnUR S5R 5nUR S5R 5nSnU bU b[U R55S:aU R S5n [U R55S:aU R S5n URS5nU RSU5 U RSU5 [US9nU"UU 5nU"UU 5nUU-S- nU (dUU4USS-nUbU4U-$U$[UUUURURS 9$) a xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. Examples: ```python >>> from transformers import AutoProcessor, MarkupLMForQuestionAnswering >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base-finetuned-websrc") >>> model = MarkupLMForQuestionAnswering.from_pretrained("microsoft/markuplm-base-finetuned-websrc") >>> html_string = " My name is Niels " >>> question = "What's his name?" >>> encoding = processor(html_string, questions=question, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> predict_answer_tokens = encoding.input_ids[0, answer_start_index : answer_end_index + 1] >>> processor.decode(predict_answer_tokens).strip() 'Niels' ```N rBrCrrrerr{rr\r]rrr<r=) ignore_indexr)loss start_logits end_logitsrrf)r6rrxrsplitsqueezerr3ryclamp_rrrrf)r5rYrBrCrrrerr{rrrr\r]r"rlogitsrr total_loss ignored_indexloss_fct start_lossend_lossr.s r9rH$MarkupLMForQuestionAnswering.forwards^&1%` 'tn` d^`  uU\\"$@@ A` ` rJrzC MarkupLM Model with a `token_classification` head on top. ) custom_introc^\rSrSrU4Sjr\SS\\RS\\RS\\RS\\RS\\RS\\RS \\RS \\RS \\RS \\ S \\ S\\ S\ \ \R\ 44Sjj5r SrU=r$)MarkupLMForTokenClassificationicd>[TU]U5 URUl[USS9UlUR b UR O UR n[R"U5Ul [R"URUR5Ul UR5 gr)rr rrrxclassifier_dropoutr'rr&r(r"r$ classifierrr5r6rr8s r9r 'MarkupLMForTokenClassification.__init__s   ++%fF )/)B)B)NF % %TZTnTn zz"45 ))F$6$68I8IJ rJrYrBrCrrrerr{labelsrr\r]rc U bU OURRn URUUUUUUUUU U U S9 n U SnURU5nSnU bF[ 5nU"UR SURR 5U R S55nU (dU4U SS-nUbU4U-$U$[UUU RU RS9$)a xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. Examples: ```python >>> from transformers import AutoProcessor, AutoModelForTokenClassification >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> processor.parse_html = False >>> model = AutoModelForTokenClassification.from_pretrained("microsoft/markuplm-base", num_labels=7) >>> nodes = ["hello", "world"] >>> xpaths = ["/html/body/div/li[1]/div/span", "/html/body/div/li[1]/div/span"] >>> node_labels = [1, 2] >>> encoding = processor(nodes=nodes, xpaths=xpaths, node_labels=node_labels, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> loss = outputs.loss >>> logits = outputs.logits ```Nrrr<rrrrrf) r6rrxrrrrrrrf)r5rYrBrCrrrerr{rrr\r]r"rrrrr.s r9rH&MarkupLMForTokenClassification.forwardsZ&1%P 'tnP d^P  uU\\"N2 3P P rJrz MarkupLM Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. c^\rSrSrU4Sjr\SS\\RS\\RS\\RS\\RS\\RS\\RS \\RS \\RS \\RS \\ S \\ S\\ S\ \ \R\ 44Sjj5r SrU=r$)!MarkupLMForSequenceClassificationiLcr>[TU]U5 URUlXl[ U5UlUR b UR O URn[R"U5Ul [R"URUR5Ul UR5 gr)rr rr6rrxrr'rr&r(r"r$rrrs r9r *MarkupLMForSequenceClassification.__init__Ts   ++ %f- )/)B)B)NF % %TZTnTn zz"45 ))F$6$68I8IJ rJrYrBrCrrrerr{rrr\r]rc VU bU OURRn URUUUUUUUUU U U S9 n U SnURU5nUR U5nSnU GbURR cUR S:XaSURlOoUR S:aNU R[R:XdU R[R:XaSURlOSURlURR S:XaI[5nUR S:Xa&U"UR5U R55nOU"X5nOURR S:Xa=[5nU"URSUR 5U RS55nO,URR S:Xa[5nU"X5nU (dU4U SS-nUbU4U-$U$[!UUU R"U R$S 9$) a  xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoProcessor, AutoModelForSequenceClassification >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> model = AutoModelForSequenceClassification.from_pretrained("microsoft/markuplm-base", num_labels=7) >>> html_string = " Page Title " >>> encoding = processor(html_string, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> loss = outputs.loss >>> logits = outputs.logits ```Nrr regressionsingle_label_classificationmulti_label_classificationr<rr)r6rrxr(r problem_typerrwr@rXrUr rrrrrrrf)r5rYrBrCrrrerr{rrr\r]r"rrrrr.s r9rH)MarkupLMForSequenceClassification.forwardcsX&1%_ 'tn_ d^_  uU\\"$<< =_ _ rJr)rrrrrw)r):rOr rtypingrrrr@torch.utils.checkpointrtorch.nnrrr activationsr modeling_outputsr r rrrrmodeling_utilsrrrrutilsrrconfiguration_markuplmr get_loggerrKriModulerr^r`rrrrrrrrr,r)r>rUrwrrrr__all__r_rJr9rs )) AA! -2   H %/ bii/ f 4 __F   299  RYY  RYY  bii$ryy0!"))!CBIICN "# 0 0hSBIISnZ biiZ z o  <R+RRjm #:m m ` a %<a  a H q (?q  q h rJ