fThSrSSKrSSKrSSKrSSKrSSKJrJrJrJ r SSK r SSK J r SSK J r JrJr SSKJr SSKJrJrJr SS KJr SS KJr SS KJrJrJrJrJrJ r J!r! SS K"J#r# SS K$J%r%J&r&J'r' SSK(J)r)J*r*J+r+J,r,J-r-J.r.J/r/J0r0 SSK1J2r2J3r3 SSK4J5r5 \-"5(a SSK6J7r7 SSK8J9r9 \0Rt"\;5rSr?"SS\ R5rASSKBJCrC \CrA\S?\Q55rY/S@QrZg!\Ea GNN\Fa \"U RAURB55Ul"U R#U S5 GM [RS+S,R!U RG55S-35 U$![a [R S5 ef=f![.a, [RS SR!U 535 GM`f=f![8a1nU=R:UR2U R24- sleSnAff=f).z'Load tf checkpoints in a pytorch model.rNzLoading a TensorFlow model in PyTorch, requires TensorFlow to be installed. Please see https://www.tensorflow.org/install/ for installation instructions.z&Converting TensorFlow checkpoint from zLoading TF weight z with shape /c3,# UH nUS;v M g7f))adam_vadam_mAdamWeightDecayOptimizerAdamWeightDecayOptimizer_1 global_stepN).0ns Z/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/t5/modeling_t5.py (load_tf_weights_in_t5..hs  n nsz Skipping _slot_z [A-Za-z]+_\d+z_(\d+))kernelscale embeddingweightself_attentionlayerenc_dec_attentionr'dense_relu_denserms_norm layer_normfinal_layer_normr< output_biasbetabiassquad classifierdecoderlogitslm_headwiwi_r=z"Transposing numpy weight of shape z for zPointer shape z and array shape mismatchedzInitialize PyTorch weight z%Weights not copied to PyTorch model: z, .)$renumpy tensorflow ImportErrorloggererrorospathabspathinfotrainlist_variables load_variableappendsplitanyjoinpop fullmatchgetattrhasattrlenisdigitAttributeErrorintshape transpose ValueErrorAssertionErrorargstorch from_numpyastypefloat32datakeys)modelconfigtf_checkpoint_pathrSnptftf_path init_varsnames tf_weightsnamerlarraytxt_namepointerm_name scope_namesnumes r6load_tf_weights_in_t5rKs ggoo01G KK8 BC''0I EJ  (l5'BC&&w5 T 4 ! ~~c"      KK)CHHTN#34 5 NN8T *  tBx  KK)CHHTN#34 5 NN8T * $F||,f55 hhy&9 %h 1~!AA!'84Q#33!'73!!*Q#66!'73!!*Q#55!'73!!*Q:-7L11%g|G;1$+a.)!#,UV q>!A Agx0G q>[ ( KK'--@QRWR]R]Q^^i!jkk,  078'' RZZ(@A x&UX KK7 *//BS8T7UUVWX L   Q   R&KK)CHHTN+; <=  FFw}}ekk2 2F  s6 Q6 RAS6!R1SS T,T  TaR This is an experimental feature and is a subject to change at a moment's notice. Uses a device map to distribute attention modules of the model across several devices. If no device map is given, it will evenly distribute blocks across all devices. Args: device_map (`Dict[int, list]`, *optional*): A dictionary that maps attention modules to devices. Note that the embedding module and LMHead are always automatically mapped to the first device (for esoteric reasons). That means that the first device should have fewer attention modules mapped to it than other devices. For reference, the t5 models have the following number of attention modules: - google-t5/t5-small: 6 - google-t5/t5-base: 12 - google-t5/t5-large: 24 - google-t5/t5-3b: 24 - google-t5/t5-11b: 24 Example: ```python # Here is an example of a device map on a machine with 4 GPUs using google-t5/t5-3b, which has a total of 24 attention modules: model = T5ForConditionalGeneration.from_pretrained("google-t5/t5-3b") device_map = { 0: [0, 1, 2], 1: [3, 4, 5, 6, 7, 8, 9], 2: [10, 11, 12, 13, 14, 15, 16], 3: [17, 18, 19, 20, 21, 22, 23], } model.parallelize(device_map) ``` a4 Moves the model to cpu from a model parallel state. Example: ```python # On a 4 GPU machine with google-t5/t5-3b: model = T5ForConditionalGeneration.from_pretrained("google-t5/t5-3b") device_map = { 0: [0, 1, 2], 1: [3, 4, 5, 6, 7, 8, 9], 2: [10, 11, 12, 13, 14, 15, 16], 3: [17, 18, 19, 20, 21, 22, 23], } model.parallelize(device_map) # Splits the model across several devices model.deparallelize() # Put the model back on cpu and cleans memory by calling torch.cuda.empty_cache() ``` c2^\rSrSrSU4SjjrSrSrU=r$) T5LayerNormc>[TU]5 [R"[R "U55UlX lg)zS Construct a layernorm module in the T5 style. No bias and no subtraction of mean. N)super__init__r Parameterrqonesr>variance_epsilon)self hidden_sizeeps __class__s r6rT5LayerNorm.__init__s/ ll5::k#:; #cUR[R5RS5R SSS9nU[R "X R -5-nURR[R[R4;a%URURR5nURU-$)NrCr:T)keepdim) torqrtpowmeanrsqrtrr>dtypefloat16bfloat16)r hidden_statesvariances r6forwardT5LayerNorm.forwards !##EMM266q9>>r4>P% H?T?T4T(UU  ;;   ? ?),,T[[->->?M{{]**r)rr>)gư>)__name__ __module__ __qualname____firstlineno__rr__static_attributes__ __classcell__rs@r6rrs$ + +rr) FusedRMSNormzODiscovered apex.normalization.FusedRMSNorm - will use it instead of T5LayerNormzBdiscovered apex but it failed to load, falling back to T5LayerNormc6^\rSrSrS\4U4SjjrSrSrU=r$)T5DenseActDenseirxcX>[TU]5 [R"URUR SS9Ul[R"UR URSS9Ul[R"UR5Ul [URUl gNFrI)rrrLineard_modeld_ffrOwoDropout dropout_ratedropoutr dense_act_fnactrrxrs r6rT5DenseActDense.__init__sn ))FNNFKKeD))FKKeDzz&"5"56 &--.rcURU5nURU5nURU5n[URR [ R5(aURURR R:waaURR R[ R:wa/URURR R5nUR U5nU$N) rOrr isinstancerr>rqTensorrint8rrrs r6rT5DenseActDense.forwards . /  ]3 tww~~u|| 4 4##tww~~';';;$$ 2),,TWW^^-A-ABM . r)rrrOr rrrrr(rrrrrs@r6rrs/x/  rrc6^\rSrSrS\4U4SjjrSrSrU=r$)T5DenseGatedActDensei-rxc>[TU]5 [R"URUR SS9Ul[R"URUR SS9Ul[R"UR URSS9Ul[R"UR5Ul [URUlgr)rrrrrrwi_0wi_1rrrrr rrrs r6rT5DenseGatedActDense.__init__.s IIfnnfkkF IIfnnfkkF ))FKKeDzz&"5"56 &--.rc8URURU55nURU5nX#-nURU5n[ UR R [R5(aURUR R R:waaUR R R[R:wa/URUR R R5nUR U5nU$r) rrrrrrr>rqrrrr)rr hidden_gelu hidden_linears r6rT5DenseGatedActDense.forward6shhtyy78  -0 #3  ]3 tww~~u|| 4 4##tww~~';';;$$ 2),,TWW^^-A-ABM . r)rrrrrrrs@r6rr-s/x/rrc6^\rSrSrS\4U4SjjrSrSrU=r$) T5LayerFFiJrxc>[TU]5 UR(a[U5UlO[ U5Ul[ URURS9Ul [R"UR5Ul g)Nr)rr is_gated_actrDenseReluDenserrrlayer_norm_epsilonrErrrrrs r6rT5LayerFF.__init__Ks_    "6v">D "1&"9D %fnn&:S:STzz&"5"56 rcpURU5nURU5nXRU5-nU$r)rErr)rrforwarded_statess r6rT5LayerFF.forwardUs;??=9../?@% 5E(FF r)rrrErrs@r6rrJs7x7rrc^\rSrSrS S\S\\4U4SjjjrSr\ S Sj5r S Sjr S Sjr S r U=r$) T5Attentioni\rx layer_idxc >[TU]5 URUlX lURUlUR UlUR UlURUlURUl URUl URUR-Ul X0lUc>UR(a-[R!SUR"R$S35 [&R("UR URSS9Ul[&R("UR URSS9Ul[&R("UR URSS9Ul[&R("URUR SS9UlUR(a0[&R2"URUR5Ul[75UlSUlg)NzInstantiating a decoder z without passing `layer_idx` is not recommended and will to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` when creating this class.Fr)rr is_decoderhas_relative_attention_biasrelative_attention_num_bucketsrelative_attention_max_distancerd_kvkey_value_proj_dim num_headsn_headsrr inner_dimrrW warning_oncerrrrqkvo Embeddingrelative_attention_biasset pruned_headsgradient_checkpointingrrxrrrs r6rT5Attention.__init__]so  +++F(.4.S.S+/5/U/U,~~ "(++'' ** (?(??"     *4>>+B+B*CD,, 4<<eD4<<eD4<<eD4>>4<=max_distance map to the same bucket. All relative positions <=-max_distance map to the same bucket. This should allow for more graceful generalization to longer sequences than the model has been trained on Args: relative_position: an int32 Tensor bidirectional: a boolean - whether the attention is bidirectional num_buckets: an integer max_distance: an integer Returns: a Tensor with the same shape as relative_position, containing int32 values in the range [0, num_buckets) rrCr') rrqlongabsmin zeros_likelogfloatmath full_likewhere)relative_position bidirectional num_buckets max_distancerelative_buckets max_exactis_smallrelative_position_if_larges r6_relative_position_bucket%T5Attention._relative_position_buckets,  A K Q!6 : :5:: F T T  % *; < !&+N>NO`>a!b b  1$ $0&/ II'--/); <hh|/0 1& ( "UZZ. &" &+YY &8RbcTc(d& " EKKE_``rcUc URRRnUc,[R"U[R US9SS2S4nOUSS2S4R U5n[R"U[R US9SSS24nXe- nURUUR(+URURS9nURU5n U R/SQ5RS5n U $)z%Compute binned relative position biasN)rdevice)rrr)rCrr'r) rr>rrqarangerrr rrrpermute unsqueeze) r query_length key_lengthrcache_positioncontext_positionmemory_positionrrelative_position_bucketvaluess r6 compute_biasT5Attention.compute_biass >1188??F  !$||L SYZ[\^b[bc -ag699&A ,,zFSTXZ[T[\+>#'#A#A #.;;== $B$  --.FG *44Q7 rc ~URSSupUSLn URU5nURU SURUR5R SS5nUbEUR RUR5nU (a URnO URnU (aUOUnU (a=Ub:W(a3WRURnURURnOURU5nURU5nURU SURUR5R SS5nURU SURUR5R SS5nUbNU (dU OSn WRUUURSU 05unnU (aSUR UR'[ R""UUR SS55nUcURSnUbUOU SS-nUR$(db[ R&"SURU U4UR(UR*S 9nUR,(aUR.(aSUlO.UR3UUUR(U S 9nUSS2SS2U *S2SS24nUb#USS2SS2SS2SURS24nUU-nUR4(aS[ R6"URS5nS U[9UR45'USS2UR;54nOUnUU- n[<R>RAURC5SS 9REU5n[<R>RGUURFUR.S 9nUbUU-n[ R""UU5nUR SS5RI5nURU SURJ5nURMU5nUXT4nU (aUU4-nU$)zp Self-attention (if key_value_states is None) or attention over source sentence (provided by key_value_states). NrCr:r'rTr rr)rrrr)ptraining)'rlrviewrrrm is_updatedgetrcross_attention_cacheself_attention_cache key_cache value_cacherrupdaterqmatmulrzerosrrrr! requires_gradrrrlistboolr functionalsoftmaxrtype_asr contiguousrr)rrmaskkey_value_states position_biaspast_key_valuelayer_head_maskr use_cacheoutput_attentionsr batch_size seq_lengthis_cross_attention query_statesr#curr_past_key_valuecurrent_states key_states value_statesscoresrreal_seq_length causal_maskposition_bias_masked attn_weights attn_outputoutputss r6rT5Attention.forwards$"/!4!4Ra!8 .T9vvm, #((RtG^G^_iijkmno  %'2266t~~FJ!&4&J&J#&4&I&I#-?)] ."<,66t~~FJ.::4>>JL/J66.1L#RtG^G^_iijkmnoJ',,ZT\\4KbKbcmmnoqrsL)7It+>+E+E dnn?OQ_>`,( L&@DN--dnn=lJ,@,@A,FG  #))"-J.:.FlN[]L^abLbO33 %  j*=fmm[a[g[g! ..4==26M/ $ 1 1#Z Vd!2! !.aZKL!.C D "1a,Bj.>.>r.B,B#BC - ;   ::m11!45D,-Dd''( )#0DIIK#@ #0 &&}},,V\\^,DLLVT }},,\T\\TXTaTa,b   &'/9Lll<> !++Aq1<<> !&&z2t~~F ff[) > /Gr)rrrrrrrrrrrrrrrrrFN)T )NN) NNNNNNFFN)rrrrr(rrkrr staticmethodr rrrrrs@r6rr\st%*#' !,!,C= !,!,F; - - ^.iirrcR^\rSrSrSS\\4U4SjjjrSSjrSrU=r $)T5LayerSelfAttentioni@rc>[TU]5 [XUS9Ul[ UR UR S9Ul[R"UR5Ul g)Nrrr) rrr SelfAttentionrrrrErrrrrs r6rT5LayerSelfAttention.__init__AsQ ( W` &fnn&:S:STzz&"5"56 rc URU5n URU UUUUUUUS9n XRU S5-nU4U SS-n U $)N)r3r5r7r6r8r9rrr')rErRr) rrattention_maskr5r7r6r8r9rnormed_hidden_statesattention_outputrHs r6rT5LayerSelfAttention.forwardIst $}=-- '+)/).  & 5Ea5H(II  "%5ab%99r)rRrrErJ)NNNNFFN rrrrrrkrrrrrs@r6rOrO@s:7XVY]77rrOcT^\rSrSrSS\\4U4SjjjrSSjrSrU=r $)T5LayerCrossAttentionidrc>[TU]5 [USUS9Ul[ UR UR S9Ul[R"UR5Ul g)NFrQr) rrrEncDecAttentionrrrrErrrr)rrxrrs r6rT5LayerCrossAttention.__init__esN *6u`ij%fnn&:S:STzz&"5"56 rc URU5n URU UUUUUUUU U S9 n XRU S5-n U 4U SS-nU$)N) r3r4r5r7r6r8rr9rrr')rEr]r)rrr4rUr5r7r6r8rr9rrVrW layer_outputrHs r6rT5LayerCrossAttention.forwardksy $}=// -'+)%/)0  %||4DQ4G'HH /$4QR$88r)r]rrEr)NNNNFNFNrYrs@r6r[r[ds<7(3-77rr[c\^\rSrSrSS\\4U4SjjjrSSjrSrU=r $)T5Blockircl>[TU]5 URUl[R"5UlUR R [XUS95 UR(a"UR R [XS95 UR R [U55 g)NrQ)r) rrrr ModuleListr@r`rOr[rrs r6rT5Block.__init__s}  ++]]_   lu v  ?? JJ  3FP Q )F+,rc URS"UUUUU U U U S9nUSSupUSSnUR[R:Xa[R"[R "U5R 5[R"UR5RS- [R"UR5R5n[R"UU*US9nUR=(a USLnU(aURS"UUUUUU U SS-U U S9 nUSSupUR[R:Xa[R"[R "U5R 5[R"UR5RS- [R"UR5R5n[R"UU*US9nUUSS-nURS"U5nUR[R:Xa[R"[R "U5R 5[R"UR5RS- [R"UR5R5n[R"UU*US9nU4nU (a UU 4-U-nU$UU-nU$) Nr)rUr5r7r6r8r9rrCi)rmaxr'r:)r4rUr5r7r6rr8r9) r@rrqrrisinfrbfinforhclampr)rrrUr5encoder_hidden_statesencoder_attention_maskencoder_decoder_position_biasr7cross_attn_layer_head_maskr6r8r9 return_dictrself_attention_outputsattention_outputs clamp_valuedo_cross_attentioncross_attention_outputsrHs r6rT5Block.forwards "&A )'+)/) " )?r(B% 2126   %-- /++ M*..0 M//044t; M//044K "KK K<[YM!__R1Fd1R &*jjm!65; :-+B/!3#"3 ' #-DBQ,G )M""emm3#kkKK .224KK 3 34884?KK 3 3488 !& M |Q\ ] !24KAB4O O  2}5    %-- /++ M*..0 M//044t; M//044K "KK K<[YM "  114EEG 11Gr)rr@rJ) NNNNNNNNFFTNrYrs@r6rcrcsK -XVY] - -"#&*#'TTrrccn^\rSrSrSrS\4U4SjjrS\RS\R4Sjr Sr U=r $) T5ClassificationHeadiz-Head for sentence-level classification tasks.rxc(>[TU]5 [R"URUR5Ul[R "URS9Ul[R"URUR5Ul g)N)r ) rrrrrdenserclassifier_dropoutr num_labelsout_projrs r6rT5ClassificationHead.__init__sZ YYv~~v~~> zzF$=$=>  &..&2C2CD rrreturncURU5nURU5n[R"U5nURU5nUR U5nU$r)rrzrqtanhr}rs r6rT5ClassificationHead.forwardsN ]3  =1  =1  ]3  m4 r)rzrr}) rrrr__doc__r(rrqrrrrrs@r6rxrxs47ExE U\\ellrrxc\\rSrSr\r\rSrSr Sr Sr Sr Sr S/rS/r\S5rSrS rS rg ) T5PreTrainedModeli transformerTFrcrcz[R"[5n[R"[5nUUUS.nU$)N)decoder_input_ids input_idsdecoder_attention_mask)rqtensorrr)rr input_mask dummy_inputss r6rT5PreTrainedModel.dummy_inputs s6LL. \\*- !*"&0 rcURRn[U[5(a)URR R US-5 g [U[[[[45(GaURRR RSUS-S9 [US5(aLURR(d1URRR RSUS-S9 [US5(avUR RR RSX RR"S--S9 UR R$R R'5 g g [U[(5(ar[US5(a`UR*RR RSUS-S9 UR*R$R R'5 g g [U[,5(GaQUR.RR RSX RR"S--S9 [UR.S5(aEUR.R$b.UR.R$R R'5 UR0RR RSX RR"S--S9 [UR0S5(aGUR0R$b/UR0R$R R'5 g g g [U[25(GaQUR4RR RSX RR"S--S9 [UR4S5(aEUR4R$b.UR4R$R R'5 UR6RR RSX RR8S--S9 [UR6S5(aGUR6R$b/UR6R$R R'5 g g g [U[:5(GaUR<RR RSX RR"S--S9 [UR<S5(aEUR<R$b.UR<R$R R'5 UR>RR RSX RR"S--S9 [UR>S5(aEUR>R$b.UR>R$R R'5 UR6RR RSX RR8S--S9 [UR6S5(aGUR6R$b/UR6R$R R'5 g g g [U[@5(GaZURR"nURRBnURRDnURFRR RSX#U-S--S9 URHRR RSX#S--S9 URJRR RSX#S--S9 URLRR RSX%U-S--S9 URN(a4URPRR RSX#S--S9 g g g ) zInitialize the weights?)rstdrN qa_outputsrKrIN))rxinitializer_factorrrr>rufill_T5ModelT5ForConditionalGenerationT5EncoderModelT5ForQuestionAnsweringsharednormal_rgtie_word_embeddingsrNrrrIzero_T5ForTokenClassificationrKrxrzr}rrOrrrrrrrrrrrrrr)rmodulefactorrrrs r6 _init_weightsT5PreTrainedModel._init_weightss// fk * * MM   $ $Vc\ 2   0.BX Y  MM % % - -3FSL - Ivy))$++2Q2Q%%**22#2Nv|,,!!((--553F{{ObObgkNkDl5m!!&&++113- 8 9 9v|,,!!((--553FSL5Q!!&&++113- 4 5 5 LL   $ $ , ,#6kkFYFY^bEb;c , dv||V,,1B1B1N !!&&,,. OO " " ' ' / /SfI\I\aeHe>f / gv//FOO4H4H4T$$))//15U/  0 0 II   ! ! ) )s;;CVCV[_B_8` ) avyy&))fiinn.H ##))+ II   ! ! ) )s;;CSCSX\B\8] ) ^vyy&))fiinn.H ##))+/I)  4 5 5 KK   # # + +&[[EXEX]aDa:b + cv{{F++ 0@0@0L   %%++- KK   # # + +&[[EXEX]aDa:b + cv{{F++ 0@0@0L   %%++- II   ! ! ) )s;;CSCSX\B\8] ) ^vyy&))fiinn.H ##))+/I)  , ,kk))G!%!1!1 kk++G HHOO ( (cvL^B^cgAg7h ( i HHOO ( (cv$7O ( P HHOO ( (cv$7O ( P HHOO ( (cvL^B^cgAg7h ( i11..55::BBQWhl[lQmBn2-rcURRnURRnUc [S5e[ U5(aE[ R "URSSS-U5n[ R"XASSS24/SS9nO=URUR5nUSSS24R5USSS24'X$S'Uc [S5eURUS :HU5 U$) Nzself.model.config.decoder_start_token_id has to be defined. In T5 it is usually set to the pad_token_id. See T5 docs for more information.r:)r'.rr').rz1self.model.config.pad_token_id has to be defined.) rxdecoder_start_token_id pad_token_idrnr"rqfullrlcat new_zerosclone masked_fill_)rrrrshifted_input_idss r6 _shift_rightT5PreTrainedModel._shift_rightUs!%!C!C{{// ! )4  Y ' ' % 9??3B+?$+FH^ _  % +<SbS>Q*RXZ [  ) 3 3IOO D )238)<)B)B)D c12g &(>f %  PQ Q&&'8D'@,O  rr3N)rrrrr( config_classrload_tf_weightsbase_model_prefixis_parallelizablesupports_gradient_checkpointing_supports_quantized_cache_supports_static_cache_supports_cache_class_no_split_modules_keep_in_fp32_modulespropertyrrrrr3rr6rrs^L+O%&*# %! " !F ;oz!rrc j^\rSrSrSU4Sjjr\"\5SSj5r\"\5S5r Sr Sr SSjr SS\ \RS 4S \RS \RS \S \4 Sjjr\S\RS\S\S\R*S \RS\4 Sj5rSrU=r$)T5Stackiqc >[TU]U5 X lURUl[R "[ UR5Vs/sHn[U[US:H5US9PM sn5Ul [URURS9Ul[R"UR 5UlUR%5 SUlSUlSUlgs snf)NrrQrF)rr embed_tokensrrrerange num_layersrcr.blockrrrrFrrr post_initmodel_parallel device_mapr)rrxrirs r6rT5Stack.__init__rs  ( ++]]]bcictct]u v]uXYWVa1fQR S]u v  !,FNN@Y@Y Zzz&"5"56  #&+# ws!C*c[R"S[5 UcD[[ UR 5[ [RR555OUUl [UR[ UR 55 SUl SURR5;aSO.S[[URR555-UlS[[#URR555-UlURR'5HGup#UH<nS[U5-nUR UR)U5UR U'M> MI UR*R)UR 5UlUR,R)UR$5Ulg)Na<`T5Stack.parallelize` is deprecated and will be removed in v5 of Transformers, you should load your model with `device_map='balanced'` in the call to `from_pretrained`. You can also provide your own `device_map` but it needs to be a dictionary module_name to device, so for instance {'block.0': 0, 'block.1': 1, ...}Tcpucuda:)warningswarn FutureWarningr&rhrrrqcuda device_countrr%rrvstrr first_devicerh last_deviceitemsrrrF)rrrrr@ cuda_devices r6 parallelizeT5Stack.parallelizese  "   R\QcN3tzz?E%**2I2I2K,L Mis  $//3tzz?;"%*doo.B.B.D%DE'TWX[\`\k\k\p\p\rXsTtJt"ST__-A-A-C)D%EEOO))+DA%A. $(JJu$5$8$8$E 5!, !--001B1BC $ 5 5 8 89I9I Jrc[R"S[5 SUlSUlSUlSUl[[UR55H.nURURS5URU'M0 URRS5Ul URRS5Ul [RR5 g)N\Like `parallelize`, `deparallelize` is deprecated and will be removed in v5 of Transformers.Fr)rrrrrrrrrhrrrrFrqr empty_cache)rrs r6 deparallelizeT5Stack.deparallelizes j  $! s4::'A JJqM,,U3DJJqM( --007 $ 5 5 8 8 ?  rcUR$rrrs r6get_input_embeddingsT5Stack.get_input_embeddingss   rcXlgrrrnew_embeddingss r6set_input_embeddingsT5Stack.set_input_embeddingss*rcUR(aS[RRUR5 UR R UR5UlU bU OURRn U bU OURRn U bU OURRn U bU OURRn Ub*Ub'UR(aSOSn[SUSUS35eUb&UR5nURSUS5nO>UbUR5SSnO'UR(aSOSn[SUSUS 35eUR (a/UR"(aU (a[$R'S 5 S n Uc)UR c [S 5eUR U5nUunnU S La UR(d[SUS35eS nS nUR(aU (dUb[)U[*5(a,[)U[,5(dS n[-U[/55nOv[)U[,5(d.S n[$R'S5 [,R0"U5nO3Uc[-[/5[/55nOUR(dSnUbUR35OSnU c#[R4"UUU-UR6S9n Uc4[95(d%UU-n[R:"UUUR6S9nURR(a%UR=UUU Ub UR>OSU 5nO\UbWUSS2SSSS24nUR UR@S9nSU- [RB"UR@5RD-nOSnUR(a^Ub[UR5unnnUU4nUc.[R:"UUR6[RFS9nURIU5nOSnURKX`RRL5nURKXpRRL5nU (aSOSnU (aSOSnU (aUR(aSOSnSnSn UROU5n![QURR5GHtun"n#UU"n$UU"n%UR(a[RRU!R65 UbUR U!R65nUbUR U!R65nUbUR U!R65nUbUR U!R65nU bU R U!R65n U$bU$R U!R65n$U%bU%R U!R65n%U (aUU!4-nUR (a:UR"(a)URUU#RVU!UUUUU U$U%SU U U U 5n&OU#"U!UUUUU U$U%UU U U U S9 n&U S LaU&SSS-U&SS-n&U&SSun!n'U&SnUR(aUbU&U (aSOSn U (a#UU&S4-nUR(a UU&S4-nUR(dGMURXR[5HOun(n)U"U)S:XdMS[]U(5-UR^:wdM/U!R S[]U(S-5-5n!MQ GMw URaU!5n!UROU!5n!U (aUU!4-nU (aW'OSn*U(a UR>n*U(aURc5n*U (d[eSU!U*UUU455$[gU!U*UUUS 9$)!Ndecoder_zYou cannot specify both zinput_ids and zinputs_embeds at the same timer:zYou have to specify either z input_ids or inputs_embedszZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...Fz.s" As )last_hidden_statepast_key_valuesr attentionscross_attentions)4rrqr set_devicerrrrxr8r9output_hidden_statesuse_return_dictrrnsizer"rr!rWrrr rrfrom_legacy_cacheget_seq_lengthrrr#r_update_causal_maskr&rrjrrinvert_attention_mask get_head_maskrr enumerater_gradient_checkpointing_funcrrrrrrFto_legacy_cachetupler)+rrrUrlrmr head_maskcross_attn_head_maskrr8r9rrprerr_msg_prefix input_shaper:r;return_legacy_cachereturn_self_attention_cachepast_key_values_lengthmask_seq_lengthrDencoder_batch_sizeencoder_sequence_length_encoder_hidden_shapeencoder_extended_attention_maskall_hidden_statesall_attentionsall_cross_attentionsr5rnrr layer_moduler7ro layer_outputsnext_decoder_cacherr next_caches+ r6rT5Stack.forwards"    JJ ! !$"3"3 4 $ 1 1 4 4T5F5F GD !*!6IDKK+/??ZN*>*:.HXXvw  "#..*K!r;r?;I  &',,.s3K+/??ZN:>:J-XfWggtuv v  & &4==##p"    ( !_`` --i8M!, J  ?? #LTFRg!hii$&+# ?? _-H/511*_Vi:j:j.2+"5o|~"V1DEE&*###` #6"G"G"X ("5lnln"U#OETE`!?!?!Afg  !"\\&(>(KTaThThN  !*B*D*D4zAO"ZZ OML`L`aN ;; ! !228G8S44Y]! K '(D$)9:K%..}/B/B.CK, M6U6X6XYfYmYm6n30<4Q4T4TUbUiUi4j1".&5&8&89M9M&NO-91K1N1N}OcOc1d.#$58H$H!**t}} $ A A ((!!)31#.%"! "!-!#."/*?+J2O$3/I#2'&7 +#1! $E! -bq 1G ;mAB>O O 0=bq0A -M- *!,M#8#D0=CTaZ[0\- !/=3C2E!E??+?=QRCSBU+U(""" OO113DAqAbEzgA&6$:J:J&J(5(8(83q1u:9M(N 4_ 5f--m<  ]3  1]4D D +4'$ &(==J (88:J "%"(    9+&+%1   rrUr) input_tensorrrr9c URRS:XaUbUS:HR5(aU$gURRS:Xa,[U[R 5(a [ U5nU$UbUR5OSnUb UROSnURRS:Xa5U(d.U(d'[R"UUUURS9(agURnURSn U(aUR5n O5[U[R 5(aURS OXi-S-n URUU U UUURSS 9n URRS:XaZUbWUR R"S ;a=U(d6[R$"U5R&n [R("X5n U $) Nflash_attention_2rflex_attentionrFsdpa)rr is_trainingr'r:)sequence_length target_lengthrrr:)rxpunpu)rx_attn_implementationrbrrqrr*ris_compileabler_ignore_causal_mask_sdpar!rrlget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionrtyperjr_unmask_unattended) rrUrrrr9past_seen_tokensusing_compilable_cacherrrrD min_dtypes r6rT5Stack._update_causal_masks ;; + +/B B)~/D.I.I.K.K%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`aCRC^!?!?di ;; + +v 5>T]n%>>*'7 MM  ""&,,Q/ !+??AMnell;;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCK[Krrrrr:c UbUR5S:XaUnU$[R"U5Rn[R"X4XUR S9nUS:wa[R "USS9nU[R"X$R S9URSS5:-nUSSSS2SS24RUSSS5nUbUR5nURSn USS2SS2SS2SU 24USS2SSSS24RUR 5-n U S:Hn USS2SS2SS2SU 24RX5USS2SS2SS2SU 24'U$) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. Nr) fill_valuerrr')diagonalrr:r)rrqrjrrrtriurreshapeexpandrrlr masked_fill) rUrrrrr:kwargsrDr' mask_length padding_masks r6r"=T5Stack._prepare_4d_causal_attention_mask_with_cache_positions}>  %.*<*<*>!*C(K*' E*..I** 0Y\j\q\qK!##jjqA 5<< >S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r) rrrrrFrrrrrr) NNNNNNNNNNNNN)F)rrrrrrPARALLELIZE_DOCSTRINGrDEPARALLELIZE_DOCSTRINGrrrrrrqrr r.rrMrkrr"rrrs@r6rrqs',&/0K1K612 !3 !!+ "#!!u |#( BellK78BllB B  B  BH4 444{{ 4  4  44rra_ The input argument `head_mask` was split into two arguments `head_mask` and `decoder_head_mask`. Currently, `decoder_head_mask` is set to copy `head_mask`, but this feature is deprecated and will be removed in future versions. If you do not want to use any `decoder_head_mask` now, please set `decoder_head_mask = torch.ones(num_layers, num_heads)`. c&^\rSrSrS/rSS/rS\4U4Sjjr\"\ 5S"Sj5r \"\ 5S5r S r S rS rS rS rSr\S#S\\R,S\\R.S\\R,S\\R0S\\R.S\\R.S\\R2S\\\\R.S\\\\R.S\\R2S\\R2S\\S\\S\\S\\S\\R,S\\\R.\44"S jj5rS!rU=r $)$ri5Fdecoder.block.0.layer.1.EncDecAttention.relative_attention_bias.weightencoder.embed_tokens.weightdecoder.embed_tokens.weightrxc>[TU]U5 [R"URUR 5Ul[R"U5nSUl SUl SUl [X R 5Ul [R"U5nSUl SUl URUl[X0R 5UlUR#5 SUlSUlgNFT)rrrr vocab_sizerrcopydeepcopyrr8is_encoder_decoderrencodernum_decoder_layersrrLrrrrrxencoder_configdecoder_configrs r6rT5Model.__init__<s  ll6#4#4fnnE v.$)!#( ,1)~{{; v.$(!,1)$*$=$=!~{{;  $rc[R"S[5 UcN[[ UR R 5[[RR555OUUl [UR[ UR R 55 UR RUR5 URRUR5 SUlg)NaL`T5Model.parallelize` is deprecated and will be removed in v5 of Transformers, you should load your model with `device_map='balanced'` in the call to `from_pretrained`. You can also provide your own `device_map` but it needs to be a dictionary module_name to device, so for instance {'encoder.block.0': 0, 'encoder.block.1': 1, ...}T)rrrr&rhr@rrrqrrrr%rrLrrrs r6rT5Model.parallelizeSs  -   ! 3t||112E%**:Q:Q:S4T U  $//3t||/A/A+BC   1   1"rcz[R"S[5 URR 5 UR R 5 URR S5UlUR R S5UlSUlSUl[RR5 gNrrF) rrrr@rrLrrrrqrrrs r6rT5Model.deparallelizefs} j   ""$ ""$||u- ||u- #  rcUR$rrrs r6rT5Model.get_input_embeddingst {{rc|XlURRU5 URRU5 grrr@rrLrs r6rT5Model.set_input_embeddingsw+$  )).9 )).9rcURR(aaURURRUR 5 URUR RUR 5 ggrrxr_tie_or_clone_weightsr@rrrLrs r6 _tie_weightsT5Model._tie_weights|P ;; * *  & &t||'@'@$++ N  & &t||'@'@$++ N +rcUR$rr@rs r6 get_encoderT5Model.get_encoder ||rcUR$rrLrs r6 get_decoderT5Model.get_decoderr^rcUR5H7up#URRURR U5 M9 g) Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)rr@r@ attentionrrheads_to_pruner@rs r6 _prune_headsT5Model._prune_headss< +002LE LL  u % / / ; ;E B3rrrUrrrdecoder_head_maskrencoder_outputsrrdecoder_inputs_embedsr8r9rrprrcU bU OURRn UbUOURRnUbRUcOURRURR:Xa![ R "[[5 UnUcURUUU UU UUS9nORU(aK[U[5(d6[US[U5S:aUSOS[U5S:aUSOSS9nUSnUR(a[RR!UR"R$5 UR'UR"R$5nUb%UR'UR"R$5nUb%UR'UR"R$5nUb%UR'UR"R$5nUR#UUU U UUUUU U UUUS9 nU(dUU-$[)UR*UR,UR.UR0UR2UR*UR.UR0S9$) a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. decoder_head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. Example: ```python >>> from transformers import AutoTokenizer, T5Model >>> tokenizer = AutoTokenizer.from_pretrained("google-t5/t5-small") >>> model = T5Model.from_pretrained("google-t5/t5-small") >>> input_ids = tokenizer( ... "Studies have been shown that owning a dog is good for you", return_tensors="pt" ... ).input_ids # Batch size 1 >>> decoder_input_ids = tokenizer("Studies show that", return_tensors="pt").input_ids # Batch size 1 >>> # preprocess: Prepend decoder_input_ids with start token which is pad token for T5Model. >>> # This is not needed for torch's T5ForConditionalGeneration as it does this internally using labels arg. >>> decoder_input_ids = model._shift_right(decoder_input_ids) >>> # forward pass >>> outputs = model(input_ids=input_ids, decoder_input_ids=decoder_input_ids) >>> last_hidden_states = outputs.last_hidden_state ```NrrUrrr9rrprr'rCrrr rrUrrrlrmrrr8r9rrpr)rrdecoder_hidden_statesdecoder_attentionsrencoder_last_hidden_staterlencoder_attentions)rxr8rrrArr_T5Model__HEAD_MASK_WARNING_MSGrr@rrrhrrqrrrLrrrrrrrr)rrrUrrrrjrrkrrrlr8r9rrprrdecoder_outputss r6rT5Model.forwards@b"+!6IDKK{{%%)G)GG 5}E$-!  ""ll#-+#"3%9'+OO_!M!M-"1!"4474H14Loa0RV14_1E1I?1-tO (*     JJ ! !$,,";"; <),,T\\-F-FGM ,$5$8$89R9R$S!)!/!2!24<<3L3L!M%1)?)B)B4<U $D>U 'tnU d^!U "!!1!12#U $ uU&&');; <%U U rrz: T5 Model with a `language modeling` head on top. ) custom_introc(^\rSrSrS/r/SQrS\4U4Sjjr\"\ 5S%Sj5r \"\ 5S5r Sr S rS rS rS rS rSr\S&S\\R.S\\R0S\\R.S\\R2S\\R0S\\R0S\\R4S\\\\R4S\\\\R4S\\R0S\\R0S\\R.S\\S\\S\\S\\S\\R.S \\\R0\44$S!jj5rS\R44S"jr S#r!S$r"U=r#$)'ri(r7)r8r9zlm_head.weightrxch>[TU]U5 URUl[R "UR UR5Ul[R"U5nSUl SUl SUl [X R5Ul[R"U5nSUl SUl URUl[X0R5Ul[R$"URUR SS9UlUR)5 SUlSUlg)NFTr)rrr model_dimrrr<rr=r>rr8r?rr@rArrLrrNrrrrBs r6r#T5ForConditionalGeneration.__init__3s  ll6#4#4fnnE v.$)!#( ,1)~{{; v.$(!,1)$*$=$=!~{{; yy1B1BO  $rcZ[R"S[5 UcN[[ UR R 5[[RR555OUUl [UR[ UR R 55 UR RUR5 URRUR5 URR!URR"5UlSUlg)Na_`T5ForConditionalGeneration.parallelize` is deprecated and will be removed in v5 of Transformers, you should load your model with `device_map='balanced'` in the call to `from_pretrained`. You can also provide your own `device_map` but it needs to be a dictionary module_name to device, so for instance {'encoder.block.0': 0, 'encoder.block.1': 1, ...}T)rrrr&rhr@rrrqrrrr%rrLrNrrrrGs r6r&T5ForConditionalGeneration.parallelizeNs  A   ! 3t||112E%**:Q:Q:S4T U  $//3t||/A/A+BC   1   1||t||'@'@A "rc[R"S[5 URR 5 UR R 5 URR S5UlUR R S5UlURR S5UlSUlSUl [RR5 grJ) rrrr@rrLrrNrrrqrrrs r6r(T5ForConditionalGeneration.deparallelizebs j   ""$ ""$||u- ||u- ||u- #  rcUR$rrMrs r6r/T5ForConditionalGeneration.get_input_embeddingsqrOrc|XlURRU5 URRU5 grrQrs r6r/T5ForConditionalGeneration.set_input_embeddingstrSrcURR(aaURURRUR 5 URUR RUR 5 ggrrUrs r6rW'T5ForConditionalGeneration._tie_weightsyrYrcXlgrrNrs r6set_output_embeddings0T5ForConditionalGeneration.set_output_embeddings~s% rcUR$rrrs r6get_output_embeddings0T5ForConditionalGeneration.get_output_embeddingsr^rcUR$rr[rs r6r\&T5ForConditionalGeneration.get_encoderr^rcUR$rr`rs r6ra&T5ForConditionalGeneration.get_decoderr^rrrUrrrrjrrkrrrllabelsr8r9rrprrc*U bU OURRn UbUOURRnUbRUcOURRURR:Xa![ R "[[5 UnUcURUUU UUUUS9nORU(aK[U[5(d6[US[U5S:aUSOS[U5S:aUSOSS9nUSnUR(a3[RR!UR"R$5 U bUcU cUR'U 5nUR(a[RR!UR"R$5 UR)UR"R$5nUb%UR)UR"R$5nUb%UR)UR"R$5nUb%UR)UR"R$5nUR#UUU U UUUUU UUUUS9 nUSnUR(a[RR!URR$5 UR*R)URR$5UlUR)UR*R,R.5nURR0(aUUR2S--nUR+U5nSnU b[[5S S 9nU R)UR.5n U"UR7S UR9S 55U R7S 55nU(dU4USS-U-nUbU4U-$U$[;UUUR<UR>UR@URBURDUR>UR@S 9 $) a$ input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. decoder_head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[-100, 0, ..., config.vocab_size - 1]`. All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size]` Examples: ```python >>> from transformers import AutoTokenizer, T5ForConditionalGeneration >>> tokenizer = AutoTokenizer.from_pretrained("google-t5/t5-small") >>> model = T5ForConditionalGeneration.from_pretrained("google-t5/t5-small") >>> # training >>> input_ids = tokenizer("The walks in park", return_tensors="pt").input_ids >>> labels = tokenizer(" cute dog the ", return_tensors="pt").input_ids >>> outputs = model(input_ids=input_ids, labels=labels) >>> loss = outputs.loss >>> logits = outputs.logits >>> # inference >>> input_ids = tokenizer( ... "summarize: studies have shown that owning a dog is good for you", return_tensors="pt" ... ).input_ids # Batch size 1 >>> outputs = model.generate(input_ids) >>> print(tokenizer.decode(outputs[0], skip_special_tokens=True)) >>> # studies have shown that owning a dog is good for you. ```Nrnrr'rCrorprr ignore_indexr: lossrMrrqrrrrsrlrt)#rxr8rrrArr2_T5ForConditionalGeneration__HEAD_MASK_WARNING_MSGrr@rrrhrrqrrrLrrrrNr>rrrr r"rrrrrrr)rrrUrrrrjrrkrrrlrr8r9rrprrrvsequence_output lm_logitsrloss_fctoutputs r6r"T5ForConditionalGeneration.forwardsp"+!6IDKK{{%%)G)GG 5}E$-!  ""ll#-+#"3%9'+OO_!M!M-"1!"4474H14Loa0RV14_1E1I?1-tO (*    JJ ! !$,,";"; <  "3";@U@] $ 1 1& 9     JJ ! !$,,";"; <),,T\\-F-FGM ,$5$8$89R9R$S!)!/!2!24<<3L3L!M%1)?)B)B4<\%\ "'"1(&%r)rLrr@rNrrrr)NNNNNNNNNNNNNNNNN)$rrrrryrzr(rrr4rr5rrrrWrrr\rar rrqr{r|r}rrr.rrrrrrrrs@r6rr(sj Q*&jx6/0#1#&12 !3 !: O &156:8<=A159=7;@D@D59=A-1$(,0/3&*59%} E,,-} !!2!23} $E$4$45 } !))9)9 : } E--. } $E$5$56} 'u||4} "%ell(;"<=} "%ell(;"<=}   1 12}  ((9(9:} ))*} D>} $D>} 'tn!} "d^#} $!!1!12%} & uU&&'8 9'} } ~)ELL)&&rrcz^\rSrSrS/rS/rS\4U4Sjjr\"\ 5SSj5r \"\ 5S5r Sr S rS rS rS r\SS \\R*S\\R,S\\R,S\\R,S\\S\\S\\S\\\R,\44Sjj5rSrU=r$)rilr8rLrxc<>[TU]U5 [R"URUR 5Ul[R"U5nSUl SUl [X R 5Ul UR5 SUlSUlgNF)rrrrr<rrr=r>r8r?rr@rrr)rrxrCrs r6rT5EncoderModel.__init__qsw  ll6#4#4fnnE v.#( ,1)~{{;  $rc[R"S[5 UcN[[ UR R 5[[RR555OUUl [UR[ UR R 55 UR RUR5 SUlg)NaC`T5EncoderModel.parallelize` is deprecated and will be removed in v5 of Transformers, you should load your model with `device_map='balanced'` in the call to `from_pretrained`. You can also provide your own `device_map` but it needs to be a dictionary module_name to device, so for instance {'block.0': 0, 'block.1': 1, ...}T)rrrr&rhr@rrrqrrrr%rrrGs r6rT5EncoderModel.parallelizes  "   ! 3t||112E%**:Q:Q:S4T U  $//3t||/A/A+BC   1"rc[R"S[5 URR 5 URR S5UlSUlSUl[RR5 grJ) rrrr@rrrrrqrrrs r6rT5EncoderModel.deparallelizesY j   ""$||u- #  rcUR$rrMrs r6r#T5EncoderModel.get_input_embeddingsrOrcFXlURRU5 gr)rr@rrs r6r#T5EncoderModel.set_input_embeddingss$  )).9rcURR(a1URURRUR 5 ggr)rxrrVr@rrrs r6rWT5EncoderModel._tie_weightss2 ;; * *  & &t||'@'@$++ N +rcUR$rr[rs r6r\T5EncoderModel.get_encoderr^rcUR5HDup#URRURSRR U5 MF g)rdrN)rr@rr@rRrrfs r6rhT5EncoderModel._prune_headssG +002LE LL  u % + +A . < < H H O3rrrUrrr9rrprc fUbUOURRnURUUUUUUUS9nU$)a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). Example: ```python >>> from transformers import AutoTokenizer, T5EncoderModel >>> tokenizer = AutoTokenizer.from_pretrained("google-t5/t5-small") >>> model = T5EncoderModel.from_pretrained("google-t5/t5-small") >>> input_ids = tokenizer( ... "Studies have been shown that owning a dog is good for you", return_tensors="pt" ... ).input_ids # Batch size 1 >>> outputs = model(input_ids=input_ids) >>> last_hidden_states = outputs.last_hidden_state ```rn)rxrr@) rrrUrrr9rrprks r6rT5EncoderModel.forwardsKD&1% -'tn-d^- uU&&'8 9--rrz T5 model with a sequence classification/head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. c$^\rSrSrS/rSS/rS\4U4Sjjr\SS\ \ RS\ \ RS \ \ RS \ \ RS \ \ RS \ \ RS \ \ RS\ \ \ RS\ \ RS\ \ RS\ \ RS\ \S\ \S\ \S\ \S\\\44 Sjj5rSrU=r$)T5ForSequenceClassificationir7r8r9rxc>[TU]U5 [U5Ul[ U5UlUR 5 SUlgr)rrrrrxclassification_headrrrs r6r$T5ForSequenceClassification.__init__s=  "6?#7#?  #rrrUrrrrjrrkrrlrr8r9rrprc4UbUOURRnU bSn Uc%U b"[SURR35eUc"U cUc [ S5eUR U5nURUUUUUUUUU U U U UUS9nUSnURURR5RUR5n[[R"URS555S:a [ S5eUR unnnUUSS24R#US U5SS2S SS24nUR%U5nSnU GbU RUR5n URR&cURR(S:XaS URlOyURR(S:aNU R*[R,:XdU R*[R.:XaS URlOS URlURR&S :XaT[15nURR(S:Xa&U"UR35U R355nOU"UU 5nOURR&S :XaG[55nU"UR#S URR(5U R#S 55nO-URR&S :Xa[75nU"UU 5nU(dU4USS-nUbU4U-$U$[9UUUR:UR<UR>UR@URBURDURFS 9 $)a_ input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. decoder_head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NFz8Passing input embeddings is currently not supported for If no `decoder_input_ids` or `decoder_inputs_embeds` are passed, `input_ids` cannot be `None`. Please pass either `input_ids` or `decoder_input_ids` or `decoder_inputs_embeds`.) rUrrrrjrrkrrlr8r9rrprr'z7All examples must have the same number of tokens.r: regressionsingle_label_classificationmulti_label_classificationr)$rxrNotImplementedErrorrrrnrreq eos_token_idrrrhrqunique_consecutivesumrlr"r problem_typer|rrrkr squeezer rrrrqrrrrsrlrt)rrrUrrrrjrrkrrlrr8r9rrprHreos_maskr:r rsentence_representationrMrrrs r6r#T5ForSequenceClassification.forwardsR|&1%>KbKbJcd   $)>)F  U !% 1 1) < "" )/#9/!5+'"7/!5##  "!*<< 8 89<<_=S=ST u'' Q8 9A =VW W%4%:%:" A{"1(A+">"C"CJPRT_"`abdfhiai"j))*AB  YYv}}-F{{''/;;))Q./;DKK,[[++a/V\\UZZ5OSYS_S_chclclSl/LDKK,/KDKK,{{''<7"9;;))Q.#FNN$4fnn6FGD#FF3D))-JJ+- B 0F0F GUWY))-II,./Y,F)-)9TGf$ Ev E.#33")"?"?&99$55&-&G&G")"?"?&99  r)rrr)NNNNNNNNNNNNNNN)rrrrryrzr(rr rrqr{rrr|r.rrrrrrrs@r6rrs+s)s&79VW$x$15158<=A,0487;=A59=A-1$(,0/3&*!P E,,-P !.P $E$4$45 P !))9)9 : P ELL) P $ELL1P 'u||4P "$u'8'8"9:P   1 12P  ((9(9:P ))*P D>P $D>P 'tnP d^!P " u55 6#P P rrc:^\rSrSrS/rS\4U4Sjjr\SS\\ RS\\ RS\\ RS\\ RS \\ RS \\ S \\ S \\ S \ \ \ R\44Sjj5rSrU=r$)riz'transformer.encoder.embed_tokens.weightrxc0>[TU]U5 URUl[U5Ul[ R "UR5Ul[ R"URUR5Ul UR5 gr) rrr|rrrrr{rrrrKrrs r6r!T5ForTokenClassification.__init__sj   ++)&1zz&";";< ))F$6$68I8IJ rrrUrrrr9rrprc UbUOURRnURUUUUUUUS9n U Sn URU 5n UR U 5n Sn Ub<[ 5n U "U R SUR5UR S55n U(dXSS4nU bU 4U-$U$[U U U RU RS9$)a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. N)rUrrr9rrprr:rC)rrMrr) rxrrrrKr r"r|rrr)rrrUrrrr9rrprHrrMrrrs r6r T5ForTokenClassification.forwards4&1%7 'tn7 d^7  uU\\"$99 :7 7 rrc&X^\rSrSrS/rSS/rS\4U4SjjrSrSr S r S r S r \ SS \\R S \\R"S\\R S\\R$S\\R"S\\R"S\\R&S\\\\R&S\\R S\\R S\\R"S\\R"S\\S\\S\\S\\S\\\R"\44"Sjj5rSrU=r$) rir7r8r9rxc~>[TU]U5 URUl[R "UR UR5Ul[R"U5nSUl SUl SUl [X R5Ul[R"U5nSUl SUl URUl[X0R5UlUR$Ul[R&"UR(UR$5UlUR-5 SUlgr;)rrrrrrr<rr=r>rr8r?rr@rArrLr|rrrrrrBs r6rT5ForQuestionAnswering.__init__s  ll6#4#4fnnE v.$)!#( ,1)~{{; v.$(!,1)$*$=$=!~{{;  ++))F$6$68I8IJ #rcUR$rrMrs r6r+T5ForQuestionAnswering.get_input_embeddingsrOrc|XlURRU5 URRU5 grrQrs r6r+T5ForQuestionAnswering.set_input_embeddingsrSrcURR(aaURURRUR 5 URUR RUR 5 ggrrUrs r6rW#T5ForQuestionAnswering._tie_weightsrYrcUR$rr[rs r6r\"T5ForQuestionAnswering.get_encoder r^rcUR$rr`rs r6ra"T5ForQuestionAnswering.get_decoder r^rrrUrrrrjrrkstart_positions end_positionsrrlr8r9rrprcUbUOURRnU bU OURRn U bU bSn Uc"U cUc [S5eUR U5nU bU OURRn UbUOURRnUbRUcOURR URR :Xa![R"[[5 UnUcURUUU UUUUS9nORU(aK[U[5(d6[US[U5S:aUSOS[U5S:aUSOSS9nUSnURUUU SUUUUU UUUS 9 nUSnUR!U5nUR#SS S 9unnUR%S 5R'5nUR%S 5R'5nSnU bU b[U R)55S:a*U R%S 5R+UR,5n [U R)55S:a*U R%S 5R+UR,5n UR)S5nU R/SU5n U R/SU5n [1US 9nU"UU 5nU"UU 5nUU-S- nU(dUU4USS-U-nUbU4U-$U$[3UUUUR4UR6UR8UR:UR<UR6UR8S 9 $)aM input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. decoder_head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. NFrrnrr'rCro) rrUrrrlrmrrr8r9rrpr:rr) r start_logits end_logitsrrqrrrrsrlrt)rxrr8rnrrrArr._T5ForQuestionAnswering__HEAD_MASK_WARNING_MSGrr@rrrhrLrrarr2rrrrkr rrrrrr)rrrUrrrrjrrkrrrrlr8r9rrprrvrrMrr total_loss ignored_indexr start_lossend_lossrs r6rT5ForQuestionAnswering.forward s{x&1%)F  U !% 1 1) < !*!6IDKK{{%%)G)GG 5}E$-!  ""ll#-+#"3%9'+OO_!M!M-"1!"4474H14Loa0RV14_1E1I?1-tO (* ,,'1/ "/#1'!5/!5#'  *!,1#)<<r<#: j#++B/::< ''+668   &=+D?'')*Q."1"9"9""="@"@ATAT"U=%%'(1, - 5 5b 9 < ` $D>` 'tn!` "d^#` $ uU&&')LL M%` ` rr)rrrrrrrr)[rr=rrYrtypingrrrrrqrtorch.nnrr r activationsr cache_utilsr rr generationrmodeling_attn_mask_utilsrmodeling_outputsrrrrrrrmodeling_utilsr pytorch_utilsrrrutilsrrrr r!r"r#r$utils.model_parallel_utilsr%r&configuration_t5r(!torch.nn.attention.flex_attentionr)integrations.flex_attentionr* get_loggerrrWrr4r5Modulerapex.normalizationrr\rV Exceptionrr`rrrrrOr[rcrxrr__HEAD_MASK_WARNING_MSGrrrrrr__all__r3rr6r s // AA!CC)>.gg   L& !!;J   H %f\ B(+"))+2  /K KKabK(bii,299: $a"))aH!299!H#BII#LabiiaH299$m!m!m!`xxx o o o d |&!2O|&  |&~ v&vvr _ "3_  _ DF 0F F RR .R R j CE   NNWX sH88III