fThISSKrSSKrSSKJr SSKJr SSKJrJrJ r J r SSK J r J r \"5(a SSKrSSKJr \ "5(aSS KJr \ R("\5r"S S \R.5r\"\ "S S 95"SS\ 55r\"\ "S S 95"SS\55r\"\ "S S 95"SS\55rg)N)GenerationConfig)TruncationStrategy)add_end_docstringsis_tf_availableis_torch_availablelogging)Pipelinebuild_pipeline_init_args)/TF_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING_NAMES),MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING_NAMESc\rSrSrSrSrSrg) ReturnTyperr N)__name__ __module__ __qualname____firstlineno__TENSORSTEXT__static_attributes__rc/var/www/auris/envauris/lib/python3.13/site-packages/transformers/pipelines/text2text_generation.pyrrs G DrrT) has_tokenizerc^\rSrSrSrSr\"SSS9rSrU4Sjr SS jr S \ S \ S \ 4S jr Sr U4Sjr\R 4SjrSr\R(S4SjrSrU=r$)Text2TextGenerationPipelinea Pipeline for text to text generation using seq2seq models. Unless the model you're using explicitly sets these generation parameters in its configuration files (`generation_config.json`), the following default values will be used: - max_new_tokens: 256 - num_beams: 4 Example: ```python >>> from transformers import pipeline >>> generator = pipeline(model="mrm8488/t5-base-finetuned-question-generation-ap") >>> generator( ... "answer: Manuel context: Manuel has created RuPERTa-base with the support of HF-Transformers and Google" ... ) [{'generated_text': 'question: Who created the RuPERTa-base?'}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial). You can pass text generation parameters to this pipeline to control stopping criteria, decoding strategy, and more. Learn more about text generation parameters in [Text generation strategies](../generation_strategies) and [Text generation](text_generation). This Text2TextGenerationPipeline pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"text2text-generation"`. The models that this pipeline can use are models that have been fine-tuned on a translation task. See the up-to-date list of available models on [huggingface.co/models](https://huggingface.co/models?filter=text2text-generation). For a list of available parameters, see the [following documentation](https://huggingface.co/docs/transformers/en/main_classes/text_generation#transformers.generation.GenerationMixin.generate) Usage: ```python text2text_generator = pipeline("text2text-generation") text2text_generator("question: What is 42 ? context: 42 is the answer to life, the universe and everything") ```T)max_new_tokens num_beams generatedc>[TU]"U0UD6 URURS:Xa [5 g[ 5 g)Ntf)super__init__check_model_type frameworkr rselfargskwargs __class__s rr($Text2TextGenerationPipeline.__init__Os> $)&) ~~% < > rc 0nUbXXS'Un 0n Ub*Uc'U(a[RO[RnUbX:S'UbXJS'UbGURR USS9n [ U 5S:a[ R"S5 U SUS 'URbURU S 'URbURU S 'URU S 'XU 4$) N truncation return_typeclean_up_tokenization_spacesF)add_special_tokensr zStopping on a multiple token sequence is not yet supported on transformers. The first token of the stop sequence will be used as the stop sequence string in the interim.r eos_token_idassistant_model tokenizerassistant_tokenizer) rrrr8encodelenwarningswarnr7r9) r,return_tensors return_textr3r4r2 stop_sequencegenerate_kwargspreprocess_paramsforward_paramspostprocess_paramsstop_sequence_idss r_sanitize_parameters0Text2TextGenerationPipeline._sanitize_parametersXs  !.8l +(  %+*=0>*,,JOOK  "0;} - ' 3A]= >  $ $ 5 5mX] 5 ^ $%) b/@.BON +    +040D0DN, -  # # /*...N; '484L4LN0 1 2DDDr input_length min_length max_lengthcg)Z Checks whether there might be something wrong with given input with regard to the model. Trr,rHrIrJs r check_inputs(Text2TextGenerationPipeline.check_inputssrcURb UROSn[US[5(a>URRc [ S5eUSVs/sHoCU-PM sn4nSnO5[US[ 5(a X2S-4nSnO[ SUSS35eUR"X%XRS.6nS U;aUS U$s snf) NrzOPlease make sure that the tokenizer has a pad_token_id when using a batch inputTFz `args[0]`: zI have the wrong format. The should be either of type `str` or type `list`)paddingr2r>token_type_ids)prefix isinstancelistr8 pad_token_id ValueErrorstrr*)r,r2r-rTargrRinputss r_parse_and_tokenize/Text2TextGenerationPipeline._parse_and_tokenizes $ 7R d1gt $ $~~**2 !rss-1!W5Wcc\W57DG Q % %!W$&DGtAwi'pq :^l^lm v %'( 6sCc>[TU]"U0UD6n[US[5(aF[ SUS55(a,[ SU55(aUVs/sHoDSPM sn$U$s snf)a Generate the output text(s) using text(s) given as inputs. Args: args (`str` or `List[str]`): Input text for the encoder. return_tensors (`bool`, *optional*, defaults to `False`): Whether or not to include the tensors of predictions (as token indices) in the outputs. return_text (`bool`, *optional*, defaults to `True`): Whether or not to include the decoded texts in the outputs. clean_up_tokenization_spaces (`bool`, *optional*, defaults to `False`): Whether or not to clean up the potential extra spaces in the text output. truncation (`TruncationStrategy`, *optional*, defaults to `TruncationStrategy.DO_NOT_TRUNCATE`): The truncation strategy for the tokenization within the pipeline. `TruncationStrategy.DO_NOT_TRUNCATE` (default) will never truncate, but it is sometimes desirable to truncate the input to fit the model's max_length instead of throwing an error down the line. generate_kwargs: Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework [here](./text_generation)). Return: A list or a list of list of `dict`: Each result comes as a dictionary with the following keys: - **generated_text** (`str`, present when `return_text=True`) -- The generated text. - **generated_token_ids** (`torch.Tensor` or `tf.Tensor`, present when `return_tensors=True`) -- The token ids of the generated text. rc3B# UHn[U[5v M g7f)N)rUrY).0els r 7Text2TextGenerationPipeline.__call__..s:'BJr3'''sc3># UHn[U5S:Hv M g7f)r N)r;)r`ress rrbrcs4VcCHMVs)r'__call__rUrVall)r,r-r.resultrer/s rrf$Text2TextGenerationPipeline.__call__sq:!4262 tAw % %:$q':::4V444&,-fsFf- - .sA2c 0UR"U4SU0UD6nU$)Nr2)r\)r,r[r2r.s r preprocess&Text2TextGenerationPipeline.preprocesss!))&RZR6R rc URS:XaUSRup4O9URS:Xa)[R"US5R5up4UR WUR SUR R5UR SUR R55 SU;aUR US'URR"S 0UDUD6nURSnURS:Xa'UR"WXc-/URSSQ76nS U0$URS:Xa+[R"UWXc-/URSSQ75nS U0$) Npt input_idsr&rIrJgeneration_configrr output_idsr) r*shaper&numpyrNgetrprIrJmodelgeneratereshape)r, model_inputsrAin_brHrqout_bs r_forward$Text2TextGenerationPipeline._forwardsc >>T !!-k!:!@!@ D, ^^t #!#,{*C!D!J!J!L D      d.D.D.O.O P    d.D.D.O.O P  o 5373I3IO/ 0ZZ((K<K?K   # >>T !#++D%-W*BRBRSTSUBVWJj))^^t #Ju}0\zGWGWXYXZG[0\]Jj))rFc/nUSSHxnU[R:XaURS3U0nO>U[R:Xa*URS3URR USUS90nUR W5 Mz U$)Nrqr _token_ids_textT)skip_special_tokensr4)rr return_namerr8decodeappend)r, model_outputsr3r4recordsrqrecords r postprocess'Text2TextGenerationPipeline.postprocesss' 5a8Jj000!--.j9:F /''(.0E0E",05Q1F1 NN6 "9rr)NNNNNN)rrrr__doc___pipeline_calls_generater_default_generation_configrr(rFintrNr\rfrDO_NOT_TRUNCATErkr{rrrr __classcell__r/s@rrrs' R $!1" K %)(ET#3 *$L-?,N,N*06@__chrrcJ^\rSrSrSrSrU4SjrS\S\S\S\4S jr S r U=r $) SummarizationPipelinea/ Summarize news articles and other documents. This summarizing pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"summarization"`. The models that this pipeline can use are models that have been fine-tuned on a summarization task, which is currently, '*bart-large-cnn*', '*google-t5/t5-small*', '*google-t5/t5-base*', '*google-t5/t5-large*', '*google-t5/t5-3b*', '*google-t5/t5-11b*'. See the up-to-date list of available models on [huggingface.co/models](https://huggingface.co/models?filter=summarization). For a list of available parameters, see the [following documentation](https://huggingface.co/docs/transformers/en/main_classes/text_generation#transformers.generation.GenerationMixin.generate) Unless the model you're using explicitly sets these generation parameters in its configuration files (`generation_config.json`), the following default values will be used: - max_new_tokens: 256 - num_beams: 4 Usage: ```python # use bart in pytorch summarizer = pipeline("summarization") summarizer("An apple a day, keeps the doctor away", min_length=5, max_length=20) # use t5 in tf summarizer = pipeline("summarization", model="google-t5/t5-base", tokenizer="google-t5/t5-base", framework="tf") summarizer("An apple a day, keeps the doctor away", min_length=5, max_length=20) ```summaryc$>[TU]"U0UD6$)a~ Summarize the text(s) given as inputs. Args: documents (*str* or `List[str]`): One or several articles (or one list of articles) to summarize. return_text (`bool`, *optional*, defaults to `True`): Whether or not to include the decoded texts in the outputs return_tensors (`bool`, *optional*, defaults to `False`): Whether or not to include the tensors of predictions (as token indices) in the outputs. clean_up_tokenization_spaces (`bool`, *optional*, defaults to `False`): Whether or not to clean up the potential extra spaces in the text output. generate_kwargs: Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework [here](./text_generation)). Return: A list or a list of list of `dict`: Each result comes as a dictionary with the following keys: - **summary_text** (`str`, present when `return_text=True`) -- The summary of the corresponding input. - **summary_token_ids** (`torch.Tensor` or `tf.Tensor`, present when `return_tensors=True`) -- The token ids of the summary. r'rfr+s rrfSummarizationPipeline.__call__s0w000rrHrIrJreturnc X2:a[RSUSUS35 X:a#[RSUSUSUS-S35 g g ) rLzYour min_length=z' must be inferior than your max_length=.zYour max_length is set to z , but your input_length is only z. Since this is a summarization task, where outputs shorter than the input are typically wanted, you might consider decreasing max_length manually, e.g. summarizer('...', max_length=r)NloggerwarningrMs rrN"SummarizationPipeline.check_inputs,sm  " NN-j\9`ak`llmn o  $ NN,ZL8XYeXfg^^jno^o]ppqs  %rr) rrrrrrrfrboolrNrrrs@rrrs; <K14  # 3 SW  rrc^\rSrSrSrSrS\S\S\4Sjr\RSSS .U4S jjr SU4S jjr U4S jr S r U=r$)TranslationPipelinei;a Translates from one language to another. This translation pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"translation_xx_to_yy"`. The models that this pipeline can use are models that have been fine-tuned on a translation task. See the up-to-date list of available models on [huggingface.co/models](https://huggingface.co/models?filter=translation). For a list of available parameters, see the [following documentation](https://huggingface.co/docs/transformers/en/main_classes/text_generation#transformers.generation.GenerationMixin.generate) Unless the model you're using explicitly sets these generation parameters in its configuration files (`generation_config.json`), the following default values will be used: - max_new_tokens: 256 - num_beams: 4 Usage: ```python en_fr_translator = pipeline("translation_en_to_fr") en_fr_translator("How old are you?") ``` translationrHrIrJcNUSU-:a[RSUSUS35 g)Ng?zYour input_length: z" is bigger than 0.9 * max_length: z`. You might consider increasing your max_length manually, e.g. translator('...', max_length=400)TrrMs rrN TranslationPipeline.check_inputsWs= # * * NN%l^3UV`Uab^^ rN)r2src_langtgt_langc>[URSS5(a'URR"X@RXUS.6$[TU]"USU06$)N_build_translation_inputs)r>r2rrr2)getattrr8rr*r'r\)r,r2rrr-r/s rrkTranslationPipeline.preprocess_sQ 4>>#> E E>>;;nniq 7.LL Lrc >[T U]"S0UD6upEnUbXS'UbX$S'UcVUcSURSUR5nUR S5nU(a[ U5S:XaUSUS'USUS'XEU4$) Nrrtask_r!r r)r'rFrtrsplitr;) r,rrr.rBrCrDritemsr/s rrF(TranslationPipeline._sanitize_parametersgs@E@\@f_e@f=+=  ,4j )  ,4j )   0::fdii0DJJsOEE a05a!*-05a!*- 2DDDrc$>[TU]"U0UD6$)a Translate the text(s) given as inputs. Args: args (`str` or `List[str]`): Texts to be translated. return_tensors (`bool`, *optional*, defaults to `False`): Whether or not to include the tensors of predictions (as token indices) in the outputs. return_text (`bool`, *optional*, defaults to `True`): Whether or not to include the decoded texts in the outputs. clean_up_tokenization_spaces (`bool`, *optional*, defaults to `False`): Whether or not to clean up the potential extra spaces in the text output. src_lang (`str`, *optional*): The language of the input. Might be required for multilingual models. Will not have any effect for single pair translation models tgt_lang (`str`, *optional*): The language of the desired output. Might be required for multilingual models. Will not have any effect for single pair translation models generate_kwargs: Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework [here](./text_generation)). Return: A list or a list of list of `dict`: Each result comes as a dictionary with the following keys: - **translation_text** (`str`, present when `return_text=True`) -- The translation. - **translation_token_ids** (`torch.Tensor` or `tf.Tensor`, present when `return_tensors=True`) -- The token ids of the translation. rr+s rrfTranslationPipeline.__call__ws<w000rr)NN)rrrrrrrrNrrrkrFrfrrrs@rrr;sV 0 K#3,>+M+MX\gkMME 11rr)enumr< generationrtokenization_utilsrutilsrrrr baser r tensorflowr&models.auto.modeling_tf_autor models.auto.modeling_autor get_loggerrrEnumrrrrrrrrs )3TT4^X   H %  ,4@AR(RBRj,4@AG7GBGT,4@AY15Y1BY1r