fThd2SSKrSSKrSSKrSSKJr SSKJr SSKJrJ r J r J r SSK J r Jr \ "5(aSSKrSSKJr SS KJr \ "5(a SSKrSS KJr "S S \R05r"S S5r\ "\"SS95"SS\ 55rg)N)Dict)GenerationConfig) ModelOutputadd_end_docstringsis_tf_availableis_torch_available)Pipelinebuild_pipeline_init_args)!MODEL_FOR_CAUSAL_LM_MAPPING_NAMES) KeyDataset)$TF_MODEL_FOR_CAUSAL_LM_MAPPING_NAMESc \rSrSrSrSrSrSrg) ReturnTyperr rN)__name__ __module__ __qualname____firstlineno__TENSORSNEW_TEXT FULL_TEXT__static_attributes__r^/var/www/auris/envauris/lib/python3.13/site-packages/transformers/pipelines/text_generation.pyrrsGHIrrc&\rSrSrSrS\4SjrSrg)ChataThis class is intended to just be used internally in this pipeline and not exposed to users. We convert chats to this format because the rest of the pipeline code tends to assume that lists of messages are actually a batch of samples rather than messages in the same conversation.messagescPUHnSU;aSU;aM[S5e Xlg)NrolecontentzQWhen passing chat dicts as input, each dict must have a 'role' and 'content' key.) ValueErrorr!)selfr!messages r__init__ Chat.__init__"s.Gg%)w*> !tuu ! r)r!N)rrrr__doc__rr(rrrrrrsR!!rrT) has_tokenizerc^\rSrSrSrSrSr\"SSSS9rU4Sjr SS jr U4S jr U4S jr SS jr Sr\R SS 4SjrSrU=r$)TextGenerationPipeline)a+ Language generation pipeline using any `ModelWithLMHead` or `ModelForCausalLM`. This pipeline predicts the words that will follow a specified text prompt. When the underlying model is a conversational model, it can also accept one or more chats, in which case the pipeline will operate in chat mode and will continue the chat(s) by adding its response(s). Each chat takes the form of a list of dicts, where each dict contains "role" and "content" keys. Unless the model you're using explicitly sets these generation parameters in its configuration files (`generation_config.json`), the following default values will be used: - max_new_tokens: 256 - do_sample: True - temperature: 0.7 Examples: ```python >>> from transformers import pipeline >>> generator = pipeline(model="openai-community/gpt2") >>> generator("I can't believe you did such a ", do_sample=False) [{'generated_text': "I can't believe you did such a icky thing to me. I'm so sorry. I'm so sorry. I'm so sorry. I'm so sorry. I'm so sorry. I'm so sorry. I'm so sorry. I"}] >>> # These parameters will return suggestions, and only the newly created text making it easier for prompting suggestions. >>> outputs = generator("My tart needs some", num_return_sequences=4, return_full_text=False) ``` ```python >>> from transformers import pipeline >>> generator = pipeline(model="HuggingFaceH4/zephyr-7b-beta") >>> # Zephyr-beta is a conversational model, so let's pass it a chat instead of a single string >>> generator([{"role": "user", "content": "What is the capital of France? Answer in one word."}], do_sample=False, max_new_tokens=2) [{'generated_text': [{'role': 'user', 'content': 'What is the capital of France? Answer in one word.'}, {'role': 'assistant', 'content': 'Paris'}]}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial). You can pass text generation parameters to this pipeline to control stopping criteria, decoding strategy, and more. Learn more about text generation parameters in [Text generation strategies](../generation_strategies) and [Text generation](text_generation). This language generation pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"text-generation"`. The models that this pipeline can use are models that have been trained with an autoregressive language modeling objective. See the list of available [text completion models](https://huggingface.co/models?filter=text-generation) and the list of [conversational models](https://huggingface.co/models?other=conversational) on [huggingface.co/models]. a In 1991, the remains of Russian Tsar Nicholas II and his family (except for Alexei and Maria) are discovered. The voice of Nicholas's young son, Tsarevich Alexei Nikolaevich, narrates the remainder of the story. 1883 Western Siberia, a young Grigori Rasputin is asked by his father and a group of men to perform magic. Rasputin has a vision and denounces one of the men as a horse thief. Although his father initially slaps him for making such an accusation, Rasputin watches as the man is chased outside and beaten. Twenty years later, Rasputin sees a vision of the Virgin Mary, prompting him to become a priest. Rasputin quickly becomes famous, with people, even a bishop, begging for his blessing. Tgffffff?)max_new_tokens do_sample temperaturec>[TU]"U0UD6 URURS:Xa[O[ 5 SUR ;aSnURb URnUc0URRRS;a URnUbLUR"SSU0URD6upEn0UR EUEUl0UREUEUl ggg)Ntfprefix)XLNetLMHeadModelTransfoXLLMHeadModelTFXLNetLMHeadModelTFTransfoXLLMHeadModelr)superr(check_model_type frameworkrr _preprocess_paramsr5model __class__r XL_PREFIX_sanitize_parameters_forward_params)r&argskwargsr5preprocess_paramsforward_params_r?s rr(TextGenerationPipeline.__init__qs $)&) 48NNd4J 0Pq  422 2 F{{&~$**"6"6"?"?D#!7;7P7P7wX^7wbfbvbv7w4!1*ZT-D-D*ZHY*Z''Q$*>*>'Q.'Q$ "! 3rNc  h0n SnSU ;aU RS5=oS'SU ;aU RS5U S'U bXS'U bXS'XS'UbXmS'U(a0URUSXRS9nUSRS U S 'UbUS ;a[ US 35eX}S 'U bXS'U R U 5 U n0nUbFUcCUb [ S5eUb [ S5eU(a[ RO[ RnUb!UcUb [ S5e[ RnUbUUS'UbUUS'U bU US'UbURRUSS9nUU S'URbURUS'URbURUS'URUS'U UU4$)NFadd_special_tokenspadding truncation max_lengthr5)rKrJreturn_tensors input_ids prefix_length>holezT is not a valid value for `handle_long_generation` parameter expected [None, 'hole']handle_long_generationcontinue_final_messagez;`return_text` is mutually exclusive with `return_full_text`z>`return_full_text` is mutually exclusive with `return_tensors`z9`return_text` is mutually exclusive with `return_tensors` return_typeclean_up_tokenization_spaces)rJ eos_token_idassistant_model tokenizerassistant_tokenizer) poprYr<shaper%updaterrrrencoderXrZ)r&return_full_textrN return_textrUrVr5rS stop_sequencerLrMrTgenerate_kwargsrErJ prefix_inputsrFpostprocess_paramsstop_sequence_idss rrA+TextGenerationPipeline._sanitize_parameterssE" ? 2KZK^K^_sKt t 3G!H  '+:+>+>y+I i (  !.8l +  !.8l +,6L )  *0h '  NN:L]k]k+M0=[/I/O/OPR/SOO , ! -%X5 -./&&;Q6 7 ! -:P6 7  1(  'K,?& !^__) !abb2B*.. H[H[K  %+*=& !\]]$,,K  "0; } - ' 3A] = > ! -;Q 7 8  $ $ 5 5mX] 5 ^ .?ON +    +040D0DN, -  # # /*...N; '484L4LN0 1 .2DDDrc>URRRS;aURSS05 [TU]"U0UD6$)z Parse arguments and tokenize )r7add_space_before_punct_symbolT)r>r?rr]r:_parse_and_tokenize)r&rCrDr?s rri*TextGenerationPipeline._parse_and_tokenizesE ::   ( (,D D MM:DA Bw*D;F;;rc >[U[5(a [[[R [ 4O[[[R 45(a[U[R 5(a,[R"U5upSU5[U5pAOUSn[U[[[45(a[U[5(a[TU]0"[U540UD6$SU5n[U[R 5(a[TU]0"U40UD6$[TU]0"[U540UD6$[TU]0"U40UD6$)a Complete the prompt(s) given as inputs. Args: text_inputs (`str`, `List[str]`, List[Dict[str, str]], or `List[List[Dict[str, str]]]`): One or several prompts (or one list of prompts) to complete. If strings or a list of string are passed, this pipeline will continue each prompt. Alternatively, a "chat", in the form of a list of dicts with "role" and "content" keys, can be passed, or a list of such chats. When chats are passed, the model's chat template will be used to format them before passing them to the model. return_tensors (`bool`, *optional*, defaults to `False`): Returns the tensors of predictions (as token indices) in the outputs. If set to `True`, the decoded text is not returned. return_text (`bool`, *optional*): Returns the decoded texts in the outputs. return_full_text (`bool`, *optional*, defaults to `True`): If set to `False` only added text is returned, otherwise the full text is returned. Cannot be specified at the same time as `return_text`. clean_up_tokenization_spaces (`bool`, *optional*, defaults to `True`): Whether or not to clean up the potential extra spaces in the text output. continue_final_message( `bool`, *optional*): This indicates that you want the model to continue the last message in the input chat rather than starting a new one, allowing you to "prefill" its response. By default this is `True` when the final message in the input chat has the `assistant` role and `False` otherwise, but you can manually override that behaviour by setting this flag. prefix (`str`, *optional*): Prefix added to prompt. handle_long_generation (`str`, *optional*): By default, this pipelines does not handle long generation (ones that exceed in one form or the other the model maximum length). There is no perfect way to address this (more info :https://github.com/huggingface/transformers/issues/14033#issuecomment-948385227). This provides common strategies to work around that problem depending on your use case. - `None` : default strategy where nothing in particular happens - `"hole"`: Truncates left of input, and leaves a gap wide enough to let generation happen (might truncate a lot of the prompt and not suitable when generation exceed the model capacity) generate_kwargs (`dict`, *optional*): Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework [here](./text_generation)). Return: A list or a list of lists of `dict`: Returns one of the following dictionaries (cannot return a combination of both `generated_text` and `generated_token_ids`): - **generated_text** (`str`, present when `return_text=True`) -- The generated text. - **generated_token_ids** (`torch.Tensor` or `tf.Tensor`, present when `return_tensors=True`) -- The token ids of the generated text. c3$# UHov M g7fNr).0xs r 2TextGenerationPipeline.__call__..!s*Bk1ksrc38# UHn[U5v M g7frm)r)rnchats rrprq)s@KDT$ZZKs) isinstancer listtupletypes GeneratorTyper itertoolsteenextdictr:__call__r)r& text_inputsrDrG first_itemchatsr?s rr}TextGenerationPipeline.__call__s^  !##5%--z :u223   +u':':;;!*{!; *Bk*BDGZ(^ *tUD&9::j$// 7+D,=HHH@K@E!+u/B/BCC$w/@@@$w/U FvFFw 6v66rc  NUUUUS.n U R5V V s0sH upU cM X_M n n n [U[5(afU RSS5 UcURSSS:HnUR R "UR4U(+USURS.U D6n O!UR "X!-4SUR0U D6n XS 'US :XaU S RSnS U ;aU S nO:U RS URR5U- nUS:a [S5eX-UR R:aTUR RU- nUS::a [S5eU S SS2U*S24U S 'SU ;aU SSS2U*S24U S'U $s sn n f)N)rJrLrKrMrJrPr# assistantT)add_generation_promptrT return_dictrNrN prompt_textrRrOr0rMrz0We cannot infer how many new tokens are expectedziWe cannot use `hole` to handle this generation the number of desired tokens exceeds the models max lengthattention_mask)itemsrtrr[r!rYapply_chat_templater<r\getgeneration_configrMr%model_max_length)r&rr5rSrJrLrKrMrTrbtokenizer_kwargskeyvalueinputscur_len new_tokens keep_lengths r preprocess!TextGenerationPipeline.preprocess0s#5$$   :J9O9O9Qg9Q:3UZJCJ9Qg k4 ( (  !5t <&-)4)=)=b)A&)I[)X&^^77$$*@&@'= #~~  # F^^F$8ll[klF +} !V +[)//3G?2,-=> ,00t?U?U?`?`adkk >$%WXX#dnn&E&EE"nn== J !#$- '-[&9!k\]:J&K{##v-/56F/GK<=HX/YF+, Shs F!F!c USnURSS5nURSS:XaSnSnSnOURSnURS5nURSS5nUS:aSU;=(d SU;=(a USRSLnU(d>URS 5=(d URR US 'US ==U- ss'S U;=(d SU;=(a USR SLn U (dS U;a US ==U- ss'SU;aURUS'URR"SX4S .UD6n [U [5(GaU Rn U R5V V s0sHupU S ;dM X_M nn n U RSnURS:XaUR5Hunn[U[R5(a9URSU:Xa&UR "X_U-/URSSQ76UU'[U["5(dMu[%US5U:XdM[R&"U5R)SS5nUUU'M OURS:XaUR5Hunn[U[*R5(aAURSU:Xa.[*R "UX_U-/URSSQ75UU'[U["5(dM}[%US5U:XdM[*R&"U5R)SS5nUUU'M OU n 0nU RSnURS:Xa$U R "X_U-/U RSSQ76n O:URS:Xa*[*R "XX-/U RSSQ75n U UUS.nU(aUR-SU05 U$s sn n f)NrOrr rrrQr0rrMmin_new_tokens min_length)rOr> sequencespast_key_valuesptr4)generated_sequencerOradditional_outputsr)rr\r[r0rrMrr>generatertrrrr<torchTensorreshapervlenstackswapaxesr4r])r& model_inputsrbrOrin_brrQhas_max_new_tokenshas_min_new_tokensoutputrkv other_outputsout_brr model_outputss r_forwardTextGenerationPipeline._forwardns - %))*:DA ??1  "I!ND??1%D"&&}5 (++OQ? 1 !1_!D"#6T#$78GGtS &0?0C0CL0Q0vUYUkUkUvUv - ->-!1_!D"#6T#$78GGtS &,/*I ->- o 5373I3IO/ 0$$kyk[jk fk * *!'!1!1 .4llnjndaIi@iTQTnMj&,,Q/E~~%"/"5"5"7JC!%665;;q>U;R-2]]4$-aQVQ\Q\]^]_Q`-a c*!%//CaMU4J % E 2 ; ;Aq A-2 c* #8 4'"/"5"5"7JC!%33 A%8O-/ZZtm?fV[VaVabcbdVe?f-g c*!%//CaMU4J " 8 8A >-2 c* #8"( M"((+ >>T !!3!;!;D4-!oRdRjRjklkmRn!o  ^^t #!#,>u}@tWiWoWopqprWs@t!u #5"&   "6 !F GEks  PPc USSnUSnUSnUR5R5n/nURS05n 0n U (GaURS:XayU R 5Hdup[ U [ R5(dM&U RS[U5:XdMDU R5R5X'Mf OURS:XaxU R 5Hdup[ U [R5(dM&U RS[U5:XdMDU R5R5X'Mf [U5GHupU[R:XaSU0nGOZU[R[R1;Ga5UR R#US US 9nUcSnO'[UR R#USS US 95nUUSnU[R:Xa[ U[$5(aUU-nO[ U[&5(a~UcUR(S S S :HnU(aC[+UR(5SS UR(S S UR(S SU-S./-nO[+UR(5S US./-nSU0nU R 5HunnUU UU'M UR-W5 GM U$)NrrrOrrrr4generated_token_idsT)skip_special_tokensrVrPr#rr$)r#r$generated_text)numpytolistrr<rrtrrr\rr4 enumeraterrrrrYdecodestrrr!ruappend)r&rrUrVrTrrOrrecordsr splitted_keysrridxsequencerecordtext prompt_lengthall_textrvaluess r postprocess"TextGenerationPipeline.postprocesss++?@C!+. #M2 /557>>@%))*>C  ~~%)//1DA!!U\\22qwwqzSI[E\7\+,779+;+;+= (24')//1DA!!RYY//AGGAJ#FXBY4Y+,779+;+;+= (2''9:MCj000/:!4!4j6J6J KK~~,,(,1M-$$%M$'--%aL049U.%M /*"6"66!+s33#.#9#K66196A5I5I"5Mf5UYd5d21'+K,@,@'A#2'F,7,@,@,DV,L/:/C/CB/G /RU]/]!"J(H(,K,@,@'AkfnEoDp'pH*H5#0#6#6#8KC"(+F3K$9 NN6 "];`r)rBr=) NNNNNNNNNNN)NNNNNN)rrrrr*r@_pipeline_calls_generater_default_generation_configr(rArir}rrrrrr __classcell__)r?s@rr-r-)s.hI $!1" R:%)##QEh<D7R##<|HZ((%)# HHrr-)enumryrwtypingr generationrutilsrrrr baser r rmodels.auto.modeling_autor pt_utilsr tensorflowr4models.auto.modeling_tf_autorEnumrrr-rrrrs  )XX4M$S ! !,4@AVXVBVr