fThod SSKrSSKJr SSKJrJrJrJrJrJ r SSK r SSK J r SSK JrJrJrJr SSKJr SSKJrJr SSKJr SS KJr \(aSS KJr \R:"\5r\"S S \55r "S S\5r!g)N) dataclass) TYPE_CHECKINGDictListOptionalTupleUnion)GenerateDecoderOnlyOutputGenerationConfigGenerationMixinGenerationMode)LogitsProcessorList)MaxLengthCriteriaStoppingCriteriaList)GenerateNonBeamOutput)logging) BaseStreamercH\rSrSr%SrSr\\\R\ S'Sr g)CsmGenerateOutput*a Outputs of CsmForConditionalGeneration.generate. Args: sequences (`torch.LongTensor` of shape `(batch_size, sequence_length)`): The generated sequences. The second dimension (sequence_length) is either equal to `max_length` or shorter if all batches finished early due to the `eos_token_id`. scores (`tuple(torch.FloatTensor)` *optional*, returned when `output_scores=True`): Processed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax) at each generation step. Tuple of `torch.FloatTensor` with up to `max_new_tokens` elements (one element for each generated token), with each tensor of shape `(batch_size, config.vocab_size)`. logits (`tuple(torch.FloatTensor)` *optional*, returned when `output_logits=True`): Unprocessed prediction scores of the language modeling head (scores for each vocabulary token before SoftMax) at each generation step. Tuple of `torch.FloatTensor` with up to `max_new_tokens` elements (one element for each generated token), with each tensor of shape `(batch_size, config.vocab_size)`. attentions (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `output_attentions=True`): Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of `torch.FloatTensor` of shape `(batch_size, num_heads, generated_length, sequence_length)`. hidden_states (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `output_hidden_states=True`): Tuple (one element for each generated token) of tuples (one element for each layer of the decoder) of `torch.FloatTensor` of shape `(batch_size, generated_length, hidden_size)`. past_key_values (`tuple(tuple(torch.FloatTensor)))`, *optional*, returned when `use_cache=True`): Returns the model cache, used to speed up decoding. Different models have a different cache format, check audio (`list(torch.FloatTensor)` of length `batch_size`): The generated audio. Naudio) __name__ __module__ __qualname____firstlineno____doc__rrrtorchTensor__annotations____static_attributes__r^/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/csm/generation_csm.pyrr*s!6+/E8D& '.r#rc^\rSrSrS\4U4SjjrSS\\S\\S\ S\ \\ 44U4Sjjjr S\ RS \S \S\S \S \S S\\\ R44SjrSS\\ R$S\\ R$S\\ R$S\\S \\S \\S \\S \S S\\S\\\ R44U4SjjjrSrU=r$)CsmGenerationMixinJreturnc>[TU]"U0UD6n[5nUHXn[U[5(d/[ R SURRS35 MGURU5 MZ U$)NzCsm does not support z' stopping criteria, it will be ignored.) super_get_stopping_criteriar isinstancerloggerwarning __class__rappend)selfargskwargscriteria kept_criteria criterionr/s r$r+)CsmGenerationMixin._get_stopping_criteriaKsw 714B6B,. !Ii):;;+I,?,?,H,H+IIpq$$Y/ "r#generation_configuse_model_defaultsr3c |>^ UR5VVs0sH,upEURS5(dMU[S5SU_M. nnnUR5VVs0sHupEURS5(aMXE_M! nnn[T U]"X40UD6upUR R R"S 0UD6 [UR R S5=(d URRS- n[UR R S5=(d URRS- n X1URRS- 1:wa,[SUSU SURRS- S 35eUR R R(a0[RS 5 S UR R l XR R lXR R lUR"m SU 4S jjn XlX4$s snnfs snnf)z This method overrides [~generation.utils.GenerationMixin._prepare_generation_config]. It ensures that the depth decoder generation config is initialized and that passed args as depth_decoder_* are properly handled. depth_decoder_Nmin_new_tokensmax_new_tokensz2depth_decoder_generation_config's min_new_tokens (z) and max_new_tokens (z2) must be equal to self.config.num_codebooks - 1 ()zdepth_decoder_generation_config.return_dict_in_generate is set to True, but this will be ignored as the depth decoder model does not return a dictionary in generateFc~>T"U5nU[R[R4;a[SUS35eU$)NzGeneration mode zg is not supported for CSM model. Please set generation parameters to use greedy or sampling generation.)r GREEDY_SEARCHSAMPLE ValueError)assistant_modelgeneration_modeoriginal_get_generation_modes r$patched_get_generation_modeRCsmGenerationMixin._prepare_generation_config..patched_get_generation_modesO:?KO~'C'C^EZEZ&[[ &&78_`# "r#rN)items startswithlenr*_prepare_generation_config depth_decoderr8updategetattrconfig num_codebooksrCreturn_dict_in_generater-r.r<r>get_generation_mode) r1r8r9r3kvdepth_decoder_kwargs model_kwargsdepth_decoder_min_new_tokensdepth_decoder_max_new_tokensrGrFr/s @r$rM-CsmGenerationMixin._prepare_generation_config\sL7=lln 6Dda UeHf )Ac"#% & )n  $*<<>X>41FV9W$!$>X+0'*L + 5;+ ' ,,33K6JK(/t/A/A/S/SUe'f( KK % % ) %(/t/A/A/S/SUe'f( KK % % ) % ) GDKKLeLehiLiKj jDEaDbbxzVyWWIJNJUJUJcJcfgJgIhhij    1 1 I I NNw LQD   0 0 H>Z,,;>Z,,;(9'L'L$ #1L- ..a YsH2H2H8?H8 input_idslogits_processorstopping_criteria synced_gpusstreamerrc URRnURSLn URn URn UR n UR n URnURnU(a U (aSOSnU(a U (aSOSnU(a U (aSOSnU(a U (aSOSnURSSunnSn[R"U[RURS9nURUURU5nURS:XaGUR!S5c5UH/n[#U[$5(dMU=R&U-slM1 UR(nUR+Xt5nU(a.S[,R.S'UR1UR25nS nUR5UUURS 9(Ga@UR6"U40UD6nUR9U (aS U 0O05 UR9S S 05 U(aU"S0UDS S 0D6nSnO U"S0UDS S 0D6nUR;UU5nU(a U(aMUR<SS2SSS24R?5RA5nURCUR5nU"UU5nU(aHU (aUU4- nU (aUU4- nU (aUURD4- nU (aUURF4- nU(aC[HRJRMUSS9n [RN"U SS9RQS5n!O[RR"USS9n!U!SS2S4n"[HRJRUU"SSS9n#URFSSS2SSS24n$URVRYU#U$R?5S9n%[#U%[RZ5(aU%O U%R\n&U&SS2SS24n&U&n!U (a,U!UR_S5-USUR_S5- --n!URS:Xa U!SS2SSS24nO![R`"UU!SS2SSS24/SS9nUbURcU!Re55 UUSS2SSS24URRf:HRiS5)-nUU"UU5)-nURk5S:HnUS- nAA%UR5UUURS 9(aGM@UbURm5 U(a[oUUUUUUR!S5S9$U$)a This method overrides [~generation.utils.GenerationMixin._sample]. To ease maintenance, modifications are marked with the comment "Csm specific". Indeed, Csm model requires a custom generation sampling step: 1. Infer the backbone model to sample the first codebook token 2. Call generate on the depth decoder with the first codebook token as input_ids to sample the next codebook tokens 3. Use these generated codebook tokens as input_ids to sample the next first codebook token using the backbone model 4. Repeat until stopping criteria is met Csm supports two stopping criterias: - stop when the generated sequence is at max_length - stop when all the generated codebook tokens are the codebook_eos_token_id NrF)dtypedevice inputs_embeds0TOKENIZERS_PARALLELISMT)rdoutput_attentionsoutput_hidden_states return_dictdimr=) num_samples)r=rr)value)r\backbone_last_hidden_statepast_key_values) sequencesscoreslogits attentions hidden_statesrq)8rQcodebook_pad_token_id_eos_token_tensorrhri output_scores output_logitsrS do_sampleshaperoneslongrd_get_initial_cache_positionndimgetr,r max_length__call___valid_auto_compile_criteriaosenvironget_compiled_callcompile_config_has_unfinished_sequencesprepare_inputs_for_generationrO#_update_model_kwargs_for_generationrtclonefloattorurvnn functionalsoftmax multinomialsqueezeargmaxpadrNgenerater rr unsqueezecatputcpucodebook_eos_token_idallmaxendr )'r1r\r]r^r8r_r`rX pad_token_idhas_eos_stopping_criteriarhriryrzrSr{rs raw_logitsdecoder_attentionsdecoder_hidden_states batch_sizecur_lenthis_peer_finishedunfinished_sequencesr6 model_forwardcompile_forward is_prefill model_inputsoutputsnext_token_logitsnext_token_scoresprobs next_tokensfirst_codebook_idsdepth_decoder_input_idsrpdepth_decoder_outputs codebook_idss' r$_sampleCsmGenerationMixin._samplesk4{{88 $5$G$Gt$S!-??0EE)77 )77 "3"K"K%// 0M3 RD $;@QRX\'>CW^b(oobq1 G"$zz*EJJyO_O_`77AQAQS_`  >>Q <#3#3O#D#L/ i):;;((G3(/  ;;L\ 36BJJ/ 0 223D3S3STM ,,  ##-  ==iX<XL   L]!46G Hce f   !7 > ?@@4@" 'I,IDI CCL1!(q"ax 8 > > @ F F H  1 4 4Y5E5E F !1)>r)B1b!8)L &$($6$6$?$?1NhNnNnNp%@% ! 3U\\BB&*44  (12.L&K)),@,J,J2,NNQ],66r::R ~~"'4 3 !IIy+aqj2I&JPQR # [__./$8!R"*%)J)JJc"g;$ $8;LYX^;_:_#_ !5!9!9!;q!@  qLG&U,,  ##-  Z   LLN ",#!-3 , 0 01B C   r# input_valuesinput_values_cutoffs output_audioc >[TU]"S UUUUUUUUS.U D6n [U [R5(+n Sn U (GaU (a U R OU n/n [R "5 UHnXRR:HRSS9R5nUR5S:waUR5nOURSnUSUnURRUR!SS5R#S55nU R%UR&S5 M SSS5 U (a [)S SU 0U D6$U (aU $U $!,(df  N-=f) a This method overrides [`~generation.utils.GenerationMixin.generate`] to match the specifics of the Csm model. Indeed, Csm model requires a custom generation sampling step: 1. Infer the backbone model to sample the first codebook token 2. Call generate on the depth decoder with the first codebook token as `input_ids` to sample the next codebook tokens 3. Use these generated codebook tokens as `input_ids` to sample the next first codebook token using the backbone model 4. Repeat until stopping criteria is met Most generation-controlling parameters are set in `generation_config` which, if not passed, will be set to the model's default generation configuration. You can override any `generation_config` by passing the corresponding parameters to generate(), e.g. `.generate(inputs, do_sample=True)`. Parameters: inputs_ids (`torch.Tensor` of shape (batch_size, seq_length), *optional*): The sequence used as a prompt for the backbone model. input_values (`torch.Tensor` of shape (batch_size, channels, max_concatenated_audio_length), *optional*): The batched audio input values, where each batch entry contains the concatenation of all audio segments for that entry. These values will be encoded into codebook tokens using the codec model and merged with the text input ids provided in `input_ids`. input_values_cutoffs (`torch.Tensor` of shape (batch_size, max_num_audio), *optional*): Specify the end positions of audio segments within each batch entry, relative to the concatenated audio input. If a batch entry has fewer segments than the maximum, it is padded with -1. For example, in a batch of 2 sequences where the first contains 2 audio segments of length l1, and the second contains 1 audio segment of length l2, the input_values_cutoffs would be: [[l1, 2 * l1], [l2, -1]]. generation_config ([`~generation.GenerationConfig`], *optional*): The generation configuration to be used as base parametrization for the generation call. `**kwargs` passed to generate matching the attributes of `generation_config` will override them. If `generation_config` is not provided, the default will be used, which has the following loading priority: 1) from the `generation_config.json` model file, if it exists; 2) from the model configuration. Please note that unspecified parameters will inherit [`~generation.GenerationConfig`]'s default values, whose documentation should be checked to parameterize generation. logits_processor (`LogitsProcessorList`, *optional*): Custom logits processors that complement the default logits processors built from arguments and generation config. If a logit processor is passed that is already created with the arguments or a generation config an error is thrown. This feature is intended for advanced users. stopping_criteria (`StoppingCriteriaList`, *optional*): Custom stopping criteria that complements the default stopping criteria built from arguments and a generation config. If a stopping criteria is passed that is already created with the arguments or a generation config an error is thrown. If your stopping criteria depends on the `scores` input, make sure you pass `return_dict_in_generate=True, output_scores=True` to `generate`. This feature is intended for advanced users. synced_gpus (`bool`, *optional*): Whether to continue running the while loop until max_length. Unless overridden, this flag will be set to `True` if using `FullyShardedDataParallel` or DeepSpeed ZeRO Stage 3 with multiple GPUs to avoid deadlocking if one GPU finishes generating before other GPUs. Otherwise, defaults to `False`. streamer (`BaseStreamer`, *optional*): Streamer object that will be used to stream the generated sequences. Generated tokens are passed through `streamer.put(token_ids)` and the streamer is responsible for any further processing. output_audio (`bool`, *optional*): Whether to return the generated audio. kwargs (`Dict[str, Any]`, *optional*): Ad hoc parametrization of `generation_config` and/or additional model-specific kwargs that will be forwarded to the `forward` function of the model. Depth decoder specific kwargs should be prefixed with *depth_decoder_*. Return: [`CsmGenerateOutput`] or `torch.LongTensor` or `List[torch.FloatTensor]`: A [`CsmGenerateOutput`] (if `return_dict_in_generate=True` or when `config.return_dict_in_generate=True`) or a `torch.LongTensor` when `output_audio=False` or a `List[torch.FloatTensor]` otherwise. Example: ```python >>> from transformers import CsmProcessor, CsmForConditionalGeneration >>> from datasets import load_dataset, Audio >>> model_id = "eustlb/csm-1b" >>> torch_device = "cuda" if torch.cuda.is_available() else "cpu" >>> processor = AutoProcessor.from_pretrained(model_id) >>> ds = load_dataset("hf-internal-testing/dailytalk-dummy", split="train") >>> # ensure the audio is 24kHz >>> ds = ds.cast_column("audio", Audio(sampling_rate=24000)) >>> conversation = [] >>> # prepare a conversation with text and corresponding audio >>> for text, audio, speaker_id in zip(ds[:4]["text"], ds[:4]["audio"], ds[:4]["speaker_id"]): ... conversation.append( ... { ... "role": f"{speaker_id}", ... "content": [{"type": "text", "text": text}, {"type": "audio", "path": audio["array"]}], ... } ... ) >>> # text prompt >>> conversation.append({"role": f"{ds[4]['speaker_id']}", "content": [{"type": "text", "text": ds[4]["text"]}]}) >>> inputs = processor.apply_chat_template( ... conversation, ... tokenize=True, ... return_dict=True, ... ).to(torch_device) >>> model = CsmForConditionalGeneration.from_pretrained(model_id, device_map=torch_device) >>> audio = model.generate(**inputs, output_audio=True) >>> processor.save_audio(audio, "output.wav") ``` )r\rrr8r]r^r_r`Nrkrlrr=)rrrr)r*rr,rr rrno_gradrQrrnonzeronumelminr| codec_modeldecode transposerr0 audio_valuesr)r1r\rrr8r]r^r_r`rr3generate_outputgenerate_returned_dictrgenerated_audio_codesaudio_codes_batcheos_idxs cutoff_idxcodec_decode_outputr/s r$rCsmGenerationMixin.generateRsgb '*  %!5/-/#    &0%N!N AWO$=$=]l !E*?% 1[[5V5V V[[`b[ckkmH~~'1,%-\\^ %6%<%*z $e&6&66 7z|-1/37;8<:><@&*-1',Y#ELL)Y#u||,Y#'u||4 Y# $$45 Y# ##67 Y#$$89Y#d^Y#>*Y#tnY# $e&6&66 7Y#Y#r#r&)"r dataclassesrtypingrrrrrr rtorch.nnr generationr r r rgeneration.logits_processrgeneration.stopping_criteriarrgeneration.utilsrutilsrgeneration.streamersr get_loggerrr-rr&rr#r$rsu  !DD  =S54   H % /1/ />a#a#r#