fTheSrSSKJr SSKJrJrJr SSKrSSKrSSKJ r SSK J r SSK J r SS KJrJr S S KJr \R&"\5r\"S S \ 55r"SS\ R.5r"SS\ R.5r"SS\ R.5r"SS\ R.5r\"SS\ 55rS/rg)zPyTorch UnivNetModel model.) dataclass)OptionalTupleUnionN)nn) ModelOutput)PreTrainedModel)auto_docstringlogging) UnivNetConfigcj\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr g)UnivNetModelOutput a Output class for the [`UnivNetModel`], which includes the generated audio waveforms and the original unpadded lengths of those waveforms (so that the padding can be removed by [`UnivNetModel.batch_decode`]). Args: waveforms (`torch.FloatTensor` of shape `(batch_size, sequence_length)`): Batched 1D (mono-channel) output audio waveforms. waveform_lengths (`torch.FloatTensor` of shape `(batch_size,)`): The batched length in samples of each unpadded waveform in `waveforms`. N waveformswaveform_lengths) __name__ __module__ __qualname____firstlineno____doc__rrtorch FloatTensor__annotations__r__static_attributes__rd/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/univnet/modeling_univnet.pyrr s4 .2Ix))*148hu0018rrcb^\rSrSrSrS\4U4SjjrS\R4Sjr Sr Sr S r U=r $) #UnivNetKernelPredictorResidualBlock1z Implementation of the residual block for the kernel predictor network inside each location variable convolution block (LVCBlock). Parameters: config: (`UnivNetConfig`): Config for the `UnivNetModel` model. configc>[TU]5 URUlURUlUR UlURUlUR S- S-n[R"UR5Ul [R"URURUR USS9Ul [R"URURUR USS9Ulg)Nr Tpaddingbias)super__init__model_in_channelschannelskernel_predictor_conv_size kernel_sizekernel_predictor_dropout dropout_probleaky_relu_sloperDropoutdropoutConv1dconv1conv2)selfr#r' __class__s rr*,UnivNetKernelPredictorResidualBlock.__init__;s 00 !<<";; & 7 7##a'A-zz$"3"34 YYt}}dmmT=M=MW^eij YYt}}dmmT=M=MW^eij r hidden_statescUnURU5nURU5n[RR XR 5nUR U5n[RR XR 5nX-$N)r3r5r functional leaky_relur1r6)r7r:residuals rforward+UnivNetKernelPredictorResidualBlock.forwardKsn  ]3  =1  00@U@UV  =1  00@U@UV ''rc[RRn[[RRS5(a$[RRRnU"UR 5 U"UR 5 gN weight_norm)rutilsrDhasattrparametrizationsr5r6r7rDs rapply_weight_norm5UnivNetKernelPredictorResidualBlock.apply_weight_normUsUhh** 288,,m < <((33??KDJJDJJrc[RRUR5 [RRUR5 gr<)rrEremove_weight_normr5r6r7s rrL6UnivNetKernelPredictorResidualBlock.remove_weight_norm]s. ##DJJ/ ##DJJ/r)r,r5r6r3r0r.r1)rrrrrrr*rrr@rIrLr __classcell__r8s@rr!r!1s8kk (U%6%6( 00rr!cr^\rSrSrSrS S\S\S\4U4SjjjrS\R4Sjr S r S r S r U=r$) UnivNetKernelPredictorba Implementation of the kernel predictor network which supplies the kernel and bias for the location variable convolutional layers (LVCs) in each UnivNet LVCBlock. Based on the KernelPredictor implementation in [maum-ai/univnet](https://github.com/maum-ai/univnet/blob/9bb2b54838bb6d7ce767131cc7b8b61198bc7558/model/lvcnet.py#L7). Parameters: config: (`UnivNetConfig`): Config for the `UnivNetModel` model. conv_kernel_size (`int`, *optional*, defaults to 3): The kernel size for the location variable convolutional layer kernels (convolutional weight tensor). conv_layers (`int`, *optional*, defaults to 4): The number of location variable convolutional layers to output kernels and biases for. r#conv_kernel_size conv_layersc>[TU]5 URUlSUR-UlX lX0lURUR-UR -UR -UlURUR -UlURUl URUl URUlURUlUR"UlURS- S-n[$R&"URURSSSS9Ul[$R*"[-UR 5Vs/sHn[/U5PM sn5Ul[$R&"URURURUSS9Ul[$R&"URURURUSS9Ulgs snf)Nr%r Tr&)r)r*model_hidden_channelsconv_in_channelsconv_out_channelsrTrUkernel_channels bias_channels num_mel_binsresnet_in_channels kernel_predictor_hidden_channelsresnet_hidden_channelsr-resnet_kernel_sizekernel_predictor_num_blocks num_blocksr1rr4 input_conv ModuleListranger! resblocks kernel_conv bias_conv)r7r#rTrUr'_r8s rr*UnivNetKernelPredictor.__init__ss  & < <!"V%A%A!A 0&  ! !D$:$: :T=R=R RUYUeUe e "33d6F6FF"("5"5&,&M&M#"("C"C << & 7 7**Q.14))D$;$;T=X=XZ[efmqr]bcgcrcr]s't]sXY(KF(S]s'tu99  ' ')=)=t?V?V`gnr   ' '););T=T=T^elp  (usG. spectrogramcURup#nURU5n[RR XPR 5nUR H nU"U5nM URU5nURU5nURUURURURURU5R5n URUURURU5R5n X4$)aE Maps a conditioning log-mel spectrogram to a tensor of convolutional kernels and biases, for use in location variable convolutional layers. Note that the input spectrogram should have shape (batch_size, input_channels, seq_length). Args: spectrogram (`torch.FloatTensor` of shape `(batch_size, input_channels, seq_length)`): Tensor containing the log-mel spectrograms. Returns: Tuple[`torch.FloatTensor, `torch.FloatTensor`]: tuple of tensors where the first element is the tensor of location variable convolution kernels of shape `(batch_size, self.conv_layers, self.conv_in_channels, self.conv_out_channels, self.conv_kernel_size, seq_length)` and the second element is the tensor of location variable convolution biases of shape `(batch_size, self.conv_layers. self.conv_out_channels, seq_length)`. )shaperdrr=r>r1rgrhriviewrUrYrZrT contiguous) r7rl batch_sizerj seq_lengthr:resblockkernel_hidden_statesbias_hidden_stateskernelsbiasess rr@UnivNetKernelPredictor.forwards"%0$5$5! z 4  00@U@UV H$]3M' $// >!^^M:'++      ! !  " "  ! !    *, $((      " "   *, rc[RRn[[RRS5(a$[RRRnU"UR 5 UR HnUR5 M U"UR5 U"UR5 grC) rrErDrFrGrdrgrIrhrir7rDlayers rrI(UnivNetKernelPredictor.apply_weight_norms}hh** 288,,m < <((33??KDOO$^^E  # # %$D$$%DNN#rc@[RRUR5 URHnUR5 M [RRUR 5 [RRUR 5 gr<)rrErLrdrgrhrir7r{s rrL)UnivNetKernelPredictor.remove_weight_norms` ##DOO4^^E  $ $ &$ ##D$4$45 ##DNN3r)r\rirYrTrUrZrdr[rhr1rcrgr`r^ra)rrrrrrrintr*rrr@rIrLrrOrPs@rrRrRbsY&!" $ $ $  $ $ L,5#4#4,\ $44rrRc ^\rSrSrSrS\S\S\4U4SjjrSSjrSS\ RS \ RS \ RS\S \4 S jjr S r Sr SrU=r$)UnivNetLvcResidualBlockaq Implementation of the location variable convolution (LVC) residual block for the UnivNet residual network. Parameters: config: (`UnivNetConfig`): Config for the `UnivNetModel` model. kernel_size (`int`): The kernel size for the dilated 1D convolutional layer. dilation (`int`): The dilation for the dilated 1D convolutional layer. r#r.dilationcF>[TU]5 URUlX lX0lUR UlUR URS- -S-n[R"URURURUUR S9Ul g)Nr r%)r'r) r)r*rXhidden_channelsr.rr1rr4conv)r7r#r.rr'r8s rr* UnivNetLvcResidualBlock.__init__s %;;& & 7 7--4#3#3a#78A=II     ]]   rcUn[RRXR5nUR U5n[RRXR5nUR XX4S9n[ R"USS2SUR2SS245[ R"USS2URS2SS245-nXQ-nU$N)hop_size) rr=r>r1rlocation_variable_convolutionrsigmoidrtanh)r7r:kernelr(rr?s rr@UnivNetLvcResidualBlock.forwards  00@U@UV  -0  00@U@UV ::=RV:j  mA7M9M9M7Mq4P&QRUZU_U_ !T113Q6 7V  !0 rr:rr(rc*URupgnURupgpn XU-:wa[SX-SUS35eU[U S- S- 5-n [RR XU 4SS5nUR SUSU --U5nXT:a$[RR USU4SS5nUR SXD5nUS S 2S S 2S S 2S S 2S U24nURSS 5nUR S U S5n[R"S X5n U R[RS 9n URS 5RS 5R[RS 9nX-n U R5RXiS 5n U $)u$ Performs location-variable convolution operation on the input sequence (hidden_states) using the local convolution kernel. This was introduced in [LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation](https://arxiv.org/abs/2102.10815) by Zhen Zheng, Jianzong Wang, Ning Cheng, and Jing Xiao. Time: 414 μs ± 309 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each), test on NVIDIA V100. Args: hidden_states (`torch.FloatTensor` of shape `(batch_size, in_channels, in_length)`): The input sequence of shape (batch, in_channels, in_length). kernel (`torch.FloatTensor` of shape `(batch_size, in_channels, out_channels, kernel_size, kernel_length)`): The local convolution kernel of shape (batch, in_channels, out_channels, kernel_size, kernel_length). bias (`torch.FloatTensor` of shape `(batch_size, out_channels, kernel_length)`): The bias for the local convolution of shape (batch, out_channels, kernel_length). dilation (`int`, *optional*, defaults to 1): The dilation of convolution. hop_size (`int`, *optional*, defaults to 256): The hop_size of the conditioning sequence. Returns: `torch.FloatTensor`: the output sequence after performing local convolution with shape (batch_size, out_channels, in_length). z#Dim 2 of `hidden_states` should be z ) but got zX. Please check `hidden_states` or `kernel` and `hop_size` to make sure they are correct.r r%constantrrNrzbildsk,biokl->bolsd) memory_format)rn ValueErrorrrr=padunfold transposereinsumtochannels_last_3d unsqueezerpro)r7r:rr(rrbatchrj in_length out_channelsr. kernel_lengthr'output_hidden_statess rr5UnivNetLvcResidualBlock.location_variable_convolution s<,11)=C\\:,] 1 25m6N5OzZcYde]]  S+/Q!677 ))-79KZYZ[ %,,Q1w;0FQ  MM--ma]JXYZM%,,QC %aAq)8)&;< %//15 %,,Q Q?  %||,A=Y366UE[E[6\~~b!++B/22AWAW2X3:3>>@EEe[]^##rc[RRn[[RRS5(a$[RRRnU"UR 5 grC)rrErDrFrGrrHs rrI)UnivNetLvcResidualBlock.apply_weight_normNsIhh** 288,,m < <((33??KDIIrcV[RRUR5 gr<)rrErLrrMs rrL*UnivNetLvcResidualBlock.remove_weight_normUs ##DII.r)rrrr.r1)r r)rrrrrrrr*r@rrrrIrLrrOrPs@rrrs      , * ?$((?$!!?$ ?$  ?$  ?$B//rrc^\rSrSrSrS S\S\S\4U4SjjjrS\RS\R4S jr S r S r S r U=r$)UnivNetLvcBlockiYa Implementation of the location variable convolution (LVC) residual block of the UnivNet residual block. Includes a `UnivNetKernelPredictor` inside to predict the kernels and biases of the LVC layers. Based on LVCBlock in [maum-ai/univnet](https://github.com/maum-ai/univnet/blob/9bb2b54838bb6d7ce767131cc7b8b61198bc7558/model/lvcnet.py#L98) Parameters: config (`UnivNetConfig`): Config for the `UnivNetModel` model. layer_id (`int`): An integer corresponding to the index of the current LVC resnet block layer. This should be between 0 and `len(config.resblock_stride_sizes) - 1)` inclusive. lvc_hop_size (`int`, *optional*, defaults to 256): The hop size for the location variable convolutional layers. r#layer_id lvc_hop_sizec >[TU]5 URUlURUUlUR UUlURUUl X0l URUl [UR5Ul [R"URURSUR-URURS-URS--URS-S9Ul[#XR UR5Ul[R&"[)UR5Vs/sH&n[+XR URU5PM( sn5Ulgs snf)Nr%)strider'output_padding)r)r*rXrresblock_kernel_sizesr.resblock_stride_sizesrresblock_dilation_sizes dilationscond_hop_lengthr1lenrcrConvTranspose1d convt_prerRkernel_predictorrerfrrg)r7r#rrir8s rr*UnivNetLvcBlock.__init__ks< %;;!77A228< 77A+ & 7 7dnn-++    O;;KK1$t{{Q6;;?  !7v?O?OQUQ`Q` a[`aeapap[q r[qVW $V-=-=t~~a?P Q[q r  rs -Fr:rlc F[RRXR5nUR U5nUR U5up4[ UR5H:upVUSS2USS2SS2SS2SS24nUSS2USS2SS24nU"XXRS9nM< U$r) rr=r>r1rr enumeratergr) r7r:rlrvrwrrsrr(s rr@UnivNetLvcBlock.forwards 00@U@UV }5 // <$T^^4KAQ1aA-.F!Q1*%D$]DK_K_`M5 rcp[RRn[[RRS5(a$[RRRnU"UR 5 UR R5 URHnUR5 M grC) rrErDrFrGrrrIrgrzs rrI!UnivNetLvcBlock.apply_weight_normsuhh** 288,,m < <((33??KDNN# //1^^E  # # %$rc[RRUR5 URR5 UR HnUR5 M gr<)rrErLrrrgr~s rrL"UnivNetLvcBlock.remove_weight_normsD ##DNN3 002^^E  $ $ &$r) rrrrrr.r1rcrgrrrrPs@rrrYsa*       < U%6%6 UEVEV &''rrc^\rSrSr\rSrS\4U4Sjjr\SS\ RS\ \ RS\ \ RS\ \ RS\ \ S \\\ R\44 S jj5rS rS rS rSrU=r$) UnivNetModeliinput_featuresr#c X>[TU]U5 [UR5UlUR Ul[ R"URURSSSSS9Ul [UR5nSn/nURHnX5-nURU5 M [ R"[U5Vs/sHn[UUXFS9PM sn5Ul[ R"URSSSSS9UlUR%5 gs snf)Nr rreflect)r.rr' padding_mode)rr)r'r)r)r*rr num_kernelsr1rr4r+rXconv_prerappendrerfrrg conv_post post_init)r7r# num_layers hop_length hop_lengthsrrr8s rr*UnivNetModel.__init__s  v;;< & 7 7  $ $  ( ("   556   22F#,J   z *3z*   +A  !, +    6#?#?Aq_hi  s D'noise_sequence padding_mask generator return_dictreturncUbUOURRnUR5S:HnU(dURS5nURupxn Ub,UR5S:Hn U (dURS5nOBXxURR 4n [ R"XURURS9nURSn US:aU S:XaURUSS5nOU S:aUS:XaURU SS5nX:wa[SU SUS35eUbKUR5S:XaURS5nURSn X:wa[S U SUS35eURS S5nURS S5nURU5nURH nU"X5nM [R R#XR$5nUR'U5n[ R("U5nUR+S5nSnUb[ R,"USS 9nU(dUU4nU$[/UUS 9$) a input_features (`torch.FloatTensor`): Tensor containing the log-mel spectrograms. Can be batched and of shape `(batch_size, sequence_length, config.num_mel_channels)`, or un-batched and of shape `(sequence_length, config.num_mel_channels)`. noise_sequence (`torch.FloatTensor`, *optional*): Tensor containing a noise sequence of standard Gaussian noise. Can be batched and of shape `(batch_size, sequence_length, config.model_in_channels)`, or un-batched and of shape (sequence_length, config.model_in_channels)`. If not supplied, will be randomly generated. padding_mask (`torch.BoolTensor`, *optional*): Mask indicating which parts of each sequence are padded. Mask values are selected in `[0, 1]`: - 1 for tokens that are **not masked** - 0 for tokens that are **masked** The mask can be batched and of shape `(batch_size, sequence_length)` or un-batched and of shape `(sequence_length,)`. generator (`torch.Generator`, *optional*): A [torch generator](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. return_dict: Whether to return a [`~utils.ModelOutput`] subclass instead of a plain tuple. Example: ```python >>> from transformers import UnivNetFeatureExtractor, UnivNetModel >>> from datasets import load_dataset, Audio >>> model = UnivNetModel.from_pretrained("dg845/univnet-dev") >>> feature_extractor = UnivNetFeatureExtractor.from_pretrained("dg845/univnet-dev") >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> # Resample the audio to the feature extractor's sampling rate. >>> ds = ds.cast_column("audio", Audio(sampling_rate=feature_extractor.sampling_rate)) >>> inputs = feature_extractor( ... ds[0]["audio"]["array"], sampling_rate=ds[0]["audio"]["sampling_rate"], return_tensors="pt" ... ) >>> audio = model(**inputs).waveforms >>> list(audio.shape) [1, 140288] ``` Nrr)rdtypedevicer z&The batch size of `noise_sequence` is z+ and the batch size of `input_features` is z', but the two are expected to be equal.z$The batch size of `padding_mask` is r%)dim)rr)r#use_return_dictrrrnr+rrandnrrrepeatrrrrgrr=r>r1rrsqueezesumr)r7rrrrrspectrogram_batchedspectrogram_batch_sizespectrogram_lengthrjnoise_sequence_batchednoise_sequence_shapenoise_sequence_batch_sizepadding_mask_batch_sizer:rswaveformroutputss rr@UnivNetModel.forwardsf&1% ")!/!9!9!!<%;PTP[P[PmPm#n "[[$AUAU^l^s^sN%3$8$8$;! !A %*Cq*H+223I1aPN & */E/J+223LaQRSN $ >89R8ST((>'??fh   #!Q&+55a8 &2&8&8&; #&@ :;R:ST,,B+CCjl '00A6 '11!Q7 m4 H$]CM' 00@U@UV }5  =1 !((+   #$yy1= !12GN!-  rcV[U[R[R[R45(akUR R RSURRS9 URb%URR R5 ggg)zInitialize the weights.g)meanstdN) isinstancerLinearr4rweightdatanormal_r#initializer_ranger(zero_)r7modules r _init_weightsUnivNetModel._init_weightsSsv fryy"))R5G5GH I I MM   & &CT[[5R5R & S{{&   &&(' Jrc`[RRn[[RRS5(a$[RRRnU"UR 5 UR HnUR5 M U"UR5 grC) rrErDrFrGrrgrIrrzs rrIUnivNetModel.apply_weight_normZsohh** 288,,m < <((33??KDMM"^^E  # # %$DNN#rc[RRUR5 URHnUR5 M [RRUR 5 gr<)rrErLrrgrr~s rrLUnivNetModel.remove_weight_normdsH ##DMM2^^E  $ $ &$ ##DNN3r)rrr1rrg)NNNN)rrrrr config_classmain_input_namer*r rrr Generatorboolrrrr@rrIrLrrOrPs@rrrs L&O%}%N7;48/3&* { )){ !!2!23{ u001 { EOO, { d^ {  uU&&');; <{ { z)$44rr)r dataclassesrtypingrrrrtorch.utils.checkpointrmodeling_outputsr modeling_utilsr rEr r configuration_univnetr get_loggerrloggerrModuler!rRrrr__all__rrrr s"!)) +-,0   H %  9 9  9 .0")).0bu4RYYu4p|/bii|/~M'biiM'`~4?~4~4B  r