fThn~SrSSKrSSKJr SSKJr SSKrSSKrSSK J r SSK J s J r SSKJr SSKJrJr SS KJr \"S S \55r\"S S \55r\"SS\55r"SS\ R05r"SS\ R05r"SS\ R05r"SS\ R05r"SS\ R05r"SS\ R05r"SS\ R05r"SS\ R05r \"S S!\55r!\"S"S#9"S$S%\!55r"S%S!/r#g)&zTransformers DAC model.N) dataclass)Optional)PreTrainedModel) ModelOutputauto_docstring) DacConfigc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr\\R\ S'S rg) DacOutputa Args: loss (`torch.Tensor`): Loss from the encoder model, comprising the weighted combination of the commitment and codebook losses. audio_values (`torch.Tensor` of shape `(batch_size, input_length)`): Reconstructed audio data. quantized_representation (`torch.Tensor` of shape `(batch_size, dimension, time_steps)`): Quantized continuous representation of input. audio_codes (`torch.LongTensor` of shape `(batch_size, num_codebooks, time_steps)`): Codebook indices for each codebook (quantized discrete representation of input). projected_latents (`torch.Tensor` of shape `(batch_size, num_codebooks * dimension, time_steps)`): Projected latents (continuous representation of input before quantization). Nloss audio_valuesquantized_representation audio_codesprojected_latents)__name__ __module__ __qualname____firstlineno____doc__rrtorch FloatTensor__annotations__rrr LongTensorr__static_attributes__r\/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/dac/modeling_dac.pyr r st )-D(5$$ %,04L(5,,-4<@hu'8'89@.2K%**+259x 1 129rr c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr g) DacEncoderOutput6a Args: loss (`torch.Tensor`): Loss from the encoder model, comprising the weighted combination of the commitment and codebook losses. quantized_representation (`torch.Tensor` of shape `(batch_size, dimension, time_steps)`, *optional*): Quantized continuous representation of input. audio_codes (`torch.Tensor` of shape `(batch_size, num_codebooks, time_steps)`, *optional*): Codebook indices for each codebook (quantized discrete representation of input). projected_latents (`torch.Tensor` of shape `(batch_size, num_codebooks * dimension, time_steps)`, *optional*): Projected latents (continuous representation of input before quantization). Nrrrrr)rrrrrrrrrrrrrrrrrr!r!6s_ )-D(5$$ %,<@hu'8'89@/3K%++,359x 1 129rr!cB\rSrSr%SrSr\\R\ S'Sr g)DacDecoderOutputJz Args: audio_values (`torch.FloatTensor` of shape `(batch_size, input_length)`, *optional*): Decoded audio values, obtained using the decoder part of Dac. Nrr) rrrrrrrrrrrrrrr$r$Js 15L(5,,-4rr$c2^\rSrSrSrU4SjrSrSrU=r$)Snake1dVz3 A 1-dimensional Snake activation function module. c>[TU]5 [R"[R "SUS55Ulg)Nr )super__init__nn Parameterronesalpha)self hidden_dim __class__s rr+Snake1d.__init__[s+ \\%**Q A">? rcURnURUSUSS5nXRS-R5[R "URU-5R S5--nURU5nU$)Nrr g& .>)shapereshaper/ reciprocalrsinpow)r0 hidden_statesr7s rforwardSnake1d.forward_s##%--eAha"E %d):(F(F(H599UYU_U_boUoKpKtKtuvKw(ww %--e4 r)r/) rrrrrr+r=r __classcell__r2s@rr'r'Vs@rr'c@^\rSrSrSrS\4U4SjjrSrSrSr U=r $)DacVectorQuantizega Implementation of VQ similar to Karpathy's repo (https://github.com/karpathy/deep-vector-quantization) Additionally uses following tricks from improved VQGAN (https://arxiv.org/pdf/2110.04627.pdf): 1. Factorized codes: Perform nearest neighbor lookup in low-dimensional space for improved codebook usage 2. l2-normalized codes: Converts euclidean distance to cosine similarity which improves training stability configc>>[TU]5 [R"URUR SS9Ul[R"UR URSS9Ul[R"URUR 5Ul g)Nr  kernel_size) r*r+r,Conv1d hidden_size codebook_dimin_projout_proj Embedding codebook_sizecodebookr0rDr2s rr+DacVectorQuantize.__init__ssn yy!3!3V5H5HVWX  &"5"5v7I7IWXY  V%9%96;N;NO rc,URU5nURU5up4[R"X#R 5SS9n[R"X2R 5SS9nX#U- R 5-nUR U5nX5XdU4$)a Quantizes the input tensor using a fixed codebook and returns the corresponding codebook vectors. Args: hidden_state (`torch.FloatTensor` of shape `(batch_size, dimension, time_steps)`): Input tensor. Returns: quantized_representation (`torch.Tensor`of shape `(batch_size, dimension, time_steps)`): Quantized continuous representation of input. commitment_loss (`torch.FloatTensor`of shape `(1)`): Commitment loss to train encoder to predict vectors closer to codebook entries. codebook_loss (`torch.FloatTensor`of shape `(1)`): Codebook loss to update the codebook. audio_codes (`torch.LongTensor` of shape `(batch_size, time_steps)`): Codebook indices for each codebook, quantized discrete representation of input. projected_latents (torch.FloatTensor of shape `(batch_size, num_codebooks * dimension, time_steps)`): Projected latents (continuous representation of input before quantization). mean) reduction)rKdecode_latentsFmse_lossdetachrL)r0 hidden_staterrrcommitment_loss codebook_losss rr=DacVectorQuantize.forwardzs*!LL6040C0CDU0V- **%68W8W8Yekl #;=U=U=Wcij #4Sd8d7l7l7n#n #'==1I#J '-VgggrcnURup#nURSSS5RX$-U5nURRn[ R "U5n[ R "U5nURS5RSSS9nUSU-UR5-- *URS5RSSS9R5-nURS5Sn U RURS5S5n URU 5RSS5n X4$)Nrr6r T)keepdimr5) r7permuter8rOweightrV normalizer;sumtmaxsize transpose) r0r< batch_sizer1sequence_length encodingsrOl2_normdistindicesrs rrU DacVectorQuantize.decode_latentss2?2E2E/ !))!Q2:::;WYcd ==''KK * ;;x(--"&&q$&71y=8::<7788<<?;N;NqZ^;N;_;a;a;cc((1+a.//-"4"4Q"7<#'==#9#C#CAq#I '00r)rOrKrL) rrrrrr r+r=rUrr?r@s@rrBrBgs' PyPh@11rrBcB^\rSrSrSrSS\S\4U4SjjjrSrSrU=r $) DacResidualUnitzY A residual unit composed of Snake1d and weight-normalized Conv1d layers with dilations. dimensiondilationc>[TU]5 SU-S-n[U5Ul[R "XSX#S9Ul[U5Ul[R "XSS9Ulg)Nr6)rGrrpaddingr rF) r*r+r'snake1r,rHconv1snake2conv2)r0rqrrpadr2s rr+DacResidualUnit.__init__s[ !a'i( YYyXc i( YYyC rcUnURURU55nURURU55nURSURS- S-nUS:a USX3*24nX-nU$)a2 Forward pass through the residual unit. Args: hidden_state (`torch.Tensor` of shape `(batch_size, channels, time_steps)`): Input tensor . Returns: output_tensor (`torch.Tensor` of shape `(batch_size, channels, time_steps)`): Input tensor after passing through the residual unit. r5r6r.)rxrwrzryr7)r0rY output_tensorrvs rr=DacResidualUnit.forwards%  4;;}#=>  4;;}#=> %%b)M,?,?,CCI Q;'WX-=(=>L$4 r)rxrzrwry)r ) rrrrrintr+r=rr?r@s@rroros.D#DcDDrrocF^\rSrSrSrS S\S\S\4U4SjjjrSrSr U=r $) DacEncoderBlockz"Encoder block used in DAC encoder.rDstride stride_indexc L>[TU]5 URSU--n[US-SS9Ul[US-SS9Ul[US-SS9Ul[US-5Ul[R"US-USU-U[R"US- 5S9Ul g)Nr6r rrr rGrrv)r*r+encoder_hidden_sizero res_unit1 res_unit2 res_unit3r'rwr,rHmathceilrx)r0rDrrrqr2s rr+DacEncoderBlock.__init__s ..L@ (a!D(a!D(a!Di1n- YY NI1v:fVZV_V_`fij`jVk  rcURU5nURU5nURURU55nUR U5nU$N)rrrwrrxr0rYs rr=DacEncoderBlock.forwardsI~~l3 ~~l3 {{4>>,#?@ zz,/ r)rxrrrrwr r rrrrrr rr+r=rr?r@s@rrrs/,  y  #     rrcF^\rSrSrSrS S\S\S\4U4SjjjrSrSr U=r $) DacDecoderBlockz"Decoder block used in DAC decoder.rDrrc X>[TU]5 URSU--nURSUS---n[U5Ul[ R "UUSU-U[R"US- 5S9Ul [USS9Ul [USS9Ul [USS9Ul g)Nr6r rrrr)r*r+decoder_hidden_sizer'rwr,ConvTranspose1drrconv_t1rorrr)r0rDrr input_dim output_dimr2s rr+DacDecoderBlock.__init__s ..!\/A //19I3JJ i( ))  F IIfqj)   )a@(a@(a@rcURU5nURU5nURU5nURU5nUR U5nU$r)rwrrrrrs rr=DacDecoderBlock.forwardsN{{<0 ||L1 ~~l3 ~~l3 ~~l3 r)rrrrrwrrr@s@rrrs4,AyA#AAA$rrc^\rSrSrSrS\4U4SjjrS S\\4Sjjr S\ R4Sjr S \ R4S jr S rU=r$) DacResidualVectorQuantizei z| ResidualVectorQuantize block - Introduced in SoundStream: An end2end neural audio codec (https://arxiv.org/abs/2107.03312) rDc>[TU]5 URnURnX l[R "[ UR5Vs/sHn[U5PM sn5UlX0lgs snfr) r*r+ n_codebooksquantizer_dropoutr, ModuleListrangerB quantizers)r0rDrrir2s rr+"DacResidualVectorQuantize.__init__sj (( "44&--ERXRdRdLe(fLeq):6)BLe(fg!2)gsA= n_quantizerscSnUnSnSn/n/nUbUO URnUR(a[R"URS45UR-S-n[R "SURS-URS45n [ URSUR-5n U SU USU &URUR5n[UR5HupURSLaX:a OU "U5upnnn[R"URS4XRS9U:nX=USS2SS4--nXM- nX^U-- nXoU-- nURU5 URU5 M [R"USS9n[R"USS9nX7XU4$)a Quantizes the input tensor using a fixed set of codebooks and returns corresponding codebook vectors. Args: hidden_state (`torch.Tensor` of shape `(batch_size, dimension, time_steps)`): Input tensor to be quantized. n_quantizers (`int`, *optional*): Number of quantizers to use. If specified and `self.quantizer_dropout` is True, this argument is ignored during training, and a random number of quantizers is used. Returns: quantized_representation (`torch.Tensor` of shape `(batch_size, dimension, time_steps)`): Quantized continuous representation of input. audio_codes (`torch.Tensor` of shape `(batch_size, num_codebooks, time_steps)`): Codebook indices for each codebook (quantized discrete representation of input). projected_latents (`torch.Tensor` of shape `(batch_size, num_codebooks * dimension, time_steps)`): Projected latents (continuous representation of input before quantization). commitment_loss (`torch.Tensor` of shape `(1)`): Commitment loss to train the encoder to predict vectors closer to codebook entries. codebook_loss (`torch.Tensor` of shape `(1)`): Codebook loss to update the codebook. rNr F) fill_valuedevicedim)rtrainingrr.r7randintrrtor enumeraterfullappendstackcat)r0rYrrresidualrZr[rrdropout n_dropoutr quantizerquantized_representation_icommitment_loss_icodebook_loss_i indices_iprojected_latents_imasks rr=!DacResidualVectorQuantize.forwards.$%   '3'?|TEUEU == ::|'9'9!'<&>?$BRBRRUVVLmmAt'7'7!';l>P>PQR>S=UVGL..q1D4J4JJKI'.z ':L) $'??<+>+>?L%doo6LA}}%!*;mvn j &?IWj ::|11!461M`M`adppD'?_cdegkmqdq_rBr'r $>?2[,[\||0a8hht}}Q'778;??QQU?VWXY {#A)-$1u+,+/??1+=+L+LWUVXdXqstUtMu+v (   $ $%8 9 LL !)-);)D)DEX)Y &'?'\ $$(3D!)LLL3\sD5 )rrrr)rrrrrr r+rrr=rTensorrrrr?r@s@rrr sQ 3y 3>h(3->h@ZellZ4MELLMMrrc:^\rSrSrSrS\4U4SjjrSrSrU=r $) DacDecoderiz DAC DecoderrDc>[T U]5 URnURnURn[ R "X#SSS9Ul/n[U5HupgU[XU5/- nM [ R"U5Ul URSWS---n[U5Ul [ R "USSSS9Ul[ R"5Ulg)NrurrGrvr6r )r*r+rIrupsampling_ratiosr,rHrxrrrblockr'rwrzTanhtanh) r0rD input_channelchannelsstridesrrrrr2s rr+DacDecoder.__init__s ** --**YY}AqQ $-g$6 L oflCD DE%7]]5) //19I3JJ j) YYz1!QG GGI rcURU5nURH nU"U5nM URU5nURU5nUR U5nU$r)rxrrwrzr)r0rYlayers rr=DacDecoder.forwardsZzz,/ ZZE .L {{<0 zz,/ yy. r)rrxrzrwr rrrrrr r+r=rr?r@s@rrrsy*  rrc:^\rSrSrSrS\4U4SjjrSrSrU=r $) DacEncoderiz DAC EncoderrDc >[TU]5 URn[R"SUR SSS9Ul/Ul[U5H(up4US-nU=R[XUS9/- slM* [R"UR5UlUR SW--n[U5Ul [R"XQRSSS9Ulg)Nr rurr)rrr6)r*r+downsampling_ratiosr,rHrrxrrrrr'rwrIrz)r0rDrrrd_modelr2s rr+DacEncoder.__init__s ,,YYq&"<"g& YYw(:(:STU rcURU5nURH nU"U5nM URU5nURU5nU$r)rxrrwrz)r0rYmodules rr=DacEncoder.forwardsLzz,/ jjF!,/L!{{<0 zz,/ r)rrxrzrwrr@s@rrrsVyV$  rrc2\rSrSr\rSrSrSrSr Sr Sr g) DacPreTrainedModelidac input_valuesc[U[R5(aS[RR UR SS9 [RR URS5 gg)Ng{Gz?)stdr) isinstancer,rHinit trunc_normal_r` constant_bias)r0rs r _init_weights DacPreTrainedModel._init_weightssK fbii ( ( GG ! !&--T ! : GG  fkk1 - )rc[RRn[[RRS5(a$[RRRnUR R H'nU"UR5 U"UR5 M) U"URR5 U"URR5 URRHnU"UR5 U"URR5 U"URR5 U"URR5 U"URR5 U"URR5 U"URR5 M U"UR R5 U"UR R5 UR RHnU"UR"5 U"URR5 U"URR5 U"URR5 U"URR5 U"URR5 U"URR5 M g)N weight_norm)r,utilsrhasattrparametrizationsrrrKrLencoderrxrzrrrrdecoderr)r0rrs rapply_weight_norm$DacPreTrainedModel.apply_weight_normshh** 288,,m < <((33??K^^..E  &  '/ DLL&&'DLL&&'\\''E  $ -- . -- . -- . -- . -- . -- .( DLL&&'DLL&&'\\''E  & -- . -- . -- . -- . -- . -- .(rcbURRHUn[RR UR 5 [RR UR 5 MW [RR URR5 [RR URR5 URRGH_n[RR UR5 [RR URR5 [RR URR5 [RR URR5 [RR URR5 [RR URR5 [RR URR5 GMb [RR URR5 [RR URR5 URRGH_n[RR UR5 [RR URR5 [RR URR5 [RR URR5 [RR URR5 [RR URR5 [RR URR5 GMb gr)rrr,rremove_weight_normrKrLr rxrzrrrrr r)r0rs rr%DacPreTrainedModel.remove_weight_normsZ^^..E HH ' ' 6 HH ' ' 7/ ##DLL$6$67 ##DLL$6$67\\''E HH ' ' 4 HH ' '(=(= > HH ' '(=(= > HH ' '(=(= > HH ' '(=(= > HH ' '(=(= > HH ' '(=(= >( ##DLL$6$67 ##DLL$6$67\\''E HH ' ' 6 HH ' '(=(= > HH ' '(=(= > HH ' '(=(= > HH ' '(=(= > HH ' '(=(= > HH ' '(=(= >(rrN) rrrrr config_classbase_model_prefixmain_input_namerr rrrrrrrs"L$O. /B?rrz/ The DAC (Descript Audio Codec) model. ) custom_introc $^\rSrSrS\4U4Sjjr\S S\RS\ \ S\ \ 4Sjj5r \SS\ \RS \ \RS\ \ 4S jj5r \S S\RS\ \ S\ \ 4S jj5rS rU=r$)DacModeli#rDc>[TU]U5 Xl[U5Ul[ U5Ul[U5Ul[[R"URR55Ul SUR-URR:wa [S5eUR5 g)Nr6z'The codebook_size must be a power of 2.)r*r+rDrr rr rrrrlog2rNbits_per_codebook ValueError post_initrPs rr+DacModel.__init__)s   !&) !&) 26:!$TYYt{{/H/H%I!J d$$ $ (A(A AFG G rrr return_dictcUbUOURRnURU5nURXB5upEpgnURRU-URR U--n U(dXXV4$[ XXV5$)z input_values (`torch.Tensor of shape `(batch_size, 1, time_steps)`): Input audio data to encode, n_quantizers (int, *optional*): Number of quantizers to use. If None, all quantizers are used. Default is None. )rDrr rcommitment_loss_weightcodebook_loss_weightr!) r0rrrrrrrZr[rs rencodeDacModel.encode9s&1%>> from datasets import load_dataset, Audio >>> from transformers import DacModel, AutoProcessor >>> librispeech_dummy = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> model = DacModel.from_pretrained("descript/dac_16khz") >>> processor = AutoProcessor.from_pretrained("descript/dac_16khz") >>> librispeech_dummy = librispeech_dummy.cast_column("audio", Audio(sampling_rate=processor.sampling_rate)) >>> audio_sample = librispeech_dummy[-1]["audio"]["array"] >>> inputs = processor(raw_audio=audio_sample, sampling_rate=processor.sampling_rate, return_tensors="pt") >>> encoder_outputs = model.encode(inputs["input_values"]) >>> # Get the intermediate audio codes >>> audio_codes = encoder_outputs.audio_codes >>> # Reconstruct the audio from its quantized representation >>> audio_values = model.decode(encoder_outputs.quantized_representation) >>> # or the equivalent with a forward pass >>> audio_values = model(inputs["input_values"]).audio_values ```Nr5F)rr.)rDrr7r r$r ) r0rrrlengthrrrrrs rr=DacModel.forwardssF&1%r4s` ! -0( : : :, :{: :& 5{5 5bii"C1 C1L"bii"Jbii0bii>GM GMT""JBF?F?F?R xg!xg  xgv + ,r