JTh" SSKrSSKrSSKrSSKrSSKJrJr SSKJr SSK r SSK J r J r SSK J r SSKJr SSKJr SS KJr /S Qr\ R*\ R,S .rS$S \ S \ S\S\ 4Sjjr\"S\S9S$S \ S \ S\S\ 4Sjj5r"SS\5r"SS\5r"SS\5r"SS\5r"SS\5r "SS\ 5r!"S S!\ 5r""S"S#\ 5r#g)%N)Optionaloverload) deprecated)_VFTensor)init) Parameter)PackedSequence)Module)RNNBaseRNNLSTMGRU RNNCellBaseRNNCellLSTMCellGRUCell)RNN_TANHRNN_RELUtensor permutationdimreturnc$URX!5$N) index_selectrrrs L/var/www/auris/envauris/lib/python3.13/site-packages/torch/nn/modules/rnn.py_apply_permutationr $s   s 00z]`apply_permutation` is deprecated, please use `tensor.index_select(dim, permutation)` instead)categoryc[XU5$rr rs rapply_permutationr%(s f3 77r!c ^\rSrSr%Sr/SQrS/r\\S'\ \S'\ \S'\ \S'\ \S '\ \S '\ \S '\ \S '\ \S 'S*S\S\ S\ S\ S \ S \ S \ S \ S \ SS4U4Sjjjr Sr U4SjrS+SjrS,U4SjjrS+SjrS\S\\SS4SjrS\S\\S\\ \ \ 44SjrS-S\S\\ \ \ 4S\SS4SjjrSrS\S\S\\4S jrS\S!\\4S"jrS\4S#jrS$rS%rU4S&jr\S\ \ \!4S'j5r"U4S(jr#S)r$U=r%$).r 0adBase class for RNN modules (RNN, LSTM, GRU). Implements aspects of RNNs shared by the RNN, LSTM, and GRU classes, such as module initialization and utility methods for parameter storage management. .. note:: The forward method is not implemented by the RNNBase class. .. note:: LSTM and GRU classes override some methods implemented by RNNBase. ) mode input_size hidden_size num_layersbias batch_firstdropout bidirectional proj_size all_weightsr(r)r*r+r,r-r.r/r0Nrc >XS.n [TU]5 XlX lX0lX@lXPlX`l[U5Ul Xl Xl /Ul U(aSOSn [U[R5(a%SUs=::aS::aO O[U[ 5(a [#S5eUS:a"US:Xa[$R&"SUSU35 [U[(5(d![+S[-U5R.35eUS::a [#S 5eUS::a [#S 5eU S:a [#S 5eX:a [#S 5eUS :XaSU-nO,US:XaSU-nO US:XaUnOUS:XaUnO[#SU-5e/Ul/Ul[5U5GHn[5U 5GHnU S:aU OUnUS:XaUOUU -n[7[8R:"UU440U D65n[7[8R:"UU440U D65n[7[8R:"U40U D65n[7[8R:"U40U D65nSnURS:XaU(aUUUU4nO:UU4nO5[7[8R:"X440U D65nU(aUUUUU4nOUUU4nUS:XaSOSnSS/nU(aUSS/- nURS:aUS/- nUVs/sHnUR=UU5PM nn[?UU5Hunn[AUUU5 M UR0RCU5 UR2REU5 GM GM URG5 URI5 gs snf)Ndevicedtyper rzbdropout should be a number in range [0, 1] representing the probability of an element being zeroedzdropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=z and num_layers=z(hidden_size should be of type int, got: z%hidden_size must be greater than zeroz$num_layers must be greater than zerozEproj_size should be a positive integer or zero to disable projectionsz,proj_size has to be smaller than hidden_sizerrrrzUnrecognized RNN mode: _reverseweight_ih_l{}{}weight_hh_l{}{} bias_ih_l{}{} bias_hh_l{}{}weight_hr_l{}{})%super__init__r(r)r*r+r,r-floatr.r/r0_flat_weight_refs isinstancenumbersNumberbool ValueErrorwarningswarnint TypeErrortype__name___flat_weights_names _all_weightsranger torchemptyformatzipsetattrextendappend_init_flat_weightsreset_parameters)selfr(r)r*r+r,r-r.r/r0r4r5factory_kwargsnum_directions gate_sizelayer directionreal_hidden_sizelayer_input_sizew_ihw_hhb_ihb_hh layer_paramsw_hrsuffix param_namesxnameparam __class__s rrBRNNBase.__init__Ts%+;  $&$ &W~ *"SU+7GNN33$1$'4((  Q;:? MM>>EYG(\+ +s++:4 ;L;U;U:VW  ! DE E ?CD D q=W   #KL L 6>KI U]KI Z #I Z #I6=> >#% :&E">2 09A 9; "'1*J2B^2S!!KK,< =PP!KK,< =PP!Y!I.!IJ!Y!I.!IJ35 >>Q&(,dD$'? (,d| $ Y$<OOD(,dD$'E (,dD'9 '0A~202CD O_#EEK>>A%$5#66K@KL 1qxxv6 L#&{L#AKD%D$.$B((// <!!((5U3'Z ! Ms%Nc&URVs/sH n[X5(a [X5OSPM" snUlURVs/sHo"b[R "U5OSPM snUlUR5 gs snfs snfr)rPhasattrgetattr _flat_weightsweakrefrefrDflatten_parameters)r\wnws rrZRNNBase._init_flat_weightss.. .")!2!2GD  <.  @D?Q?Q" ?Q!mGKKN 5?Q"  ! " s 'B  $Bc>[US5(a8XR;a)URRU5nX RU'[TU]X5 g)NrP)rrrPindexrtrA __setattr__)r\attrvalueidxros rr}RNNBase.__setattr__sM 4. / /D/Ul[TU] X5nUR5 U$r)rDrA_applyrZ)r\fnrecurseretros rrRNNBase._applys,!#gnR) ! r!cURS:a#S[R"UR5- OSnUR5Hn[R "X!*U5 M gNrg?r*mathsqrt parametersruniform_r\stdvweights rr[RNNBase.reset_parameters&K484D4Dq4HsTYYt//00aoo'F MM&% .(r!input batch_sizescH[RR5(d|URURSR:waU[R R 5(d2[SURSRSUR35eUbSOSnUR5U:wa[SUSUR535eURURS5:wa*[S URS URS535eg) Nrzinput must have the type z , got type r6r8zinput must have z dimensions, got z5input.size(-1) must be equal to input_size. Expected z, got ) rSjit is_scriptingr5rt_C_is_any_autocast_enabledrIr RuntimeErrorr)size)r\rrexpected_input_dims r check_inputRNNBase.check_input+s yy%%'' t11!4:::99;; /0B0B10E0K0K/LKX]XcXcWde#."9Qq 99;, ,"#5"66G }U  ??ejjn ,GGXX^_d_i_ijl_m^no  -r!cNUb[US5nO3UR(aURS5OURS5nUR(aSOSnURS:aUR U-UUR4nU$UR U-UUR 4nU$Nrr r6)rLr-rr/r0r+r*r\rr mini_batchr^expected_hidden_sizes rget_expected_hidden_size RNNBase.get_expected_hidden_size>s  "[^,J*.*:*:A 1 J"00a >>A .0$ $# .0  $ $#r!hxrmsgc UR5U:wa2[URU[UR5555egr)rrrUlist)r\rrrs rcheck_hidden_sizeRNNBase.check_hidden_sizeTs9 779, ,szz*>RWWYPQ Q -r!cSn[URUR5H;up#[X5(a [ X5OSnUcM'UcM,U"5ULdM8Sn U$ U$)NFT)rVrDrPrrrs)r\weights_changedrvrmrs r_weights_have_changedRNNBase._weights_have_changed]sh T33T5M5MNIC,3D,?,?WT(TF!co#%v:M"& O r!hiddencjURX5 URX5nURX$5 gr)rrr)r\rrrrs rcheck_forward_argsRNNBase.check_forward_argshs1 ,#<>Q  * *A ??a  , ,A 99D  A   5 ( . .A <<1  & &A   U * 2 2Axx($--((r!c[RR5(d'UR5(aUR 5 gggr)rSrrrrZ)r\s r_update_flat_weightsRNNBase._update_flat_weightss:yy%%''))++''),(r!c`UR5 URR5nUS U$)NrD)rrcopy)r\states r __getstate__RNNBase.__getstate__s. !!# ""$ % & r!c >[T U]U5 SU;a USUlSU;aSUl[ URSS[ 5(GdUR nUR(aSOSn/Ul/Ul[U5GHbn[U5GHNnUS:XaSOSn/SQnUVs/sHoRXF5PM nnUR(a|URS:a3U=RU/- slURRU5 MU=RUSS /- slURRUSS 5 MURS:aHU=RUSS/US S/-- slURRUSSUS S/-5 GMU=RUSS/- slURRUSS5 GMQ GMe URV s/sH n [X 5(a [X 5OSPM" sn UlURV s/sHob[ R""U 5OSPM sn Ulgs snfs sn fs sn f) Nr1r0rr6r r:r;)r<r=r>r?r@r7r)rA __setstate__rQr0rEstrr+r/rPrRrUr,rXrrrsrtrurvrD) r\dr+r^r`rarjweightsrlrxryros rrRNNBase.__setstate__sL Q A  !- 0D  a DN$++A.q1377J"&"4"4Q!N')D $ "D z*!&~!6I+4>ZrFGAHH1xx6GHyy>>A- --':- 44;;GD --'"1+>- 44;;GBQKH>>A- --'"1+'"#,1OO- 44;; ' wrs|n <!--'"1+>- 44;;GBQKH3"7+:22"2B&-T%6%6!D@2"D  @D?Q?Q" ?Q!mGKKN 5?Q" -I"" " s3I-'I2$I7c URVVs/sHnUVs/sHn[X5PM snPM! snn$s snfs snnfr)rQrs)r\rrs rr1RNNBase.all_weightssF ,, ,29 9vWT " 9,  9 s ?: ??ct>[TU]5nURSSUlURSSUlU$r)rA_replicate_for_data_parallelrtrP)r\replicaros rr$RNNBase._replicate_for_data_parallels='68!( 5 5a 8&-&A&A!&D#r!) rQrDrtrPr-r,r/r.r*r)r(r+r0r TFFrNNrN)T)zExpected hidden size {}, got {})&rO __module__ __qualname____firstlineno____doc__ __constants____jit_unused_properties__r__annotations__rLrHrCrBrZr}rwrr[rrrtuplerrrrrrrrrpropertyrr r1r__static_attributes__ __classcell__ros@rr r 0s   M"/ IOO J NN!#y y y  y  y  y y y y y  y y v")9v / hv6F4&$$*26*:$ sC} $45 R R$CcM2R R  R ==%+=:B6:J=33hv6F3 )C) * 1 f T$y/2  r!r cb^\rSrSrSr\SS\S\S\S\S\S \S \ S \S S4S jj5r \S5r U4Sjr \\ RRSS\S\\S \\\44Sjj55r\\ RRSS\S\\S \\\44Sjj55rSSjrSrU=r$)riad__init__(input_size,hidden_size,num_layers=1,nonlinearity='tanh',bias=True,batch_first=False,dropout=0.0,bidirectional=False,device=None,dtype=None) Apply a multi-layer Elman RNN with :math:`\tanh` or :math:`\text{ReLU}` non-linearity to an input sequence. For each element in the input sequence, each layer computes the following function: .. math:: h_t = \tanh(x_t W_{ih}^T + b_{ih} + h_{t-1}W_{hh}^T + b_{hh}) where :math:`h_t` is the hidden state at time `t`, :math:`x_t` is the input at time `t`, and :math:`h_{(t-1)}` is the hidden state of the previous layer at time `t-1` or the initial hidden state at time `0`. If :attr:`nonlinearity` is ``'relu'``, then :math:`\text{ReLU}` is used instead of :math:`\tanh`. .. code-block:: python # Efficient implementation equivalent to the following with bidirectional=False def forward(x, hx=None): if batch_first: x = x.transpose(0, 1) seq_len, batch_size, _ = x.size() if hx is None: hx = torch.zeros(num_layers, batch_size, hidden_size) h_t_minus_1 = hx h_t = hx output = [] for t in range(seq_len): for layer in range(num_layers): h_t[layer] = torch.tanh( x[t] @ weight_ih[layer].T + bias_ih[layer] + h_t_minus_1[layer] @ weight_hh[layer].T + bias_hh[layer] ) output.append(h_t[-1]) h_t_minus_1 = h_t output = torch.stack(output) if batch_first: output = output.transpose(0, 1) return output, h_t Args: input_size: The number of expected features in the input `x` hidden_size: The number of features in the hidden state `h` num_layers: Number of recurrent layers. E.g., setting ``num_layers=2`` would mean stacking two RNNs together to form a `stacked RNN`, with the second RNN taking in outputs of the first RNN and computing the final results. Default: 1 nonlinearity: The non-linearity to use. Can be either ``'tanh'`` or ``'relu'``. Default: ``'tanh'`` bias: If ``False``, then the layer does not use bias weights `b_ih` and `b_hh`. Default: ``True`` batch_first: If ``True``, then the input and output tensors are provided as `(batch, seq, feature)` instead of `(seq, batch, feature)`. Note that this does not apply to hidden or cell states. See the Inputs/Outputs sections below for details. Default: ``False`` dropout: If non-zero, introduces a `Dropout` layer on the outputs of each RNN layer except the last layer, with dropout probability equal to :attr:`dropout`. Default: 0 bidirectional: If ``True``, becomes a bidirectional RNN. Default: ``False`` Inputs: input, hx * **input**: tensor of shape :math:`(L, H_{in})` for unbatched input, :math:`(L, N, H_{in})` when ``batch_first=False`` or :math:`(N, L, H_{in})` when ``batch_first=True`` containing the features of the input sequence. The input can also be a packed variable length sequence. See :func:`torch.nn.utils.rnn.pack_padded_sequence` or :func:`torch.nn.utils.rnn.pack_sequence` for details. * **hx**: tensor of shape :math:`(D * \text{num\_layers}, H_{out})` for unbatched input or :math:`(D * \text{num\_layers}, N, H_{out})` containing the initial hidden state for the input sequence batch. Defaults to zeros if not provided. where: .. math:: \begin{aligned} N ={} & \text{batch size} \\ L ={} & \text{sequence length} \\ D ={} & 2 \text{ if bidirectional=True otherwise } 1 \\ H_{in} ={} & \text{input\_size} \\ H_{out} ={} & \text{hidden\_size} \end{aligned} Outputs: output, h_n * **output**: tensor of shape :math:`(L, D * H_{out})` for unbatched input, :math:`(L, N, D * H_{out})` when ``batch_first=False`` or :math:`(N, L, D * H_{out})` when ``batch_first=True`` containing the output features `(h_t)` from the last layer of the RNN, for each `t`. If a :class:`torch.nn.utils.rnn.PackedSequence` has been given as the input, the output will also be a packed sequence. * **h_n**: tensor of shape :math:`(D * \text{num\_layers}, H_{out})` for unbatched input or :math:`(D * \text{num\_layers}, N, H_{out})` containing the final hidden state for each element in the batch. Attributes: weight_ih_l[k]: the learnable input-hidden weights of the k-th layer, of shape `(hidden_size, input_size)` for `k = 0`. Otherwise, the shape is `(hidden_size, num_directions * hidden_size)` weight_hh_l[k]: the learnable hidden-hidden weights of the k-th layer, of shape `(hidden_size, hidden_size)` bias_ih_l[k]: the learnable input-hidden bias of the k-th layer, of shape `(hidden_size)` bias_hh_l[k]: the learnable hidden-hidden bias of the k-th layer, of shape `(hidden_size)` .. note:: All the weights and biases are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{hidden\_size}}` .. note:: For bidirectional RNNs, forward and backward are directions 0 and 1 respectively. Example of splitting the output layers when ``batch_first=False``: ``output.view(seq_len, batch, num_directions, hidden_size)``. .. note:: ``batch_first`` argument is ignored for unbatched inputs. .. include:: ../cudnn_rnn_determinism.rst .. include:: ../cudnn_persistent_rnn.rst Examples:: >>> rnn = nn.RNN(10, 20, 2) >>> input = torch.randn(5, 3, 10) >>> h0 = torch.randn(2, 3, 20) >>> output, hn = rnn(input, h0) Nr)r*r+ nonlinearityr,r-r.r/rc grr9) r\r)r*r+rr,r-r.r/r4r5s rrB RNN.__init__V r!cgrr9r\argskwargss rrBrf r!cF>SU;a [S5e[U5S:aUSUlUSSUSS-nOURSS5UlURS:XaSnO,URS:XaS nO[S URS 35e[TU]"U/UQ70UD6 g) Nr0=proj_size argument is only supported for LSTM, not RNN or GRUr8r7rtanhrrelurzUnknown nonlinearity 'z '. Select from 'tanh' or 'relu'.)rIrrpoprArB)r\rrr(ros rrBrjs & O  t9q= $QD 8d12h&D & >6 BD     &D   & (D():):(;;[\  ///r!rrcgrr9r\rrs rforward RNN.forward~ r!cgrr9r s rr r r r!c UR5 UR(aSOSnUn[U[5(aeUuppgUSnUcE[R "UR U-UURURURS9nGOURX&5nGO|SnUR5S;a[SUR5S35eUR5S:Hn UR(aSOSn U (dWURU 5nUbBUR5S:wa[S UR5S 35eURS5nO4Ub1UR5S:wa[S UR5S 35eUR(aUR!S5OUR!S5nSnSnUcD[R "UR U-UURURURS9nOURX&5nUceUR#XU5 UR$S :XdUR$S :XdeUcUR$S :Xaf[&R("UUUR*UR,UR UR.UR0URUR5 n GO*[&R2"UUUR*UR,UR UR.UR0URUR5 n OUR$S :Xa[[&R("UUUUR*UR,UR UR.UR0UR5 n OZ[&R2"UUUUR*UR,UR UR.UR0UR5 n U Sn U Sn [U[5(a[XXg5nXRX54$W (d"U R5W 5n U R5S5n XRX54$)Nr6r rr5r4r6r8z(RNN: Expected input to be 2D or 3D, got zD tensor insteadr87For unbatched 2-D input, hx should also be 2-D but got -D tensor5For batched 3-D input, hx should also be 3-D but got rr)rr/rEr rSzerosr+r*r5r4rrrIr- unsqueezerrrr(rrnn_tanhrtr,r.trainingrnn_relusqueeze)r\rrr^ orig_inputrsorted_indicesunsorted_indicesmax_batch_size is_batched batch_dimresultoutputr output_packeds rr r s !!#"00a j. 1 1CH @E(^Nz[[OOn4"$$++ << ((<Kyy{&( >uyy{mK[\)J!--1I 2>vvx1}*UVXV\V\V^U__hiaB>bffh!m&OPRPVPVPXzYbc/3.>.>UZZ]EJJqMN!N# z[[OOn4"$$++ << ((<~~ ;7yyJ&$))z*AAA  yyJ&&&IIOOLLMM&&$$ &&IIOOLLMM&&$$ yyJ&&&IIOOLLMM&& &&IIOOLLMM&&  j. 1 1*^M!"5"5f"OO O^^I.F^^A&F**6DDDr!r)r rTFrFNNr)rOrrrrrrLrrHrCrBrS _jit_internal_overload_methodrrrr r rrrs@rrrsZ~@ "!#                         0( ))48  !)&!1 vv~  *  ))<@ # )1&)9 ~v% & * ~E~Er!rc^\rSrSrSr\SS\S\S\S\S\S \S \S \S S4S jj5r \S5r U4Sjr S\ S\ \ S \ \\\44Sjr S\ S\ \ \ 4S\ \ 4SjrS\ \ \ 4S\ \ S \ \ \ 44Sjr\\R"R$SS\ S\ \ \ \ 4S \ \ \ \ \ 444Sjj55r\\R"R$SS\S\ \ \ \ 4S \ \\ \ \ 444Sjj55rSSjrSrU=r$)ria#__init__(input_size,hidden_size,num_layers=1,bias=True,batch_first=False,dropout=0.0,bidirectional=False,proj_size=0,device=None,dtype=None) Apply a multi-layer long short-term memory (LSTM) RNN to an input sequence. For each element in the input sequence, each layer computes the following function: .. math:: \begin{array}{ll} \\ i_t = \sigma(W_{ii} x_t + b_{ii} + W_{hi} h_{t-1} + b_{hi}) \\ f_t = \sigma(W_{if} x_t + b_{if} + W_{hf} h_{t-1} + b_{hf}) \\ g_t = \tanh(W_{ig} x_t + b_{ig} + W_{hg} h_{t-1} + b_{hg}) \\ o_t = \sigma(W_{io} x_t + b_{io} + W_{ho} h_{t-1} + b_{ho}) \\ c_t = f_t \odot c_{t-1} + i_t \odot g_t \\ h_t = o_t \odot \tanh(c_t) \\ \end{array} where :math:`h_t` is the hidden state at time `t`, :math:`c_t` is the cell state at time `t`, :math:`x_t` is the input at time `t`, :math:`h_{t-1}` is the hidden state of the layer at time `t-1` or the initial hidden state at time `0`, and :math:`i_t`, :math:`f_t`, :math:`g_t`, :math:`o_t` are the input, forget, cell, and output gates, respectively. :math:`\sigma` is the sigmoid function, and :math:`\odot` is the Hadamard product. In a multilayer LSTM, the input :math:`x^{(l)}_t` of the :math:`l` -th layer (:math:`l \ge 2`) is the hidden state :math:`h^{(l-1)}_t` of the previous layer multiplied by dropout :math:`\delta^{(l-1)}_t` where each :math:`\delta^{(l-1)}_t` is a Bernoulli random variable which is :math:`0` with probability :attr:`dropout`. If ``proj_size > 0`` is specified, LSTM with projections will be used. This changes the LSTM cell in the following way. First, the dimension of :math:`h_t` will be changed from ``hidden_size`` to ``proj_size`` (dimensions of :math:`W_{hi}` will be changed accordingly). Second, the output hidden state of each layer will be multiplied by a learnable projection matrix: :math:`h_t = W_{hr}h_t`. Note that as a consequence of this, the output of LSTM network will be of different shape as well. See Inputs/Outputs sections below for exact dimensions of all variables. You can find more details in https://arxiv.org/abs/1402.1128. Args: input_size: The number of expected features in the input `x` hidden_size: The number of features in the hidden state `h` num_layers: Number of recurrent layers. E.g., setting ``num_layers=2`` would mean stacking two LSTMs together to form a `stacked LSTM`, with the second LSTM taking in outputs of the first LSTM and computing the final results. Default: 1 bias: If ``False``, then the layer does not use bias weights `b_ih` and `b_hh`. Default: ``True`` batch_first: If ``True``, then the input and output tensors are provided as `(batch, seq, feature)` instead of `(seq, batch, feature)`. Note that this does not apply to hidden or cell states. See the Inputs/Outputs sections below for details. Default: ``False`` dropout: If non-zero, introduces a `Dropout` layer on the outputs of each LSTM layer except the last layer, with dropout probability equal to :attr:`dropout`. Default: 0 bidirectional: If ``True``, becomes a bidirectional LSTM. Default: ``False`` proj_size: If ``> 0``, will use LSTM with projections of corresponding size. Default: 0 Inputs: input, (h_0, c_0) * **input**: tensor of shape :math:`(L, H_{in})` for unbatched input, :math:`(L, N, H_{in})` when ``batch_first=False`` or :math:`(N, L, H_{in})` when ``batch_first=True`` containing the features of the input sequence. The input can also be a packed variable length sequence. See :func:`torch.nn.utils.rnn.pack_padded_sequence` or :func:`torch.nn.utils.rnn.pack_sequence` for details. * **h_0**: tensor of shape :math:`(D * \text{num\_layers}, H_{out})` for unbatched input or :math:`(D * \text{num\_layers}, N, H_{out})` containing the initial hidden state for each element in the input sequence. Defaults to zeros if (h_0, c_0) is not provided. * **c_0**: tensor of shape :math:`(D * \text{num\_layers}, H_{cell})` for unbatched input or :math:`(D * \text{num\_layers}, N, H_{cell})` containing the initial cell state for each element in the input sequence. Defaults to zeros if (h_0, c_0) is not provided. where: .. math:: \begin{aligned} N ={} & \text{batch size} \\ L ={} & \text{sequence length} \\ D ={} & 2 \text{ if bidirectional=True otherwise } 1 \\ H_{in} ={} & \text{input\_size} \\ H_{cell} ={} & \text{hidden\_size} \\ H_{out} ={} & \text{proj\_size if } \text{proj\_size}>0 \text{ otherwise hidden\_size} \\ \end{aligned} Outputs: output, (h_n, c_n) * **output**: tensor of shape :math:`(L, D * H_{out})` for unbatched input, :math:`(L, N, D * H_{out})` when ``batch_first=False`` or :math:`(N, L, D * H_{out})` when ``batch_first=True`` containing the output features `(h_t)` from the last layer of the LSTM, for each `t`. If a :class:`torch.nn.utils.rnn.PackedSequence` has been given as the input, the output will also be a packed sequence. When ``bidirectional=True``, `output` will contain a concatenation of the forward and reverse hidden states at each time step in the sequence. * **h_n**: tensor of shape :math:`(D * \text{num\_layers}, H_{out})` for unbatched input or :math:`(D * \text{num\_layers}, N, H_{out})` containing the final hidden state for each element in the sequence. When ``bidirectional=True``, `h_n` will contain a concatenation of the final forward and reverse hidden states, respectively. * **c_n**: tensor of shape :math:`(D * \text{num\_layers}, H_{cell})` for unbatched input or :math:`(D * \text{num\_layers}, N, H_{cell})` containing the final cell state for each element in the sequence. When ``bidirectional=True``, `c_n` will contain a concatenation of the final forward and reverse cell states, respectively. Attributes: weight_ih_l[k] : the learnable input-hidden weights of the :math:`\text{k}^{th}` layer `(W_ii|W_if|W_ig|W_io)`, of shape `(4*hidden_size, input_size)` for `k = 0`. Otherwise, the shape is `(4*hidden_size, num_directions * hidden_size)`. If ``proj_size > 0`` was specified, the shape will be `(4*hidden_size, num_directions * proj_size)` for `k > 0` weight_hh_l[k] : the learnable hidden-hidden weights of the :math:`\text{k}^{th}` layer `(W_hi|W_hf|W_hg|W_ho)`, of shape `(4*hidden_size, hidden_size)`. If ``proj_size > 0`` was specified, the shape will be `(4*hidden_size, proj_size)`. bias_ih_l[k] : the learnable input-hidden bias of the :math:`\text{k}^{th}` layer `(b_ii|b_if|b_ig|b_io)`, of shape `(4*hidden_size)` bias_hh_l[k] : the learnable hidden-hidden bias of the :math:`\text{k}^{th}` layer `(b_hi|b_hf|b_hg|b_ho)`, of shape `(4*hidden_size)` weight_hr_l[k] : the learnable projection weights of the :math:`\text{k}^{th}` layer of shape `(proj_size, hidden_size)`. Only present when ``proj_size > 0`` was specified. weight_ih_l[k]_reverse: Analogous to `weight_ih_l[k]` for the reverse direction. Only present when ``bidirectional=True``. weight_hh_l[k]_reverse: Analogous to `weight_hh_l[k]` for the reverse direction. Only present when ``bidirectional=True``. bias_ih_l[k]_reverse: Analogous to `bias_ih_l[k]` for the reverse direction. Only present when ``bidirectional=True``. bias_hh_l[k]_reverse: Analogous to `bias_hh_l[k]` for the reverse direction. Only present when ``bidirectional=True``. weight_hr_l[k]_reverse: Analogous to `weight_hr_l[k]` for the reverse direction. Only present when ``bidirectional=True`` and ``proj_size > 0`` was specified. .. note:: All the weights and biases are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{hidden\_size}}` .. note:: For bidirectional LSTMs, forward and backward are directions 0 and 1 respectively. Example of splitting the output layers when ``batch_first=False``: ``output.view(seq_len, batch, num_directions, hidden_size)``. .. note:: For bidirectional LSTMs, `h_n` is not equivalent to the last element of `output`; the former contains the final forward and reverse hidden states, while the latter contains the final forward hidden state and the initial reverse hidden state. .. note:: ``batch_first`` argument is ignored for unbatched inputs. .. note:: ``proj_size`` should be smaller than ``hidden_size``. .. include:: ../cudnn_rnn_determinism.rst .. include:: ../cudnn_persistent_rnn.rst Examples:: >>> rnn = nn.LSTM(10, 20, 2) >>> input = torch.randn(5, 3, 10) >>> h0 = torch.randn(2, 3, 20) >>> c0 = torch.randn(2, 3, 20) >>> output, (hn, cn) = rnn(input, (h0, c0)) Nr)r*r+r,r-r.r/r0rc grr9) r\r)r*r+r,r-r.r/r0r4r5s rrB LSTM.__init__rr!cgrr9rs rrBr)rr!c.>[TU]"S/UQ70UD6 g)NrrArBr\rrros rrBr)s 1$1&1r!rrcUb[US5nO3UR(aURS5OURS5nUR(aSOSnURU-UUR 4nU$r)rLr-rr/r+r*rs rget_expected_cell_sizeLSTM.get_expected_cell_sizesn  "[^,J*.*:*:A 1 J"00a OOn ,      $#r!rcURX5 URUSURX5S5 URUSURX5S5 g)Nrz"Expected hidden[0] size {}, got {}r z"Expected hidden[1] size {}, got {})rrrr/)r\rrrs rrLSTM.check_forward_argss` ,  1I  ) )% = 0  1I  ' ' ; 0 r!rrcHUcU$[USU5[USU54$)Nrr r$rs rrLSTM.permute_hiddens8  I!"Q%57I qE;8   r!cgrr9r s rr  LSTM.forwardr r!cgrr9r s rr r6 r r!c n UR5 UnSnUR(aSOSnURS:a URO URn[ U[ 5(aUuppxUSn Uc[ R"URU-U UURURS9n [ R"URU-U URURURS9n X4nGOlURX'5nGOYUR5S;a[SUR5S35eUR5S:Hn UR(aSOSn U (dURU 5nUR(aUR!S5OUR!S5n SnSnUc[ R"URU-U UURURS9n [ R"URU-U URURURS9n X4nUR#XU5 GOU (aeUSR5S:wdUSR5S:wa6S USR5S USR5S 3n[%U5eOUSR5S:wdUSR5S:wa6S USR5S USR5S 3n[%U5eUSRS5USRS54nUR#XU5 URX'5nUce[&R("UUUR*UR,URUR.UR0URUR5 nOZ[&R("UUUUR*UR,URUR.UR0UR5 nUSnUSSn[ U[ 5(a![ UXGU5nUURUU54$W (d9UR3W 5nUSR3S5USR3S54nUURUU54$) Nr6r rrrz)LSTM: Expected input to be 2D or 3D, got D insteadr8z=For batched 3-D input, hx and cx should also be 3-D but got (z-D, z -D) tensorsz?For unbatched 2-D input, hx and cx should also be 2-D but got ()rr/r0r*rEr rSrr+r5r4rrrIr-rrrrrlstmrtr,r.rr)r\rrrrr^rbrrrh_zerosc_zerosrr rr!r"rr#s rr r6sC !!#  "00a-1^^a-?4>>TEUEU j. 1 1CH @E(^Nz++OOn4"$++ <<  ++OOn4"$$++ << '((<yy{&( ? }IV)J!--1I 2.2.>.>UZZ]EJJqMN!N# z++OOn4"$++ <<  ++OOn4"$$++ << ''';?!uyy{a'2a599;!+;446qEIIK=RUYY[MQ\^+3// ,<!uyy{a'2a599;!+;446qEIIK=RUYY[MQ\^+3//Q%//!,beooa.@AB'';?((<  XX""   ""   FXX""   "" F j. 1 1* 5EM!$"5"5f>N"OO O 2 )++A.q 0A0A!0DE4..v7GHH Hr!r9rr)rOrrrrrrLrHrCrBrrrr/rrrSr%r&r r rrrs@rrrs^@ !#                         2 $ $*26*: $ sC}  $"  ffn% f%  &  &&. !  f%   vv~    ))CG  !)%*?!@ vuVV^,, - *  ))KO # )1%2G)H ~uVV^44 5 * uIuIr!rc\^\rSrSrSr\SS\S\S\S\S\S \S \S S4S jj5r \S 5r U4Sjr \\ RRSS\ S\\ S \\ \ 44Sjj55r\\ RRSS\S\\ S \\\ 44Sjj55rSSjrSrU=r$)ria__init__(input_size,hidden_size,num_layers=1,bias=True,batch_first=False,dropout=0.0,bidirectional=False,device=None,dtype=None) Apply a multi-layer gated recurrent unit (GRU) RNN to an input sequence. For each element in the input sequence, each layer computes the following function: .. math:: \begin{array}{ll} r_t = \sigma(W_{ir} x_t + b_{ir} + W_{hr} h_{(t-1)} + b_{hr}) \\ z_t = \sigma(W_{iz} x_t + b_{iz} + W_{hz} h_{(t-1)} + b_{hz}) \\ n_t = \tanh(W_{in} x_t + b_{in} + r_t \odot (W_{hn} h_{(t-1)}+ b_{hn})) \\ h_t = (1 - z_t) \odot n_t + z_t \odot h_{(t-1)} \end{array} where :math:`h_t` is the hidden state at time `t`, :math:`x_t` is the input at time `t`, :math:`h_{(t-1)}` is the hidden state of the layer at time `t-1` or the initial hidden state at time `0`, and :math:`r_t`, :math:`z_t`, :math:`n_t` are the reset, update, and new gates, respectively. :math:`\sigma` is the sigmoid function, and :math:`\odot` is the Hadamard product. In a multilayer GRU, the input :math:`x^{(l)}_t` of the :math:`l` -th layer (:math:`l \ge 2`) is the hidden state :math:`h^{(l-1)}_t` of the previous layer multiplied by dropout :math:`\delta^{(l-1)}_t` where each :math:`\delta^{(l-1)}_t` is a Bernoulli random variable which is :math:`0` with probability :attr:`dropout`. Args: input_size: The number of expected features in the input `x` hidden_size: The number of features in the hidden state `h` num_layers: Number of recurrent layers. E.g., setting ``num_layers=2`` would mean stacking two GRUs together to form a `stacked GRU`, with the second GRU taking in outputs of the first GRU and computing the final results. Default: 1 bias: If ``False``, then the layer does not use bias weights `b_ih` and `b_hh`. Default: ``True`` batch_first: If ``True``, then the input and output tensors are provided as `(batch, seq, feature)` instead of `(seq, batch, feature)`. Note that this does not apply to hidden or cell states. See the Inputs/Outputs sections below for details. Default: ``False`` dropout: If non-zero, introduces a `Dropout` layer on the outputs of each GRU layer except the last layer, with dropout probability equal to :attr:`dropout`. Default: 0 bidirectional: If ``True``, becomes a bidirectional GRU. Default: ``False`` Inputs: input, h_0 * **input**: tensor of shape :math:`(L, H_{in})` for unbatched input, :math:`(L, N, H_{in})` when ``batch_first=False`` or :math:`(N, L, H_{in})` when ``batch_first=True`` containing the features of the input sequence. The input can also be a packed variable length sequence. See :func:`torch.nn.utils.rnn.pack_padded_sequence` or :func:`torch.nn.utils.rnn.pack_sequence` for details. * **h_0**: tensor of shape :math:`(D * \text{num\_layers}, H_{out})` or :math:`(D * \text{num\_layers}, N, H_{out})` containing the initial hidden state for the input sequence. Defaults to zeros if not provided. where: .. math:: \begin{aligned} N ={} & \text{batch size} \\ L ={} & \text{sequence length} \\ D ={} & 2 \text{ if bidirectional=True otherwise } 1 \\ H_{in} ={} & \text{input\_size} \\ H_{out} ={} & \text{hidden\_size} \end{aligned} Outputs: output, h_n * **output**: tensor of shape :math:`(L, D * H_{out})` for unbatched input, :math:`(L, N, D * H_{out})` when ``batch_first=False`` or :math:`(N, L, D * H_{out})` when ``batch_first=True`` containing the output features `(h_t)` from the last layer of the GRU, for each `t`. If a :class:`torch.nn.utils.rnn.PackedSequence` has been given as the input, the output will also be a packed sequence. * **h_n**: tensor of shape :math:`(D * \text{num\_layers}, H_{out})` or :math:`(D * \text{num\_layers}, N, H_{out})` containing the final hidden state for the input sequence. Attributes: weight_ih_l[k] : the learnable input-hidden weights of the :math:`\text{k}^{th}` layer (W_ir|W_iz|W_in), of shape `(3*hidden_size, input_size)` for `k = 0`. Otherwise, the shape is `(3*hidden_size, num_directions * hidden_size)` weight_hh_l[k] : the learnable hidden-hidden weights of the :math:`\text{k}^{th}` layer (W_hr|W_hz|W_hn), of shape `(3*hidden_size, hidden_size)` bias_ih_l[k] : the learnable input-hidden bias of the :math:`\text{k}^{th}` layer (b_ir|b_iz|b_in), of shape `(3*hidden_size)` bias_hh_l[k] : the learnable hidden-hidden bias of the :math:`\text{k}^{th}` layer (b_hr|b_hz|b_hn), of shape `(3*hidden_size)` .. note:: All the weights and biases are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{hidden\_size}}` .. note:: For bidirectional GRUs, forward and backward are directions 0 and 1 respectively. Example of splitting the output layers when ``batch_first=False``: ``output.view(seq_len, batch, num_directions, hidden_size)``. .. note:: ``batch_first`` argument is ignored for unbatched inputs. .. note:: The calculation of new gate :math:`n_t` subtly differs from the original paper and other frameworks. In the original implementation, the Hadamard product :math:`(\odot)` between :math:`r_t` and the previous hidden state :math:`h_{(t-1)}` is done before the multiplication with the weight matrix `W` and addition of bias: .. math:: \begin{aligned} n_t = \tanh(W_{in} x_t + b_{in} + W_{hn} ( r_t \odot h_{(t-1)} ) + b_{hn}) \end{aligned} This is in contrast to PyTorch implementation, which is done after :math:`W_{hn} h_{(t-1)}` .. math:: \begin{aligned} n_t = \tanh(W_{in} x_t + b_{in} + r_t \odot (W_{hn} h_{(t-1)}+ b_{hn})) \end{aligned} This implementation differs on purpose for efficiency. .. include:: ../cudnn_persistent_rnn.rst Examples:: >>> rnn = nn.GRU(10, 20, 2) >>> input = torch.randn(5, 3, 10) >>> h0 = torch.randn(2, 3, 20) >>> output, hn = rnn(input, h0) Nr)r*r+r,r-r.r/rc grr9) r\r)r*r+r,r-r.r/r4r5s rrB GRU.__init__ s r!cgrr9rs rrBr?rr!cP>SU;a [S5e[TU]"S/UQ70UD6 g)Nr0rr)rIrArBr-s rrBr?s4 & O  000r!rrcgrr9r s rr  GRU.forward&r r!cgrr9r s rr rC-r r!c UR5 Un[U[5(azUuppVUSnUcZUR(aSOSn[R "UR U-UURURURS9nGOURX%5nGOSnUR5S;a[SUR5S35eUR5S:Hn UR(aSOSn U (dWURU 5nUbBUR5S:wa[S UR5S 35eURS5nO4Ub1UR5S:wa[S UR5S 35eUR(aUR!S5OUR!S5nSnSnUcYUR(aSOSn[R "UR U-UURURURS9nOURX%5nUR#XU5 Uce[$R&"UUUR(UR*UR UR,UR.URUR5 n OZ[$R&"UUUUR(UR*UR UR,UR.UR5 n U Sn U Sn [U[5(a[XXV5nXRX54$W (d"U R1W 5n U R1S5n XRX54$) Nrr6r rrz(GRU: Expected input to be 2D or 3D, got r9r8rrr)rrEr r/rSrr+r*r5r4rrrIr-rrrrrgrurtr,r.rr)r\rrrrrrrr^rr r!r"rr#s rr rC4s !!# j. 1 1CH @E(^Nz&*&8&8a[[OOn4"$$++ << ((<Kyy{&( >uyy{m9U)J!--1I 2>vvx1}*UVXV\V\V^U__hiaB>bffh!m&OPRPVPVPXzYbc/3.>.>UZZ]EJJqMN!N# z&*&8&8a[[OOn4"$$++ << ((< ;7  WW""   ""   FWW""   "" F j. 1 1*^M!"5"5f"OO O 2*..vHH Hr!r9)r TFrFNNr)rOrrrrrrLrHrCrBrSr%r&rrrr r rrrs@rrrsMB !#                       1 ))48  !)&!1 vv~  *  ))<@ # )1&)9 ~v% & * bIbIr!rc ^\rSrSr%/SQr\\S'\\S'\\S'\\S'\\S'SS\S\S\S \S S4 U4S jjjr S \ 4S jr SS jr Sr U=r$)ri)r)r*r,r)r*r, weight_ih weight_hhN num_chunksrc>XVS.n[TU]5 XlX lX0l[ [ R"XB-U440UD65Ul[ [ R"XB-U440UD65Ul U(aO[ [ R"XB-40UD65Ul [ [ R"XB-40UD65Ul O$URSS5 URSS5 UR5 g)Nr3bias_ihbias_hh)rArBr)r*r,r rSrTrHrIrLrMregister_parameterr[) r\r)r*r,rJr4r5r]ros rrBRNNCellBase.__init__s%+; $& " KK1:> Q. Q # KK1;? R> R  $ J4GGDL% J4GGDL  # #It 4  # #It 4 r!cSnSUR;aURSLaUS- nSUR;aURS:waUS- nUR"S0URD6$) Nrr,Trrrz, nonlinearity={nonlinearity}r9)rr,rrUrs rrRNNCellBase.extra_reprsb ) T]] "tyy'<  A T]] *t/@/@F/J 0 0Axx($--((r!cURS:a#S[R"UR5- OSnUR5Hn[R "X!*U5 M grrrs rr[RNNCellBase.reset_parametersrr!)r,rMrLr*r)rIrH)NNr)rOrrrrrLrrHrrBrrr[rrrs@rrrs}9MO J          B)C)//r!rc ^\rSrSr%Sr/SQr\\S'SS\S\S\ S\S S4 U4S jjjr SS \ S \ \ S \ 4S jjr SrU=r$)riaAn Elman RNN cell with tanh or ReLU non-linearity. .. math:: h' = \tanh(W_{ih} x + b_{ih} + W_{hh} h + b_{hh}) If :attr:`nonlinearity` is `'relu'`, then ReLU is used in place of tanh. Args: input_size: The number of expected features in the input `x` hidden_size: The number of features in the hidden state `h` bias: If ``False``, then the layer does not use bias weights `b_ih` and `b_hh`. Default: ``True`` nonlinearity: The non-linearity to use. Can be either ``'tanh'`` or ``'relu'``. Default: ``'tanh'`` Inputs: input, hidden - **input**: tensor containing input features - **hidden**: tensor containing the initial hidden state Defaults to zero if not provided. Outputs: h' - **h'** of shape `(batch, hidden_size)`: tensor containing the next hidden state for each element in the batch Shape: - input: :math:`(N, H_{in})` or :math:`(H_{in})` tensor containing input features where :math:`H_{in}` = `input_size`. - hidden: :math:`(N, H_{out})` or :math:`(H_{out})` tensor containing the initial hidden state where :math:`H_{out}` = `hidden_size`. Defaults to zero if not provided. - output: :math:`(N, H_{out})` or :math:`(H_{out})` tensor containing the next hidden state. Attributes: weight_ih: the learnable input-hidden weights, of shape `(hidden_size, input_size)` weight_hh: the learnable hidden-hidden weights, of shape `(hidden_size, hidden_size)` bias_ih: the learnable input-hidden bias, of shape `(hidden_size)` bias_hh: the learnable hidden-hidden bias, of shape `(hidden_size)` .. note:: All the weights and biases are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{hidden\_size}}` Examples:: >>> rnn = nn.RNNCell(10, 20) >>> input = torch.randn(6, 3, 10) >>> hx = torch.randn(3, 20) >>> output = [] >>> for i in range(6): ... hx = rnn(input[i], hx) ... output.append(hx) )r)r*r,rrNr)r*r,rcB>XVS.n[TU]"XU4SS0UD6 X@lg)Nr3rJr )rArBr) r\r)r*r,rr4r5r]ros rrBRNNCell.__init__ s,%+; $W1WW(r!rrcUR5S;a[SUR5S35eUb1UR5S;a[SUR5S35eUR5S:HnU(dURS5nUcE[R"UR S5UR URURS9nOU(dURS5OUnURS:XaD[R"UUURURURUR5nOnURS :XaD[R "UUURURURUR5nOUn[#S UR35eU(dUR%S5nU$) Nr r6z,RNNCell: Expected input to be 1D or 2D, got r9z-RNNCell: Expected hidden to be 1D or 2D, got r6rrrrzUnknown nonlinearity: )rrIrrSrrr*r5r4rr rnn_tanh_cellrHrIrLrM rnn_relu_cellrrr\rrrrs rr RNNCell.forwards 99;f $>uyy{m9U  >bffhf4?zS YY[A% OOA&E : 1 t//u{{5<<B)3aB    &##  C  & (##  CC!78I8I7JKL L++a.C r!r$)TrNNr)rOrrrrrrrrLrHrBrrr rrrs@rrrs4lJM " ) ) ) )  )  ) )-V-&)9-V--r!rc ^\rSrSrSrS S\S\S\SS4U4SjjjrSS \S \ \ \\4S\ \\44S jjr S r U=r $)riJa~ A long short-term memory (LSTM) cell. .. math:: \begin{array}{ll} i = \sigma(W_{ii} x + b_{ii} + W_{hi} h + b_{hi}) \\ f = \sigma(W_{if} x + b_{if} + W_{hf} h + b_{hf}) \\ g = \tanh(W_{ig} x + b_{ig} + W_{hg} h + b_{hg}) \\ o = \sigma(W_{io} x + b_{io} + W_{ho} h + b_{ho}) \\ c' = f \odot c + i \odot g \\ h' = o \odot \tanh(c') \\ \end{array} where :math:`\sigma` is the sigmoid function, and :math:`\odot` is the Hadamard product. Args: input_size: The number of expected features in the input `x` hidden_size: The number of features in the hidden state `h` bias: If ``False``, then the layer does not use bias weights `b_ih` and `b_hh`. Default: ``True`` Inputs: input, (h_0, c_0) - **input** of shape `(batch, input_size)` or `(input_size)`: tensor containing input features - **h_0** of shape `(batch, hidden_size)` or `(hidden_size)`: tensor containing the initial hidden state - **c_0** of shape `(batch, hidden_size)` or `(hidden_size)`: tensor containing the initial cell state If `(h_0, c_0)` is not provided, both **h_0** and **c_0** default to zero. Outputs: (h_1, c_1) - **h_1** of shape `(batch, hidden_size)` or `(hidden_size)`: tensor containing the next hidden state - **c_1** of shape `(batch, hidden_size)` or `(hidden_size)`: tensor containing the next cell state Attributes: weight_ih: the learnable input-hidden weights, of shape `(4*hidden_size, input_size)` weight_hh: the learnable hidden-hidden weights, of shape `(4*hidden_size, hidden_size)` bias_ih: the learnable input-hidden bias, of shape `(4*hidden_size)` bias_hh: the learnable hidden-hidden bias, of shape `(4*hidden_size)` .. note:: All the weights and biases are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{hidden\_size}}` On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. Examples:: >>> rnn = nn.LSTMCell(10, 20) # (input_size, hidden_size) >>> input = torch.randn(2, 3, 10) # (time_steps, batch, input_size) >>> hx = torch.randn(3, 20) # (batch, hidden_size) >>> cx = torch.randn(3, 20) >>> output = [] >>> for i in range(input.size()[0]): ... hx, cx = rnn(input[i], (hx, cx)) ... output.append(hx) >>> output = torch.stack(output, dim=0) Nr)r*r,rc6>XES.n[TU]"XU4SS0UD6 g)Nr3rJr7r,r\r)r*r,r4r5r]ros rrBLSTMCell.__init__&%+; $W1WWr!rrc0UR5S;a[SUR5S35eUbH[U5H9up4UR5S;dM[SUSUR5S35e UR5S:HnU(dURS5nUcH[R "UR S5URURURS9nXf4nO1U(d(USRS5US RS54OUn[R"UUURURURUR5nU(d(USR!S5US R!S54nU$) NrXz-LSTMCell: Expected input to be 1D or 2D, got r9zLSTMCell: Expected hx[z] to be 1D or 2D, got r6rrr )rrI enumeraterrSrrr*r5r4r lstm_cellrHrIrLrMr)r\rrrrrrrs rr LSTMCell.forwardsb 99;f $? }IV  >'m 99;f,$05KEIIK=Xab, YY[A% OOA&E :KK 1 t//u{{5<<EBAK"Q%//!$beooa&89QSBmm   NN NN LL LL  q6>>!$c!fnnQ&78C r!r9TNNr)rOrrrrrLrHrBrrrr rrrs@rrrJs9~ X X X X  X XDH$$!)%*?!@$ vv~ $$r!rc j^\rSrSrSrS S\S\S\SS4U4SjjjrSS \S \ \S\4S jjr S r U=r $)riaA gated recurrent unit (GRU) cell. .. math:: \begin{array}{ll} r = \sigma(W_{ir} x + b_{ir} + W_{hr} h + b_{hr}) \\ z = \sigma(W_{iz} x + b_{iz} + W_{hz} h + b_{hz}) \\ n = \tanh(W_{in} x + b_{in} + r \odot (W_{hn} h + b_{hn})) \\ h' = (1 - z) \odot n + z \odot h \end{array} where :math:`\sigma` is the sigmoid function, and :math:`\odot` is the Hadamard product. Args: input_size: The number of expected features in the input `x` hidden_size: The number of features in the hidden state `h` bias: If ``False``, then the layer does not use bias weights `b_ih` and `b_hh`. Default: ``True`` Inputs: input, hidden - **input** : tensor containing input features - **hidden** : tensor containing the initial hidden state for each element in the batch. Defaults to zero if not provided. Outputs: h' - **h'** : tensor containing the next hidden state for each element in the batch Shape: - input: :math:`(N, H_{in})` or :math:`(H_{in})` tensor containing input features where :math:`H_{in}` = `input_size`. - hidden: :math:`(N, H_{out})` or :math:`(H_{out})` tensor containing the initial hidden state where :math:`H_{out}` = `hidden_size`. Defaults to zero if not provided. - output: :math:`(N, H_{out})` or :math:`(H_{out})` tensor containing the next hidden state. Attributes: weight_ih: the learnable input-hidden weights, of shape `(3*hidden_size, input_size)` weight_hh: the learnable hidden-hidden weights, of shape `(3*hidden_size, hidden_size)` bias_ih: the learnable input-hidden bias, of shape `(3*hidden_size)` bias_hh: the learnable hidden-hidden bias, of shape `(3*hidden_size)` .. note:: All the weights and biases are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{hidden\_size}}` On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. Examples:: >>> rnn = nn.GRUCell(10, 20) >>> input = torch.randn(6, 3, 10) >>> hx = torch.randn(3, 20) >>> output = [] >>> for i in range(6): ... hx = rnn(input[i], hx) ... output.append(hx) Nr)r*r,rc6>XES.n[TU]"XU4SS0UD6 g)Nr3rJr8r,r_s rrBGRUCell.__init__rar!rrcUR5S;a[SUR5S35eUb1UR5S;a[SUR5S35eUR5S:HnU(dURS5nUcE[R"UR S5UR URURS9nOU(dURS5OUn[R"UUURURURUR5nU(dURS5nU$)NrXz,GRUCell: Expected input to be 1D or 2D, got r9z-GRUCell: Expected hidden to be 1D or 2D, got r6rr)rrIrrSrrr*r5r4rgru_cellrHrIrLrMrr[s rr GRUCell.forwards 99;f $>uyy{m9U  >bffhf4?zS YY[A% OOA&E : 1 t//u{{5<<B)3aBll   NN NN LL LL  ++a.C r!r9rfr) rOrrrrrLrHrBrrr rrrs@rrrsn;B X X X X  X X V &)9 V  r!r)r )$rrFrJrutypingrrtyping_extensionsrrSrrtorch.nnrtorch.nn.parameterr torch.nn.utils.rnnr moduler __all__rr _rnn_implsrLr FutureWarningr%r rrrrrrrr9r!rrvs  %( (-    1v1F11V1 c 8f8688F8 8bfbJ uE'uEL lI7lI^ LI'LI^7/&7/ttktnk{k\ikir!