eThhG S r SSKJrJr SSKrSSKJrJr SSK J
r
J r J r J
r
JrJr \R " \5 r " S S\5 rg) zZ
Sequence feature extraction class for common feature extractors to preprocess sequences.
)OptionalUnionN ) BatchFeatureFeatureExtractionMixin)PaddingStrategy
TensorType is_tf_tensoris_torch_tensorloggingto_numpyc ^ \ rS rSrSrS\S\S\4U 4S jjr SS\\ \
\ \ \ \ 4 \ \ \
\ 4 \
\ \ \ 4 4 S \\
\ \4 S
\\ S \
S \\ S
\\
S\\\ \4 S\ 4S jjrS\R$ SS4S\\ \ \R( 4 \ 4 S
\\ S\S \\ S
\\
S\ 4 S jjr SS\\ \ \R( 4 \ 4 S
\\ S \\ S \\
4S jjrSS jrSrU =r$ )SequenceFeatureExtractor a}
This is a general feature extraction class for speech recognition.
Args:
feature_size (`int`):
The feature dimension of the extracted features.
sampling_rate (`int`):
The sampling rate at which the audio files should be digitalized expressed in hertz (Hz).
padding_value (`float`):
The value that is used to fill the padding values / vectors.
feature_size
sampling_rate
padding_valuec > Xl X l X0l UR SS5 U l UR SS5 U l [ TU ] " S0 UD6 g )N padding_siderightreturn_attention_maskT )r r r popr r super__init__)selfr r r kwargs __class__s f/var/www/auris/envauris/lib/python3.13/site-packages/transformers/feature_extraction_sequence_utils.pyr !SequenceFeatureExtractor.__init__* sL ( * * "JJ~w ?%+ZZ0G%N"
"6" Nprocessed_featurespadding
max_length
truncationpad_to_multiple_ofr return_tensorsreturnc
^ ^ [ U[ [ 45 ( aV [ US [ [ 45 ( a8 US R 5 VV s0 s H oU V s/ s H oU PM sn _M nnn T R S U; a5 [ ST R S S[ UR 5 5 35 eUT R S n
Ub UO T R n[ U
5 S:X a U( a / US' U$ U
S n [ U [ [ 45 ( aA Sn [ X 5 S:X a U S-
n [ X 5 S:X a M U [ U
5 : a X S n Ucx [ U 5 ( a SnOe[ U 5 ( a SnOR[ U [ [ [ [ [ R 45 ( a SnO[ S U S
[! U 5 S 35 eUR# 5 HP u p[ U
S [ [ 45 ( a [% U
5 X' M2 U
Vs/ s H n[% U5 PM snX' MR T R' X#S 9nUT R S n
[ U
5 m[) U4S
jUR+ 5 5 5 ( d [ S5 e/ n[- T5 HO nUR# 5 VVs0 s H u nnUUU _M nnnT R/ UUUUS9nUR1 U5 MQ U[2 R4 :X a% [7 U 4S jU 5 5 n[2 R8 n0 n[- T5 H nT R; UU UUUUS9nUR# 5 Ht u pUU; a / UU' U
R< [ R< " [ R> 5 L a U
RA [ RB 5 n
UU R1 U
5 Mv M [ UUS9$ s sn f s sn nf s snf s snnf )a=
Pad input values / input vectors or a batch of input values / input vectors up to predefined length or to the
max sequence length in the batch.
Padding side (left/right) padding values are defined at the feature extractor level (with `self.padding_side`,
`self.padding_value`)
If the `processed_features` passed are dictionary of numpy arrays, PyTorch tensors or TensorFlow tensors, the
result will use the same type unless you provide a different tensor type with `return_tensors`. In the case of
PyTorch tensors, you will lose the specific device of your tensors however.
Args:
processed_features ([`BatchFeature`], list of [`BatchFeature`], `Dict[str, List[float]]`, `Dict[str, List[List[float]]` or `List[Dict[str, List[float]]]`):
Processed inputs. Can represent one input ([`BatchFeature`] or `Dict[str, List[float]]`) or a batch of
input values / vectors (list of [`BatchFeature`], *Dict[str, List[List[float]]]* or *List[Dict[str,
List[float]]]*) so you can use this method during preprocessing as well as in a PyTorch Dataloader
collate function.
Instead of `List[float]` you can have tensors (numpy arrays, PyTorch tensors or TensorFlow tensors),
see the note above for the return type.
padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*, defaults to `True`):
Select a strategy to pad the returned sequences (according to the model's padding side and padding
index) among:
- `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single
sequence if provided).
- `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum
acceptable input length for the model if that argument is not provided.
- `False` or `'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of different
lengths).
max_length (`int`, *optional*):
Maximum length of the returned list and optionally padding length (see above).
truncation (`bool`):
Activates truncation to cut input sequences longer than `max_length` to `max_length`.
pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
`>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default.
[What are attention masks?](../glossary#attention-mask)
return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are:
- `'tf'`: Return TensorFlow `tf.constant` objects.
- `'pt'`: Return PyTorch `torch.Tensor` objects.
- `'np'`: Return Numpy `np.ndarray` objects.
r zYou should supply an instance of `transformers.BatchFeature` or list of `transformers.BatchFeature` to this method that includes z, but you provided attention_maskr tfptnpztype of z
unknown: zA. Should be one of a python, numpy, pytorch or tensorflow object.)r# r$ c 3 @ ># U H n[ U5 T:H v M g 7f)N)len).0v
batch_sizes r /SequenceFeatureExtractor.pad.. s M1LA3q6Z'1Ls zLSome items in the output dictionary have a different batch size than others.)r$ r& r% c 3 Z ># U H n[ UTR S 5 v M" g7f)r N)r/ model_input_names)r0 input_slicer s r r3 r4 s+ m\l[ST-C-CA-F!G H H\ls (+)r$ padding_strategyr& r ) tensor_type)"
isinstancelisttupledictr keysr6
ValueErrorr r/ r
r intfloatr- ndarraytypeitemsr
_get_padding_strategiesallvaluesrange _truncateappendr LONGESTmax
MAX_LENGTH_paddtypefloat64astypefloat32)r r" r# r$ r% r& r r' keyexamplerequired_input
first_elementindexvaluer1 r8 truncated_inputsikinputs inputs_slice
batch_outputsoutputsr2 s ` @r pad SequenceFeatureExtractor.pad4 s R (4- 8 8ZHZ[\H]`dfr_s=t=tQcdeQfQkQkQm"Qm#2DE2Dwcl2DEEQm
"
! !! $,> > 1151G1G1J0K L+002346
,D,B,B1,EF%:%F !DLfLf ~ ! #$79"#34% % 'q)
mdE] 3 3En+,1
n+,1s>** . 5a 8
!M**!% //!%MCeRZZ+PQQ!% }oZ]8K7L MV V
-224JC%(S%L11*25/"'@E*F18A;*F"' 5 777_+D,B,B1,EF (
M1C1J1J1LMMM kl l z"A*<*B*B*DE*D$!Qa1g*DFE >>%#5% * L
# #L 1 # 6 6 6 m\l mmJ . 9 9
z"Aii #%!1#5&; G &mmo
m+)+M#&;;"((2::"66 !LL 4E c"))%0 . #" M~FFC F"h +G Fs$
O5 O0*O5 O;>P 0O5r8 c XR S nU[ R :X a [ U5 nUb Ub X$- S:w a
X$- S- U- nU[ R :g =( a [ U5 U: nU( a5 SU; a/ [
R " [ U5 [
R S9US' U( Ga1 U[ U5 -
nU R S:X aq U( a [
R " US SU45 US' U R S: a SU4S4OSU4n [
R " XiSU R S9XR S ' U$ U R S :X aq U( a [
R " US US45 US' U R S: a US4S4OUS4n [
R " XiSU R S9XR S ' U$ [ S
[ U R 5 - 5 eU$ ) a
Pad inputs (on left/right and up to predefined length or max length in the batch)
Args:
processed_features (`Union[Dict[str, np.ndarray], BatchFeature]`):
Dictionary of input values (`np.ndarray[float]`) / input vectors (`List[np.ndarray[float]]`) or batch
of inputs values (`List[np.ndarray[int]]`) / input vectors (`List[np.ndarray[int]]`)
max_length (`int`, *optional*):
Maximum length of the returned list and optionally padding length (see below)
padding_strategy (`PaddingStrategy`, *optional*, default to `PaddingStrategy.DO_NOT_PAD`):
PaddingStrategy to use for padding.
- PaddingStrategy.LONGEST Pad to the longest sequence in the batch
- PaddingStrategy.MAX_LENGTH: Pad to the max length (default)
- PaddingStrategy.DO_NOT_PAD: Do not pad
The feature_extractor padding sides are defined in self.padding_side:
- 'left': pads on the left of the sequences
- 'right': pads on the right of the sequences
pad_to_multiple_of (`int`, *optional*):
Integer if set will pad the sequence to a multiple of the provided value. This is especially useful to
enable the use of Tensor Core on NVIDIA hardware with compute capability `>= 7.5` (Volta), or on TPUs
which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*):
Set to False to avoid returning attention mask (default: set to model specifics)
r r r* )rO r )r r constant)constant_valuesleftzInvalid padding strategy:)r6 r rK r/
DO_NOT_PADr- onesint32r r` r r r? str)
r r" r$ r8 r& r rU needs_to_be_padded
difference
padding_shapes
r rN SequenceFeatureExtractor._pad s D ,,B,B1,EF 6 6 6 ^,J !&8&D*JimnJn%;q@DVVJ - 1K1K K pPSTbPcfpPp %5=O%O35773~;NVXV^V^3_ / 0 #c.&99J G+(;=66*+;B=N=NQR=R!Z & 9YZ\fXg
@B":tOaOaA"#9#9!#<= "! ""f,(;=66*+;B=N=NQR=R*a & 9YcefXg
@B":tOaOaA"#9#9!#<= "! !!= 7.5` (Volta), or on TPUs
which benefit from having sequence lengths be a multiple of 128.
truncation (`bool`, *optional*):
Activates truncation to cut input sequences longer than `max_length` to `max_length`.
NzKWhen setting ``truncation=True``, make sure that ``max_length`` is defined.r r r* )r? r6 r/ )r r" r$ r& r% rU needs_to_be_truncateds r rI "SequenceFeatureExtractor._truncate* s . % %
J . jk k+,B,B1,EF !&8&D*JimnJn%;q@DVVJ #N 3j @ r s< # J ` `
H %V 5 V r!