o ZhY@s8ddlZddlZddlmZddlmZddlmZddl m Z ddl m Z e e ZdZGdddZGd d d ZGd d d eZGd ddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdd d eZGd!d"d"eZdS)#N)sparse)add_start_docstrings) get_loggerad Args: input_ids (`jnp.ndarray` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) scores (`jnp.ndarray` of shape `(batch_size, config.vocab_size)`): Prediction scores of a language modeling head. These can be logits for each vocabulary when not using beam search or log softmax for each vocabulary token when using beam search kwargs (`Dict[str, Any]`, *optional*): Additional logits processor specific kwargs. Return: `jnp.ndarray` of shape `(batch_size, config.vocab_size)`: The processed prediction scores. c@4eZdZdZeedejdejdejfddZdS)FlaxLogitsProcessorzSAbstract base class for all logit processors that can be applied during generation. input_idsscoresreturncCt|jd)z"Flax method for processing logits.H is an abstract class. Only classes inheriting this class can be called.NotImplementedError __class__selfrr rZ/var/www/auris/lib/python3.10/site-packages/transformers/generation/flax_logits_process.py__call__6 zFlaxLogitsProcessor.__call__N __name__ __module__ __qualname____doc__r!LOGITS_PROCESSOR_INPUTS_DOCSTRINGjnpndarrayrrrrrr3"rc@r)FlaxLogitsWarperzjAbstract base class for all logit warpers that can be applied during generation with multinomial sampling.rr r cCr )zFlax method for warping logits.r r rrrrrArzFlaxLogitsWarper.__call__Nrrrrrr>rrc @s8eZdZdZeedejdejdedejfddZ dS) FlaxLogitsProcessorLista. This class can be used to create a list of [`FlaxLogitsProcessor`] or [`FlaxLogitsWarper`] to subsequently process a `scores` input tensor. This class inherits from list and adds a specific *__call__* method to apply each [`FlaxLogitsProcessor`] or [`FlaxLogitsWarper`] to the inputs. rr cur_lenr c s|D]C}t|jj}t|dkr?tfddt|ddDs4tdt|d|j d||||fi}q||||}q|S)Nc3s|]}|vVqdSNr).0argkwargsrr Usz3FlaxLogitsProcessorList.__call__..rz,Make sure that all the required parameters: z for z$ are passed to the logits processor.) inspect signaturer parameterslenalllistkeys ValueErrorr)rrr r!r' processorZ function_argsrr&rrPs &z FlaxLogitsProcessorList.__call__N) rrrrrrrrintrrrrrr Is&r c@>eZdZdZdefddZdejdejdedejfd d Z d S) FlaxTemperatureLogitsWarperz [`FlaxLogitsWarper`] for temperature (exponential scaling output probability distribution). Args: temperature (`float`): The value used to module the logits distribution. temperaturecCs*t|tr |dkstd|||_dS)Nrz:`temperature` has to be a strictly positive float, but is ) isinstancefloatr0r5)rr5rrr__init__is z$FlaxTemperatureLogitsWarper.__init__rr r!r cCs||j}|Sr#)r5rrr r!rrrros z$FlaxTemperatureLogitsWarper.__call__N) rrrrr7r8rrr2rrrrrr4`s$r4c@sReZdZdZed dfdededefddZd ejd ejd ed ejfd dZ dS)FlaxTopPLogitsWarpera= [`FlaxLogitsWarper`] that performs top-p, i.e. restricting to top tokens summing to prob_cut_off <= prob_cut_off. Args: top_p (`float`): If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation. filter_value (`float`, *optional*, defaults to -inf): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. Inftop_p filter_valuemin_tokens_to_keepcCs^t|tr |dks |dkrtd|t|tr|dkr$td|||_||_||_dS)Nrg?z.`top_p` has to be a float > 0 and < 1, but is r<z:`min_tokens_to_keep` has to be a positive integer, but is )r6r7r0r2r=r>r?)rr=r>r?rrrr8s zFlaxTopPLogitsWarper.__init__rr r!r c Cst||jd\}}t||j}tjj|ddj dd}||j k}t |d}||j dddf dO}|j ddd|jf d}t|||} tj|| d} | S)NZaxisr<rT)laxtop_kshaperZ full_liker>jaxnnZsoftmaxZcumsumr=Zrollatsetr?whereZ sort_key_val) rrr r! topk_scores topk_indicesZ mask_scoresZcumulative_probsZ score_maskZtopk_next_scores next_scoresrrrrs  zFlaxTopPLogitsWarper.__call__N rrrrr7r2r8rrrrrrrr:ts" $ r:c@sReZdZdZed dfdededefddZd ejd ejd ed ejfd dZ dS)FlaxTopKLogitsWarpera [`FlaxLogitsWarper`] that performs top-k, i.e. restricting to the k highest probability elements. Args: top_k (`int`): The number of highest probability vocabulary tokens to keep for top-k-filtering. filter_value (`float`, *optional*, defaults to -inf): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. r;r<rCr>r?cCs6t|tr |dkrtd|t|||_||_dS)Nrz6`top_k` has to be a strictly positive integer, but is )r6r2r0maxrCr>)rrCr>r?rrrr8s  zFlaxTopKLogitsWarper.__init__rr r!r cCs|j\}}t|||j}t|j|jd}t||\}} tt||dddf||f } | } | | } |j |  | }| ||} | S)Nr@) rDrfullr>minrCrBZ broadcast_toarangeflattenrGrHZreshape)rrr r! batch_size vocab_sizeZnext_scores_flatZtopkrJrKshiftZtopk_scores_flatZtopk_indices_flatrLrrrrs *  zFlaxTopKLogitsWarper.__call__NrMrrrrrNs" $rNc@s>eZdZdZdefddZdejdejdedejfd d Zd S) !FlaxForcedBOSTokenLogitsProcessorz [`FlaxLogitsProcessor`] that enforces the specified token as the first generated token. Args: bos_token_id (`int`): The id of the token to force as the first generated token. bos_token_idcCs ||_dSr#)rX)rrXrrrr8s z*FlaxForcedBOSTokenLogitsProcessor.__init__rr r!r cCsNt|jtd }dt|d}t||jdd|jfd|}|SNinfr<r) rrPrDr7bool_rIrGrXrHrrr r! new_scores apply_penaltyrrrrs$z*FlaxForcedBOSTokenLogitsProcessor.__call__N rrrrr2r8rrrrrrrrWs$rWc@BeZdZdZdedefddZdejdejded ejfd d Zd S) !FlaxForcedEOSTokenLogitsProcessorae [`FlaxLogitsProcessor`] that enforces the specified token as the last generated token when `max_length` is reached. Args: max_length (`int`): The maximum length of the sequence to be generated. eos_token_id (`int`): The id of the token to force as the last generated token when `max_length` is reached. max_length eos_token_idcCs||_||_dSr#)rbrc)rrbrcrrrr8s z*FlaxForcedEOSTokenLogitsProcessor.__init__rr r!r cCsTt|jtd }dt||jd}t||jdd|jf d|}|SrY) rrPrDr7r[rbrIrGrcrHr\rrrrs$z*FlaxForcedEOSTokenLogitsProcessor.__call__Nr_rrrrras $rac@r`) FlaxMinLengthLogitsProcessora3 [`FlaxLogitsProcessor`] enforcing a min-length by setting EOS probability to 0. Args: min_length (`int`): The minimum length below which the score of `eos_token_id` is set to `-float("Inf")`. eos_token_id (`int`): The id of the *end-of-sequence* token. min_lengthrccCsPt|tr |dkrtd|t|tr|dkr td|||_||_dS)Nrz2`min_length` has to be a positive integer, but is z4`eos_token_id` has to be a positive integer, but is )r6r2r0rerc)rrercrrrr8s  z%FlaxMinLengthLogitsProcessor.__init__rr r!r cCsFdt||jdd}t||jdd|jftd |}|S)Nr<rrZ)rZcliprerIrGrcrHr7rrr r!r^rrrrs*z%FlaxMinLengthLogitsProcessor.__call__Nr_rrrrrds $ rdc@s&eZdZdZddZdefddZdS)(FlaxSuppressTokensAtBeginLogitsProcessora [`FlaxLogitsProcessor`] suppressing a list of tokens as soon as the `generate` function starts generating using `begin_index` tokens. This should ensure that the tokens defined by `begin_suppress_tokens` are not sampled at the beginning of the generation. Args: begin_suppress_tokens (`List[int]`): Tokens to not sample. begin_index (`int`): Index where the tokens are suppressed. cCst||_||_dSr#)r.begin_suppress_tokens begin_index)rrhrirrrr8s  z1FlaxSuppressTokensAtBeginLogitsProcessor.__init__r!cCsBdt||j}t||jdd|jftd |}|S)Nr<rZ)rr[rirIrGrhrHr7rfrrrr!s*z1FlaxSuppressTokensAtBeginLogitsProcessor.__call__N)rrrrr8r2rrrrrrgs rgc@r3) !FlaxSuppressTokensLogitsProcessorz [`FlaxLogitsProcessor`] suppressing a list of tokens at each decoding step. The processor will set their log probs to be `-inf` so they are not sampled. Args: suppress_tokens (`list`): Tokens to not sample. suppress_tokenscCst||_dSr#)r.rk)rrkrrrr83sz*FlaxSuppressTokensLogitsProcessor.__init__rr r!r cCs |jd|jftd }|S)N.rZ)rGrkrHr7r9rrrr6sz*FlaxSuppressTokensLogitsProcessor.__call__N) rrrrr.r8rrr2rrrrrrj)s $rjc@s8eZdZdZddZdejdejdedejfdd Zd S) FlaxForceTokensLogitsProcessora [`FlaxLogitsProcessor`] that takes a list of pairs of integers which indicates a mapping from generation indices to token indices that will be forced before sampling. The processor will set their log probs to 0 and all other tokens to `-inf` so that they are sampled at their corresponding index. Args: force_token_map (`list`): Map giving token ids and indices where they will be forced to be sampled. cCsbt|}tjt|dtjdd}|D]\}}|dur(|j||}qt||_ dS)Nr<dtyper@) dictrZonesrOr/Zint32itemsrGrHforce_token_array)rZforce_token_maprqindextokenrrrr8Gs z'FlaxForceTokensLogitsProcessor.__init__rr r!r csBfddtjjdkfddfddS)NcsXjd}j|}tjjdtd }tj|dfjd}t||d|f}|S)NrrmrZr<) rDrqrZ ones_likernr7zerosrBdynamic_update_slice)Zgeneration_idxrT current_tokenr]Zupdates)r rrr _force_tokenSs  z=FlaxForceTokensLogitsProcessor.__call__.._force_tokenrcSr#rrr rr_z9FlaxForceTokensLogitsProcessor.__call__..cs*tjdkfddfddS)NrcsSr#rr)rwr!rrrzdszKFlaxForceTokensLogitsProcessor.__call__....crxr#rrryrrrzfr{)rBcondrqrrwr!r rrrrzas    )rBr|rqrDr9rr}rrRs   z'FlaxForceTokensLogitsProcessor.__call__N) rrrrr8rrr2rrrrrrl<s $ rlc@s eZdZdZddZddZdS)#FlaxWhisperTimeStampLogitsProcessora{ Whisper specific Processor. This processor can be used to force a list of tokens. The processor will set their log probs to `inf` so that they are sampled at their corresponding index. Args: generate_config (`GenerateConfig`): The generate config used to generate the output. The following parameters are required: eos_token_id (`int`, *optional*, defaults to 50257): The id of the *end-of-sequence* token. no_timestamps_token_id (`int`, *optional*, defaults to 50363): The id of the `"<|notimestamps|>"` token. max_initial_timestamp_index (`int`, *optional*, defaults to 1): Used to set the maximum value of the initial timestamp. This is used to prevent the model from predicting timestamps that are too far in the future. cCsp|j|_|j|_|jd|_|d|_|jr|jd7_t|dr'|j|_n|j|_|jdur6|j|_dSdS)Nr<rmax_initial_timestamp_index)rcno_timestamps_token_idtimestamp_beginriZis_multilingualhasattrrrU)rZgenerate_configZ model_configZdecoder_input_lengthrrrr8}s      z,FlaxWhisperTimeStampLogitsProcessor.__init__c s|jddjftd }fdd}t|||}tjkdd}tj du|d}j j }t||jdd|ddftd |}tj j |dd}fd d }t|||}|S) NrZc stjdkdd}t|djk|d}tjdkdd}t|djkd|}t|t|dk|jjdtd |jdjtd |S)Nr<TFrrrZ)rrIrirrGrHr7rc)Z input_ids_kscores_kZlast_was_timestampZpenultimate_was_timestampr!rrr handle_pairss*zBFlaxWhisperTimeStampLogitsProcessor.__call__..handle_pairsTFr<r@rAcsXtjj|jddd}t|dj}t||k|jdjt d |S)Nr@rArZ) rErFZ logsumexprrrOrIrGrHr7)Z logprobs_krZtimestamp_logprobZmax_text_token_logprob)rrrhandle_cumulative_probsszMFlaxWhisperTimeStampLogitsProcessor.__call__..handle_cumulative_probs) rGrrHr7rEvmaprrIrirrrFZ log_softmax) rrr r!rZapply_max_initial_timestampZ last_allowedZlogprobsrrrrrs&  $  z,FlaxWhisperTimeStampLogitsProcessor.__call__N)rrrrr8rrrrrr~ls r~c@sleZdZdZdefddZdejdedefdd Zd ejd ejfd d Z dejdejded ejfddZ dS) FlaxNoRepeatNGramLogitsProcessora9 [`FlaxLogitsProcessor`] that enforces no repetition of n-grams. See [Fairseq](https://github.com/pytorch/fairseq/blob/a07cb6f40480928c9e0548b737aadd36ee66ac76/fairseq/sequence_generator.py#L345). Args: ngram_size (`int`): All ngrams of size `ngram_size` can only occur once. ngram_sizecCs*t|tr |dkrtd|||_dS)Nrz;`ngram_size` has to be a strictly positive integer, but is )r6r2r0r)rrrrrr8s z)FlaxNoRepeatNGramLogitsProcessor.__init__rrUr!c sj\}|jd}|jd}fdd}|jdf}tjd||tj|jd} t||k d} t j | | ff|fjdS)a get a matrix of size (batch_size,) + (vocab_size,)*n (for n-grams) that represent the n-grams that occurred previously. The BCOO representation allow to store only the few non-zero entries, instead of the full (huge) matrix r<csD|||j|tgfddtjDS)Ncs"g|] }t|fqSr)rarray)r$j)brposrr s"zZFlaxNoRepeatNGramLogitsProcessor.get_previous_ngrams..body_fun..)rGrHrrranger)ivalrTrr)rrrbody_funs zFFlaxNoRepeatNGramLogitsProcessor.get_previous_ngrams..body_funrrmZfloat32)rD) rDrrErBZ fori_looprrtrnrRastyperZBCOO) rrrUr!Zseq_lenZ seq_ngramsZ cur_ngramsrrDZall_update_indicesdatarrrget_previous_ngramss   z4FlaxNoRepeatNGramLogitsProcessor.get_previous_ngrams latest_tokensr cCs$tjtjdd}t|||S)zt Determines which tokens must be banned given latest tokens and the previously seen ngrams. cSs |t|Sr#)tuple)rprevious_ngramsrrrinner_fns zIFlaxNoRepeatNGramLogitsProcessor.get_banned_tokens_mask..inner_fn)rZsparsifyrErZ bcoo_todense)rrrrrrrget_banned_tokens_masks z7FlaxNoRepeatNGramLogitsProcessor.get_banned_tokens_maskr cs8fdd}tjjdk|fdd}|S)Nc sj\}}|}tjjdjdfjd}tj|tj djdfjdjdfd} || d}t |t d S)Nrr<rm)rrboolrZ)rDrrrtrrnrErBruZ dynamic_slicerrrIr7)_rUrrZbanned_tokens_indices_maskr!rr rrrtrue_fn s  $ z:FlaxNoRepeatNGramLogitsProcessor.__call__..true_fnr<crxr#rrryrrrzr{z;FlaxNoRepeatNGramLogitsProcessor.__call__..)rErBr|r)rrr r!routputrrrr s"z)FlaxNoRepeatNGramLogitsProcessor.__call__N) rrrrr2r8rrrrrrrrrrs  "$ r)r)rEZjax.laxrBZ jax.numpynumpyrZjax.experimentalrutilsrZ utils.loggingrrloggerrrrr.r r4r:rNrWrardrgrjrlr~rrrrrs.       ,#0a