fThSrSSKJr SSKJr SSKJr SSKJr \R"\ 5r "SS\5r "S S \5r SS /rg ) zLongT5 model configuration)Mapping)PretrainedConfig)OnnxSeq2SeqConfigWithPast)loggingcp^\rSrSrSrSrS/rSSSSS .rS U4S jjrS r U=r $) LongT5Configa> This is the configuration class to store the configuration of a [`LongT5Model`] or a [`FlaxLongT5Model`]. It is used to instantiate a LongT5 model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the LongT5 [google/long-t5-local-base](https://huggingface.co/google/long-t5-local-base) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Arguments: vocab_size (`int`, *optional*, defaults to 32128): Vocabulary size of the LongT5 model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`LongT5Model`]. d_model (`int`, *optional*, defaults to 512): Size of the encoder layers and the pooler layer. d_kv (`int`, *optional*, defaults to 64): Size of the key, query, value projections per attention head. `d_kv` has to be equal to `d_model // num_heads`. d_ff (`int`, *optional*, defaults to 2048): Size of the intermediate feed forward layer in each `LongT5Block`. num_layers (`int`, *optional*, defaults to 6): Number of hidden layers in the Transformer encoder. num_decoder_layers (`int`, *optional*): Number of hidden layers in the Transformer decoder. Will use the same value as `num_layers` if not set. num_heads (`int`, *optional*, defaults to 8): Number of attention heads for each attention layer in the Transformer encoder. local_radius (`int`, *optional*, defaults to 127) Number of tokens to the left/right for each token to locally self-attend in a local attention mechanism. global_block_size (`int`, *optional*, defaults to 16) Length of blocks an input sequence is divided into for a global token representation. Used only for `encoder_attention_type = "transient-global"`. relative_attention_num_buckets (`int`, *optional*, defaults to 32): The number of buckets to use for each attention layer. relative_attention_max_distance (`int`, *optional*, defaults to 128): The maximum distance of the longer sequences for the bucket separation. dropout_rate (`float`, *optional*, defaults to 0.1): The ratio for all dropout layers. layer_norm_eps (`float`, *optional*, defaults to 1e-6): The epsilon used by the layer normalization layers. initializer_factor (`float`, *optional*, defaults to 1): A factor for initializing all weight matrices (should be kept to 1, used internally for initialization testing). feed_forward_proj (`string`, *optional*, defaults to `"relu"`): Type of feed forward layer to be used. Should be one of `"relu"` or `"gated-gelu"`. LongT5v1.1 uses the `"gated-gelu"` feed forward projection. Original LongT5 implementation uses `"gated-gelu"`. encoder_attention_type (`string`, *optional*, defaults to `"local"`): Type of encoder attention to be used. Should be one of `"local"` or `"transient-global"`, which are supported by LongT5 implementation. use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should return the last key/values attentions (not used by all models). longt5past_key_valuesd_model num_heads num_layersd_kv) hidden_sizenum_attention_headsnum_hidden_layershead_dimc >XlX lX0lX@lXPlUbUO URUlXplXlXlXl Xl Xl Xl Xl XlUUlUUlURR#S5nUSUlUSS:HUl[)U5S:a USS:wd[)U5S:a[+SUS35eUS :XaS Ul[,TU]\"S UUUS .UD6 g) N-rgatedz`feed_forward_proj`: z is not a valid activation function of the dense layer. Please make sure `feed_forward_proj` is of the format `gated-{ACT_FN}` or `{ACT_FN}`, e.g. 'gated-gelu' or 'relu'z gated-gelugelu_new) pad_token_id eos_token_idis_encoder_decoder) vocab_sizer rd_ffrnum_decoder_layersr local_radiusglobal_block_sizerelative_attention_num_bucketsrelative_attention_max_distance dropout_ratelayer_norm_epsiloninitializer_factorfeed_forward_projencoder_attention_type use_cachesplit dense_act_fn is_gated_actlen ValueErrorsuper__init__)selfr r rr!rr"rr#r$r%r&r'r(r)r*rr+r,rrkwargsact_info __class__s g/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/longt5/configuration_longt5.pyr3LongT5Config.__init__Ys%0%   $8J8V"4\`\k\k"(!2.L+/N,("4"4!2&<#"))//4$RL$QK72 x=1 !!73x=1;L'(9':;))   , *D   %%1   )r!rr r.r'r+r*r$r)r/r(r#r"rrr&r%r,r )i}i@iN g?gư>g?reluTlocalTrr) __name__ __module__ __qualname____firstlineno____doc__ model_typekeys_to_ignore_at_inference attribute_mapr3__static_attributes__ __classcell__)r7s@r8r r st2hJ#4"5 *) M  ')(+ &+? ? r:r cX\rSrSr\S\\\\\444Sj5r\S\4Sj5r Sr g)LongT5OnnxConfigreturncSSS.SSS.S.nUR(aSUSS'SS0US 'SS S.US 'OSS S.US 'SS S.US 'UR(aURUS S9 U$)Nbatchencoder_sequence)rr) input_idsattention_maskz past_encoder_sequence + sequencerVrrdecoder_input_idsz past_decoder_sequence + sequencedecoder_attention_maskdecoder_sequenceinputs) direction)use_pastfill_with_past_key_values_)r4 common_inputss r8rZLongT5OnnxConfig.inputss%);<").@A  ==1SM* +A .23WM- .:AFh6iM2 35rnsR!3-   H %} #} @02 - .r: