fThgLSrSSKJrJrJrJrJr SSKrSSKrSSKJ r SSK J r J r J r SSKJr SSKJrJr SS KJr SS KJr SS KJrJr SS KJrJrJrJr SS KJ r J!r! SSK"J#r# SSK$J%r%J&r&J'r'J(r(J)r) SSK*J+r+ \("5(a SSK,J-r- SSK.J/r/ \"5(a\)R`"\15r2"SS\ Rf5r4S3S\ RjS\RlS\RlS\RlS\\RlS\7S\74Sjjr8"SS\ Rj5r9"S S!\ Rj5r:\&"S"S#\!55r;"S$S%\;5r<\&"S&S'\;55r="S(S)\\%5r>"S*S+\;\5r?\&"S,S-9"S.S/\;55r@\&"S0S1\;55rA/S2QrBg)4zPyTorch OPT model.)CallableListOptionalTupleUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)Cache DynamicCache)GenerationMixin)AttentionMaskConverter)FlashAttentionKwargsis_flash_attn_available)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPast)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) LossKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging) OPTConfig) BlockMask)make_flex_block_causal_maskc^\rSrSrSrS\S\4U4SjjrS S\RS\S\ \R4U4S jjjr S r U=r $) OPTLearnedPositionalEmbedding6zF This module learns positional embeddings up to a fixed maximum size. num_embeddings embedding_dimcL>SUl[TU] XR-U5 gN)offsetsuper__init__)selfr'r( __class__s \/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/opt/modeling_opt.pyr.&OPTLearnedPositionalEmbedding.__init__;s"  ++5}Eattention_maskpast_key_values_length position_idsc>Uc5[R"USS9nX1-S- R5nUSS2US24n[TU]X0R -5$)z3`input_ids_shape` is expected to be [bsz x seqlen].Nr dim)torchcumsumlongr-forwardr,)r/r4r5r6r0s r1r=%OPTLearnedPositionalEmbedding.forwardAsZ   <<A>L(9A=CCEL'+A+B(BCLw|kk9::r3)r,rN) __name__ __module__ __qualname____firstlineno____doc__intr.r: LongTensorrr=__static_attributes__ __classcell__r0s@r1r%r%6s]FsF3F'(37 ;((;!$;u//0 ;;r3r%modulequerykeyvaluer4scalingdropoutc [R"XRSS55U-nUbX-n[RR US[R S9RUR5n[RRXURS9n[R"X5n U RSS5R5n X4$)N)r9dtypeptrainingr r+) r:matmul transposer functionalsoftmaxfloat32torSrOrV contiguous) rJrKrLrMr4rNrOkwargs attn_weights attn_outputs r1eager_attention_forwardraSs<<}}R'<=GL!#4 ==((2U]](SVVW\WbWbcL==((6??([L,,|3K''1-88:K  $$r3c:^\rSrSrSrSS\S\\4U4SjjjrSS\ RS\\ \ RS\\ RS \\ RS \ S \\ RS \ \ R\\ R\\ 44S jjrSrU=r$) OPTAttentionjz=Multi-headed attention from 'Attention Is All You Need' paperconfig layer_idxc >[TU]5 XlURUlUR UlURUlURUl X l Uc-[RSURRS35 URUR -UlSUlURUR -UR:wa&[#SURSUR S35eURS-Ul[&R("URURURS9Ul[&R("URURURS9Ul[&R("URURURS9Ul[&R("URURURS9Ulg) NzInstantiating z without passing a `layer_idx` is not recommended and will lead to errors during the forward call if caching is used. Please make sure to provide a `layer_idx` when creating this class.Tz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).g࿩bias)r-r.re hidden_size embed_dimnum_attention_heads num_headsattention_dropoutrO enable_biasrflogger warning_oncer0r@head_dim is_causal ValueErrorrNrLineark_projv_projq_projout_proj)r/rerfr^r0s r1r.OPTAttention.__init__ms{  ++33// !--"      !8!8 9:,,  $..8  MMDNN *t~~ =MdnnM]$T^^$4B8 }}d* iiTEUEUV iiTEUEUV iiTEUEUV  $..$..tGWGWX r3 hidden_statespast_key_valuer4layer_head_maskoutput_attentionscache_positionreturnc UR5upn URU5UR-n U RUSURUR 5R SS5n URU5n URU5n U RUSURUR 5R SS5n U RUSURUR 5R SS5n Ub!URXURSU05up[nURRS:waTURRS:XaU(a[RS5 O[ URRnU"UU U U U4UR"(dS O UR$URS .UD6unnUR'XS5R)5nUR+U5nU(dSnUUU4$) z#Input shape: Batch x Time x ChannelrQr r+Nreagersdpaz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)rOrN)sizerxrNviewrmrrrXrvrwupdaterfrare_attn_implementationrprqrrVrOreshaper]ry)r/r{r|r4r}r~rr^bsztgt_len_ query_states key_states value_statesattention_interfacer`r_s r1r=OPTAttention.forwards(,,.a{{=1DLL@ #((b$..$--PZZ[\^_` [[/ {{=1 __S"dnndmmLVVWXZ[\ #((b$..$--PZZ[\^_`  %'5'<'<$..;K^:\( $J)@ ;; + +w 6{{//69>O##L '>dkk>^>^&_#$7      % $}}C$,,LL %  % ! \"))#;FFH mmK0  LL.88r3) rerOrkrorrrsrvrfrmryrxrNrwN)NNNFN)r@rArBrCrDr!rrEr.r:Tensorrboolrr=rGrHrIs@r1rcrcjsG $(!Y!YC=!Y!YL9=1526"'1589||89!u||!4589!. 89 "%,,/ 89  89!.89 u||Xell3Xe_D E8989r3rcc^\rSrSrSS\S\\4U4SjjjrSS\RS\\RS\\RS\\ \RS \\ S \\ S \\RS \\RS \ \S\ \R\\ \R\R444SjjrSrU=r$)OPTDecoderLayerrerfcp>[TU]5 URUl[ XS9UlUR UlURUl[URUl [R"URURS9Ul[R"URUR UR"S9Ul[R"UR URUR"S9Ul[R"URURS9Ulg)N)rerfelementwise_affinerh)r-r.rjrkrc self_attndo_layer_norm_beforerOr activation_function activation_fnr LayerNormlayer_norm_elementwise_affineself_attn_layer_normruffn_dimrofc1fc2final_layer_norm)r/rerfr0s r1r.OPTDecoderLayer.__init__s ++%VI$*$?$?!~~ #F$>$>?$&LL NNv/S/S% !99T^^V^^&BTBTU99V^^T^^&BTBTU " T^^PVPtPt ur3r{r4r}r|r~ use_cacher6rr^rc DUn UR(aURU5nUR"SUUUUUUUS.U D6upn [RR XR UR S9nX-nUR(dURU5nURn URSURS55nUn UR(aURU5nURU5nURU5nURU5n[RR XR UR S9nX-RU 5nUR(dURU5nU4nU(aX4- nU(aX4- nU$)a4 Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`, *optional*): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_value (`Tuple(torch.FloatTensor)`, *optional*): cached past key and value projection states cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence.. )r{r|r6r4r}r~rrTrQ)rrrrrYrOrVshaperrrrrrr)r/r{r4r}r|r~rr6rr^residualself_attn_weightspresent_key_valuehidden_states_shapeoutputss r1r=OPTDecoderLayer.forwards<!  $ $ 55mDM?Cnn ? ')%)+/) ?  ? ; *; --m||VZVcVc-d  0 (( 55mDM,11%--b-2D2DR2HI    $ $ 11-@M/ **=9 /  --m||VZVcVc-d !1778KL (( 11-@M "  + +G  + +Gr3) rrrOrkrrrrrr)NNNFFNN)r@rArBrCr!rrEr.r:rrrrFrr FloatTensorr=rGrHrIs@r1rrs#vyvXc]vv(26268<,1$)3715S||S!.S"%,,/ S !u||!45 S $D> SD>Su//0S!.S-.S u  (51B1BEDUDU1U+V"WW XSSr3rcH\rSrSr\rSrSrS/rSr Sr Sr Sr Sr SrSrSrSrg)OPTPreTrainedModeli3modelTrcURRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[R5(aJUR R RS5 URR R5 gg)Nr)meanstdg?)reinit_std isinstancerruweightdatanormal_rizero_ Embedding padding_idxrfill_)r/rJrs r1 _init_weights OPTPreTrainedModel._init_weightsAskk"" fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>.  - - MM   $ $S ) KK   " " $.r3rN)r@rArBrCr! config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_supports_attention_backend_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cacherrGrr3r1rr3sJL&*#*+"&!N  $! %r3rcX^\rSrSrSrS\4U4SjjrSrSrSS\ \ RS4S \ RS \ RS \ S \ 4 S jjr\S\ RS\S\S\ R"S \ RS\4 Sj5r\SS\\ R*S\\ RS\\ RS \\\ R.S\\ R.S\\ S \\ S\\ S\\ S\\ R*S \\ RS\\S\ \\44Sjj5rSrU=r$) OPTDecoderiPz Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`OPTDecoderLayer`] Args: config: OPTConfig rec F>[TU]U5 URUlURUlURUlUR UlURUl[R"URURUR 5Ul [UR UR5UlURUR:wa0[R "URURSS9UlOSUlURUR:wa0[R "URURSS9UlOSUlUR&(a@UR((d/[R*"URUR,S9UlOSUl[R0"[3UR45Vs/sH n[7XS9PM sn5UlSUlUR=5 gs snf)NFrhr)rf)r-r.rO layerdrop pad_token_idrmax_position_embeddingsmax_target_positions vocab_sizerrword_embed_proj_dim embed_tokensr%rjembed_positionsru project_out project_inr_remove_final_layer_normrrr ModuleListrangenum_hidden_layersrlayersgradient_checkpointing post_init)r/reir0s r1r.OPTDecoder.__init__Xs  ~~ ))!..$*$B$B! ++LL):):FT]n%>>*'7 MM  ""&,,Q/ !+??AMnell;;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCK[Kr3rrrSrc UbUR5S:XaUnU$[R"U5Rn[R"X4XUR S9nUS:wa[R "USS9nU[R"X$R S9URSS5:-nUSSSS2SS24RUSSS5nUbUR5nURSn USS2SS2SS2SU 24USS2SSSS24RUR 5-n U S:Hn USS2SS2SS2SU 24RX5USS2SS2SS2SU 24'U$) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuerSrr )diagonalrrQr)r9r:rrfullrtriuarangerexpandclonerr\ masked_fill) r4rrrSrrr^rr mask_length padding_masks r1r@OPTDecoder._prepare_4d_causal_attention_mask_with_cache_positions}>  %.*<*<*>!*C(K*' E*..I** 0Y\j\q\qK!##jjqA 5<< >S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r3 input_ids head_maskrroutput_hidden_states return_dictr6r^rc UbUOURRnUbUOURRnUbUOURRnU bU OURRn USLUSL- (a [ S5eUR (a/UR(aU(a[RS5 SnUbURSURS5nUcURU5nSn U(aE[U[5(d0Sn [R "U5nUc[RS5 UbUR#5OSnU c/[$R&"XURS -UR(S 9n Uc=XRS -n[$R*"URSXR(S 9nUR-X%XU5nU c5[$R."US S 9n X-S - R15n U SS2US24n UR3X.U S 9nUR4bUR5U5nUUR7UR(5-nU(aS OSnU(aS OSnSn[9U/S/5HnunnUcM UR;5S[=UR>5:wdM7[ SUS[=UR>5SUR;5SS35e [AUR>5HunnU(aUU4- nUR(a([$RB"/5nUURD:aMLUR (a=UR(a,URGURHUUUbUUOSSUUU U 5 nOU"U4UU UbUUOSUUUU S.U D6nUSnU(aUU(aSOS nU(dMUUS 4- nM URJbURKU5nURLbURMU5nU(aUU4- nU(aUOSnU (aURO5n[QUUUUS9$)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(num_hidden_layers, num_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. for padding use -1. [What are position IDs?](../glossary#position-ids) cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. Contrarily to `position_ids`, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length. Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FrQTzPassing a tuple of `past_key_values` is deprecated and will be removed in Transformers v4.53.0. You should pass an instance of `DynamicCache` instead, e.g. `past_key_values=DynamicCache.from_legacy_cache(past_key_values)`.rr rr8)r6rrzThe `z` should be specified for z layers, but it is for .)r4r6r}r|r~rrr+last_hidden_staterr{ attentions))rer~rruse_return_dictrtrrVrprqrrrrrrfrom_legacy_cacherr:rronesr r;r<rrr\ziprlenr enumeraterandr_gradient_checkpointing_func__call__rrto_legacy_cacher)r/rr4rrrrr~rrr6rr^return_legacy_cacher seq_lengthr pos_embedsr{all_hidden_statesall_self_attnsnext_decoder_cache attn_mask mask_nameidx decoder_layerdropout_probability layer_outputs next_caches r1r=OPTDecoder.forwardsP2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?I   --i8M# Z??"& *<<_MO&##Y @O?Z?99;`a  !"\\ ]5H5H5K"KTaThThN  !),?,?,BBJ"ZZ (;(;A(> SgSghN.. >L]   <<A>L(9A=CCEL'+;+<(<=L)).Ye)f ?? & OOM:M% m6J6J(KK #7BD0d!%( k]$C Iy$>>#A&3t{{+;<$ {*DSEUDVW%NN,Q/03%D#,DKK"8 C#!m%55!}}&+jjn#&7**t}} $ A A!**!&/&;IcN% " ! !.! !#.!-7@7LYs^RV#2&7'#1 ! ! *!,M%28I1q%Q"  =#3"55S#9V  , 11-@M    ' ,,];M -!1 1 +4'$ #335J&+&+%   r3) rOrrrrrrrrrrr)F NNNNNNNNNNN)r@rArBrCrDr!r.rrrr:rrrr  staticmethodrErSrrrrFrrrrrrr=rGrHrIs@r1rrPs#y#J!"#( BellK78BllB B  B  BH4 444{{ 4  4  44l1515,0=A59$(,0/3&*3715R E,,-R !.R ELL) R "$u'8'8"9: R   1 12 R D>R $D>R 'tnR d^R u//0R !.R -.R  u-- .R R r3rc^\rSrSrS\4U4SjjrSrSrSr\ \ SS\ \ RS\ \ RS \ \ RS \ \\\ R"\4S \ \ R"S \ \S \ \S\ \S\ \S\ \ RS\ \ RS\\S\\\44Sjj55rSrU=r$)OPTModelirecd>[TU]U5 [U5UlUR 5 gr)r-r.rdecoderrr/rer0s r1r.OPTModel.__init__s&  !&)  r3c.URR$rr?rrs r1rOPTModel.get_input_embeddingss||(((r3c$XRlgrrCrs r1rOPTModel.set_input_embeddingss $) !r3cUR$rr?rs r1 get_decoderOPTModel.get_decoder ||r3rr4rrrrr~rrr6rr^rc  ~UbUOURRnUbUOURRnUbUOURRnU bU OURRn UR "SUUU UUUUUUSU S. U D6n [ U RU RU RU RS9$)NT rr4r6rrrrr~rrrrr) rer~rrr"r?rr rr{r!)r/rr4rrrrr~rrr6rr^decoder_outputss r1r=OPTModel.forwards"2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK->(?(F"GH +   1 12 + D>+ $D>+ 'tn+ d^+ u//0+ !.+ -.+  u-- .+ + r3r=c\rSrSrSrg)KwargsForCausalLMirN)r@rArBrCrGrr3r1rQrQs3r3rQc!^\rSrSrS/rU4SjrSrSrSrSr Sr S r \ \ SS \\R S \\R"S \\R"S \\\\R(\4S\\R(S\\R S\\S\\S\\S\\S\\R S\\R"S\\S\\\44Sjj55r\S5rSrU=r$)OPTForCausalLMizlm_head.weightc>[TU]U5 [U5Ul[R "UR URSS9UlUR5 gNFrh) r-r.r=rrrurrlm_headrr@s r1r.OPTForCausalLM.__init__sK  f% yy!;!;V=N=NUZ[  r3cBURRR$rrr?rrs r1r#OPTForCausalLM.get_input_embeddings'zz!!...r3c8XRRlgrrYrs r1r#OPTForCausalLM.set_input_embeddings**/ 'r3cUR$rrVrs r1get_output_embeddings$OPTForCausalLM.get_output_embeddings-rKr3cXlgrr`)r/new_embeddingss r1set_output_embeddings$OPTForCausalLM.set_output_embeddings0s% r3c$XRlgrrr?)r/r?s r1 set_decoderOPTForCausalLM.set_decoder3s $ r3c.URR$rrhrs r1rIOPTForCausalLM.get_decoder6szz!!!r3rr4rrrlabelsrr~rrr6rr^rc  "UbUOURRnU bU OURRn U bU OURRn URR "SUUU UUUUUU SU S. U D6nUR US5R5nSnUbEURUR5nUR"UU4SURR0U D6n[UUURURURS9$)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, OPTForCausalLM >>> model = OPTForCausalLM.from_pretrained("facebook/opt-350m") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious. I'm just a little bit of a weirdo." ```NTrMrrlosslogitsrr{r!r)rer~rr"rr?rVr]r\r loss_functionrrrr{r!)r/rr4rrrrmrr~rrr6rr^rrqrps r1r=OPTForCausalLM.forward9s7R2C1N-TXT_T_TqTq$8$D $++JjJj &1%# UH/oRSTRUR55v M1 g7fr?) index_selectr\r).0 past_statebeam_idxs r1 0OPTForCausalLM._reorder_cache..s1ncmU_--aZ=N=N1OPPcms7:)tuple)rryreordered_past layer_pasts ` r1_reorder_cacheOPTForCausalLM._reorder_caches8)J ncmnn N*r3)rVr NNNNNNNNNNNN) r@rArBrC_tied_weights_keysr.rrrarerirIrrrr:rFrrrrrrrrQrrr=r;rrGrHrIs@r1rSrSs*+/0&%"1515,0KO59-1$(,0/3&*3715P E,,-P !.P ELL) P "%U->->(?(F"GH P   1 12 P ))*P D>P $D>P 'tnP d^P u//0P !.P *+P  u,, -P P dr3rSa The OPT Model transformer with a sequence classification head on top (linear layer). [`OPTForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introc^\rSrSrS\4U4Sjjr\SS\\RS\\RS\\RS\\ \ \R\ 4S\\RS \\RS \\S \\S \\S \\S\\RS\ \\44Sjj5rSrSrSrU=r$)OPTForSequenceClassificationirec>[TU]U5 URUl[U5Ul[ R "URURSS9UlUR5 grU) r-r. num_labelsr=rrrurscorerr@s r1r.%OPTForSequenceClassification.__init__sT   ++f% YYv994??QVW  r3rr4rrrrmrr~rrr6rc U bU OURRn URUUUU UUUUU U S9 n U Sn URU 5nUbURSSunnOURSSunnURR cUS:wa [ S5eURR cSnOUbXRR :gRUR[R5n[R"URSUR[RS9nUU-RS5nO.Sn[RURR S 35 U[R"XRS 9U4nSnUGbURR"cUR$S:XaS URlOoUR$S:aNUR&[R(:XdUR&[R*:XaS URlOS URlURR"S :XaJ[-5nUR$S:Xa&U"UR/5UR/55nOU"UU5nOURR"S :Xa=[15nU"UR3SUR$5UR3S55nO-URR"S :Xa[55nU"UU5nU (dU4U SS-nUbU4U-$U$[7UUU R8U R:U R<S9$)ae labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). N rr4r6rrrr~rrrr+r z=Cannot handle batch sizes > 1 if no padding token is defined.rQ)rrSz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r regressionsingle_label_classificationmulti_label_classificationro)rer"rrrrrtr\rr:int32rargmaxrprqr0r@ problem_typerrSr<rEr squeezer rr rrr{r!)r/rr4rrrrmrr~rrr6transformer_outputsr{rqrrlast_non_pad_token non_pad_mask token_indices pooled_logitsrploss_fctoutputs r1r=$OPTForSequenceClassification.forwards*&1%!F!Fr!J !#    >>**+,ZZ  u||J}}MOaab   {{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#M$9$9$;V^^=MND#M6:D))-JJ+- 2 22t GUWY))-II,. v6#%(;AB(??F)-)9TGf$ Ev E/ /??-;;*55   r3cBURRR$rrYrs r1r1OPTForSequenceClassification.get_input_embeddingsr[r3c8XRRlgrrYrs r1r1OPTForSequenceClassification.set_input_embeddingsr^r3)rrrr:)r@rArBrCr!r.rrr:rFrrrrrrrr=rrrGrHrIs@r1rrsQy156:15KO59-1$(,0/3&*37\ E,,-\ !!2!23\ E--. \ "%U->->(?(F"GH \   1 12 \ ))*\ D>\ $D>\ 'tn\ d^\ u//0\  u66 7\ \ |/00r3rc^\rSrSrS\4U4Sjjr\SS\\RS\\RS\\RS\\ \ \R\ 4S\\RS \\RS \\RS \\S \\S \\S\\S\\RS\ \\44Sjj5rSrSrSrU=r$)OPTForQuestionAnsweringirec>[TU]U5 [U5Ul[R "UR S5UlUR5 gr*) r-r.r=rrrur qa_outputsrr@s r1r. OPTForQuestionAnswering.__init__s@  f% ))F$>$>B r3rr4rrrstart_positions end_positionsrr~rrr6rc U bU OURRn URUUUU UUUU U U S9 n U SnURU5nUR SSS9unnUR S5R 5nUR S5R 5nSnUbUb[UR55S:aUR S5n[UR55S:aUR S5nURS5nURSU5RUR5nURSU5RUR5n[US9nU"UU5nU"UU5nUU-S- nU (dUU4U SS-nUbU4U-$U$[UUUU RU RS 9$) a Example: ```python >>> from transformers import AutoTokenizer, OPTForQuestionAnswering >>> import torch >>> torch.manual_seed(4) # doctest: +IGNORE_RESULT >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> # note: we are loading a OPTForQuestionAnswering from the hub here, >>> # so the head will be randomly initialized, hence the predictions will be random >>> model = OPTForQuestionAnswering.from_pretrained("facebook/opt-350m") >>> question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" >>> inputs = tokenizer(question, text, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> answer_offset = len(tokenizer(question)[0]) >>> predict_answer_tokens = inputs.input_ids[ ... 0, answer_offset + answer_start_index : answer_offset + answer_end_index + 1 ... ] >>> predicted = tokenizer.decode(predict_answer_tokens) >>> predicted ' a nice puppet' ```Nrrr rQr8) ignore_indexr+)rp start_logits end_logitsr{r!)rer"rrsplitrr]r&rclampr\rr rr{r!)r/rr4rrrrrrr~rrr6rr{rqrr total_loss ignored_indexr start_lossend_lossrs r1r=OPTForQuestionAnswering.forwards`&1%->(?(F"GH _   1 12 _ "%"2"23_   0 01_ D>_ $D>_ 'tn_ d^_ u//0_  u22 3_ _ B/00r3r)rSr=rrr)r)CrDtypingrrrrrr:torch.utils.checkpointrtorch.nnr r r activationsr cache_utilsrr generationrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrrmodeling_outputsrrrrmodeling_utilsrrprocessing_utilsrutilsrrrrrconfiguration_optr!!torch.nn.attention.flex_attentionr"integrations.flex_attentionr# get_loggerr@rprr%Modulerfloatrarcrrrr=rQrSrr__all__rr3r1rs99 AA!.)>[ G&hh( !!;J   H %;BLL;H% II% <<% % << % U\\* %  %%.^9299^9BebiieP%%%8C #C L = != = @?,j>z'zz  m0#5m0 m0`o00o0o0d r3