o Zhb@s<dZddlZddlmZddlmZmZmZmZddl Z ddl Z ddl m Z ddl m Z ddlmZdd lmZmZmZdd lmZdd lmZmZmZdd lmZmZmZd dlmZe e!Z"eGdddeZ#Gddde j$Z%Gddde j$Z&Gddde j$Z'Gddde j$Z(Gddde j$Z)Gddde j$Z*Gddde j$Z+Gdd d e j$Z,Gd!d"d"e j$Z-Gd#d$d$e j$Z.Gd%d&d&e j$Z/Gd'd(d(e j$Z0Gd)d*d*e j$Z1eGd+d,d,eZ2eGd-d.d.e2Z3eGd/d0d0e2Z4ed1d2Gd3d4d4e2Z5ed5d2Gd6d7d7e2Z6gd8Z7dS)9zPyTorch Bros model.N) dataclass)ListOptionalTupleUnion)nn)CrossEntropyLoss)ACT2FN))BaseModelOutputWithPastAndCrossAttentions,BaseModelOutputWithPoolingAndCrossAttentionsTokenClassifierOutput)PreTrainedModel)apply_chunking_to_forward find_pruneable_heads_and_indicesprune_linear_layer) ModelOutputauto_docstringlogging) BrosConfigc@steZdZUdZdZeejed<dZ eejed<dZ eejed<dZ ee ejed<dZ ee ejed<dS)BrosSpadeOutputa Base class for outputs of token classification models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided) : Classification loss. initial_token_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.num_labels)`): Classification scores for entity initial tokens (before SoftMax). subsequent_token_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, sequence_length+1)`): Classification scores for entity sequence tokens (before SoftMax). hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. Nlossinitial_token_logitssubsequent_token_logits hidden_states attentions)__name__ __module__ __qualname____doc__rrtorch FloatTensor__annotations__rrrrrr$r$U/var/www/auris/lib/python3.10/site-packages/transformers/models/bros/modeling_bros.pyr)s rc2eZdZfddZdejdejfddZZS)BrosPositionalEmbedding1DcsDtt||j|_ddtd|jd|j}|d|dS)Nri'g@inv_freq)superr'__init__dim_bbox_sinusoid_emb_1dr!arangeregister_buffer)selfconfigr) __class__r$r%r+Ls z"BrosPositionalEmbedding1D.__init__pos_seqreturncCsX|}|\}}}||||d|jddd|jd}tj||gdd}|S)Nrdim)sizeviewr)r,r!catsincos)r/r3Zseq_sizeb1b2Zb3Z sinusoid_inpZpos_embr$r$r%forwardVs  (z!BrosPositionalEmbedding1D.forwardrrrr+r!Tensorr@ __classcell__r$r$r1r%r'Is  r'cr&)BrosPositionalEmbedding2Dcs.tt||j|_t||_t||_dSN)r*rDr+dim_bboxr' x_pos_emb y_pos_embr/r0r1r$r%r+_s z"BrosPositionalEmbedding2D.__init__bboxr4cCsdg}t|jD]!}|ddkr|||d|fq|||d|fqtj|dd}|S)Nr5r.r6r7)rangerFappendrGrHr!r;)r/rJstacki bbox_pos_embr$r$r%r@fs z!BrosPositionalEmbedding2D.forwardrAr$r$r1r%rD^s rDcs,eZdZfddZdejfddZZS)BrosBboxEmbeddingscs2tt|t||_tj|j|jdd|_ dS)NF)bias) r*rPr+rDbbox_sinusoid_embrLinearZdim_bbox_sinusoid_emb_2dZdim_bbox_projectionbbox_projectionrIr1r$r%r+rs zBrosBboxEmbeddings.__init__rJcCs\|dd}|dddddddf|dddddddf}||}||}|S)Nrr) transposerRrT)r/rJZbbox_tZbbox_posrOr$r$r%r@ws 8  zBrosBboxEmbeddings.forwardrAr$r$r1r%rPqs rPc sheZdZdZfddZ     ddeejdeejdeejd eejd ed ejf d d Z Z S)BrosTextEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.csttj|j|j|jd|_t|j|j|_ t|j |j|_ tj |j|j d|_ t|j|_t|dd|_|dt|jd|jdtj|jtj|jjdd d dS) N) padding_idxZepsposition_embedding_typeabsolute position_ids)rr6token_type_idsdtypedeviceF) persistent)r*r+r EmbeddingZ vocab_size hidden_sizeZ pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingsZtype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsDropouthidden_dropout_probdropoutgetattrrYr.r!r-expandzerosr[r9longr_rIr1r$r%r+s"  zBrosTextEmbeddings.__init__Nr input_idsr\r[ inputs_embedspast_key_values_lengthr4c Cs|dur |}n|dd}|d}|dur&|jdd|||f}|durPt|drE|jddd|f}||d|} | }n tj|tj|jjd}|durY| |}| |} || } |j dkrp| |} | | 7} | | } || } | S)Nr6rr\rr]rZ)r9r[hasattrr\rmr!rnror_rcrfrYrergrk) r/rpr\r[rqrr input_shape seq_lengthbuffered_token_type_ids buffered_token_type_ids_expandedrf embeddingsrer$r$r%r@s,        zBrosTextEmbeddings.forward)NNNNr) rrrr r+rr!rBintr@rCr$r$r1r%rVs* rVcseZdZfddZdejfddZ      ddejd ejd eejd eejd eejd eejdeeeej deejdeejfddZ Z S)BrosSelfAttentioncst|j|jdkrt|dstd|jd|jd|j|_t|j|j|_|j|j|_t |j|j|_ t |j|j|_ t |j|j|_ t |j|_t|dd|_|jdksf|jd krw|j|_t d |jd |j|_|j|_dS) NrZembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()rYrZ relative_keyrelative_key_queryr5r)r*r+rbnum_attention_headsrs ValueErrorryattention_head_size all_head_sizerrSquerykeyvalueriZattention_probs_dropout_probrkrlrYrdradistance_embedding is_decoderrIr1r$r%r+s&   zBrosSelfAttention.__init__xcCs6|dd|j|jf}|j|}|ddddS)Nr6rr5rr )r9r~rr:permute)r/rZ new_x_shaper$r$r%transpose_for_scoress  z&BrosSelfAttention.transpose_for_scoresNFrrOattention_mask head_maskencoder_hidden_statesencoder_attention_maskpast_key_valueoutput_attentionsr4c Cs||} |du} | r|dur|d} |d} |}nP| r/|||} |||} |}n;|durZ|||} |||} tj|d| gdd} tj|d| gdd} n|||} |||} || } |jrv| | f}t| | dd}|j dks|j dkr| d}tj |tj |j d dd}tj |tj |j d dd}||}|||jd}|j| jd }|j dkrtd | |}||}n|j dkrtd | |}td | |}|||}| j\}}}}|||||}|gd }td| |f}||}|t|j}|dur||}tjdd|}||}|dur4||}t|| }|dddd}| dd|jf}|j|}|r\||fn|f}|jrh||f}|S)Nrrr5r7r6r|r}r])r^zbhld,lrd->bhlrzbhrd,lrd->bhlr)r5rrr zbnid,bijd->bnijr )rrrrr!r;rmatmulrUrYr9r-ror_r:rrdtor^ZeinsumshapermathsqrtrrZSoftmaxrk contiguousr)r/rrOrrrrrrZmixed_query_layerZis_cross_attention key_layerZ value_layer query_layerZattention_scoresruZposition_ids_lZposition_ids_rZdistanceZpositional_embeddingZrelative_position_scoresZrelative_position_scores_queryZrelative_position_scores_key batch_sizeZn_headZd_headZbbox_pos_scoresZattention_probsZ context_layerZnew_context_layer_shapeoutputsr$r$r%r@sn              zBrosSelfAttention.forwardNNNNNF) rrrr+r!rBrrrr"r@rCr$r$r1r%rzs8    rzc8eZdZfddZdejdejdejfddZZS)BrosSelfOutputcsBtt|j|j|_tj|j|jd|_t|j |_ dSNrX) r*r+rrSrbdensergrhrirjrkrIr1r$r%r+N zBrosSelfOutput.__init__r input_tensorr4cC&||}||}|||}|SrErrkrgr/rrr$r$r%r@T  zBrosSelfOutput.forwardrAr$r$r1r%rM $rcseZdZfddZddZ      ddejdejd eejd eejd eejd eejd eeeej dee deejfddZ Z S) BrosAttentioncs*tt||_t||_t|_dSrE)r*r+rzr/routputset pruned_headsrIr1r$r%r+\s    zBrosAttention.__init__cCst|dkrdSt||jj|jj|j\}}t|jj||j_t|jj||j_t|jj ||j_ t|j j |dd|j _ |jjt||j_|jj|jj|j_ |j ||_dS)Nrrr7)lenrr/r~rrrrrrrrrunion)r/headsindexr$r$r% prune_headsbs zBrosAttention.prune_headsNFrrOrrrrrrr4c Cs@|j||||||||d} || d|} | f| dd} | S)NrrOrrrrrrrr)r/r) r/rrOrrrrrrZ self_outputsattention_outputrr$r$r%r@ws  zBrosAttention.forwardr) rrrr+rr!rBrrr"boolr@rCr$r$r1r%r[s8   rcr&)BrosIntermediatecsDtt|j|j|_t|jt rt |j|_ dS|j|_ dSrE) r*r+rrSrbintermediate_sizer isinstanceZ hidden_actstrr intermediate_act_fnrIr1r$r%r+s   zBrosIntermediate.__init__rr4cCs||}||}|SrE)rr)r/rr$r$r%r@s  zBrosIntermediate.forwardrAr$r$r1r%rs rcr) BrosOutputcsBtt|j|j|_tj|j|jd|_t |j |_ dSr) r*r+rrSrrbrrgrhrirjrkrIr1r$r%r+rzBrosOutput.__init__rrr4cCrrErrr$r$r%r@rzBrosOutput.forwardrAr$r$r1r%rrrcseZdZfddZ      ddejdejdeejdeejd eejd eejd eeeejd ee d eejfddZ ddZ Z S) BrosLayercsnt|j|_d|_t||_|j|_|j|_|jr+|js&t|dt||_ t ||_ t ||_ dS)Nrz> should be used as a decoder model if cross attention is added)r*r+chunk_size_feed_forward seq_len_dimr attentionradd_cross_attention Exceptioncrossattentionr intermediaterrrIr1r$r%r+s    zBrosLayer.__init__NFrrOrrrrrrr4c  Cs|dur |ddnd} |j|||||| d} | d} |jr)| dd} | d} n| dd} d}|jrp|durpt|drEtd|d|durO|d dnd}|| ||||||}|d} | |dd} |d}| |} t|j|j|j| }|f| } |jr| | f} | S) Nr5)rOrrrrrrr6rz'If `encoder_hidden_states` are passed, z` has to be instantiated with cross-attention layers by setting `config.add_cross_attention=True`r) rrrsrrrfeed_forward_chunkrr)r/rrOrrrrrrZself_attn_past_key_valueZself_attention_outputsrrZpresent_key_valueZcross_attn_present_key_valueZcross_attn_past_key_valueZcross_attention_outputs layer_outputr$r$r%r@sX         zBrosLayer.forwardcCs||}|||}|SrE)rr)r/rZintermediate_outputrr$r$r%rs  zBrosLayer.feed_forward_chunkr) rrrr+r!rBrr"rrr@rrCr$r$r1r%rs8   ErcseZdZfddZ         ddejdejdeejd eejd eejd eejd eeeejd ee dee dee dee de eeje ffddZ Z S) BrosEncodercs4t|_tfddtjD|_dS)Ncsg|]}tqSr$)r).0_r0r$r% sz(BrosEncoder.__init__..)r*r+r0rZ ModuleListrKnum_hidden_layerslayerrIr1rr%r+ s $zBrosEncoder.__init__NFTrrOrrrrpast_key_values use_cacheroutput_hidden_states return_dictr4c  Cs^| rdnd} | r dnd} | r|jjrdnd}|rdnd}t|jD]m\}}| r,| |f} |dur4||nd}|dur>||nd}t|jddra|jra|rStdd}||j ||||||| }n ||||||||| d}|d}|rz||df7}| r| |df} |jjr||d f}q!| r| |f} | st d d ||| | |fDSt ||| | |d S) Nr$Zgradient_checkpointingFzh`use_cache=True` is incompatible with `config.gradient_checkpointing=True`. Setting `use_cache=False`...rrr6rr5css|] }|dur|VqdSrEr$)rvr$r$r% Rsz&BrosEncoder.forward..)last_hidden_staterrrcross_attentions) r0r enumeraterrlZtrainingloggerwarningZ_gradient_checkpointing_func__call__tupler )r/rrOrrrrrrrrrZall_hidden_statesZall_self_attentionsZall_cross_attentionsZnext_decoder_cacherNZ layer_moduleZlayer_head_maskrZ layer_outputsr$r$r%r@sz        zBrosEncoder.forward) NNNNNNFFT)rrrr+r!rBrr"rrrr r@rCr$r$r1r%r sH       rcr&) BrosPoolercs*tt|j|j|_t|_dSrE)r*r+rrSrbrZTanh activationrIr1r$r%r+hs zBrosPooler.__init__rr4cCs(|dddf}||}||}|S)Nr)rr)r/rZfirst_token_tensor pooled_outputr$r$r%r@ms  zBrosPooler.forwardrAr$r$r1r%rgs rcr&)BrosRelationExtractorcst|j|_|j|_|j|_|j|_t|j|_ t |j|j|j|_ t |j|j|j|_ t td|j|_dS)Nr)r*r+ n_relationsrbbackbone_hidden_sizehead_hidden_sizeZclassifier_dropout_probrridroprSrr Parameterr!rn dummy_noderIr1r$r%r+ws zBrosRelationExtractor.__init__rrc Cs|||}|jdd|dd}tj||gdd}|||}| |d|d|j |j }| |d|d|j |j }t | dddd| dddd}|S)NrrZaxisr5r )rrrZ unsqueezerepeatr9r!r;rr:rrrr)r/rrZ dummy_vecZrelation_scorer$r$r%r@s zBrosRelationExtractor.forwardrAr$r$r1r%rvs rc@seZdZeZdZddZdS)BrosPreTrainedModelbroscCst|tjr |jjjd|jjd|jdur|jj dSdSt|tj rC|jjjd|jjd|j durA|jj|j  dSdSt|tj rX|jj |jj ddSdS)zInitialize the weightsr()meanZstdNg?)rrrSweightdataZnormal_r0Zinitializer_rangerQZzero_rarWrgZfill_)r/moduler$r$r% _init_weightss      z!BrosPreTrainedModel._init_weightsN)rrrrZ config_classZbase_model_prefixrr$r$r$r%rs rc"seZdZdfdd ZddZddZdd Ze dd ee j d ee j d ee j dee j dee j dee j dee j dee j dee j dee e j dee dee dee dee deee j effddZZS) BrosModelTcsNt|||_t||_t||_t||_|rt |nd|_ | dS)zv add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer N) r*r+r0rVrxrPbbox_embeddingsrencoderrpooler init_weights)r/r0Zadd_pooling_layerr1r$r%r+s     zBrosModel.__init__cCs|jjSrErxrc)r/r$r$r%get_input_embeddingsszBrosModel.get_input_embeddingscCs ||j_dSrEr)r/rr$r$r%set_input_embeddingss zBrosModel.set_input_embeddingscCs*|D]\}}|jj|j|qdS)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsrrrr)r/Zheads_to_prunerrr$r$r% _prune_headsszBrosModel._prune_headsNrprJrr\r[rrqrrrrrrrr4c" Cs| dur| n|jj} | dur| n|jj} |dur|n|jj}|jjr-| dur(| n|jj} nd} |dur;|dur;td|durD|}n|durQ|dd}ntd|dur]td|\}}|durh|jn|j}| durx| ddj dnd}|durt j ||d }|durt |j d r|j jddd|f}|||}|}n t j|t j|d }||||}|jjr|dur|\}}}||f}| durt j ||d } || }nd}|||jj}|j |||||d }|j dd kr|ddddgdf}||jj}||}|j||||||| | | | |d }|d} |jdur+|| nd}!|s:| |!f|ddSt| |!|j|j|j|jdS)a bbox ('torch.FloatTensor' of shape '(batch_size, num_boxes, 4)'): Bounding box coordinates for each token in the input sequence. Each bounding box is a list of four values (x1, y1, x2, y2), where (x1, y1) is the top left corner, and (x2, y2) is the bottom right corner of the bounding box. Examples: ```python >>> import torch >>> from transformers import BrosProcessor, BrosModel >>> processor = BrosProcessor.from_pretrained("jinho8345/bros-base-uncased") >>> model = BrosModel.from_pretrained("jinho8345/bros-base-uncased") >>> encoding = processor("Hello, my dog is cute", add_special_tokens=False, return_tensors="pt") >>> bbox = torch.tensor([[[0, 0, 1, 1]]]).repeat(1, encoding["input_ids"].shape[-1], 1) >>> encoding["bbox"] = bbox >>> outputs = model(**encoding) >>> last_hidden_states = outputs.last_hidden_state ```NFzDYou cannot specify both input_ids and inputs_embeds at the same timer6z5You have to specify either input_ids or inputs_embedszYou have to specify bboxrr5)r_r\r])rpr[r\rqrr)rrr5rr5r rr ) rOrrrrrrrrrr)rZ pooler_outputrrrr)r0rruse_return_dictrrrr9r_rr!Zonesrsrxr\rmrnroZget_extended_attention_maskZinvert_attention_maskZ get_head_maskrZ bbox_scalerrrr rrrr)"r/rprJrr\r[rrqrrrrrrrrtrrur_rrrvrwZextended_attention_maskZencoder_batch_sizeZencoder_sequence_lengthrZencoder_hidden_shapeZencoder_extended_attention_maskZembedding_outputZ scaled_bboxZbbox_position_embeddingsZencoder_outputssequence_outputrr$r$r%r@s)        zBrosModel.forward)T)NNNNNNNNNNNNNN)rrrr+rrrrrr!rBrr"rrrr r@rCr$r$r1r%rsf     rceZdZdgZfddZe            ddeejdeejdeejdeejd eejd eejd eejd eejd eejdee dee dee de e eje ffddZ ZS)BrosForTokenClassificationrcs^t||j|_t||_t|dr|jn|j}t ||_ t |j |j|_ |dSNclassifier_dropout)r*r+ num_labelsrrrsrrjrrirkrSrb classifierrr/r0rr1r$r%r+ds    z#BrosForTokenClassification.__init__NrprJrbbox_first_token_maskr\r[rrqlabelsrrrr4c  Cs| dur| n|jj} |j|||||||| | | d } | d}||}||}d}| durXt}|durK|d}||d|j|| d|}n ||d|j| d}| sn|f| dd}|durl|f|S|St||| j | j dS)a bbox ('torch.FloatTensor' of shape '(batch_size, num_boxes, 4)'): Bounding box coordinates for each token in the input sequence. Each bounding box is a list of four values (x1, y1, x2, y2), where (x1, y1) is the top left corner, and (x2, y2) is the bottom right corner of the bounding box. bbox_first_token_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to indicate the first token of each bounding box. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. Examples: ```python >>> import torch >>> from transformers import BrosProcessor, BrosForTokenClassification >>> processor = BrosProcessor.from_pretrained("jinho8345/bros-base-uncased") >>> model = BrosForTokenClassification.from_pretrained("jinho8345/bros-base-uncased") >>> encoding = processor("Hello, my dog is cute", add_special_tokens=False, return_tensors="pt") >>> bbox = torch.tensor([[[0, 0, 1, 1]]]).repeat(1, encoding["input_ids"].shape[-1], 1) >>> encoding["bbox"] = bbox >>> outputs = model(**encoding) ```N) rJrr\r[rrqrrrrr6r5rlogitsrr) r0rrrkrrr:rr rr)r/rprJrrr\r[rrqrrrrrrrrloss_fctrr$r$r%r@qsD,   z"BrosForTokenClassification.forward NNNNNNNNNNNNrrr"_keys_to_ignore_on_load_unexpectedr+rrr!rBrrrr r@rCr$r$r1r%r`sV       ra Bros Model with a token classification head on top (initial_token_layers and subsequent_token_layer on top of the hidden-states output) e.g. for Named-Entity-Recognition (NER) tasks. The initial_token_classifier is used to predict the first token of each entity, and the subsequent_token_classifier is used to predict the subsequent tokens within an entity. Compared to BrosForTokenClassification, this model is more robust to serialization errors since it predicts next token from one token. )Z custom_introc seZdZdgZfddZe             ddeejdeejdeejdeejd eejd eejd eejd eejd eejdeejdee dee dee de e eje ffddZ ZS)!BrosSpadeEEForTokenClassificationrc st|||_|j|_|j|_|j|_t||_t |dr"|j n|j }t t |t |j|jt |t |j|j|_t||_|dSr)r*r+r0rrrbrrrrsrrjrZ SequentialrirSinitial_token_classifierrsubsequent_token_classifierrrr1r$r%r+s    z*BrosSpadeEEForTokenClassification.__init__NrprJrrr\r[rrqinitial_token_labelssubsequent_token_labelsrrrr4c Cs| dur| n|jj} |j|||||||| | | d }|d}|dd}||dd}|||d}d|}|j\}}|j }t j |t |dg |gdd}||dddddft |jj}t ||dj |t jd}||dddddft |jj}|d}d}| dur| durt}| d} |dur|d}||d|j|| |}n ||d|j| }| d} ||d|d|| |}||}| s||f|dd}|dur|f|S|St||||j|jd S) a> bbox ('torch.FloatTensor' of shape '(batch_size, num_boxes, 4)'): Bounding box coordinates for each token in the input sequence. Each bounding box is a list of four values (x1, y1, x2, y2), where (x1, y1) is the top left corner, and (x2, y2) is the bottom right corner of the bounding box. bbox_first_token_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to indicate the first token of each bounding box. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. initial_token_labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for the initial token classification. subsequent_token_labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for the subsequent token classification. Examples: ```python >>> import torch >>> from transformers import BrosProcessor, BrosSpadeEEForTokenClassification >>> processor = BrosProcessor.from_pretrained("jinho8345/bros-base-uncased") >>> model = BrosSpadeEEForTokenClassification.from_pretrained("jinho8345/bros-base-uncased") >>> encoding = processor("Hello, my dog is cute", add_special_tokens=False, return_tensors="pt") >>> bbox = torch.tensor([[[0, 0, 1, 1]]]).repeat(1, encoding["input_ids"].shape[-1], 1) >>> encoding["bbox"] = bbox >>> outputs = model(**encoding) ```N rprJrr\r[rrqrrrrrrr_r^r6r5)rrrrr)r0rrrUrrrsqueezerr_r!r;rnrr masked_fillfinfor^mineyer:rrrrr)r/rprJrrr\r[rrqr r rrrrlast_hidden_statesrrZinv_attention_maskrmax_seq_lengthr_Zinvalid_token_maskself_token_maskZsubsequent_token_maskrrZinitial_token_lossZsubsequent_token_lossrr$r$r%r@sp1  &     z)BrosSpadeEEForTokenClassification.forward) NNNNNNNNNNNNN)rrrrr+rrr!rBrrrrr@rCr$r$r1r%rs\      rz Bros Model with a token classification head on top (a entity_linker layer on top of the hidden-states output) e.g. for Entity-Linking. The entity_linker is used to predict intra-entity links (one entity to another entity). cr)!BrosSpadeELForTokenClassificationrcsxt|||_|j|_|j|_|j|_t||_t |dr"|j n|j t ||_ |dSt ||_ |dSr)r*r+r0rrrbrrrrsrrjr entity_linkerrrIr1r$r%r+ms      z*BrosSpadeELForTokenClassification.__init__NrprJrrr\r[rrqrrrrr4c  Cs~| dur| n|jj} |j|||||||| | | d } | d}|dd}|||d}d}| durt}|j\}}|j }t ||dj |t j d}|d}t j|t j|dgt j |dgdd}||dddddft |jj}||dddddft |jj}||d|d|| d|}| s|f| d d}|dur|f|S|St||| j| jd S) a bbox ('torch.FloatTensor' of shape '(batch_size, num_boxes, 4)'): Bounding box coordinates for each token in the input sequence. Each bounding box is a list of four values (x1, y1, x2, y2), where (x1, y1) is the top left corner, and (x2, y2) is the bottom right corner of the bounding box. bbox_first_token_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to indicate the first token of each bounding box. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. Examples: ```python >>> import torch >>> from transformers import BrosProcessor, BrosSpadeELForTokenClassification >>> processor = BrosProcessor.from_pretrained("jinho8345/bros-base-uncased") >>> model = BrosSpadeELForTokenClassification.from_pretrained("jinho8345/bros-base-uncased") >>> encoding = processor("Hello, my dog is cute", add_special_tokens=False, return_tensors="pt") >>> bbox = torch.tensor([[[0, 0, 1, 1]]]).repeat(1, encoding["input_ids"].shape[-1], 1) >>> encoding["bbox"] = bbox >>> outputs = model(**encoding) ```Nr rrr r6r]rr5r)r0rrrUrrr rrr_r!rrrr:r;rnrrr^rr rr)r/rprJrrr\r[rrqrrrrrrrrrrrr_rmaskrr$r$r%r@{sR+   (($z)BrosSpadeELForTokenClassification.forwardrrr$r$r1r%rdsV      r)rrrrr)8r r dataclassesrtypingrrrrr!Ztorch.utils.checkpointrZtorch.nnrZ activationsr Zmodeling_outputsr r r Zmodeling_utilsrZ pytorch_utilsrrrutilsrrrZconfiguration_brosrZ get_loggerrrrModuler'rDrPrVrzrrrrrrrrrrrrr__all__r$r$r$r%s`       B 7Z^"2g  r