fTh`cSrSSKJr SSKJrJrJr SSKrSSKJrJ r SSK J r SSK J r SS KJrJrJr S S KJr S S KJr \R,"\5r\"SS\55r\"SS\55r\"SS\55r\"SS\ 55r"SS\5r"SS\5r"SS\5r"SS\5r "SS\5r!\"S S!9"S"S#\55r"\"S$S!9"S%S&\ 55r#\"S'S!9"S(S)\!55r$/S*Qr%g)+z5PyTorch DPR model for Open Domain Question Answering.) dataclass)OptionalTupleUnionN)Tensornn)BaseModelOutputWithPooling)PreTrainedModel) ModelOutputauto_docstringlogging) BertModel) DPRConfigc\rSrSr%Sr\R \S'Sr\ \ \R S4\S'Sr \ \ \R S4\S'Sr g) DPRContextEncoderOutput*a# Class for outputs of [`DPRQuestionEncoder`]. Args: pooler_output (`torch.FloatTensor` of shape `(batch_size, embeddings_size)`): The DPR encoder outputs the *pooler_output* that corresponds to the context representation. Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer. This output is to be used to embed contexts for nearest neighbors queries with questions embeddings. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. pooler_outputN. hidden_states attentions __name__ __module__ __qualname____firstlineno____doc__torch FloatTensor__annotations__rrrr__static_attributes__r\/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/dpr/modeling_dpr.pyrr*S*$$$=AM8E%"3"3S"89:A:>Ju00#567>r$rc\rSrSr%Sr\R \S'Sr\ \ \R S4\S'Sr \ \ \R S4\S'Sr g) DPRQuestionEncoderOutputFa# Class for outputs of [`DPRQuestionEncoder`]. Args: pooler_output (`torch.FloatTensor` of shape `(batch_size, embeddings_size)`): The DPR encoder outputs the *pooler_output* that corresponds to the question representation. Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer. This output is to be used to embed questions for nearest neighbors queries with context embeddings. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. rN.rrrrrr$r%r(r(Fr&r$r(c\rSrSr%Sr\R \S'Sr\ \R \S'Sr \ \R \S'Sr \ \ \R S4\S'Sr \ \ \R S4\S 'S rg) DPRReaderOutputba Class for outputs of [`DPRQuestionEncoder`]. Args: start_logits (`torch.FloatTensor` of shape `(n_passages, sequence_length)`): Logits of the start index of the span for each passage. end_logits (`torch.FloatTensor` of shape `(n_passages, sequence_length)`): Logits of the end index of the span for each passage. relevance_logits (`torch.FloatTensor` of shape `(n_passages, )`): Outputs of the QA classifier of the DPRReader that corresponds to the scores of each passage to answer the question, compared to all the other passages. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. start_logitsN end_logitsrelevance_logits.rrr)rrrrrr r!r"r.rr/rrrr#rr$r%r+r+bs~0###.2J**+248hu0018=AM8E%"3"3S"89:A:>Ju00#567>r$r+c\rSrSrSrSrSrg)DPRPreTrainedModelTc[U[R5(akURRR SUR RS9 URb%URRR5 gg[U[R5(axURRR SUR RS9 URb2URRURR5 gg[U[R5(aJURRR5 URRRS5 gg)zInitialize the weightsg)meanstdNg?) isinstancerLinearweightdatanormal_configinitializer_rangebiaszero_ Embedding padding_idx LayerNormfill_)selfmodules r% _init_weights DPRPreTrainedModel._init_weightss fbii ( ( MM   & &CT[[5R5R & S{{&   &&('  - - MM   & &CT[[5R5R & S!!- ""6#5#56<<>.  - - KK   " " $ MM   $ $S ).r$rN)rrrr_supports_sdparEr#rr$r%r1r1s N*r$r1c^\rSrSrSrS\4U4SjjrSS\S\\S\\S\\S \ S \ S \ S \ \ \ \S 444Sjjr \S \4Sj5rSrU=r$) DPREncoder bert_modelr;c>[TU]U5 [USS9UlURRR S::a [ S5eURUlURS:aD[R"URRR UR5Ul UR5 g)NF)add_pooling_layerrz!Encoder hidden_size can't be zero) super__init__rrKr; hidden_size ValueErrorprojection_dimrr7 encode_proj post_initrCr; __class__s r%rODPREncoder.__init__s  #FeD ?? ! ! - - 2@A A$33    "!yy)?)?)K)KVMbMbcD  r$ input_idsattention_masktoken_type_ids inputs_embedsoutput_attentionsoutput_hidden_states return_dictreturn.c URUUUUUUUS9nUSn U SS2SSS24n URS:aURU 5n U(d X4USS-$[U U URUR S9$)NrXrYrZr[r\r]r^rr)last_hidden_staterrr)rKrRrSr rr) rCrXrYrZr[r\r]r^outputssequence_output pooled_outputs r%forwardDPREncoder.forwards//))'/!5#" "!*'1a0    " ,,];M#3gabkA A)-'!//))   r$cURS:aURR$URRR $)Nr)rRrS out_featuresrKr;rP)rCs r%embeddings_sizeDPREncoder.embeddings_sizes8    "##00 0%%111r$)rKrSrR)NNNFFF)rrrrbase_model_prefixrrOrrboolrr rrfpropertyintrjr# __classcell__rVs@r%rIrIs$ y ,0+/*."'%*!! ! !(! !( !  ' !  ! #! !  )5+== >! F222r$rIc^\rSrSrSrS\4U4SjjrSS\S\S\\S\ S \ S \ S \ \ \ \S 444S jjr SrU=r$)DPRSpanPredictorencoderr;c$>[TU]U5 [U5Ul[R "URR S5Ul[R "URR S5UlUR5 g)Nrr) rNrOrIrurr7rj qa_outputs qa_classifierrTrUs r%rODPRSpanPredictor.__init__s_  !&) ))DLL$@$@!DYYt||'C'CQG r$rXrYr[r\r]r^r_.c FUbUR5OUR5SSupxURUUUUUUS9n U Sn URU 5n U RSSS9upU R S5R 5n U R S5R 5n UR U SS2SSS245nU RXx5n U RXx5n URU5nU(d XU4U SS-$[U U UU RU RS9$)Nr)rYr[r\r]r^rr)dim)r-r.r/rr) sizerurwsplitsqueeze contiguousrxviewr+rr)rCrXrYr[r\r]r^ n_passagessequence_lengthrcrdlogitsr-r.r/s r%rfDPRSpanPredictor.forwardsD;D:Oinn&6UbUgUgUijlklUm# ,, )'/!5#  "!*1#)<<r<#: #++B/::< ''+668 --oaAg.FG$((E __ZA +00< .>?'!"+M M%!-!//))   r$)rurxrw)NFFF)rrrrrlrrOrrrmrr+rrfr#rprqs@r%rsrss!y+/"'%*!+ + +  ' +  + # + +  fck 22 3+ + r$rsc$\rSrSrSr\rSrSrSr g)DPRPretrainedContextEncoderiz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. N ctx_encoderr rrrrrr config_classload_tf_weightsrlr#rr$r%rrs LO%r$rc$\rSrSrSr\rSrSrSr g)DPRPretrainedQuestionEncoderirNquestion_encoderrrrr$r%rrs LO*r$rc$\rSrSrSr\rSrSrSr g)DPRPretrainedReaderi$rNspan_predictorrrrr$r%rr$s LO(r$rzf The bare DPRContextEncoder transformer outputting pooler outputs as context representations. ) custom_introc^\rSrSrS\4U4Sjjr\SS\\S\\S\\S\\S\\ S \\ S \\ S \ \ \ \S 444S jj5r SrU=r$)DPRContextEncoderi4r;cp>[TU]U5 Xl[U5UlUR 5 gN)rNrOr;rIrrTrUs r%rODPRContextEncoder.__init__:s,   %f- r$rXrYrZr[r\r]r^r_.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUbUR 5nO"UbUR 5SSnO [ S5eUb UR O UR n Uc/Uc[R"XS9OXRR:gnUc$[R"U[RU S9nURUUUUUUUS9n U(dU SS$[U RU RU R S 9$) a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. To match pretraining, DPR input sequence should be formatted with [CLS] and [SEP] tokens as follows: (a) For sequence pairs (for a pair title+text for example): ``` tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP] token_type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 ``` (b) For single sequences (for a question for example): ``` tokens: [CLS] the dog is hairy . [SEP] token_type_ids: 0 0 0 0 0 0 0 ``` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) Examples: ```python >>> from transformers import DPRContextEncoder, DPRContextEncoderTokenizer >>> tokenizer = DPRContextEncoderTokenizer.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") >>> model = DPRContextEncoder.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") >>> input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"] >>> embeddings = model(input_ids).pooler_output ```NDYou cannot specify both input_ids and inputs_embeds at the same timer{5You have to specify either input_ids or inputs_embedsdevicedtyperrarrrr)r;r\r]use_return_dictrQr}rr ones pad_token_idzeroslongrrrrr rCrXrYrZr[r\r]r^ input_shaperrcs r%rfDPRContextEncoder.forwardAslb2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "#..*K  &',,.s3KTU U%.%:!! @T@T  !$ ;6;;#;#;;   !"[[EJJvVN""))'/!5## 12; &!//w?T?Tahasas  r$)r;rNNNNNNN)rrrrrrOr rrrmrrrrfr#rprqs@r%rr4s y'++/+/*.,0/3&*X F#X !(X !( X  ' X $D> X 'tnX d^X  &fck(:: ;X X r$rzh The bare DPRQuestionEncoder transformer outputting pooler outputs as question representations. c^\rSrSrS\4U4Sjjr\SS\\S\\S\\S\\S\\ S \\ S \\ S \ \ \ \S 444S jj5r SrU=r$)DPRQuestionEncoderir;cp>[TU]U5 Xl[U5UlUR 5 gr)rNrOr;rIrrTrUs r%rODPRQuestionEncoder.__init__s,   *6 2 r$rXrYrZr[r\r]r^r_.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb"UR X5 UR 5nO"UbUR 5SSnO [ S5eUb URO URn Uc/Uc[R"XS9OXRR:gnUc$[R"U[RU S9nURUUUUUUUS9n U(dU SS$[U RU R U R"S 9$) a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. To match pretraining, DPR input sequence should be formatted with [CLS] and [SEP] tokens as follows: (a) For sequence pairs (for a pair title+text for example): ``` tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP] token_type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 ``` (b) For single sequences (for a question for example): ``` tokens: [CLS] the dog is hairy . [SEP] token_type_ids: 0 0 0 0 0 0 0 ``` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) Examples: ```python >>> from transformers import DPRQuestionEncoder, DPRQuestionEncoderTokenizer >>> tokenizer = DPRQuestionEncoderTokenizer.from_pretrained("facebook/dpr-question_encoder-single-nq-base") >>> model = DPRQuestionEncoder.from_pretrained("facebook/dpr-question_encoder-single-nq-base") >>> input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"] >>> embeddings = model(input_ids).pooler_output ``` Nrr{rrrrarr)r;r\r]rrQ%warn_if_padding_and_no_attention_maskr}rr rrrrrr(rrrrs r%rfDPRQuestionEncoder.forwardszb2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U%.%:!! @T@T  !$ ;6;;#;#;;   !"[[EJJvVN''))'/!5#( 12; '!//w?T?Tahasas  r$)r;rr)rrrrrrOr rrrmrr(rrfr#rprqs@r%rrs y'++/+/*.,0/3&*Y F#Y !(Y !( Y  ' Y $D> Y 'tnY d^Y  'vs{);; <Y Y r$rzE The bare DPRReader transformer outputting span predictions. c^\rSrSrS\4U4Sjjr\SS\\S\\S\\S\\ S\\ S \\ S \ \ \ \S 444S jj5r S rU=r$) DPRReaderir;cp>[TU]U5 Xl[U5UlUR 5 gr)rNrOr;rsrrTrUs r%rODPRReader.__init__ s,   .v6 r$rXrYr[r\r]r^r_.c UbUOURRnUbUOURRnUbUOURRnUbUb [ S5eUb"UR X5 UR 5nO"UbUR 5SSnO [ S5eUb URO URnUc[R"XxS9nURUUUUUUS9$)a input_ids (`Tuple[torch.LongTensor]` of shapes `(n_passages, sequence_length)`): Indices of input sequence tokens in the vocabulary. It has to be a sequence triplet with 1) the question and 2) the passages titles and 3) the passages texts To match pretraining, DPR `input_ids` sequence should be formatted with [CLS] and [SEP] with the format: `[CLS] [SEP] [SEP] ` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`DPRReaderTokenizer`]. See this class documentation for more details. [What are input IDs?](../glossary#input-ids) inputs_embeds (`torch.FloatTensor` of shape `(n_passages, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. Examples: ```python >>> from transformers import DPRReader, DPRReaderTokenizer >>> tokenizer = DPRReaderTokenizer.from_pretrained("facebook/dpr-reader-single-nq-base") >>> model = DPRReader.from_pretrained("facebook/dpr-reader-single-nq-base") >>> encoded_inputs = tokenizer( ... questions=["What is love ?"], ... titles=["Haddaway"], ... texts=["'What Is Love' is a song recorded by the artist Haddaway"], ... return_tensors="pt", ... ) >>> outputs = model(**encoded_inputs) >>> start_logits = outputs.start_logits >>> end_logits = outputs.end_logits >>> relevance_logits = outputs.relevance_logits ``` Nrr{rr)r[r\r]r^) r;r\r]rrQrr}rr rr) rCrXrYr[r\r]r^rrs r%rfDPRReader.forwards`2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ CN""  '/!5# #  r$)r;r)NNNNNN)rrrrrrOr rrrmrr+rrfr#rprqs@r%rrs y'++/*.,0/3&*K F#K !(K  ' K $D> K 'tn K d^K  fck 22 3K K r$r)rrr1rrrr)&r dataclassesrtypingrrrr rrmodeling_outputsr modeling_utilsr utilsr r rbert.modeling_bertrconfiguration_dprr get_loggerrloggerrr(r+r1rIrsrrrrrr__all__rr$r%rsx<!)) :- +(   H % ?k? ?6 ?{? ?6 ?k? ?@***(52#52p6 )6 |&"4&+#5+),)  a 3a  a H b 5b  b J T #T  T n r$