fTh$ S SK JrJr S SKrS SKJr S SKJrJrJr S SK J
r
SSK J r SSK
Jr SSKJrJr S S
KJrJrJr " S S \5 r\ " S
S\5 5 r " S S\\5 r\" SS9 " S S\\5 5 r/ SQrg) )OptionalUnionN)BCEWithLogitsLossCrossEntropyLossMSELoss) IJepaConfig )ImageClassifierOutput)PreTrainedModel)auto_docstring torch_int )
ViTEmbeddingsViTForImageClassificationViTModelc ^ \ rS rSrSS\S\SS4U 4S jjjrS\R S\ S \ S\R 4S
jr
SS \R S \ \R S
\S\R 4S jjr
SrU =r$ )IJepaEmbeddings configuse_mask_tokenreturnNc > [ TU ] X5 U ?U R R n[
R " [ R " SX1R 5 5 U l
g )N ) super__init__ cls_tokenpatch_embeddings num_patchesnn Parametertorchrandn hidden_sizeposition_embeddings)selfr r r __class__s _/var/www/auris/envauris/lib/python3.13/site-packages/transformers/models/ijepa/modular_ijepa.pyr IJepaEmbeddings.__init__ sH
0 N++77 #%<< A{L^L^0_#`
embeddingsheightwidthc , UR S nU R R S n[ R R 5 ( d XE:X a X#:X a U R $ U R nUR S nX R
- nX0R
- n [
US- 5 n
UR SXU5 nUR SSSS5 n[ R R UX4SSS 9nUR SSSS5 R SSU5 nU$ )
a
This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution
images. This method is also adapted to support torch.jit tracing.
Adapted from:
- https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and
- https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211
r g ?r r r bicubicF)sizemode
align_corners)
shaper$ r! jit
is_tracing
patch_sizer
reshapepermuter
functional interpolateview) r% r* r+ r, r
num_positionspatch_pos_embeddim
new_height new_widthsqrt_num_positionss r' interpolate_pos_encoding(IJepaEmbeddings.interpolate_pos_encoding s !&&q) 0066q9
yy##%%+*F6?++ + 22 r".
__, &}c'9 :)11!5G]`a)11!Q1= --33 ( 4
*11!Q1=BB1b#N r) pixel_valuesbool_masked_posrB c n UR u pEpgU R XS9nUbX UR S n U R R XIS5 n
UR S5 R U
5 n USU -
- X- - nU( a XR
XU5 - nOXR - nU R U5 nU$ )N)rB r r. ?) r3 r
mask_tokenexpand unsqueezetype_asrB r$ dropout) r% rD rE rB
batch_size_r+ r, r*
seq_length mask_tokensmasks r' forwardIJepaEmbeddings.forward> s (4'9'9$
v**<*k
&#))!,J //00LK",,R088ED#sTz2[5GGJ $#&C&CJX]&^^J#&>&>>J\\*-
r) )r$ )F)NF)__name__
__module__ __qualname____firstlineno__r boolr r! TensorintrB r
BoolTensorrR __static_attributes__
__classcell__r& s @r' r r s a{ aD aT a a% 5<< % % UX % ]b]i]i % T 7;). ll "%"2"23 #'
r) r c \ rS rSr\rSrSrSrSS/r Sr
Sr S\ \
R \
R \
R 4 SS 4S
jrS rg ) IJepaPreTrainedModelY ijeparD Tr
IJepaLayermoduler Nc [ U[ R [ R 45 ( a [ R R UR R R [ R 5 SU R R S9R UR R 5 UR l UR b% UR R R 5 gg[ U[ R 5 ( aJ UR R R 5 UR R R# S5 g[ U[$ 5 ( a [ R R UR& R R [ R 5 SU R R S9R UR& R 5 UR& l UR( b% UR( R R 5 ggg)zInitialize the weightsg )meanstdNrG )
isinstancer LinearConv2dinit
trunc_normal_weightdatator! float32r initializer_rangedtypebiaszero_ LayerNormfill_r r$ rH )r% rd s r'
_init_weights"IJepaPreTrainedModel._init_weightsc s fryy")) 4 5 5 "$!6!6
""%%emm43DKKDaDa "7 "b $$%
MM {{& &&( '
-
- KK " " $ MM $ $S )
0
0.0gg.C.C**//225==AKK11 /D / b++112
& & +
,!!&&,,. -
1r) )rT rU rV rW r config_classbase_model_prefixmain_input_name supports_gradient_checkpointing_no_split_modules_supports_sdpa_supports_flash_attn_2r r ri rj ru rw r\ ry r) r' r` r` Y sZ L $O&*#*L9N !/E"))RYY *L$M /RV /r) r` c < ^ \ rS rSrSS\S\S\4U 4S jjjrSrU =r$ )
IJepaModelz r add_pooling_layerr c L > [ TU ] U5 Xl [ XS9U l g)z
add_pooling_layer (bool, *optional*, defaults to `True`):
Whether to add a pooling layer
use_mask_token (`bool`, *optional*, defaults to `False`):
Whether to use a mask token for masked image modeling.
)r N)r r r r r* )r% r r r r& s r' r IJepaModel.__init__{ s# )&Pr) )r r* )FF) rT rU rV rW r rX r r\ r] r^ s @r' r r z s( Q{ Qt Q]a Q Qr) r a
IJepa Model transformer with an image classification head on top (a linear layer on top of the final hidden states)
e.g. for ImageNet.
Note that it's possible to fine-tune IJepa on higher resolution images than the ones it has been trained on, by
setting `interpolate_pos_encoding` to `True` in the forward of the model. This will interpolate the pre-trained
position embeddings to the higher resolution.
) custom_introc ^ \ rS rSrS\4U 4S jjr SS\\R S\\R S\\R S\\ S\\ S \\ S
\\ S \
\ \ 4 4S jjr
S
rU =r$ )IJepaForImageClassification r c b > [ TU ] U5 [ USS9U l U R 5 g )NF)r )r r r rb post_init)r% r r& s r' r $IJepaForImageClassification.__init__ s(
%@
r) rD head_masklabelsoutput_attentionsoutput_hidden_statesrB return_dictr c z Ub UOU R R nU R UUUUUUS9nUS n U R U R SS95 n
Sn UGb UR U
R 5 nU R R c U R S:X a SU R l OoU R S: aN UR [ R :X d UR [ R :X a SU R l OSU R l U R R S:X aI [ 5 n U R S:X a& U " U
R 5 UR 5 5 n OU " X5 n OU R R S:X a= [ 5 n U " U
R! S U R 5 UR! S 5 5 n O,U R R S:X a [# 5 n U " X5 n U( d U
4USS - n
U b U 4U
- $ U
$ [% U U
UR& UR( S
9$ ) ab
labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*):
Labels for computing the image classification/regression loss. Indices should be in `[0, ...,
config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If
`config.num_labels > 1` a classification loss is computed (Cross-Entropy).
N)r r r rB r r r )r>
regressionsingle_label_classificationmulti_label_classificationr. )losslogits
hidden_states
attentions)r use_return_dictrb
classifierrf ro device problem_type
num_labelsrr r! longrZ r squeezer r; r r
r r )r% rD r r r r rB r outputssequence_outputr r loss_fctoutputs r' rR #IJepaForImageClassification.forward s &1% A
'tn
A
#+4.A
d^A
u++ ,A
A
r) r )r` r r )typingr r r! torch.nnr r r r -transformers.models.ijepa.configuration_ijepar modeling_outputsr
modeling_utilsr utilsr r
vit.modeling_vitr r r r r` r r __all__ry r) r' r s " A A E 5 - . Q QGm GT /? / /@
Q%x
Q
G
"68Q G
G
T r)