h,U 4 S SK r S SKrS SKrS SKrS SKrS SKrS SKrS SKJr S SK J
r
J r J r J
r
S SKJr S SKrS SKrSSKJr SSKJrJrJrJrJr SS KJr " S
S \5 r " S S
\ 5 r " S S\ 5 r " S S\ 5 r " S S\ 5 r S\!S\"4S jr#\RH \RJ \RL \RN \RP \RR S.r*S S\+S\,S\RZ 4S jjr.S\+S\RZ 4S jr/S\+S\RZ 4S jr0g) N)Path)AnyCallableOptionalUnion)URLError )_Image_fromarray )_flip_byte_ordercheck_integrity download_and_extract_archiveextract_archiveverify_str_arg)
VisionDatasetc
~ ^ \ rS rSrSrSS/r/ SQrSrSr/ SQr \
S 5 r \
S
5 r \
S 5 r
\
S 5 r S"S\\\4 S\S\\ S\\ S\SS
4 U 4S jjjrS rS rS rS\S\\\4 4S jr S\4S jr \
S\4S j5 r \
S\4S j5 r \
S\ \\4 4S j5 r!S\4S jr"S#S jr#S\4S jr$S!r%U =r&$ )$MNIST a9 `MNIST `_ Dataset.
Args:
root (str or ``pathlib.Path``): Root directory of dataset where ``MNIST/raw/train-images-idx3-ubyte``
and ``MNIST/raw/t10k-images-idx3-ubyte`` exist.
train (bool, optional): If True, creates dataset from ``train-images-idx3-ubyte``,
otherwise from ``t10k-images-idx3-ubyte``.
transform (callable, optional): A function/transform that takes in a PIL image
and returns a transformed version. E.g, ``transforms.RandomCrop``
target_transform (callable, optional): A function/transform that takes in the
target and transforms it.
download (bool, optional): If True, downloads the dataset from the internet and
puts it in root directory. If dataset is already downloaded, it is not
downloaded again.
z.https://ossci-datasets.s3.amazonaws.com/mnist/z!http://yann.lecun.com/exdb/mnist/))train-images-idx3-ubyte.gz f68b3c2dcbeaaa9fbdd348bbdeb94873)train-labels-idx1-ubyte.gz d53e105ee54ea40749a09fcbcd1e9432)t10k-images-idx3-ubyte.gz 9fb629c4189551a2d022fa330f9573f3)t10k-labels-idx1-ubyte.gz ec29112dd5afa0611ce80d1b7f02629cz training.ptztest.pt
z0 - zeroz1 - onez2 - twoz 3 - threez4 - fourz5 - fivez6 - sixz 7 - sevenz 8 - eightz9 - ninec F [ R " S5 U R $ )Nz%train_labels has been renamed targetswarningswarntargetsselfs R/var/www/auris/envauris/lib/python3.13/site-packages/torchvision/datasets/mnist.py train_labelsMNIST.train_labels@ s
=>|| c F [ R " S5 U R $ )Nz$test_labels has been renamed targetsr r# s r% test_labelsMNIST.test_labelsE s
<=||r( c F [ R " S5 U R $ )Nz train_data has been renamed datar r! datar# s r%
train_dataMNIST.train_dataJ s
89yyr( c F [ R " S5 U R $ )Nz test_data has been renamed datar- r# s r% test_dataMNIST.test_dataO s
78yyr( Nroottrain transformtarget_transformdownloadreturnc < > [ TU ] XUS9 X l U R 5 ( a U R 5 u U l U l g U( a U R 5 U R 5 ( d [ S5 eU R 5 u U l U l g )N)r6 r7 z;Dataset not found. You can use download=True to download it) super__init__r5 _check_legacy_exist_load_legacy_datar. r" r8
_check_exists RuntimeError
_load_data)r$ r4 r5 r6 r7 r8 __class__s r% r< MNIST.__init__T s EUV
# # % %&*&<&<&> #DIt| MMO!!## \] ]"&//"3 4# U H8 n[ [ R R TR U5 5 v M: g 7fN)r
ospathjoinprocessed_folder).0filer$ s r% ,MNIST._check_legacy_exist..p s2
Sw4OBGGLL)>)> E F FSws A A)rG rH existsrJ all
training_file test_file)r$ processed_folder_existss ` r% r= MNIST._check_legacy_existk sN "$''..1F1F"G &
TXTfTfhlhvhvSw
r( c U R ( a U R O U R n[ R " [
R R U R U5 SS9$ )NT) weights_only) r5 rQ rR torchloadrG rH rI rJ )r$ data_files r% r> MNIST._load_legacy_datat sB +/**D&&$.. zz"'',,t'<'> %..%C ,**62F{r( c , [ U R 5 $ rF )lenr. r# s r% __len__
MNIST.__len__ s 499~ r( c [ R R U R U R R
S5 $ )NrawrG rH rI r4 rB __name__r# s r% r` MNIST.raw_folder s' ww||DIIt~~'>'>FFr( c [ R R U R U R R
S5 $ )N processedrw r# s r% rJ MNIST.processed_folder s' ww||DIIt~~'>'> LLr( c b [ U R 5 VVs0 s H u pX!_M snn$ s snnf rF ) enumerateclasses)r$ i_classs r% class_to_idxMNIST.class_to_idx s) +4T\\+BC+Bia +BCCCs +c B ^ [ U 4S jT R 5 5 $ )Nc 3 ># U Hw u p[ [ R R TR [ R R [ R R
U5 5 S 5 5 v My g7f)r N)r
rG rH rI r` splitextbasename)rK url_r$ s r% rM &MNIST._check_exists.. sZ
(
BGGLL "'':J:J277K[K[\_K`:abc:d e f f(s A?B)rP resourcesr# s `r% r? MNIST._check_exists s!
..
r( c U R 5 ( a g[ R " U R SS9 U R Hy u p/ nU R
H nU U 3n [
XPR XS9 M4 SU S3n[ U R
U5 H u pHUSU S[ U5 S 3-
nM [ U5 e g! [ a nUR U5 SnAM SnAff = f)
z4Download the MNIST data if it doesn't exist already.NTexist_ok)
download_rootfilenamemd5zError downloading z:
zTried z, got:
) r? rG makedirsr` r mirrorsr r appendzipstrr@ ) r$ r r errorsmirrorr eserrs r% r8 MNIST.download s
DOOd3 "^^MHF ,,
+ 0OO^fp ' )
#6#&t||V#>A $="1o% , MM!$ s C
C&
C!!C&c 2 U R SL a SOSnSU 3$ )NTTrainTestSplit: )r5 )r$ splits r%
extra_reprMNIST.extra_repr s! ::-6 r( )r. r" r5 )TNNFr9 N)'rx
__module__ __qualname____firstlineno____doc__r r rQ rR r propertyr& r* r/ r2 r r r boolr r r< r= r> rA rj tupler ro rs r` rJ dictr r? r8 r __static_attributes__
__classcell__rB s @r% r r s " 9+G
I "MI G (,/3
4CI 4 4 H% 4
#8, 4
4
4 4.
] sCx , GC G G M# M M Dd38n D D
t
&2!C ! !r( r c . \ rS rSrSrS/r/ SQr/ SQrSrg) FashionMNIST a^ `Fashion-MNIST `_ Dataset.
Args:
root (str or ``pathlib.Path``): Root directory of dataset where ``FashionMNIST/raw/train-images-idx3-ubyte``
and ``FashionMNIST/raw/t10k-images-idx3-ubyte`` exist.
train (bool, optional): If True, creates dataset from ``train-images-idx3-ubyte``,
otherwise from ``t10k-images-idx3-ubyte``.
transform (callable, optional): A function/transform that takes in a PIL image
and returns a transformed version. E.g, ``transforms.RandomCrop``
target_transform (callable, optional): A function/transform that takes in the
target and transforms it.
download (bool, optional): If True, downloads the dataset from the internet and
puts it in root directory. If dataset is already downloaded, it is not
downloaded again.
z;http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/))r 8d4fb7e6c68d591d4c3dfef9ec88bf0d)r 25c81989df183df01b3e8a0aad5dffbe)r bef4ecab320f06d8554ea6380940ec79)r bb300cfdad3c16e7a12a480ee83cd310)
z T-shirt/topTrouserPulloverDressCoatSandalShirtSneakerBagz
Ankle boot N rx r r r r r r r r r r( r% r r s! MMGI yGr( r c . \ rS rSrSrS/r/ SQr/ SQrSrg)KMNIST aG `Kuzushiji-MNIST `_ Dataset.
Args:
root (str or ``pathlib.Path``): Root directory of dataset where ``KMNIST/raw/train-images-idx3-ubyte``
and ``KMNIST/raw/t10k-images-idx3-ubyte`` exist.
train (bool, optional): If True, creates dataset from ``train-images-idx3-ubyte``,
otherwise from ``t10k-images-idx3-ubyte``.
transform (callable, optional): A function/transform that takes in a PIL image
and returns a transformed version. E.g, ``transforms.RandomCrop``
target_transform (callable, optional): A function/transform that takes in the
target and transforms it.
download (bool, optional): If True, downloads the dataset from the internet and
puts it in root directory. If dataset is already downloaded, it is not
downloaded again.
z-http://codh.rois.ac.jp/kmnist/dataset/kmnist/))r bdb82020997e1d708af4cf47b453dcf7)r e144d726b3acfaa3e44228e80efcd344)r 5c965bf0a639b31b8f53240b1b52f4d7)r 7320c461ea6c1c855c0b718fb2a4b134)
okisutsunahamayarewor Nr r r( r% r r s ??GI KGr( r c ^ \ rS rSrSrSrSrSr1 Skr\ " \
R \
R - 5 r
\" \" \
5 5 \" \" \
\-
5 5 \" \" \
\-
5 5 S/\" \
R 5 - \" \
R 5 \" \
R 5 S.rS\\\4 S \S
\S S 4U 4S
jjr\S \4S j5 r\S \4S j5 r\S \4S j5 r\S \4S j5 r \S \4S j5 r S r S \ 4S jr SS jr!Sr"U =r#$ )EMNISTi a `EMNIST `_ Dataset.
Args:
root (str or ``pathlib.Path``): Root directory of dataset where ``EMNIST/raw/train-images-idx3-ubyte``
and ``EMNIST/raw/t10k-images-idx3-ubyte`` exist.
split (string): The dataset has 6 different splits: ``byclass``, ``bymerge``,
``balanced``, ``letters``, ``digits`` and ``mnist``. This argument specifies
which one to use.
train (bool, optional): If True, creates dataset from ``training.pt``,
otherwise from ``test.pt``.
download (bool, optional): If True, downloads the dataset from the internet and
puts it in root directory. If dataset is already downloaded, it is not
downloaded again.
transform (callable, optional): A function/transform that takes in a PIL image
and returns a transformed version. E.g, ``transforms.RandomCrop``
target_transform (callable, optional): A function/transform that takes in the
target and transforms it.
z4https://biometrics.nist.gov/cs_links/EMNIST/gzip.zip 58c8d27c78d21e728a6bc7b3cc06412e)byclassbymergebalancedlettersdigitsmnist> cr jklmr pr uvwxyzzN/Ar4 r kwargsr9 Nc > [ USU R 5 U l U R U5 U l U R U5 U l [ TU ] " U40 UD6 U R U R U l
g )Nr ) r splitsr _training_filerQ
_test_filerR r; r< classes_split_dictr )r$ r4 r r rB s r% r< EMNIST.__init__' sb #E7DKK@
! 0 0 7 /
((..tzz: r( c SU S3$ )N training_.ptr r s r% r EMNIST._training_file. s 5'%%r( c SU S3$ )Ntest_r r r s r% r EMNIST._test_file2 s ugS!!r( c P SU R SU R ( a S 3$ S 3$ )Nzemnist--r5 test)r r5 r# s r% _file_prefixEMNIST._file_prefix6 s+ Ag%HII%HIIr( c p [ R R U R U R S35 $ )Nr] rG rH rI r` r r# s r% images_fileEMNIST.images_file: * ww||DOO0A0A/BBT-UVVr( c p [ R R U R U R S35 $ )Nr^ r r# s r% labels_fileEMNIST.labels_file> r r( c V [ U R 5 [ U R 5 4$ rF )r_ r ra r r# s r% rA EMNIST._load_dataB s# t / /0/$BRBR2SSSr( c R [ S U R U R 4 5 5 $ )Nc 3 8 # U H n[ U5 v M g 7frF r
rK rL s r% rM 'EMNIST._check_exists..F Z5YT?4((5Y rP r r r# s r% r? EMNIST._check_existsE $ Zd6F6FHXHX5YZZZr( c U R 5 ( a g[ R " U R SS9 [ U R
U R U R S9 [ R R U R S5 n[ R " U5 HN nUR S5 ( d M [ [ R R X5 U R 5 MP [ R " U5 g)z5Download the EMNIST data if it doesn't exist already.NTr )r r gzipz.gz)r? rG r r` r r r rH rI listdirendswithr shutilrmtree)r$ gzip_folder gzip_files r% r8 EMNIST.downloadH s
DOOd3$TXXT__RVRZRZ[ggll4??F; K0I!!%(( [ DdooV 1
k"r( )r r rR rQ r )$rx r r r r r r r _merged_classessetstringr
ascii_letters _all_classessortedlistascii_lowercaser r r r r r< staticmethodr r r r r r rA r r? r8 r r r s @r% r r sv & AC
,C
MFaOv}}v';';;?4 >?@7T&"8"8 99v}}%fmm$
;U39 - ;c ;S ;T ; & & & "S " " Jc J J WS W W WS W WT[t [
#
#r( r c
, ^ \ rS rSr% SrSSSSSS.rSS/S S
/S S /S
.r\\\ \
\\4 4 \ S' / SQr S S\
\\4 S\\ S\S\S\SS4 U 4S jjjr\S\4S j5 r\S\4S j5 rS\4S jrS rS!S jrS \S\
\\4 4S jrS\4S jrS r U =r $ )"QMNISTiX a `QMNIST `_ Dataset.
Args:
root (str or ``pathlib.Path``): Root directory of dataset whose ``raw``
subdir contains binary files of the datasets.
what (string,optional): Can be 'train', 'test', 'test10k',
'test50k', or 'nist' for respectively the mnist compatible
training set, the 60k qmnist testing set, the 10k qmnist
examples that match the mnist testing set, the 50k
remaining qmnist testing examples, or all the nist
digits. The default is to select 'train' or 'test'
according to the compatibility argument 'train'.
compat (bool,optional): A boolean that says whether the target
for each example is class number (for compatibility with
the MNIST dataloader) or a torch vector containing the
full qmnist information. Default=True.
train (bool,optional,compatibility): When argument 'what' is
not specified, this boolean decides whether to load the
training set or the testing set. Default: True.
download (bool, optional): If True, downloads the dataset from
the internet and puts it in root directory. If dataset is
already downloaded, it is not downloaded again.
transform (callable, optional): A function/transform that
takes in a PIL image and returns a transformed
version. E.g, ``transforms.RandomCrop``
target_transform (callable, optional): A function/transform
that takes in the target and transforms it.
r5 r nist)r5 r test10ktest50kr% )zbhttps://raw.githubusercontent.com/facebookresearch/qmnist/master/qmnist-train-images-idx3-ubyte.gz ed72d4157d28c017586c42bc6afe6370)z`https://raw.githubusercontent.com/facebookresearch/qmnist/master/qmnist-train-labels-idx2-int.gz 0058f8dd561b90ffdd0f734c6a30e5e4)zahttps://raw.githubusercontent.com/facebookresearch/qmnist/master/qmnist-test-images-idx3-ubyte.gz 1394631089c404de565df7b7aeaf9412)z_https://raw.githubusercontent.com/facebookresearch/qmnist/master/qmnist-test-labels-idx2-int.gz 5b5b05890a5e13444e108efe57b788aa)z[https://raw.githubusercontent.com/facebookresearch/qmnist/master/xnist-images-idx3-ubyte.xz 7f124b3b8ab81486c9d8c2749c17f834)zYhttps://raw.githubusercontent.com/facebookresearch/qmnist/master/xnist-labels-idx2-int.xz 5ed0e788978e45d4a8bd4b7caec3d79d)r5 r r% r r Nr4 whatcompatr r9 c > Uc U( a SOSn[ US[ U R R 5 5 5 U l X0l US- U l U R U l U R U l [ TU ]( " X40 UD6 g )Nr5 r r. r ) r r subsetskeysr. r/ rY rQ rR r; r< )r$ r4 r. r/ r5 r rB s r% r< QMNIST.__init__ sn < #7D"4t||7H7H7J1KL !^^
//r( c ( U R U R U R u u n n[ R R U R [ R R [ R R U5 5 S 5 $ Nr r r1 r. rG rH rI r` r r )r$ r r s r% r QMNIST.images_file sb nnT\\$))%<= a!ww||DOORWW-=-=bgg>N>Ns>S-TUV-WXXr( c & U R U R U R u nu p![ R R U R [ R R [ R R U5 5 S 5 $ r5 r6 )r$ r r s r% r QMNIST.labels_file s` nnT\\$))%<= 8Cww||DOORWW-=-=bgg>N>Ns>S-TUV-WXXr( c R [ S U R U R 4 5 5 $ )Nc 3 8 # U H n[ U5 v M g 7frF r r
s r% rM 'QMNIST._check_exists.. r r
r r# s r% r? QMNIST._check_exists r r( c [ U R 5 nUR [ R :w a [ SUR 35 eUR
5 S:w a [ S5 e[ U R 5 R 5 nUR
5 S:w a [ SUR
5 35 eU R S:X a8 USS2S S 2S S 24 R 5 nUSS2S S 24 R 5 nX4$ U R S :X a5 USS 2S S 2S S 24 R 5 nUSS 2S S 24 R 5 nX4$ )
Nz/data should be of dtype torch.uint8 instead of z :: $ Mdjj\ Z[ [ ?? ! [\ \/0@0@AFFH 1 $ KGL^L^L`Ka bc c 99 !%A
&,,.D agqj)//1G
} YY)
#1 %++-D efai(..0G} r( c U R 5 ( a g[ R " U R SS9 U R U R
U R nU H u p#[ X R US9 M g)z|Download the QMNIST data if it doesn't exist already.
Note that we only download what has been asked for (argument 'what').
NTr )r )r? rG r r` r r1 r. r )r$ r r r s r% r8 QMNIST.download s]
DOOd3t||DII 67 HC (oo3 G r( re c ( U R U U R U p2[ UR 5 SS9nU R b U R U5 nU R
( a [
US 5 nU R b U R U5 nX#4$ )Nrg rh r )r. r" r
rk r6 r/ rj r7 rl s r% ro QMNIST.__getitem__ s} ii& U(;V syy{5 >> %..%C ;;^F ,**62F{r( c SU R 3$ )Nr )r. r# s r% r QMNIST.extra_repr s $$r( )r/ rY rR rQ r. )NTTr ) rx r r r r r1 r r r r r __annotations__r r r r r r r< r r r r? rA r8 rj ro r r r r s @r% r$ r$ X sD : Fv_efG
+ 3ItCeCHo../ @ G fj
0#t)$
0,4SM
0JN
0^b
0ux
0
0
0 YS Y Y YS Y Y[t [ ( H
sCx
%C % %r( r$ br9 c D [ [ R " U S5 S5 $ )Nhex )rj codecsencode)rP s r% get_intrV s v}}Q& ++r( )
rH strictc
D [ U S5 nUR 5 nSSS5 [ R S:X d [ R S:X a [ WSS 5 nUS- nUS- nOC[ WSS 5 n[ USS 5 [ US S
5 S- - [ US
S 5 S- S- - nSUs=:: a S
:: d e eS Us=:: a S :: d e e[ U n[ U5 Vs/ s H n[ USUS- - SUS - - 5 PM n n[ R S
:X aV [ R S:X dB [ [ U 5 5 H* n[ R X R SSS9S
SS9X' M, [ R " [ U5 USUS- - S9n
[ R S:X a U
R 5 S: a [! U
5 n
U
R" S [$ R& " U 5 :X d U( a eU
R( " U 6 $ ! , ( d f GN= fs snf )zRead a SN3 file in "Pascal Vincent" format (Lush file 'libidx/idx-io.lsh').
Argument may be a filename, compressed filename, or file object.
rbNlittleaixr r r r? rW r\ big) byteorderF)re signed)rA offset)openreadsysre platformrV SN3_PASCALVINCENT_TYPEMAPrangerr rj
from_bytesto_bytesrW
frombuffer bytearray element_sizer shapenpprodview) rH r] fr. magicndty
torch_typer r parseds r% r@ r@ s
dD Qvvx
}} CLLE$9Qq "
S[
c\
T!AY
T!AY
'$q)"4s":
:WT!AY=ORU=UX[=[
[ >!$--X-"FRW`e>fAD
ioZbSTf
WF }} V%8%8%:Q%>!&) <<?bggaj (6 6 ;;?;
Fs H $$H
Hc [ U SS9nUR [ R :w a [ SUR 35 eUR 5 S:w a [
SUR 5 35 eUR 5 $ )NFr] ,x should be of dtype torch.uint8 instead of r z%x should have 1 dimension instead of )r@ rA rW rB rC rD rE rF rH r s r% ra ra ! sh %d59Aww%++ FqwwiPQQ||~@ @PQRR 668Or( c [ U SS9nUR [ R :w a [ SUR 35 eUR 5 S:w a [
SUR 5 35 eU$ )NFr~ r r? z%x should have 3 dimension instead of )r@ rA rW rB rC rD rE r s r% r_ r_ * sb %d59Aww%++ FqwwiPQQ||~@ @PQRR Hr( )T)1rT rG os.pathr r rj r pathlibr typingr r r r urllib.errorr rk rt rW utilsr
r r
r r r visionr r r r r r$ bytesrj rV rB int8int16int32float32float64rl r r Tensorr@ ra r_ r r( r% r s
1 1 ! $ s s !u!M u!py5 y8KU K8Q#U Q#hS%U S%l,u , ,
{{ zz
" "T "U\\ "J# %,,
#
%,,
r(