a h*> @sUddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl mZddlmZmZmZmZmZmZddlmZddlZddlZddlmZddlmZmZdZ dSe!ee!ej"fe#dd d d Z$dTee!ej"fe#e!dddZ%ee!ej"fe!ee&dddZ'dUee!ej"fee!e&dddZ(dVe!e#e!dddZ)e!ee!dddZ*dWe!ee!ej"feee!ej"fee!e#dddd Z+dXee!ej"fe&e,e!d"d#d$Z-dYee!ej"fe!e&e,e!d%d&d'Z.dZe!ee!ej"feee!ej"fee!d(d)d*Z/ee!ej"fee!ej"fee!dd+d,d-Z0e j1e j2d.Z3e4e!e#fe5d/<ee!ej"fee!ej"fee!dd+d0d1Z6e0e6d2Z7e4e!eee!ej"fee!ej"fee!gdffe5d3<ej8ej8ej8d4Z9e4e!ed5effe5d6<d7d7d8d9Z:e4e!e;ee!ee!ffe5d:<ee!ej"fe;e!ee!ee!fd;dd?d@Z=d\ee!ej"feee!ej"fe&ee!ej"fd>dAdBZ>d]e!ee!ej"feee!ej"feee!ej"fee!e&ddCdDdEZ?ee!dFdGdHZ@edIe!eAZBd^eBee!eeeBee!eBdJdKdLZCd_ee!ej"fe#ejDdMdNdOZEejFejFdPdQdRZGdS)`N)Iterable)AnyCallableIOOptionalTypeVarUnion)urlparse)tqdm)#_download_file_from_remote_location_is_remote_location_availablezpytorch/vision)urlfilename chunk_sizereturnc Cstjtjj|dtid}t|dh}t|jddd8}||}rd| || t |q[^/]*)id)r rematchnetlocr8group)rpartsrFr(r(r)_get_google_drive_file_idJsrJ)rrootrr0max_redirect_hopsrc Cstj|}|stj|}ttj||}tj|ddt||rLdStr^t ||nt ||d}t |}|durt ||||Szt ||WnXtjjtfy}z8|dddkr|dd}t ||n|WYd}~n d}~00t||std dS) aDownload a file from a url and place it in root. Args: url (str): URL to download file from root (str): Directory to place downloaded file in filename (str, optional): Name to save the file under. If None, use the basename of the URL md5 (str, optional): MD5 checksum of the download. If None, do not check max_redirect_hops (int, optional): Maximum number of redirect hops allowed Texist_okN)r<httpszhttps:zhttp:File not found or corrupted.)r7r8 expanduserbasenamefspathjoinmakedirsr:r r rCrJdownload_file_from_google_driver*rerrorURLErrorOSErrorreplace RuntimeError)rrKrr0rLr,file_ider(r(r) download_urlWs,        r_F)rKprefixrcsBtjfddtD}|dur>fdd|D}|S)a List all directories at a given root Args: root (str): Path to directory whose folders need to be listed prefix (bool, optional): If true, prepends the path to each result, otherwise only returns the name of the directories found cs&g|]}tjtj|r|qSr()r7r8isdirrU.0prKr(r) zlist_dir..Tcsg|]}tj|qSr(r7r8rUrcdrer(r)rfrgr7r8rRlistdir)rKr` directoriesr(rer)list_dirs  rn)rKsuffixr`rcsDtjfddtD}|dur@fdd|D}|S)aList all files ending with a suffix at a given root Args: root (str): Path to directory whose folders need to be listed suffix (str or tuple): Suffix of the files to match, e.g. '.png' or ('.jpg', '.png'). It uses the Python "str.endswith" method and is passed directly prefix (bool, optional): If true, prepends the path to each result, otherwise only returns the name of the files found cs0g|](}tjtj|r|r|qSr()r7r8r9rUendswithrbrKror(r)rfrgzlist_files..Tcsg|]}tj|qSr(rhrirer(r)rfrgrk)rKror`filesr(rqr) list_filess rs)r]rKrr0cCsz ddl}Wnty&tdYn0tj|}|s<|}ttj||}tj|ddt ||rldS|j ||dt dt ||stddS) afDownload a Google Drive file from and place it in root. Args: file_id (str): id of file to be downloaded root (str): Directory to place downloaded file in filename (str, optional): Name to save the file under. If None, use the id of the file. md5 (str, optional): MD5 checksum of the download. If None, do not check rNz`To download files from GDrive, 'gdown' is required. You can install it with 'pip install gdown'.TrMF)rDoutputquiet user_agentrQ) gdownModuleNotFoundErrorr\r7r8rRrTrUrVr:downloadr)r]rKrr0rwr,r(r(r)rWs       rW) from_pathto_path compressionrcCsPt||rd|ddnd}||Wdn1sB0YdS)Nzr:r>r)tarfiler extractall)rzr{r|tarr(r(r) _extract_tars$r).bz2.xz_ZIP_COMPRESSION_MAPcCsLtj|d|rt|ntjd}||Wdn1s>0YdS)Nr})r|)zipfileZipFiler ZIP_STOREDr)rzr{r|zipr(r(r) _extract_zips r).tarz.zip_ARCHIVE_EXTRACTORS)r.gzr._COMPRESSED_FILE_OPENERS)rr)rr)z.tbzz.tbz2z.tgz_FILE_TYPE_ALIASES)filercCst|j}|s td|d|d}|tvr@|gt|RS|tvrR||dfS|tvrt|dkr|d}|tvr||||fS|d|fStt tt tBt tB}td|d|d dS) aDetect the archive type and/or compression of a file. Args: file (str): the filename Returns: (tuple): tuple of suffix, archive type, and compression Raises: RuntimeError: if file has no suffix or suffix is not supported zFile 'zP' has no suffixes that could be used to detect the archive type and compression.Nr>z&Unknown compression or archive type: 'z'. Known suffixes are: 'z'.) pathlibPathsuffixesr\rrrr$sortedset)rrroZsuffix2Zvalid_suffixesr(r(r)_detect_file_types$     r)rzr{remove_finishedrc Cst|\}}}|s"td|d|durNtt|||durF|nd}t|}||dH}t|d}| | Wdn1s0YWdn1s0Y|rt |t|S)aDecompress a file. The compression is automatically detected from the file name. Args: from_path (str): Path to the file to be decompressed. to_path (str): Path to the decompressed file. If omitted, ``from_path`` without compression extension is used. remove_finished (bool): If ``True``, remove the file after the extraction. Returns: (str): Path to the decompressed file. z*Couldn't detect a compression from suffix r?Nr.r) rr\rrr7rTr[rrr"r!remove) rzr{rro archive_typer|Zcompressed_file_openerZrfhZwfhr(r(r) _decompress"s$J rc stjtttjfdfdd }|dur4tj}t\}}}|svttj |tj  |d|d}||St |}||||rt |t|S)aDExtract an archive. The archive type and a possible compression is automatically detected from the file name. If the file is compressed but not an archive the call is dispatched to :func:`decompress`. Args: from_path (str): Path to the file to be extracted. to_path (str): Path to the directory the file will be extracted to. If omitted, the directory of the file is used. remove_finished (bool): If ``True``, remove the file after the extraction. Returns: (str): Path to the directory the file was extracted to. )ret_pathrcsttrt|S|SdSr5) isinstancestrr7rT)rrzr(r) path_or_strZs  z$extract_archive..path_or_strNr)r)rrrrr7r8dirnamerrrUrSr[rr) rzr{rrrorr|rZ extractorr(rr)extract_archiveFs     r)r download_root extract_rootrr0rrcCsTtj|}|dur|}|s(tj|}t||||tj||}t|||dSr5)r7r8rRrSr_rUr)rrrrr0rarchiver(r(r)download_and_extract_archivevs  r)iterablercCsdddd|DdS)N'z', 'cSsg|] }t|qSr()r)rcitemr(r(r)rfrgz#iterable_to_str..)rU)rr(r(r)iterable_to_strsrT)valuearg valid_values custom_msgrcCs|t|ts6|durd}nd}|jt||d}t||durB|S||vrx|durX|}nd}|j||t|d}t||S)Nz'Expected type str, but got type {type}.z:Expected type str for argument {arg}, but got type {type}.)typerzPUnknown value '{value}' for argument {arg}. Valid values are {{{valid_values}}}.)rrr)rrformatr ValueErrorr)rrrrmsgr(r(r)verify_str_args r) file_nameslice_channelsrc Cst|d}|}|dvr(tdtd|}|sDtddd|D\}}t|}|dkr~d }| }nd }t j ||d d } Wd n1s0Y|dkrdnd} | |||  ddd} t j | dd} | d |d d d d f} | t jS)aDRead file in .pfm format. Might contain either 1 or 3 channels of data. Args: file_name (str): Path to the file. slice_channels (int): Number of channels to slice out of the file. Useful for reading different data formats stored in .pfm files: Optical Flows, Stereo Disparity Maps, etc. r.)PFsPfzInvalid PFM files^(\d+)\s(\d+)\s$zMalformed PFM header.css|]}t|VqdSr5)int)rcZdimr(r(r) rgz_read_pfm..r<>r2)dtypeNrr;r>r )Zaxis)rreadlinerstriprrErF ExceptiongroupsfloatnpfromfileZreshapeZ transposeflipZastypefloat32) rrr2headerZ dim_matchwhZscaleZendiandataZ pfm_channelsr(r(r) _read_pfms&  0r)trcCsN|tjjg|j|Rdjg|jdddR|jS)Nr) contiguousviewtorchZuint8shapeZ element_sizerr)rr(r(r)_flip_byte_ordersLr)r)r+)N)r;)NNr;)F)F)NN)NF)NF)NNNF)NNN)r )Hbz2gzipr/lzmar7os.pathrrEr~r urllib.errorurllib.requestrcollections.abcrtypingrrrrrr urllib.parser numpyrrZtorch.utils.model_zoor Z_internally_replaced_utilsr r rrrrr*r3boolr6r:rCrJr_listrnrsrWr ZIP_BZIP2ZIP_LZMArdict__annotations__rrrrrtuplerrrrrbytesrrZndarrayrZTensorrr(r(r(r)s       "  5"$  &  <&,-  &  3      $