7Th]SSKJr SSKrSSKrSSKrSSKrSSKJr SSKJ r SSK J r J r J r JrJrJr SSKJr SSKJr SSKJrJr SS KJrJrJrJr \R:"S 5r"S S 5r"S S\ 5r!SSjr"Sr#Sr$\RJ"SS5r&SSjr'S S!Sjjr(Sr)Sr*Sr+Sr,S"Sjr-Sr."SS\R^5r0g)#) annotationsN) has_magic)Path) BaseCache BlockCache BytesCache MMapCacheReadAheadCachecaches)compr)conf) filesystemget_filesystem_class)_unstrip_protocolbuild_name_functioninfer_compressionstringify_pathfsspecc`\rSrSrSrS SjrSrSrSrSr \ S 5r S r S r S rg)OpenFile a File-like object to be used in a context Can layer (buffered) text-mode and compression over any file-system, which are typically binary-only. These instances are safe to serialize, as the low-level file object is not created until invoked using ``with``. Parameters ---------- fs: FileSystem The file system to use for opening the file. Should be a subclass or duck-type with ``fsspec.spec.AbstractFileSystem`` path: str Location to open mode: str like 'rb', optional Mode of the opened file compression: str or None, optional Compression to apply encoding: str or None, optional The encoding to use if opened in text mode. errors: str or None, optional How to handle encoding errors if opened in text mode. newline: None or str Passed to TextIOWrapper in text mode, how to handle line endings. autoopen: bool If True, calls open() immediately. Mostly used by pickle pos: int If given and autoopen is True, seek to this location immediately NczXlX lX0l[X$5UlXPlX`lXpl/UlgN) fspathmodeget_compression compressionencodingerrorsnewlinefobjects)selfrrrrrr r!s C/var/www/auris/envauris/lib/python3.13/site-packages/fsspec/core.py__init__OpenFile.__init__As6  *4=    c[URURURURUR UR UR44$r)rrrrrrr r!r#s r$ __reduce__OpenFile.__reduce__TsG           r'c"SURS3$)Nz )rr)s r$__repr__OpenFile.__repr__bsTYYKr**r'cURRSS5RSS5S-nURRURUS9nU/UlURb7[URnU"X!SS9nURRU5 SUR;aD[X RURURS9nURRU5 URS$![ a7n[ UR5(a[ SUR5UeeSnAff=f) Ntb)ra%s not found. The URL contains glob characters: you maybe needed to pass expand=True in fsspec.open() or the storage_options of your library. You can also set the config value 'open_expand' before import, or fsspec.core.DEFAULT_EXPAND at runtime, to True.r)rr r!)rreplaceropenrFileNotFoundErrorrr"rr appendPickleableTextIOWrapperrr r!)r#rfecompresss r$ __enter__OpenFile.__enter__es#yy  b)11#r:S@  TYYT 2A    'T--.Ha)A MM  # dii 'MM$++t||A MM  #}}R  3! ##'XII     s$D E2EEc$UR5 gr)close)r#argss r$__exit__OpenFile.__exit__s  r'cB[URUR5$r)rrrr)s r$ full_nameOpenFile.full_names DGG44r'c"UR5$)zMaterialise this as a real open file without context The OpenFile object should be explicitly closed to avoid enclosed file instances persisting. You must, therefore, keep a reference to the OpenFile during the life of the file-like it generates. )r<r)s r$r5 OpenFile.opens~~r'c[UR5HDnSUR;a!UR(dUR 5 UR 5 MF URR 5 g)z#Close all encapsulated file objectsrN)reversedr"rclosedflushr?clear)r#r9s r$r?OpenFile.closesK$--(A$))#AHH GGI) r')rrr r"rrr!r)rbNNNN)__name__ __module__ __qualname____firstlineno____doc__r%r*r-r<rApropertyrDr5r?__static_attributes__r'r$rr sQH&  +!@55 r'rcT^\rSrSrSrSSS.U4SjjrSrSrU4S jrS r S r U=r $) OpenFilesaoList of OpenFile instances Can be used in a single context, which opens and closes all of the contained files. Normal list access to get the elements works as normal. A special case is made for caching filesystems - the files will be down/uploaded together at the start or end of the context, and this may happen concurrently, if the target filesystem supports it. rONrrcF>XlX l/Ul[TU]"U6 gr)rrfilessuperr%)r#rrr@ __class__s r$r%OpenFiles.__init__s!   $r'cXURc [S5eURn[US5(a"URU5UlUR$[US5(aURb URnOOMaUVs/sHo"R 5PM sn$s snf)NzContext has already been used open_manyr)r ValueErrorhasattrrbr]r<)r#rss r$r<OpenFiles.__enter__s 77?<= = WWr;''\\$/ zz!r4  RUU%6UU(,,t! t,,,s B'c:URnUVs/sHo3R"U6PM nSUR;a\[US5(aUR UR 5 g[US5(aURb URnOgM[gs snf)NrIrbr)rrArrd commit_manyr])r#r@rres r$rAOpenFiles.__exit__s WW$()DqT D) dii 2{++NN4::.2t$$):B  *sBc>[TU]U5n[U[5(a[ X R UR S9$U$)Nr[)r^ __getitem__ isinstanceslicerYrr)r#itemoutr_s r$rkOpenFiles.__getitem__s9g!$' dE " "SyyTWW= = r'c S[U5S3$)Nz )lenr)s r$r-OpenFiles.__repr__s3t9+%9::r')r]rr) rPrQrRrSrTr%r<rArkr-rV __classcell__r_s@r$rYrYs2 $(D  -  ;;r'rYc  j[UUUUU UU S9upnU RS:XaXlOGSU;aAU (a:UVs1sHoRU5iM nnUHnU R USS9 M [ UVs/sHn[U UUUUUUS9PM snUU S9$s snf![ a MSf=fs snf)a Given a path or paths, return a list of ``OpenFile`` objects. For writing, a str path must contain the "*" character, which will be filled in by increasing numbers, e.g., "part*" -> "part1", "part2" if num=2. For either reading or writing, can instead provide explicit list of paths. Parameters ---------- urlpath: string or list Absolute or relative filepath(s). Prefix with a protocol like ``s3://`` to read from alternative filesystems. To read from multiple files you can pass a globstring or a list of paths, with the caveat that they must all have the same protocol. mode: 'rb', 'wt', etc. compression: string or None If given, open file using compression codec. Can either be a compression name (a key in ``fsspec.compression.compr``) or "infer" to guess the compression from the filename suffix. encoding: str For text mode only errors: None or str Passed to TextIOWrapper in text mode name_function: function or None if opening a set of files for writing, those files do not yet exist, so we need to generate their names by formatting the urlpath for each sequence number num: int [1] if writing mode, number of files we expect to create (passed to name+function) protocol: str or None If given, overrides the protocol found in the URL. newline: bytes or None Used for line terminator in text mode. If None, uses system default; if blank, uses no translation. auto_mkdir: bool (True) If in write mode, this will ensure the target directory exists before writing, by calling ``fs.mkdirs(exist_ok=True)``. expand: bool **kwargs: dict Extra options that make sense to a particular storage connection, e.g. host, port, username, password, etc. Examples -------- >>> files = open_files('2015-*-*.csv') # doctest: +SKIP >>> files = open_files( ... 's3://bucket/2015-*-*.csv.gz', compression='gzip' ... ) # doctest: +SKIP Returns ------- An ``OpenFiles`` instance, which is a list of ``OpenFile`` objects that can be used as a single context Notes ----- For a full list of the available protocols and the implementations that they map across to see the latest online documentation: - For implementations built into ``fsspec`` see https://filesystem-spec.readthedocs.io/en/latest/api.html#built-in-implementations - For implementations in separate packages see https://filesystem-spec.readthedocs.io/en/latest/api.html#other-known-implementations )num name_functionstorage_optionsprotocolexpandfilerIT)exist_ok)rrrr r!r[)get_fs_token_pathsrz auto_mkdir_parentmakedirsPermissionErrorrYr)urlpathrrrr rxrwrzr!rr{kwargsrfs_tokenpathsrparentsparents r$ open_filesrs^- #B% {{f" DZ056::d#6F  FT 2   '!     7#   sBB:B0 B-,B-c jSSKJn SU;at[R"S5n/nUR S5HFnSU;dUR U5(aUR U5 M2UR US-5 MH OU/n/nSnUR5n[U5HnURSS5=(d [U5S=(d Sn [U 5n U RU5n URU 05n XSLaU RU5 [S 0U R5V Vs0sHupX;d XU :wdMX_M snn DU D6nU R!U5nSU;a[#X5(aUnUR XU45 UnM UR%5 U$s snn f) Nr)CachingFileSystemz::z .*[^a-z]+.*://rzr|target_protocolrW)fsspec.implementations.cachedrrecompilesplitmatchr7copyrJpopsplit_protocolr_get_kwargs_from_urlsupdatedictitems_strip_protocol issubclassreverse)rrrxbitspro previous_bitbitrzcls extra_kwargskwskvkws r$ _un_chainrKs? t| JJ} %D!AzQWWQZZ A AI& " v CL [[]F~::j$/S>#3Fq3ISV"8,005 jj2& q'> JJv   , 2 2 4T 4 QRV tqt 4T  !!#& B &:c+M+MC C2&'  KKM JUs 6F/ F/c [U5n1SknUR5VVs0sHup4X2;dM X4_M nnn[X5n0n[[ U55HCupxUupn U[ U5S- :Xa[ S0U DUD6nM+[ S0U DUD6US'XS'XS'ME USupn [U 40UD6nX4$s snnf)a Turn fully-qualified and potentially chained URL into filesystem instance Parameters ---------- url : str The fsspec-compatible URL **kwargs: dict Extra options that make sense to a particular storage connection, e.g. host, port, username, password, etc. Returns ------- filesystem : FileSystem The new filesystem discovered from ``url`` and created with ``**kwargs``. urlpath : str The file-systems-specific URL for ``url``. >rwrr r{r!rrrxtarget_optionsrforrW)rrr enumeraterJrrrr)urlr known_kwargsrrchaininkwargsichurlsrzrr_rs r$ url_to_fsrqs(  C L &||~ G~tq1Fdad~F G c "EH8E?+ E Q -b-H-H %)%;B%;(%;!"&."#,!8Gq H ) )B ;Hs B:B: open_expandFc pUc[OUn[SU/UUUUUUUS.UD6n U (d [U5eU S$)aGiven a path or paths, return one ``OpenFile`` object. Parameters ---------- urlpath: string or list Absolute or relative filepath. Prefix with a protocol like ``s3://`` to read from alternative filesystems. Should not include glob character(s). mode: 'rb', 'wt', etc. compression: string or None If given, open file using compression codec. Can either be a compression name (a key in ``fsspec.compression.compr``) or "infer" to guess the compression from the filename suffix. encoding: str For text mode only errors: None or str Passed to TextIOWrapper in text mode protocol: str or None If given, overrides the protocol found in the URL. newline: bytes or None Used for line terminator in text mode. If None, uses system default; if blank, uses no translation. expand: bool or None Whether to regard file paths containing special glob characters as needing expansion (finding the first match) or absolute. Setting False allows using paths which do embed such characters. If None (default), this argument takes its value from the DEFAULT_EXPAND module variable, which takes its initial value from the "open_expand" config value at startup, which will be False if not set. **kwargs: dict Extra options that make sense to a particular storage connection, e.g. host, port, username, password, etc. Examples -------- >>> openfile = open('2015-01-01.csv') # doctest: +SKIP >>> openfile = open( ... 's3://bucket/2015-01-01.csv.gz', compression='gzip' ... ) # doctest: +SKIP >>> with openfile as f: ... df = pd.read_csv(f) # doctest: +SKIP ... Returns ------- ``OpenFile`` object. Notes ----- For a full list of the available protocols and the implementations that they map across to see the latest online documentation: - For implementations built into ``fsspec`` see https://filesystem-spec.readthedocs.io/en/latest/api.html#built-in-implementations - For implementations in separate packages see https://filesystem-spec.readthedocs.io/en/latest/api.html#other-known-implementations )rrrrr rzr!r{rrW)DEFAULT_EXPANDrr6) rrrrr rzr!r{rros r$r5r5s\H &~^6F      C (( q6Mr'c SU;a [S5e[U4SU0UD6n[USRSS5(d [S5eUnUVs/sHoURPM nnSSS5 [ U[ 5(a[U5(a[ U[5(aWS$W$s snf!,(df  NT=f) a#Open file(s) which can be resolved to local For files which either are local, or get downloaded upon open (e.g., by file caching) Parameters ---------- url: str or list(str) mode: str Must be read mode storage_options: passed on to FS for or used by open_files (e.g., compression) rIz(Can only ensure local files when readingrr local_fileFzOopen_local can only be used on a filesystem which has attribute local_file=TrueN) rcrgetattrrnamerlstrrr)rrryofr]r9rs r$ open_localrs$ $CDD C 6d 6o 6B 2a588\5 1 1 -   u!&'A' 3Ys^^ 38M8MQx L( s B6B1&B61B66 Cc`US:Xa [U5nUbU[;a[SUS35eU$)NinferzCompression type z not supported)rr rc)rrs r$rrs<g'0 ;e#;,[MHII r'c[U5nSU;a&URSS5up[U5S:aX4$URS5(aURSS5$SU4$)zReturn protocol, path pairrrzdata::N)rrrr startswith)rrzrs r$rr$sfW%G  ua0 x=1 > !'""}}S!$$ =r'cT[U5up[U5nURU5$)zCReturn only path part of full URL, according to appropriate backend)rrr)rrzrrs r$strip_protocolr1s) )KH x (C   w ''r'c/n[U5nSU;a[SU55S:a [S5e[U[ U55nUH7nSU;aUR [ XdU55 M&URU5 M9 [ U5U:aUSUnU$UHFn[U5(a"UR URU55 M5URU5 MH U$)aExpand paths if they have a ``*`` in them (write mode) or any of ``*?[]`` in them (read mode). :param paths: list of paths mode: str Mode in which to open files. num: int If opening in writing mode, number of files we expect to create. fs: filesystem object name_function: callable If opening in writing mode, this callable is used to generate path names. Names are generated for each partition by ``urlpath.replace('*', name_function(partition_index))``. :return: list of paths wc36# UHnSU;dM Sv M g7f)*rNrW).0rs r$ )expand_paths_if_needed..Ls,%Q3!8qq%s  rz;When writing data, only one filename mask can be specified.rN) listsumrcmaxrrextend _expand_pathsr7rglob)rrrwrrxexpanded_paths curr_paths r$expand_paths_if_neededr8s N KE d{ ,%, ,q 0M #s5z"Ii%%mIc&RS%%i0  ~  $+DS1N I##%%bggi&89%%i0  r'c[U[[[45(a0U(d [ S5e[ [ [U555nO [ U5nU=(d 0nU(aXTS'[Xt=(d 05n0n [[U55HCupU upnU [U5S- :Xa[S 0UDU D6n M+[S 0UDU D6U S'XS'XS'ME USupn[U40U D6n[U[[[45(avUVs/sH$n[[ U5U=(d 05SPM& nn[UVs1sHnUSiM sn5S:a [ SU5eUVs/sHnUSPM nnOURU5n[U[[[45(a7U(a[XUUU5nO[U[5(d [U5nOlS U;dS U;aU(a [!XU5nOLS U;aC[#UR%U55Vs/sHnUR'U5(aMUPM nnOU/nUUR(U4$s snfs snfs snfs snf) aJFilesystem, deterministic token, and paths from a urlpath and options. Parameters ---------- urlpath: string or iterable Absolute or relative filepath, URL (may include protocols like ``s3://``), or globstring pointing to data. mode: str, optional Mode in which to open files. num: int, optional If opening in writing mode, number of files we expect to create. name_function: callable, optional If opening in writing mode, this callable is used to generate path names. Names are generated for each partition by ``urlpath.replace('*', name_function(partition_index))``. storage_options: dict, optional Additional keywords to pass to the filesystem class. protocol: str or None To override the protocol specifier in the URL expand: bool Expand string paths for writing, assuming the path is a directory zempty urlpath sequencerzrrrrrz$Protocol mismatch getting fs from %srrrrW)rlrtuplesetrcrnextiterrrrJrrrrrrrsortedrisdir _fs_token)rrrwrxryrzr{urlpath0rrrrrnested_protocolrrrrupchainspcr9s r$r~r~gsR>'D%-..56 6!$tG}"56!'*%+O&. # h 52 6EH8E?+$&!r E Q -b-H-H %)%;B%;(%;!"&5"#,qEQ H ) )B'D%-..LS LSqInQ')>B ? BG   ("1( )A -CWM M!()2A)""5)%$s+,, *5ROEE4((KE 4K3$;F!%CCjIjc=#34jI F5M ! NN*  L D5$- ( (4yCT  L  7  Js:!Dc@^\rSrSrSrSU4SjjrSrSrU=r$)r8izTextIOWrapper cannot be pickled. This solves it. Requires that ``buffer`` be pickleable, which all instances of AbstractBufferedFile are. cH>XX4XV4Ul[TU]"UR6 gr)r@r^r%)r#bufferrr r!line_buffering write_throughr_s r$r% PickleableTextIOWrapper.__init__s$f~T  $))$r'c&[UR4$r)r8r@r)s r$r*"PickleableTextIOWrapper.__reduce__s& 11r')r@)NNNFF) rPrQrRrSrTr%r*rVrtrus@r$r8r8s( %22r'r8) rONutf8NNrNNTT)rONrNNNN)rO)rz#str | list[str] | Path | list[Path]rrryrreturnzstr | list[str])rOrNNNT)1 __future__riologgingrrrrpathlibrfsspec.cachingrrrr r r fsspec.compressionr fsspec.configr fsspec.registryrr fsspec.utilsrrrr getLoggerrrrrYrrrgetrr5rrrrrr~r TextIOWrapperr8rWr'r$rs>"  %<   8 ${{|7;7;x      pf#L/d-/      Rn ,  B (,b    K#\<2b..2r'