7ThuSSKrSSKrSSKrSSKrSSKrSSKJr SSKJr SSKrSSK r SSK J r J r J r Jr SSKJr SSKJr SSKJr SSKJrJrJrJrJr S S KJr \R:"S 5r\R:"S 5r\R@"S 5r!Sr""SS\ 5r#"SS\5r$\R:"S5r%Sr&"SS\5r'"SS\ 5r(SSjr)SSjr*SSjr+\"\+5r,g)N)copy)urlparse)AbstractAsyncStreamedFileAsyncFileSystemsync sync_wrapper)DEFAULT_CALLBACK)FSTimeoutError)AbstractBufferedFile)DEFAULT_BLOCK_SIZEglob_translate isfilelike nullcontexttokenize)AllBytesz6<(a|A)\s+(?:[^>]*?\s+)?(href|HREF)=["'](?P[^"']+)z-(?Phttp[s]?://[-a-zA-Z0-9@:%_+.~#?&/=]+)z fsspec.httpc6# [R"S0UD6$7f)N)aiohttp ClientSession)kwargss S/var/www/auris/envauris/lib/python3.13/site-packages/fsspec/implementations/http.py get_clientr s  *6 **sc B^\rSrSrSrSrSSSSSSSSS\S4 U4Sjjr\S 5r S r \ S 5r S r \S 5r\U4Sj5rS"SjrS"Sjr\"\5rSrS#SjrS\4SjrS\SS4SjrSrSrS$SjrS%SjrSrSrS&Sjr Sr!S'S jr"S!r#U=r$$)(HTTPFileSystem$a Simple File-System for fetching data via HTTP(S) ``ls()`` is implemented by loading the parent page and doing a regex match on the result. If simple_link=True, anything of the form "http(s)://server.com/stuff?thing=other"; otherwise only links within HTML href tags will be used. /TNbytesFc  >[TU]"U4XxS.U D6 UbUO[UlXlX0lXPlX`lU =(d 0UlXl Xl Xl SUl [U 5n U RSS5UlU RSS5 U RSS5 U RSS5 Xl g)a  NB: if this is called async, you must await set_client Parameters ---------- block_size: int Blocks to read bytes; if 0, will default to raw requests file-like objects instead of HTTPFile instances simple_links: bool If True, will consider both HTML tags and anything that looks like a URL; if False, will consider only the former. same_scheme: True When doing ls/glob, if this is True, only consider paths that have http/https matching the input URLs. size_policy: this argument is deprecated client_kwargs: dict Passed to aiohttp.ClientSession, see https://docs.aiohttp.org/en/stable/client_reference.html For example, ``{'auth': aiohttp.BasicAuth('user', 'pass')}`` get_client: Callable[..., aiohttp.ClientSession] A callable which takes keyword arguments and constructs an aiohttp.ClientSession. It's state will be managed by the HTTPFileSystem class. storage_options: key-value Any other parameters passed on to requests cache_type, cache_options: defaults used in open ) asynchronousloopNuse_listings_cacheFlistings_expiry_time max_pathsskip_instance_cache)super__init__r block_size simple_links same_schema cache_type cache_options client_kwargsrencodedr_sessionrpopr")selfr)r( same_scheme size_policyr+r,r r!r-rr.storage_optionsrequest_options __class__s rr'HTTPFileSystem.__init__0sT WLWW(2(>*DV(&$**0b$ %  /"1"5"56JE"R2D9K.148% cg)Nhttprr1s rfsidHTTPFileSystem.fsidqsr8c>[R"XRS9$)N)r.)yarlURLr.r1urls r encode_urlHTTPFileSystem.encode_urlusxx\\22r8cUb+UR5(a[XRSS9 g[ USS5nUbUR5 gg![[[ 4a N9f=f)Ng?)timeout _connector) is_runningrclose TimeoutErrorr NotImplementedErrorgetattr_close)r!session connectors r close_sessionHTTPFileSystem.close_sessionxsm   1 1 T==#6G\48      !!.2EF  sAA)(A)c8# URc|UR"SSUR0URD6IShvN UlUR(d6[ R "XRURUR5 UR$N\7f)Nr!r)r/rr!r-r weakreffinalizerPr;s r set_sessionHTTPFileSystem.set_sessionsm == "&//"Wtyy"WDDVDV"WWDM$$  '9'9499dmmT}}Xs9BBABcU$)z-For HTTP, we always want to keep the full URLr)clspaths r_strip_protocolHTTPFileSystem._strip_protocols  r8cF>[TU]U5n[U5S:aU$g)N)r&_parentlen)rXrYparr6s rr_HTTPFileSystem._parents&god# s8a<Jr8c 4# URR5nURU5 [R U5 UR 5IShvN nUR "URU540URD6IShvN nURXa5 UR5IShvN nUR(a?[RU5[RU5Vs/sHoSPM sn-n O([RU5Vs/sHoSPM n nSSS5IShvN [5n [!U5n W GHjn [#U [$5(aU Sn U R'S5(a,[)U 5S:aU R*SU R,U 3n U R'S5(aUR.(a;U R'UR1S5S-5(aU R3U 5 MU R5SS5R'UR5SS5R1S5S-5(aU R3U 5 GMGM!U S;dGM*U R3SR7UR1S5U R9S5/55 GMm U (d=UR;S5(a'UR=UR1S5SS 9IShvN n U(a/U Vs/sH!nUSUR;S5(aS OS S .PM# sn$[?U 5$GNGNGNs snfs snf![a /n GN=f=fGN6!,IShvN (df  GNL=fNs snf7f) Nrrz://r:https)z..z../F)detail directoryfilenamesizetype) rrupdateloggerdebugrUgetrC_raise_not_found_for_statustextr)ex2findallexUnicodeDecodeErrorsetr isinstancetuple startswithr`schemenetlocr*rstripaddreplacejoinlstripendswith_ls_realsorted) r1rBrfrkwrNrrrulinksoutpartsls rrHTTPFileSystem._ls_reals [[    & S((**;;ts3Ct{{CCq  , ,Q 4 VVX~$$KK-rzz$?O0P?O!1?O0PPE+-::d+;<+;aqT+;EEsANM 4N M NM6"M 5M6AM 6MM  M6 M !M/M 1M62 N=M3>ENBNNN(N?NNM  M  M0,M6/M00M63N6N<M? = N NNc# UR(a XR;aURUnU$UR"U4SU0UD6IShvN nX@RU'U$N7f)Nrf)r"dircacher)r1rBrfrrs r_lsHTTPFileSystem._lss`  " "smm';--$C  cC&CFCCC!$MM#  DsA A# A! A#cZURS:Xa [U5eUR5 g)zE Raises FileNotFoundError for 404s, otherwise uses raise_for_status. iN)statusFileNotFoundErrorraise_for_status)r1responserBs rrq*HTTPFileSystem._raise_not_found_for_statuss' ??c !#C( (!!#r8cj# URR5nURU5 [R U5 UcUbGX#:XagUR S05R5nUR XU5IShvN US'XeS'UR5IShvN nUR"URU540UD6IShvN nUR5IShvN n URX5 SSS5IShvN U $NNlNCN-N!,IShvN (df  W $=f7f)Nr8headersRange) rrrmrnror0_process_limitsrUrprCreadrq) r1rBstartendrrrrNrrs r _cat_fileHTTPFileSystem._cat_files [[    & S  |ffY+002G%)%9%9#c%JJGG #yM((**;;ts3:r::a.C  , ,Q 4;: K*: ;:: s~BD3DD3#D$*D3DD3D&D'D< D3DD3D3D3DD3D0D!  D0+D3iPc# URR5nURU5 [R U5 UR 5IShvN nUR "URU540UD6IShvN n[URS5n URU 5 URX5 [U5(aUn O [US5n Sn U (aWUR R#U5IShvN n U R%U 5 UR'[)U 55 U (aMW[U5(dU R+5 SSS5IShvN gGNN![[4a Sn Nf=fN![U5(dU R+5 ff=fNJ!,IShvN (df  g=f7f)Nzcontent-lengthwbT)rrrmrnrorUrprCintr ValueErrorKeyErrorset_sizerqropencontentrwriterelative_updater`rI) r1rpathlpath chunk_sizecallbackrrrNrrkoutfilechunks r _get_fileHTTPFileSystem._get_fileso[[    & U((**;;tu5<<< 199%567   d #  , ,Q 6%  ud+ $"#)).."<# [T[R5(a[T5nSnO[ TS5nSnUnU(a3TR UR SS55 UR S5 OTR [USS55 URT5nU(a9U7v TR[U55 URT5nU(aM9SSS5 g!,(df  g=f7f)NFrbTrrrk) rxioIOBaserrrseekrLrrr`)contextuse_seekfrrrrs r gen_chunks,HTTPFileSystem._put_file..gen_chunkss%++%e, ud+A%%affQl3FF1I%%ga&>?z*K,,SZ8FF:.Ees>DB&C2) D2 D<D)rputz.method has to be either 'post' or 'put', not: data) rKrrrmrUlowerrrLrCrq) r1rrrrmethodmoderrrrNmethresps ` `` r _put_fileHTTPFileSystem._put_file s ; %&78 8 /.[[    &((**  (@ K w'.HZ\HRHHD  , ,T 9IHH+IHHHs[AC<CAC<5C6C<9C" C<C C<C< C<"C9(C+ ) C95C<c# URR5nURU5 [R U5 UR 5IShvN nUR "URU540UD6IShvN nUIShvN URS:sSSS5IShvN $NYN1N(N !,IShvN (df  g=f![Ra gf=f7f)NiF) rrrmrnrorUrprCrr ClientError)r1rYrrrNrs r_existsHTTPFileSystem._exists?s [[    &  LL  ,,..Gkk$//$"7>2>>Aqxx#~qq/>qqq""  s,C-(CB1)CB3 C B5 CB9 C+B7,C0C-1C3C5C7C9C?C C CC-CC*'C-)C**C-cD# UR"U40UD6IShvN $N7fN)r)r1rYrs r_isfileHTTPFileSystem._isfileKs\\$1&1111s   c lUS:wa[eUbUO URnURR5n URU S'U R U5 0n U=(d. U R UR "U40UD65=(d U Sn[URUR5n U(acU(a\U RSS5(aE[UU4U UUUU=(d URU=(d URURS.U D6$[UU4UURU S.U D6$)aSMake a file-like object Parameters ---------- path: str Full URL with protocol mode: string must be "rb" block_size: int or None Bytes to download in one request; use instance value if None. If zero, will return a streaming Requests file-like instance. kwargs: key-value Any other parameters, passed to requests calls rr rkpartialT)rNr(rrkr+r,r!)rr!rN)rKr(rrr rminforr!rUrpHTTPFiler+r,HTTPStreamFile) r1rYrr( autocommitr+r,rkrrrrNs r_openHTTPFileSystem._openNs-2 4<% %#-#9Zt [[   !..> &Mt{{499T#c$i&*iin&9DIIcNs4y h*He,\\$1&111 6M 4(B6(B"BCCII D'N "8Gn++C0G 'A+&D1'--c2Q6ED1'--c2Q6E 4<##'99T? %)*;%<%B%B3%G!%K"2X=   4 @F  !R!HI**W%"(.."23  44}}QXXc]+.LK/JJsOO   4   J9 a2#C,   sPC:K<J3=)K&J6'B>K%J9&AK.,J;>J;K6K9K;Kc# [URU5IShvN 5$N![[4a gf=f7f)NF)boolrrr)r1rYs r_isdirHTTPFileSystem._isdirs9 dhhtn,- -,!:.  s(>(&(>(;>;>cl# URU5nURS05n[[U55US'UR 5IShvN nUR "U4X&S.UD6IShvN nUR 5 SSS5IShvN gNCN'N !,IShvN (df  g=f7f)a( Write bytes to a remote file over HTTP. Parameters ---------- path : str Target URL where the data should be written value : bytes Data to be written mode : str How to write to the file - 'overwrite' or 'append' **kwargs : dict Additional parameters to pass to the HTTP request rContent-LengthN)rr)rZr0strr`rUrr) r1rYvaluerrrBrrNrs r _pipe_fileHTTPFileSystem._pipe_file s""4(**Y+$'E O !((**;;sJJ6JJa   KJJ+JJJJsZAB4BB4.B/B42B B4BB4B4B4B1 B# ! B1-B4) r/r(r,r+r-r.rrr*r)r")T)NN)rNNNNN)rNr)r)%__name__ __module__ __qualname____firstlineno____doc__seprr'propertyr<rC staticmethodrPrU classmethodrZr_rrrlsrqrr rrrrrrrrrr r__static_attributes__ __classcell__r6s@rrr$s  C ?&B3  0d c B$&(1;K$D! 0:d 2  7r 9 CDGR!!r8rc^\rSrSrSrS U4SjjrS U4SjjrSr\"\5r Sr Sr \"\ 5r Sr U=r$) ri&aC A file-like object pointing to a remote HTTP(S) resource Supports only reading, with read-ahead of a predetermined block-size. In the case that the server does not supply the filesize, only reading of the complete file in one go is supported. Parameters ---------- url: str Full URL of the remote resource, including the protocol session: aiohttp.ClientSession or None All calls will be made within this session, to avoid restarting connections where the server allows this block_size: int or None The amount of read-ahead to do, in bytes. Default is 5MB, or the value configured for the FileSystem creating this file size: None or int If given, this is the size of the file in bytes, and we don't attempt to call the server to find the value. kwargs: all other key-values are passed to requests calls. c >US:wa [S5eXlXlX lX0lX(SS.Ul[ T U]"SUUUUUUS.U D6 g)NrzFile mode not supportedrhri)fsrYrr(r+r,r)rKr r!rBrNdetailsr&r') r1r!rBrNr(rr+r,rkr!r rr6s rr'HTTPFile.__init__?sg 4<%&?@ @(  #6B   !!'   r8cJ>US:aGURS:Xa7URbURUR::dUR5 URcUS:aUR5 O#[ URUR- U5n[ TU]U5$)zRead bytes from file Parameters ---------- length: int Read up to this many bytes. If negative, read all content to end of file. If the server has not supplied the filesize, attempting to read only part of the data will raise a ValueError. r)locrk blocksize _fetch_allrr&r)r1lengthr6s rr HTTPFile.read^s}aZDHHMYY*tyyDNN/J OO  99 z!TXX-v6Fw|F##r8c># [RSU35 [UR[5(dUR R "URRUR540URD6IShvN nUIShvN UR5 UR5IShvN n[ [U5SSUS9Ul[U5UlSSS5IShvN ggNsNjNDN !,IShvN (df  g=f7f)zRead whole file in one shot, without caching This is only called when position is still at zero, and read() is called without a byte-count. zFetch all for N)rkfetcherr&r)rnrorxcacherrNrpr!rCrBrrrr`rk)r1rrs rasync_fetch_allHTTPFile.async_fetch_allus  ~dV,-$**h//ll&&tww'9'9$(('CSt{{SSAq""$FFHn%S44c  H qq0S$qqqslBDC; DC=D$D:C?;.D) D4D5D=D?DDD D DDc"URSS5n[R"SU5nU(dgUSS:XaS=pEO0USRS5Vs/sHn[ U5PM snupEUS S:XaSO [ US 5nXEU4$s snf) zParse the Content-Range header Content-Ranger^zbytes (\d+-\d+|\*)/(\d+|\*))NNNrdrN-r)rprrsplitr)r1rsmrrxtotals r_parse_content_rangeHTTPFile._parse_content_ranges KK , HH3Q 7# Q43; EC*+A$**S/:/Q#a&/:JE! QqT5  ;sB c# [RSUSUSU35 URR5nUR S05R5nSUSUS- 3US'[RUR SUS35 UR R"URRUR 54SU0UD6IS hvN nUIS hvN URS :XaS S S 5IS hvN g UR5 URS :H=(dU URUR5S U:H=(d- [URRSUS-55X!- :*nU(aUR5IS hvN nOUS :a [!S5eS n/nUR"RS5IS hvN n U (a)UR%U 5 U['U 5- nXU- :aOOOMVS R)U5S X!- nUsS S S 5IS hvN $GNbGNZGN<NNkN!,IS hvN (df  g =f7f)a Download a block of data The expectation is that the server returns only the requested bytes, with HTTP code 206. If this is not the case, we first check the headers, and then stream the output - if the data size is bigger than we requested, an exception is raised. zFetch range for z: r1rbytes=rdrz : Nir8rr zThe HTTP server doesn't appear to support range requests. Only reading this file from the beginning is supported. Open with block_size=0 for a streaming file interface.i)rnrorrr0rBrNrpr!rCrrr7rrrrrappendr`r) r1rrrrrresponse_is_rangerclrs rasync_fetch_rangeHTTPFile.async_fetch_ranges#  'vRwau=>!!#**Y+002#E7!C!G95 zWW%5$678,,"" GG  txx ( 29 =C  1xx311    CP,,QYY7:eCPqyy}}%5sQw?@CKO  !FFHn M"#)).."77E 5)c%j(e +!,hhsmMck2S11 $%8?111sCI H7 I  H:!I $I6 I H=I BII9IIAI% I 1I2I :I =I III I I II )r r,r"r!rNrkrB)NNrrNNNF)rrrrrr'rr-rr'r7r? _fetch_rangerrrs@rrr&sU8    >$.%"o.J !9v 12Lr8rz([*[])c4[RU5nUSL$r) magic_checksearch)r3rs rrrs   q !E  r8cb^\rSrSrSU4SjjrS SjrS Sjr\"\5rSr U4Sjr Sr U=r $) ric ">^^^TRSS5TlTTlUTlUTlUS:wa[ eTSS.Tl[TT] "SUTUSS.TD6 UUU4Sjn[TRU5Tl URTlg) Nr Frrnoner!rYrr+c># TRR"TRRT540TD6R 5IShvN nTRR UT5 U$N"7fr)rNrpr!rC __aenter__rq)rrr1rBs rcor$HTTPStreamFile.__init__..corsYll&&tww'9'9#'>I&ITTVVA GG / /3 7HWsAA0 A. #A0r) r0r rBr!rNrr"r&r'rr) r1r!rBrr!rNrrMr6s ` ` ` rr'HTTPStreamFile.__init__s"JJ~u=  4<  #T2  QBStQ&Q  dii%GG r8c^US:XaUS:XagXR:XaUS:Xag[S5e)NrrdzCannot seek streaming HTTP file)r%r)r1r%whences rrHTTPStreamFile.seeks0 !8!  ((?v{ :;;r8c# URRRU5IShvN nU=R[ U5- slU$N$7fr)rrrr%r`)r1numrs r_readHTTPStreamFile._reads<FFNN'',, CH -s)AA%Ac@# URR5 g7fr)rrIr;s rrMHTTPStreamFile._closes  sc>[R"UR5UR5 [TU]5 gr)asynciorun_coroutine_threadsaferMr!r&rIr1r6s rrIHTTPStreamFile.closes&(( B  r8)r r"r!rrNrB)rNN)rrA) rrrrr'rrUrrrMrIrrrs@rrrs-$<  Dr8rcD^\rSrSrSU4SjjrSSjrU4SjrSrU=r$)ric >X lXPlSUlUS:wa[eUSS.UlXpl[ TU]XUSS9 X`lg)NrrrIrJ) rBrNrrr"rr&r'rk) r1r!rBrr!rNrkrr6s rr'AsyncStreamFile.__init__sM  4<  #T2   BtG r8c# URcURR"URR UR 540UR D6R5IShvN nURRX R 5 X lURRRU5IShvN nU=R[U5- sl U$N|N&7fr) rrNrpr!rCrBrrLrqrrr%r`)r1rTrrs rrAsyncStreamFile.reads 66>ll&&""488,04 jlA GG / /88 <FFFNN'',, CH  -s%A)C,+C(,AC,C*%C,*C,c># URb!URR5 SUl[TU]5IShvN gN7fr)rrIr&r\s rrIAsyncStreamFile.closes3 66  FFLLNDFgmosAA A A )r"rrrNrkrB)rNNNrA) rrrrr'rrIrrrs@rrrs@D  r8rc.# UR5nURS05R5nSUSUS- 3US'UR"U4SU0UD6IShvN nUR5 UIShvN UR 5IShvN nSSS5IShvN U(a8[ US5n U R U5 U RW5 SSS5 gW$NNoNYNK!,IShvN (df  N`=f!,(df  g=f7f)Nrr:r1rdrzr+b)rr0rprrrrr) rNrBrrrhrrrrrs r get_rangerf%s [[]FjjB',,.Gwaay1GGkk#9w9&99AqFFHnq $ ! FF5M GGCL   :qqq sADC"D4C$5D8C* C& C* DC(D4#D D$D&C*(D*D0C3 1 D=D DDc# [RSU5 UR5nURSS5nUR S05R5nSUS'XSS'0nUS:XaUR "U4SU0UD6IShvN nO2US :XaUR"U4SU0UD6IShvN nO[ S U35eUIShvN UR5 S UR;a?S UR;dURS S ;a[URS 5US'O=SUR;a-[URSRS5S5US'SUR;a$URSRS5SUS'URR S5S:XaSUS'[UR5US'SH6nURR U5(dM%URUXh'M8 SSS5IShvN U$GNGN|GNeN!,IShvN (df  U$=f7f)zCall HEAD on the server to get details about the file (size/checksum etc.) Default operation is to explicitly allow redirects and use encoding 'identity' (no compression) to get the true size of the target. zRetrieve file size for %sallow_redirectsTridentityzAccept-EncodingrNrpz)size_policy must be "head" or "get", got r zContent-Encoding)rir^rkr0rrdz Content-Type;rmimetypez Accept-RangesrIFrrB)ETagz Content-MD5Digest)rnrorr0rpr TypeErrorrrrr2 partitionrrB) rBrNr3rarrrrchecksum_fields rrr6s  LL,c2 [[]F %t ,B ::i $ ) ) +D(D 9 Df,,sABA&A A  ++c@2@@ @CK=QRRq  qyy ("2aii@R6SX7 #199-=#>?V  )qyy9??DQGHDL QYY & yy8BB3GJD  99== )V 3$DO!%%jU ?Nyy}}^,,'(yy'@$@1q8 KC B @qqq8 KssBIH!#I(H$)IH'ID0H,:H, IH*I$I'I*I,I2H5 3 I?Ic# Uc[5IShvN n[U/UQ7SU0UD6IShvN nURS5$N.N7f)NrNrk)rrrp)rBrNargsrrs r _file_sizertisG" $CB4BB6B BD 88F % BsAAAAAAr)r)-rZrloggingrrSr urllib.parserrr? fsspec.asynrrrrfsspec.callbacksr fsspec.exceptionsr fsspec.specr fsspec.utilsr r rrrcachingrrrurs getLoggerrnrrrrErrrrfrrt file_sizerr8rrs  ! VV-,,ZZMNjjEF   = )+!_!Dk3#k3\jj"  &)&R/@"0f  $ r8