a h @stddlZddlZddlZddlZddlmZddlmZddlm Z e dZ GdddeZ dd Z dd d ZdS)N)MutableMapping)cached_property) url_to_fszfsspec.mappingc@seZdZdZd%ddZeddZdd Zd&d d Zd dZ ddZ ddZ ddZ d'ddZ d(ddZddZddZddZdd Zd!d"Zd#d$ZdS))FSMapaWrap a FileSystem instance as a mutable wrapping. The keys of the mapping become files under the given root, and the values (which must be bytes) the contents of those files. Parameters ---------- root: string prefix for all the files fs: FileSystem instance check: bool (=True) performs a touch at the location, to check for write access. Examples -------- >>> fs = FileSystem(**parameters) # doctest: +SKIP >>> d = FSMap('my-data/path/', fs) # doctest: +SKIP or, more likely >>> d = fs.get_mapper('my-data/path/') >>> d['loc1'] = b'Hello World' # doctest: +SKIP >>> list(d.keys()) # doctest: +SKIP ['loc1'] >>> d['loc1'] # doctest: +SKIP b'Hello World' FNcCs||_|||_|t|ddd|_|dur@tttf}||_ ||_ ||_ |rn|j |sn|j ||r|j |std|d|j|d|j|ddS)NxzPath z9 does not exist. Create with the ``create=True`` keywordz/a)fsZ_strip_protocolroot posixpathjoin_root_key_to_strFileNotFoundErrorIsADirectoryErrorNotADirectoryErrormissing_exceptionscheckcreateexistsmkdir ValueErrortouchrm)selfr rrrrrfz"FSMap.getitems..r$return)on_errorrNcs(i|] \}}|t|jr tn|qSr) isinstancerKeyErrorr(r)vr"rr nsz"FSMap.getitems..csHi|]@\}}dks"t|ts|dkr4|n|t|qS)r,r$)r. BaseExceptiongetr/)r(keyZk2)r-outrrr2rs)rcatr.bytesrr/itemszip)rkeysr-keys2Zoeer)r-r6rrgetitemsRs   zFSMap.getitemscs&fdd|D}j|dS)zSet the values of multiple items in the store Parameters ---------- values_dict: dict(str, bytes) cs i|]\}}|t|qSr)r& maybe_convertr0r"rrr2r+z"FSMap.setitems..N)r9rpipe)rZ values_dictvaluesrr"rsetitemsxszFSMap.setitemscsjfdd|DdS)z#Remove multiple keys from the storecsg|]}|qSrr%r'r"rrr*r+z"FSMap.delitems..N)rr)rr;rr"rdelitemsszFSMap.delitemscCsDt|ts0tdtt|tr(t|}t|}|j|dS)zGenerate full path for the keyzAfrom fsspec 2023.5 onward FSMap non-str keys will raise TypeError/) r.strwarningswarnDeprecationWarninglisttupler rstrip)rr5rrrr&s  zFSMap._key_to_strcCs|t|jddS)zStrip path of to leave key nameNrD)lenr lstrip)rsrrr _str_to_keyszFSMap._str_to_keyc Csh||}z|j|}WnH|jyb}z.|durD|WYd}~St||WYd}~n d}~00|S)z Retrieve dataN)r&rr7rr/)rr5defaultr)resultexcrrr __getitem__s  zFSMap.__getitem__cCs.|||}z ||=Wnty(Yn0|S)zPop data)rSr/)rr5rPrQrrrpops    z FSMap.popcCs8||}|jj|j|dd|j|t|dS)zStore value in keyT)exist_okN)r&rZmkdirs_parentZ pipe_filer?)rr5valuerrr __setitem__s zFSMap.__setitem__csfddjjDS)Nc3s|]}|VqdSN)rO)r(rr"rr r+z!FSMap.__iter__..)rfindr r"rr"r__iter__szFSMap.__iter__cCst|j|jSrY)rLrr[r r"rrr__len__sz FSMap.__len__c CsFz|j||Wn*ty@}zt|WYd}~n d}~00dS)z Remove keyN)rrr& Exceptionr/)rr5rRrrr __delitem__szFSMap.__delitem__cCs||}|j|S)zDoes key exist in mapping?)r&risfile)rr5rrrr __contains__s zFSMap.__contains__cCst|j|jdd|jffS)NF)rr rrr"rrr __reduce__szFSMap.__reduce__)FFN)r$)N)N)__name__ __module__ __qualname____doc__rrrr#r>rBrCr&rOrSrTrXr\r]r_rarbrrrrr s$   &   rcCsFt|tjst|drBt|dr6|jjdvr6|d}tt|}|S)NZ __array__dtypeZMmint64)r.arrayhasattrrgkindviewr8 memoryview)rWrrrr?s   r?Fc Ks6t|fi|\}}|dur |n|}t|||||dS)aCreate key-value interface for given URL and options The URL will be of the form "protocol://location" and point to the root of the mapper required. All keys will be file-names below this location, and their values the contents of each key. Also accepts compound URLs like zip::s3://bucket/file.zip , see ``fsspec.open``. Parameters ---------- url: str Root URL of mapping check: bool Whether to attempt to read from the location before instantiation, to check that the mapping does exist create: bool Whether to make the directory corresponding to the root before instantiating missing_exceptions: None or tuple If given, these exception types will be regarded as missing keys and return KeyError when trying to read data. By default, you get (FileNotFoundError, IsADirectoryError, NotADirectoryError) alternate_root: None or str In cases of complex URLs, the parser may fail to pick the correct part for the mapper root, so this arg can override Returns ------- ``FSMap`` instance, the dict-like key-value store. N)r)rr) urlrrrZalternate_rootkwargsrZurlpathr rrr get_mappers'rq)rnFFNN)riloggingr rFcollections.abcr functoolsrZ fsspec.corer getLoggerr rr?rqrrrrs     ;