o _Zh-@sPddlZddlZddlZddlmZddlmZddlmZGdddeZ dS) N)AbstractFileSysteminfer_storage_options) MemoryFilecseZdZdZdZdZdZdZ d&fdd Ze d d Z e d'd d Z e ddZ e ddZe ddZd(ddZd)ddZe fddZeddZ    d*ddZd(d d!Zd)d"d#Zd$d%ZZS)+GithubFileSystemaInterface to files in github An instance of this class provides the files residing within a remote github repository. You may specify a point in the repos history, by SHA, branch or tag (default is current master). For files less than 1 MB in size, file content is returned directly in a MemoryFile. For larger files, or for files tracked by git-lfs, file content is returned as an HTTPFile wrapping the ``download_url`` provided by the GitHub API. When using fsspec.open, allows URIs of the form: - "github://path/file", in which case you must specify org, repo and may specify sha in the extra args - 'github://org:repo@/precip/catalog.yml', where the org and repo are part of the URI - 'github://org:repo@sha/precip/catalog.yml', where the sha is also included ``sha`` can be the full or abbreviated hex of the commit you want to fetch from, or a branch or tag name (so long as it doesn't contain special characters like "/", "?", which would have to be HTTP-encoded). For authorised access, you must provide username and token, which can be made at https://github.com/settings/tokens z9https://api.github.com/repos/{org}/{repo}/git/trees/{sha}zChttps://api.github.com/repos/{org}/{repo}/contents/{path}?ref={sha}Zgithub)<r Nc  stjd i|||_||_|du|duArtd||_||_|dur(||_|durJd}tj |j ||dfd|ji|j } | | d}||_|dzddlm} | d i||_WdStypd|_YdSw) Nz%Auth required both username and tokenz)https://api.github.com/repos/{org}/{repo}orgrepotimeoutZdefault_branchr)HTTPFileSystem)super__init__r r ValueErrorusernametokenr requestsgetformatkwraise_for_statusjsonrootlshttprhttp_fs ImportError) selfr r sharrr kwargsurr __class__rL/var/www/auris/lib/python3.10/site-packages/fsspec/implementations/github.pyr.s8      zGithubFileSystem.__init__cCs|jr d|j|jfiSiS)Nauth)rrr!rrr(rLszGithubFileSystem.kwTcCs@tjdddg|d|d|jd}|dd|DS) avList repo names for given org or user This may become the top level of the FS Parameters ---------- org_or_user: str Name of the github org or user to query is_org: bool (default True) Whether the name is an organisation (True) or user (False) Returns ------- List of string zhttps://api.github.com/ZusersZorgs/z/repos)r cSg|]}|dqSnamer).0r rrr( hz*GithubFileSystem.repos..)rrr rr)clsZ org_or_userZis_orgr%rrr(reposRs zGithubFileSystem.reposcCFtjd|jd|jdfd|ji|j}|dd|DS)zNames of tags in the repohttps://api.github.com/repos/r+z/tagsr cSr,r-rr/trrr(r0sr1z)GithubFileSystem.tags..rrr r r rrrr!r%rrr(tagsjzGithubFileSystem.tagscCr4)zNames of branches in the repor5r+z /branchesr cSr,r-rr6rrr(r0~r1z-GithubFileSystem.branches..r8r9rrr(branchesur;zGithubFileSystem.branchescCs|j|jdS)z#Named references, tags and branchesr:r<r=r*rrr(refsszGithubFileSystem.refsFc  s|dkr|p |j}|duradd}d|p |j}|D]=}|jd||d}r5d|n|7fdd|D}|sGt|d}|d d kr\|rX|gSS|d }q#|jvsm||jdfvrtj|j j |j |j |d fd |j i|j} | jdkrt| d ddfdd| dD}||jdfvr||j<n|j}|r|Stdd|DS)a List files at given path Parameters ---------- path: str Location to list, relative to repo root detail: bool If True, returns list of dicts, one per file; if False, returns list of full filenames only sha: str (optional) List at the given point in the repo history, branch or tag name or commit SHA _sha: str (optional) List this specific tree object (used internally to descend into trees) rNr+T)r"_shacsg|] }|dkr|qSr-r)r/o)so_farrr(r0sz'GithubFileSystem.ls..rtypefiler")r r r"r  directory)Zblobtreec sXg|](}|dvrrd|dn|d|d|d|dd|ddqS)rBr+pathmodesizerr")r.rHrBrIr")rr/f)rGtypesrr(r0s   rFcSr,r-rrJrrr(r0r1)_strip_protocolrrstripsplitrFileNotFoundErrordircacherrurlrr r r r status_coderrsorted) r!rGdetailr"r?r#partspartoutr%r)rGrArLr(rsR            zGithubFileSystem.lscCs|jdSN)rQclear)r!rGrrr(invalidate_cachesz!GithubFileSystem.invalidate_cachecs*t|}d|vrt|S|ddS)NrrGr+)rrrMlstrip)r2rGoptsr&rr(rMs z GithubFileSystem._strip_protocolcCs>t|}d|vr iS|d|dd}|dr|d|d<|S)Nrpasswordr hostr"r)rGr]rXrrr(_get_kwargs_from_urlss z&GithubFileSystem._get_kwargs_from_urlsrbc Ks|dkrt|jj|j|j||p|jd}tj|fd|ji|j }|j dkr+t || | } | drIt| d} | dsItdd| S|jdurRtd|jj| df|||d |S) Nrar r rGr"r rDcontents#version https://git-lfs.github.com/zRPlease install fsspec[http] to access github files >1 MB or git-lfs tracked files.Z download_url)rH block_size cache_options)NotImplementedError content_urlrr r rrrr rrSrPrrbase64 b64decode startswithrrr open) r!rGrHrdrer"r#rRr%Z content_jsonrcrrr(_opens6     zGithubFileSystem._opencCs0|j|||d}t|D] }|j||dq dS)N) recursivemaxdepth)message)Z expand_pathreversedrm_file)r!rGrmrnroprrr(rms zGithubFileSystem.rmc Ks$|jstd||}||}|sC|jj|j|j|d|j d}t j |fd|j i|j }|jdkr9t|||d}|jj|j|j||j d}|j }|pYd||d|rbd |ini} t j|f| |j d |j }| d d } td | rd} t| |||dS)aW Remove a file from a specified branch using a given commit message. Since Github DELETE operation requires a branch name, and we can't reliably determine whether the provided SHA refers to a branch, tag, or commit, we assume it's a branch. If it's not, the user will encounter an error when attempting to retrieve the file SHA or delete the file. Parameters ---------- path: str The file's location relative to the repository root. message: str, optional The commit message for the deletion. zAuthentication requiredr+rbr rDr"zDelete )ror"branch)rr rorzBranch .+ not foundzTRemove only works when the filesystem is initialised from a branch or default (None)N)rrrM_get_sha_from_cachergrr r r\rrrr rrSrPrrdeleteresearchr[) r!rGror#r"rRr%Z delete_urlrtdata error_messageerrorrrr(rqs:      zGithubFileSystem.rm_filecCsL|jD]}|D]}|d}|r"||kr"d|vr"|dSq qdS)Nr.r")rQvaluesr)r!rGentriesentry entry_pathrrr(ruIs z$GithubFileSystem._get_sha_from_cache)NNNN)T)FNNrY)raNNN)__name__ __module__ __qualname____doc__rRrgprotocolr rpropertyr classmethodr3r:r<r>rr[rM staticmethodr`rlrsrqru __classcell__rrr&r(r s@       B   3 6r) rhrwrspecrutilsrZmemoryrrrrrr(s