o rZh$@sbdZddlmZddlmZmZmZGdddeZd ddZGd d d e Z Gd d d eZ dS)zACorpus reader for the XML version of the British National Corpus.)concat) ElementTreeXMLCorpusReader XMLCorpusViewc@sTeZdZdZdddZdddZdd d Zdd d Zdd dZdddZ ddZ dS)BNCCorpusReadera7Corpus reader for the XML version of the British National Corpus. For access to the complete XML data structure, use the ``xml()`` method. For access to simple word lists and tagged word lists, use ``words()``, ``sents()``, ``tagged_words()``, and ``tagged_sents()``. You can obtain the full version of the BNC corpus at https://www.ota.ox.ac.uk/desc/2554 If you extracted the archive to a directory called `BNC`, then you can instantiate the reader as:: BNCCorpusReader(root='BNC/Texts/', fileids=r'[A-K]/\w*/\w*\.xml') TcCst|||||_dSN)r__init___lazy)selfrootfileidsZlazyr E/var/www/auris/lib/python3.10/site-packages/nltk/corpus/reader/bnc.pyrs zBNCCorpusReader.__init__NFcC||dd||S)aT :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. FN_viewsr r strip_spacestemr r rwords#s zBNCCorpusReader.wordscCs|rdnd}||d|||S)a  :return: the given file(s) as a list of tagged words and punctuation symbols, encoded as tuples ``(word,tag)``. :rtype: list(tuple(str,str)) :param c5: If true, then the tags used will be the more detailed c5 tags. Otherwise, the simplified tags will be used. :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. c5posFrr r rrrtagr r r tagged_words/s zBNCCorpusReader.tagged_wordscCr)a :return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(list(str)) :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. TNrrr r rsents?s zBNCCorpusReader.sentscCs |rdnd}|j|d|||dS)a :return: the given file(s) as a list of sentences, each encoded as a list of ``(word,tag)`` tuples. :rtype: list(list(tuple(str,str))) :param c5: If true, then the tags used will be the more detailed c5 tags. Otherwise, the simplified tags will be used. :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. rrT)sentrrrrrr r r tagged_sentsLs  zBNCCorpusReader.tagged_sentscs4|jrtn|jtfdd||DS)zPA helper function that instantiates BNCWordViews or the list of words/sentences.csg|] }|qSr r ).0fileidfrrrrr r asz*BNCCorpusReader._views..)r BNCWordView_wordsrZabspaths)r r rrrrr r rr]s zBNCCorpusReader._viewsc Csg}t|}|dD]U}g} t|D]9} | j} | sd} |s#|r'| } |r/| d| } |dkr;| | df} n|dkrJ| | d| df} | | q|r^|t |j d| q| | qd|vsjJ|S)a Helper used to implement the view methods -- returns a list of words or a list of sentences, optionally tagged. :param fileid: The name of the underlying file. :param bracket_sent: If true, include sentence bracketing. :param tag: The name of the tagset to use, or None for no tags. :param strip_space: If true, strip spaces from word tokens. :param stem: If true, then substitute stems for words. z.//shwrrnN) rparseZgetrootfindall_all_xmlwords_intextstripgetappend BNCSentenceattribextend) r rZ bracket_sentrrrresultZxmldocZxmlsentrZxmlwordwordr r rr$gs,      zBNCCorpusReader._words)T)NTF)NFTF)NFFTF) __name__ __module__ __qualname____doc__rrrrrrr$r r r rrs     rNcCs:|durg}|D]}|jdvr||qt||q|S)N)cw)rr.r*)eltr2childr r rr*s   r*c@seZdZdZddZdS)r/z A list of words, augmented by an attribute ``num`` used to record the sentence identifier (the ``n`` attribute from the XML). cCs||_t||dSr)numlistr)r r<itemsr r rrszBNCSentence.__init__N)r4r5r6r7rr r r rr/s r/c@sBeZdZdZhdZ ddZddZddZd d Zd d Z d S)r#zN A stream backed corpus view specialized for use with the BNC corpus. >aligneventshiftgapZunclearZvocalpauseZpbcCs||rd}nd}||_||_||_||_d|_d|_d|_d|_t |||| | |j d|j |ddi|_dS)aG :param fileid: The name of the underlying file. :param sent: If true, include sentence bracketing. :param tag: The name of the tagset to use, or None for no tags. :param strip_space: If true, strip spaces from word tokens. :param stem: If true, then substitute stems for words. z.*/sz.*/s/(.*/)?(c|w)Nz .*/teiHeader$rr )_sent_tag _strip_space_stemtitleauthoreditorrespsrr_openZ read_block_stream handle_headercloseZ _tag_context)r rrrrrZtagspecr r rrs zBNCWordView.__init__cCs|d}|rddd|D|_|d}|r$ddd|D|_|d}|r6ddd|D|_|d }|rJd d d|D|_dSdS) NztitleStmt/title cs|]}|jVqdSrr+r,)rrHr r r z,BNCWordView.handle_header..ztitleStmt/authorcsrQrrR)rrIr r rrSrTztitleStmt/editorcsrQrrR)rrJr r rrSrTztitleStmt/respStmtz css$|] }ddd|DVqdS)rPcsrQrrR)rZresp_eltr r rrSrTz6BNCWordView.handle_header...N)join)rrespr r rrSs )r)rUrHrIrJrK)r r:contexttitlesZauthorsZeditorsrKr r rrNs     zBNCWordView.handle_headercCs|jr||S||Sr)rD handle_sent handle_word)r r:rWr r r handle_elts  zBNCWordView.handle_eltcCst|j}|sd}|js |jr|}|jr|d|}|jdkr(||df}|S|jdkr8||d|df}|S)Nr%r&rr)r+rFrGr,r-rE)r r:r3r r rrZs    zBNCWordView.handle_wordcsvg}|D].}|jdvr|fdd|D7}q|jdvr%||q|jjvr2td|jqt|jd|S)N)mwhiZcorrtrunccsg|]}|qSr )rZ)rr9r r rr"sz+BNCWordView.handle_sent..)r9r8zUnexpected element %sr')rr.rZtags_to_ignore ValueErrorr/r0)r r:rr;r r_rrYs   zBNCWordView.handle_sentN) r4r5r6r7r`rrNr[rZrYr r r rr#s   r#r) r7Znltk.corpus.reader.utilrZnltk.corpus.reader.xmldocsrrrrr*r=r/r#r r r rs