o rZh#@s|dZddlZddlZddlZddlmZddlTddlm Z ddl Tddl Tddl m Z GdddeZGd d d eZdS) zN A reader for corpora that contain chunked (and optionally tagged) documents. N tagstr2tree)*)BracketParseCorpusReader)Treec@seZdZdZdeedddeddfdd Zdd d Zdd d Z dddZ dddZ dddZ dddZ dddZdddZdddZddZdS) ChunkedCorpusReadera& Reader for chunked (and optionally tagged) corpora. Paragraphs are split using a block reader. They are then tokenized into sentences using a sentence tokenizer. Finally, these sentences are parsed into chunk trees using a string-to-chunktree conversion function. Each of these steps can be performed using a default function or a custom function. By default, paragraphs are split on blank lines; sentences are listed one per line; and sentences are parsed into chunk trees using ``nltk.chunk.tagstr2tree``.  T)Zgapsutf8Nc Cs"t||||||||f|_dS)z :param root: The root directory for this corpus. :param fileids: A list or regexp specifying the fileids in this corpus. N) CorpusReader__init___cv_args) selfrootfileids extension str2chunktreesent_tokenizerpara_block_readerencodingtagsetrI/var/www/auris/lib/python3.10/site-packages/nltk/corpus/reader/chunked.pyr &szChunkedCorpusReader.__init__ctfdd|dDS)z~ :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) c s,g|]\}}t||ddddgjRqS)rChunkedCorpusViewr .0fencrrr Az-ChunkedCorpusReader.words..TconcatZabspathsrrrr rwords:s   zChunkedCorpusReader.wordscr)z :return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(list(str)) c s,g|]\}}t||ddddgjRqSrrrr rrr!Or"z-ChunkedCorpusReader.sents..Tr#r%rr rsentsG   zChunkedCorpusReader.sentscr)z :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of word strings. :rtype: list(list(list(str))) c s,g|]\}}t||ddddgjRqSr'rrr rrr!]r"z-ChunkedCorpusReader.paras..Tr#r%rr rparasUr*zChunkedCorpusReader.parasc tfdd|dDS)z :return: the given file(s) as a list of tagged words and punctuation symbols, encoded as tuples ``(word,tag)``. :rtype: list(tuple(str,str)) c s2g|]\}}t||ddddgjRdiqSr(r target_tagsetrrrrrrr!k z4ChunkedCorpusReader.tagged_words..Tr#rrrrr/r tagged_wordsc   z ChunkedCorpusReader.tagged_wordscr,)z :return: the given file(s) as a list of sentences, each encoded as a list of ``(word,tag)`` tuples. :rtype: list(list(tuple(str,str))) c s2g|]\}}t||ddddgjRdiqSr-rrr/rrr!{r0z4ChunkedCorpusReader.tagged_sents..Tr#r1rr/r tagged_sentssr3z ChunkedCorpusReader.tagged_sentscr,)z :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of ``(word,tag)`` tuples. :rtype: list(list(list(tuple(str,str)))) c s2g|]\}}t||ddddgjRdiqSr-rrr/rrr!r0z4ChunkedCorpusReader.tagged_paras..Tr#r1rr/r tagged_parasr3z ChunkedCorpusReader.tagged_parascr,)av :return: the given file(s) as a list of tagged words and chunks. Words are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). Chunks are encoded as depth-one trees over ``(word,tag)`` tuples or word strings. :rtype: list(tuple(str,str) and Tree) c s2g|]\}}t||ddddgjRdiqSr-rrr/rrr!r0z5ChunkedCorpusReader.chunked_words..Tr#r1rr/r chunked_words   z!ChunkedCorpusReader.chunked_wordscr,)a6 :return: the given file(s) as a list of sentences, each encoded as a shallow Tree. The leaves of these trees are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). :rtype: list(Tree) c s2g|]\}}t||ddddgjRdiqSr-rrr/rrr!r0z5ChunkedCorpusReader.chunked_sents..Tr#r1rr/r chunked_sentsr7z!ChunkedCorpusReader.chunked_sentscr,)ao :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as a shallow Tree. The leaves of these trees are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). :rtype: list(list(Tree)) c s2g|]\}}t||ddddgjRdiqS)r(r.rrr/rrr!r0z5ChunkedCorpusReader.chunked_paras..Tr#r1rr/r chunked_parasr7z!ChunkedCorpusReader.chunked_parascCsddt|DS)NcSsg|]}t|qSrr)rtrrrr!sz3ChunkedCorpusReader._read_block..)read_blankline_block)rstreamrrr _read_blockszChunkedCorpusReader._read_block)NNN)__name__ __module__ __qualname____doc__rZRegexpTokenizerr;r r&r)r+r2r4r5r6r8r9r=rrrrrs&          rc@s*eZdZ  dddZddZddZdS) rNc CsJtj|||d||_||_||_||_||_||_| |_| |_ | |_ dS)N)r) StreamBackedCorpusViewr _tagged_group_by_sent_group_by_para_chunked_str2chunktree_sent_tokenizer_para_block_reader_source_tagset_target_tagset) rZfileidrZtaggedZ group_by_sentZ group_by_parachunkedrrr source_tagsetr.rrrr s zChunkedCorpusView.__init__cCsg}||D]B}g}|j|D])}|j||j|jd}|js%||}|js,| }|j r5| |q| |q|j rD| |q| |q|S)N)rNr.)rJrItokenizerHrKrLrD_untagrGleavesrEappendextendrF)rr<blockZpara_strparaZsent_strsentrrr read_blocks(     zChunkedCorpusView.read_blockcCsJt|D]\}}t|tr||qt|tr|d||<qtd|S)Nrz"expected child to be Tree or tuple) enumerate isinstancerrPtuple ValueError)rtreeichildrrrrP s   zChunkedCorpusView._untagr>)r?r@rAr rWrPrrrrrs    "r)rBcodecsZos.pathosZnltkZ nltk.chunkrZnltk.corpus.reader.apiZ nltk.corpus.reader.bracket_parserZnltk.corpus.reader.utilZ nltk.tokenizeZ nltk.treerr rrCrrrrrs   4