o rZh.@stdZddlZddlTddlTedZedZedZedZGdd d Z Gd d d Z Gd d d e Z dS)a CorpusReader for reviews corpora (syntax based on Customer Review Corpus). Customer Review Corpus information ================================== Annotated by: Minqing Hu and Bing Liu, 2004. Department of Computer Science University of Illinois at Chicago Contact: Bing Liu, liub@cs.uic.edu https://www.cs.uic.edu/~liub Distributed with permission. The "product_reviews_1" and "product_reviews_2" datasets respectively contain annotated customer reviews of 5 and 9 products from amazon.com. Related papers: - Minqing Hu and Bing Liu. "Mining and summarizing customer reviews". Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-04), 2004. - Minqing Hu and Bing Liu. "Mining Opinion Features in Customer Reviews". Proceedings of Nineteeth National Conference on Artificial Intelligence (AAAI-2004), 2004. - Xiaowen Ding, Bing Liu and Philip S. Yu. "A Holistic Lexicon-Based Appraoch to Opinion Mining." Proceedings of First ACM International Conference on Web Search and Data Mining (WSDM-2008), Feb 11-12, 2008, Stanford University, Stanford, California, USA. Symbols used in the annotated reviews: :[t]: the title of the review: Each [t] tag starts a review. :xxxx[+|-n]: xxxx is a product feature. :[+n]: Positive opinion, n is the opinion strength: 3 strongest, and 1 weakest. Note that the strength is quite subjective. You may want ignore it, but only considering + and - :[-n]: Negative opinion :##: start of each sentence. Each line is a sentence. :[u]: feature not appeared in the sentence. :[p]: feature not appeared in the sentence. Pronoun resolution is needed. :[s]: suggestion or recommendation. :[cc]: comparison with a competing product from a different brand. :[cs]: comparison with a competing product from the same brand. Note: Some of the files (e.g. "ipod.txt", "Canon PowerShot SD500.txt") do not provide separation between different reviews. This is due to the fact that the dataset was specifically designed for aspect/feature-based sentiment analysis, for which sentence-level annotation is sufficient. For document- level classification and analysis, this peculiarity should be taken into consideration. N)*z ^\[t\](.*)$z%((?:(?:\w+\s)+)?\w+)\[((?:\+|\-)\d)\]z\[(?!t)(p|u|s|cc|cs)\]z##(.*)$c@s:eZdZdZd ddZddZddZd d Zd d ZdS)Reviewz> A Review is the main block of a ReviewsCorpusReader. NcCs"||_|dur g|_dS||_dS)z :param title: the title of the review. :param review_lines: the list of the ReviewLines that belong to the Review. N)title review_lines)selfrrrI/var/www/auris/lib/python3.10/site-packages/nltk/corpus/reader/reviews.py__init__Rs  zReview.__init__cCst|tsJ|j|dS)z Add a line (ReviewLine) to the review. :param review_line: a ReviewLine instance that belongs to the Review. N) isinstance ReviewLinerappend)r review_linerrradd_line]szReview.add_linecCs g}|jD]}||jq|S)a Return a list of features in the review. Each feature is a tuple made of the specific item feature and the opinion strength about that feature. :return: all features of the review as a list of tuples (feat, score). :rtype: list(tuple) )rextendfeatures)rrr rrrrfs zReview.featurescCsdd|jDS)z Return all tokenized sentences in the review. :return: all sentences of the review as lists of tokens. :rtype: list(list(str)) cSsg|]}|jqSr)sent).0r rrr zsz Review.sents..)rrrrrsentsssz Review.sentscCsd|j|jS)Nz#Review(title="{}", review_lines={}))formatrrrrrr__repr__|szReview.__repr__NN) __name__ __module__ __qualname____doc__r rrrrrrrrrMs    rc@s"eZdZdZdddZddZdS)r z A ReviewLine represents a sentence of the review, together with (optional) annotations of its features and notes about the reviewed item. NcCs8||_|dur g|_n||_|durg|_dS||_dSNrrnotes)rrrrrrrr s  zReviewLine.__init__cCsd|j|j|jS)Nz*ReviewLine(features={}, notes={}, sent={}))rrrrrrrrrs zReviewLine.__repr__r)rrrrr rrrrrr s  r c@sleZdZdZeZedfddZdddZddd Z dd d Z dd d Z ddZ ddZ ddZddZdS)ReviewsCorpusReadera Reader for the Customer Review Data dataset by Hu, Liu (2004). Note: we are not applying any sentence tokenization at the moment, just word tokenization. >>> from nltk.corpus import product_reviews_1 >>> camera_reviews = product_reviews_1.reviews('Canon_G3.txt') >>> review = camera_reviews[0] >>> review.sents()[0] # doctest: +NORMALIZE_WHITESPACE ['i', 'recently', 'purchased', 'the', 'canon', 'powershot', 'g3', 'and', 'am', 'extremely', 'satisfied', 'with', 'the', 'purchase', '.'] >>> review.features() # doctest: +NORMALIZE_WHITESPACE [('canon powershot g3', '+3'), ('use', '+2'), ('picture', '+2'), ('picture quality', '+1'), ('picture quality', '+1'), ('camera', '+2'), ('use', '+2'), ('feature', '+1'), ('picture quality', '+3'), ('use', '+1'), ('option', '+1')] We can also reach the same information directly from the stream: >>> product_reviews_1.features('Canon_G3.txt') [('canon powershot g3', '+3'), ('use', '+2'), ...] We can compute stats for specific product features: >>> n_reviews = len([(feat,score) for (feat,score) in product_reviews_1.features('Canon_G3.txt') if feat=='picture']) >>> tot = sum([int(score) for (feat,score) in product_reviews_1.features('Canon_G3.txt') if feat=='picture']) >>> mean = tot / n_reviews >>> print(n_reviews, tot, mean) 15 24 1.6 utf8cCs t||||||_d|_dS)ad :param root: The root directory for the corpus. :param fileids: a list or regexp specifying the fileids in the corpus. :param word_tokenizer: a tokenizer for breaking sentences or paragraphs into words. Default: `WordPunctTokenizer` :param encoding: the encoding that should be used to read the corpus. z README.txtN) CorpusReaderr _word_tokenizerZ_readme)rrootfileidsZword_tokenizerencodingrrrr s  zReviewsCorpusReader.__init__Ncs>|durj}nt|tr|g}tfdd|dDS)au Return a list of features. Each feature is a tuple made of the specific item feature and the opinion strength about that feature. :param fileids: a list or regexp specifying the ids of the files whose features have to be returned. :return: all features for the item(s) in the given file(s). :rtype: list(tuple) Nc"g|] \}}j|j|dqS)r&) CorpusView_read_featuresrfileidencrrrrz0ReviewsCorpusReader.features..T)_fileidsr strconcatabspathsrr%rrrrs    zReviewsCorpusReader.featurescs,|durj}tfdd|dDS)aS Return all the reviews as a list of Review objects. If `fileids` is specified, return all the reviews from each of the specified files. :param fileids: a list or regexp specifying the ids of the files whose reviews have to be returned. :return: the given file(s) as a list of reviews. Ncr'r()r)_read_review_blockr+rrrrr.z/ReviewsCorpusReader.reviews..T)r/r1r2r3rrrreviewss   zReviewsCorpusReader.reviewsc tfdd|ddDS)aY Return all sentences in the corpus or in the specified files. :param fileids: a list or regexp specifying the ids of the files whose sentences have to be returned. :return: the given file(s) as a list of sentences, each encoded as a list of word strings. :rtype: list(list(str)) c$g|]\}}}j|j|dqSr()r)_read_sent_blockrpathr-r,rrrrz-ReviewsCorpusReader.sents..Tr1r2r3rrrr   zReviewsCorpusReader.sentscr6)aK Return all words and punctuation symbols in the corpus or in the specified files. :param fileids: a list or regexp specifying the ids of the files whose words have to be returned. :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) cr7r()r)_read_word_blockr9rrrrr;z-ReviewsCorpusReader.words..Tr<r3rrrwordsr=zReviewsCorpusReader.wordscCs<g}tdD]}|}|s|S|tt|q|S)N)rangereadlinerrefindallFEATURES)rstreamrilinerrrr*s z"ReviewsCorpusReader._read_featuresc Cs |}|s gStt|}|rt|dd}nq |}|}|s+|gStt|r9|||gSt t |}t t |}t t |}|rU|j |d}t|||d} || q)NT)rrr)rBrCmatchTITLErgroupstriptellseekrDrENOTESSENTr#tokenizer r) rrFrHZ title_matchreviewoldposZfeatsrrr rrrr4s6         z&ReviewsCorpusReader._read_review_blockcCs0g}||D]}|dd|Dq|S)NcSsg|]}|qSrr)rrrrrrAsz8ReviewsCorpusReader._read_sent_block..)r4rr)rrFrrSrrrr8>sz$ReviewsCorpusReader._read_sent_blockcCsDg}tdD]}|}tt|}|r||j|dq|S)Nr@r)rArBrCrDrQrr#rR)rrFr?rGrHrrrrr>Ds  z$ReviewsCorpusReader._read_word_blockr)rrrrZStreamBackedCorpusViewr)ZWordPunctTokenizerr rr5rr?r*r4r8r>rrrrr s     ! r ) rrCZnltk.corpus.reader.apiZ nltk.tokenizecompilerKrErPrQrr r"r rrrrs8   5