/hMxSrSSKrSSKJr SSKJrJr SSKJr SSK J r J r J r J r SSKJrJr "SS \ 5r"S S \ 5r"S S \ 5r"SS\ 5r"SS\ 5r"SS\ 5r"SS\5r"SS\5r"SS\5r"SS\5r"SS\5rS Sjr\S:Xa\"5 gg)!a Classes and interfaces for associating probabilities with tree structures that represent the internal organization of a text. The probabilistic parser module defines ``BottomUpProbabilisticChartParser``. ``BottomUpProbabilisticChartParser`` is an abstract class that implements a bottom-up chart parser for ``PCFG`` grammars. It maintains a queue of edges, and adds them to the chart one at a time. The ordering of this queue is based on the probabilities associated with the edges, allowing the parser to expand more likely edges before less likely ones. Each subclass implements a different queue ordering, producing different search strategies. Currently the following subclasses are defined: - ``InsideChartParser`` searches edges in decreasing order of their trees' inside probabilities. - ``RandomChartParser`` searches edges in random order. - ``LongestChartParser`` searches edges in decreasing order of their location's length. The ``BottomUpProbabilisticChartParser`` constructor has an optional argument beam_size. If non-zero, this controls the size of the beam (aka the edge queue). This option is most useful with InsideChartParser. N)reduce)PCFG Nonterminal)ParserI)AbstractChartRuleChartLeafEdgeTreeEdge)ProbabilisticTreeTreec\rSrSrSrSrg)ProbabilisticLeafEdge2cg)Ng?selfs I/var/www/auris/envauris/lib/python3.13/site-packages/nltk/parse/pchart.pyprobProbabilisticLeafEdge.prob3srN)__name__ __module__ __qualname____firstlineno__r__static_attributes__rrrrr2srrc0\rSrSrSrSr\S5rSrg)ProbabilisticTreeEdge7cj[R"U/UQ70UD6 XlURU4UlgN)r __init___prob_comparison_key)rrargskwargss rr"ProbabilisticTreeEdge.__init__8s2$000 $ 4 4d;rcUR$r!)r#rs rrProbabilisticTreeEdge.prob>s zzrcZ[X!U4UR5UR5S5$)Nr)rlhsrhs) productionindexps rfrom_production%ProbabilisticTreeEdge.from_productionAs*$ u~z~~/1A1  r)r$r#N) rrrrr"r staticmethodr0rrrrrr7s <   rrc\rSrSrSrSrSrg)ProbabilisticBottomUpInitRuleIrc## [UR55H;n[URU5U5nUR US5(dM7Uv M= g7f)Nr)range num_leavesrleafinsert)rchartgrammarr.new_edges rapply#ProbabilisticBottomUpInitRule.applyLsH5++-.E,UZZ->FH||Hb))/s A A ArNrrrr NUM_EDGESr>rrrrr4r4Is Irr4c\rSrSrSrSrSrg) ProbabilisticBottomUpPredictRuleSc#R# UR5(agUR5HynUR5UR5S:XdM*[R XCR 5UR55nURUS5(dMuUv M{ g7f)Nrr) is_incomplete productionsr+r,rr0startrr:)rr;r<edgeprodr=s rr>&ProbabilisticBottomUpPredictRule.applyVs{      '')DxxzTXXZ]*0@@**, <<"--"N *sA B'AB' B'rNr@rrrrCrCSs I #rrCc\rSrSrSrSrSrg)ProbabilisticFundamentalRulebc#d# UR5UR5:XaLUR5UR5:Xa*UR 5(aUR 5(dgUR 5UR 5-n[UUR5UR54UR5UR5UR5S-S9nSnURU5H!nURXhU4-5(dMSnM# U(aUv gg7f)NrE)spanr+r,dotFT) endrInextsymr+rG is_completerrr,rSchild_pointer_listsr:) rr;r< left_edge right_edger/r= changed_chartcpl1s rr>"ProbabilisticFundamentalRule.applyes MMOz//1 1!!#z~~'77''))&&((  NN z0 0( //#Z^^%56   !#   --i8D||Hj]&:;; $ 9 N s DD0D0rNr@rrrrNrNbs IrrNc2\rSrSrSr\"5rSrSrSr g)&SingleEdgeProbabilisticFundamentalRulerEc## URnUR5(aOURUR5SUR 5S9HnUR XX55ShvN M gURUR 5SUR5S9HnUR XXS5ShvN M gNXN 7f)NT)rIrVr+F)rTrVrU)_fundamental_rulerGselectrTrUr>rIr+)rr;r<edge1fredge2s rr>,SingleEdgeProbabilisticFundamentalRule.applys  # #    iikt&88EEAAA KKMueiik&88EEAAAB Bs%A'C)C*AC8C9 CCcg)NzFundamental Rulerrs r__str__.SingleEdgeProbabilisticFundamentalRule.__str__s!rrN) rrrrrArNrar>rhrrrrr^r^sI46 B"rr^cJ\rSrSrSrS SjrSrS SjrSrSr Sr S r S r g ) BottomUpProbabilisticChartParsera An abstract bottom-up parser for ``PCFG`` grammars that uses a ``Chart`` to record partial results. ``BottomUpProbabilisticChartParser`` maintains a queue of edges that can be added to the chart. This queue is initialized with edges for each token in the text that is being parsed. ``BottomUpProbabilisticChartParser`` inserts these edges into the chart one at a time, starting with the most likely edges, and proceeding to less likely edges. For each edge that is added to the chart, it may become possible to insert additional edges into the chart; these are added to the queue. This process continues until enough complete parses have been generated, or until the queue is empty. The sorting order for the queue is not specified by ``BottomUpProbabilisticChartParser``. Different sorting orders will result in different search strategies. The sorting order for the queue is defined by the method ``sort_queue``; subclasses are required to provide a definition for this method. :type _grammar: PCFG :ivar _grammar: The grammar used to parse sentences. :type _trace: int :ivar _trace: The level of tracing output that should be generated when parsing a text. ch[U[5(d [S5eXlX lX0lg)a Create a new ``BottomUpProbabilisticChartParser``, that uses ``grammar`` to parse texts. :type grammar: PCFG :param grammar: The grammar used to parse texts. :type beam_size: int :param beam_size: The maximum length for the parser's edge queue. :type trace: int :param trace: The level of tracing that should be used when parsing a text. ``0`` will generate no tracing output; and higher numbers will produce more verbose tracing output. z&The grammar must be probabilistic PCFGN) isinstancer ValueError_grammar beam_size_trace)rr<rqtraces rr")BottomUpProbabilisticChartParser.__init__s,'4((EF F " rcUR$r!)rprs rr<(BottomUpProbabilisticChartParser.grammars }}rcXlg)a Set the level of tracing output that should be generated when parsing a text. :type trace: int :param trace: The trace level. A trace level of ``0`` will generate no tracing output; and higher trace levels will produce more verbose tracing output. :rtype: None N)rr)rrss rrs&BottomUpProbabilisticChartParser.traces  rc URRU5 [[U55nURn[ 5n[ 5n[ 5n/nURX#5HUnURS:a1[SURUSS9<SSUR5<S35 URU5 MW [U5S:aURXr5 UR(aUR!Xr5 UR#5nURS:a1[SURUSS9<SSUR5<S35 UR%URX#U55 UR%URX#U55 [U5S:aM[UR'UR)5[*55n 0n UR-5H3n U R5XR/5U R154'M5 U Hn UR3X5 M U R5S S S 9 [7U 5$) NrEz rP)width50z []rTc"UR5$r!r)trees r8BottomUpProbabilisticChartParser.parse..s 499;r)reversekey)rpcheck_coveragerlistr4rCr^r>rrprintpretty_format_edgerappendlen sort_queuerq_prunepopextendparsesrIr rHr+r,_setprobsortiter) rtokensr;r<bu_initburdqueuerJr prod_probsrKparses rr&BottomUpProbabilisticChartParser.parses $$V,d6l#--01 - / 3 5MM%1D{{Q//A/> M LL  2%j1n OOE )~~ E)99;D{{Q//A/> M LL%$7 8 LL%$7 8%%j1n*ell7==?4EFG '')D15Jxxz488:- .*E MM% ,  D&> ?F|rcUR5bg[UR55n/nUHSn[U[5(a*UR [UR555 MBUR U5 MU X#[ U54nUH=n[U[5(dMURXR5 XeR5-nM? URU5 gr!) rrlabelrnr rtuplerset_prob)rrrr+r,childrs rr)BottomUpProbabilisticChartParser._setprobs 99; " $**,'E%&& ;u{{}56 5!  uSz/*E%&& e0 $ drc[5e)a< Sort the given queue of ``Edge`` objects, placing the edge that should be tried first at the beginning of the queue. This method will be called after each ``Edge`` is added to the queue. :param queue: The queue of ``Edge`` objects to sort. Each edge in this queue is an edge that could be added to the chart by the fundamental rule; but that has not yet been added. :type queue: list(Edge) :param chart: The chart being used to parse the text. This chart can be used to provide extra information for sorting the queue. :type chart: Chart :rtype: None )NotImplementedErrorrrr;s rr+BottomUpProbabilisticChartParser.sort_queue,s "##rc[U5UR:aX[U5UR- nURS:a*USUH!n[SUR US5-5 M# USU2 gg)z@Discard items in the queue if the queue is longer than the beam.rPNz %-50s [DISCARDED])rrqrrrr)rrr;splitrJs rr'BottomUpProbabilisticChartParser._prune>sk u: &J/E{{Q!&5MD/%2J2J4QR2SST*fuf 'r)rprrrqN)rr)rP) rrrr__doc__r"r<rsrrrrrrrrrkrks+4* 7r,$$rrkc\rSrSrSrSrSrg)InsideChartParseriHa% A bottom-up parser for ``PCFG`` grammars that tries edges in descending order of the inside probabilities of their trees. The "inside probability" of a tree is simply the probability of the entire tree, ignoring its context. In particular, the inside probability of a tree generated by production *p* with children *c[1], c[2], ..., c[n]* is *P(p)P(c[1])P(c[2])...P(c[n])*; and the inside probability of a token is 1 if it is present in the text, and 0 if it is absent. This sorting order results in a type of lowest-cost-first search strategy. c$URSS9 g)a Sort the given queue of edges, in descending order of the inside probabilities of the edges' trees. :param queue: The queue of ``Edge`` objects to sort. Each edge in this queue is an edge that could be added to the chart by the fundamental rule; but that has not yet been added. :type queue: list(Edge) :param chart: The chart being used to parse the text. This chart can be used to provide extra information for sorting the queue. :type chart: Chart :rtype: None c"UR5$r!r~rJs rr.InsideChartParser.sort_queue..hs DIIKrrNrrs rrInsideChartParser.sort_queueYs  / 0rrNrrrrrrrrrrrrHs   1rrc\rSrSrSrSrSrg)RandomChartParseriz A bottom-up parser for ``PCFG`` grammars that tries edges in random order. This sorting order results in a random search strategy. ch[R"S[U5S- 5nXUSsUS'X'g)NrrE)randomrandintr)rrr;is rrRandomChartParser.sort_queues2 NN1c%j1n -!&59rEHrrNrrrrrrs  6rrc\rSrSrSrSrSrg)UnsortedChartParserizN A bottom-up parser for ``PCFG`` grammars that tries edges in whatever order. cgr!rrs rrUnsortedChartParser.sort_queuesrrNrrrrrrs  rrc\rSrSrSrSrSrg)LongestChartParseriz A bottom-up parser for ``PCFG`` grammars that tries longer edges before shorter ones. This sorting order results in a type of best-first search strategy. c$URSS9 g)Nc"UR5$r!)lengthrs rr/LongestChartParser.sort_queue..s DKKMrrrrs rrLongestChartParser.sort_queues 1 2rrNrrrrrrs 3rrc RSSKnSSKnSSKJn SSKJn [ R"S5n[ R"S5nSU4SU4/n Uc[5 [[U 55H9n [U S -S S XS35 [S XS -5 [5 M; [S S [U 54-SS9 [URR5R55S - nXupU R5n UR!U 5UR#U 5UR%U 5UR'U 5UR!U [U 5S -S9/n/n/n/n0nUHn[SU SUSU 35 UR)S5 UR5n[+UR-U 55nUR/UR5U- 5 U(a[1SUS5[U5- OSnUR/U5 UR/[U55 UHnS UUR35'M M [5 [S5 [S5 [[U55H@n [SXR4R6XR8XUU UU 4-5 MB UR;5nU(aUS5[U5- nOSn[S5 [SSS[U5U4-5 UcY[5 [SSS9 URR5R5R=5R?S5nU(aSS K J!n [S!5 U"U6 UcY[5 [S"SS9 URR5R5R=5R?S5nU(aUHn[U5 M gg! [S5 g=f)#z A demonstration of the probabilistic parsers. The user is prompted to select which demo to run, and how many parses should be found; and then each parser is run on the same demo, and a summary of the results are displayed. rN)tokenize)pcharta7 S -> NP VP [1.0] NP -> Det N [0.5] | NP PP [0.25] | 'John' [0.1] | 'I' [0.15] Det -> 'the' [0.8] | 'my' [0.2] N -> 'man' [0.5] | 'telescope' [0.5] VP -> VP PP [0.1] | V NP [0.7] | V [0.2] V -> 'ate' [0.35] | 'saw' [0.65] PP -> P NP [1.0] P -> 'with' [0.61] | 'under' [0.39] a S -> NP VP [1.0] VP -> V NP [.59] VP -> V [.40] VP -> VP PP [.01] NP -> Det N [.41] NP -> Name [.28] NP -> NP PP [.31] PP -> P NP [1.0] V -> 'saw' [.21] V -> 'ate' [.51] V -> 'ran' [.28] N -> 'boy' [.11] N -> 'cookie' [.12] N -> 'table' [.13] N -> 'telescope' [.14] N -> 'hill' [.5] Name -> 'Jack' [.52] Name -> 'Bob' [.48] P -> 'with' [.61] P -> 'under' [.39] Det -> 'the' [.41] Det -> 'a' [.31] Det -> 'my' [.28] zI saw John with my telescopez:the boy saw Jack with Bob under the table with a telescoperEz>3z: z %rzWhich demo (%d-%d)?  )rTzBad sentence number)rqz s: z parser: z grammar: c&XR5-$r!r~abs rrdemo.. FFH rzC Parser Beam | Time (secs) # Parses Average P(parse)zC------------------------+------------------------------------------z%18s %4d |%11.4f%11d%19.14fc&XR5-$r!r~rs rrr'rrz%18s |%11s%11d%19.14fz (All Parses)zn/azDraw parses (y/n)? y) draw_treesz please wait...zPrint parses (y/n)? )"systimenltkr nltk.parserr fromstringrr7rintstdinreadlinestriprrrrrrsrrrrfreeze __class__rrqkeyslower startswithnltk.draw.treer)choice draw_parses print_parsesrrrr toy_pcfg1 toy_pcfg2demosrsentr<rparserstimes average_p num_parses all_parsesparsertrr/rrs rdemors!  I I: (3 EyQ E ~ s5z"A QU2Jb! . / )ehqk) * G# $3u:6C@SYY'')//12Q6  ZZ\F   )  )""7+!!'*  CK!O D G EIJJ dV:fX[ BC Q IIKfll6*+ TYY[1_%JPF,fa 83v; FVW#f+&A%&Jqxxz " G OP OP 3w<  ) $$-- $$1 !   !__ F ,fa 83v; F  OP &.%Va)P PQ  #-ii((*00288:EEcJ -  !F  $#.yy))+11399;FFsK E %LQ #$s 4P P&__main__)NNN)rr functoolsr nltk.grammarrrnltk.parse.apirnltk.parse.chartrrr r nltk.treer r rrr4rCrNr^rkrrrrrrrrrrs>*"II-H  H $$5 #'8 ##4D"->"0iwiX 18 1~ 68 6: 39 3"Qh zFr