/hEdSSKJr SSKJr SSKJrJr "SS\5rSr\ S:Xa\"5 gg) )reduce)ParserI)ProbabilisticTreeTreec\\rSrSrSrSSjrSrSSjrSrSr Sr S r S r S r S rS rg) ViterbiParsera A bottom-up ``PCFG`` parser that uses dynamic programming to find the single most likely parse for a text. The ``ViterbiParser`` parser parses texts by filling in a "most likely constituent table". This table records the most probable tree representation for any given span and node value. In particular, it has an entry for every start index, end index, and node value, recording the most likely subtree that spans from the start index to the end index, and has the given node value. The ``ViterbiParser`` parser fills in this table incrementally. It starts by filling in all entries for constituents that span one element of text (i.e., entries where the end index is one greater than the start index). After it has filled in all table entries for constituents that span one element of text, it fills in the entries for constitutants that span two elements of text. It continues filling in the entries for constituents spanning larger and larger portions of the text, until the entire table has been filled. Finally, it returns the table entry for a constituent spanning the entire text, whose node value is the grammar's start symbol. In order to find the most likely constituent with a given span and node value, the ``ViterbiParser`` parser considers all productions that could produce that node value. For each production, it finds all children that collectively cover the span and have the node values specified by the production's right hand side. If the probability of the tree formed by applying the production to the children is greater than the probability of the current entry in the table, then the table is updated with this new tree. A pseudo-code description of the algorithm used by ``ViterbiParser`` is: | Create an empty most likely constituent table, *MLC*. | For width in 1...len(text): | For start in 1...len(text)-width: | For prod in grammar.productions: | For each sequence of subtrees [t[1], t[2], ..., t[n]] in MLC, | where t[i].label()==prod.rhs[i], | and the sequence covers [start:start+width]: | old_p = MLC[start, start+width, prod.lhs] | new_p = P(t[1])P(t[1])...P(t[n])P(prod) | if new_p > old_p: | new_tree = Tree(prod.lhs, t[1], t[2], ..., t[n]) | MLC[start, start+width, prod.lhs] = new_tree | Return MLC[0, len(text), start_symbol] :type _grammar: PCFG :ivar _grammar: The grammar used to parse sentences. :type _trace: int :ivar _trace: The level of tracing output that should be generated when parsing a text. cXlX lg)ab Create a new ``ViterbiParser`` parser, that uses ``grammar`` to parse texts. :type grammar: PCFG :param grammar: The grammar used to parse texts. :type trace: int :param trace: The level of tracing that should be used when parsing a text. ``0`` will generate no tracing output; and higher numbers will produce more verbose tracing output. N_grammar_trace)selfgrammartraces J/var/www/auris/envauris/lib/python3.13/site-packages/nltk/parse/viterbi.py__init__ViterbiParser.__init__Ks  cUR$Nr rs rrViterbiParser.grammar[s }}rcXlg)a Set the level of tracing output that should be generated when parsing a text. :type trace: int :param trace: The trace level. A trace level of ``0`` will generate no tracing output; and higher trace levels will produce more verbose tracing output. :rtype: None N)r )rrs rrViterbiParser.trace^s  rc## [U5nURRU5 0nUR(a [ S5 [ [ U55H=nXnXBX3S-U4'URS:dM"URXC[ U55 M? [ S[ U5S-5H^nUR(a[ SSU--5 [ [ U5U- S-5HnXfU-4nURXrU5 M M` URS[ U5URR545nUbUv gg7f)Nz;Inserting tokens into the most likely constituents table...z$Finding the most likely constituentsz spanning %d text elements...r) listr check_coverager printrangelen_trace_lexical_insertion_add_constituents_spanninggetstart) rtokens constituentsindextokenlengthr&spantrees rparseViterbiParser.parseks4f $$V,  ;; T U3v;'EME49 50 1{{Q--eCKH (As6{Q/F{{:5>?s6{V3a78v~.//FK9 0CK1D1D1F GH  J s A7E=CEc SnU(GarSnURX5nUGHMupgUVs/sHn[U[5(dMUPM n n[SXR 55n UR 5R 5n [XU S9n URUSUSUR 545nURS:a^UbX:waVUb"UR 5U R 5:a [SS S 9 O [S S S 9 URXjU[U55 Ub%UR 5U R 5:dGM0XUSUSUR 54'SnGMP U(aGMqggs snf) aJ Find any constituents that might cover ``span``, and add them to the most likely constituents table. :rtype: None :type span: tuple(int, int) :param span: The section of the text for which we are trying to find possible constituents. The span is specified as a pair of integers, where the first integer is the index of the first token that should be included in the constituent; and the second integer is the index of the first token that should not be included in the constituent. I.e., the constituent should cover ``text[span[0]:span[1]]``, where ``text`` is the text that we are parsing. :type constituents: dict(tuple(int,int,Nonterminal) -> ProbabilisticToken or ProbabilisticTree) :param constituents: The most likely constituents table. This table records the most probable tree representation for any given span and node value. In particular, ``constituents(s,e,nv)`` is the most likely ``ProbabilisticTree`` that covers ``text[s:e]`` and has a node value ``nv.symbol()``, where ``text`` is the text that we are parsing. When ``_add_constituents_spanning`` is called, ``constituents`` should contain all possible constituents that are shorter than ``span``. :type tokens: list of tokens :param tokens: The text we are parsing. This is only used for trace output. TFc&XR5-$rprob)prts r:ViterbiParser._add_constituents_spanning..s ffhrr2rrNz Insert: endz Discard:) _find_instantiations isinstancerrr3lhssymbolrr%r r _trace_productionr") rr,r(r'changedinstantiations productionchildrencsubtreespnoder-s rr$(ViterbiParser._add_constituents_spanningsKHG"66tJN )7$ '/Gx!:a3FAxG6//BST!~~'..0(a@!$$d1gtAw 8H%IJ;;?yAI9499;(>!,C8!,C8..zdCKP9499; 6GKa$q':>>3C!CD"G%)7gHs E>E>c/nURR5H>nURUR5X5nUHnUR XF45 M M@ U$)aa :return: a list of the production instantiations that cover a given span of the text. A "production instantiation" is a tuple containing a production and a list of children, where the production's right hand side matches the list of children; and the children cover ``span``. :rtype: list of ``pair`` of ``Production``, (list of (``ProbabilisticTree`` or token. :type span: tuple(int, int) :param span: The section of the text for which we are trying to find production instantiations. The span is specified as a pair of integers, where the first integer is the index of the first token that should be covered by the production instantiation; and the second integer is the index of the first token that should not be covered by the production instantiation. :type constituents: dict(tuple(int,int,Nonterminal) -> ProbabilisticToken or ProbabilisticTree) :param constituents: The most likely constituents table. This table records the most probable tree representation for any given span and node value. See the module documentation for more information. )r productions _match_rhsrhsappend)rr,r(rvrB childlists childlists rr;"ViterbiParser._find_instantiationssW0--335J)94NJ'  :12(6  rcUupEXE:a US:Xa//$XE:dUS:Xa/$/n[XES-5HMnURXGUS45nUcMURUSSXu4U5n XiV s/sHo/U -PM sn - nMO U$s sn f)a :return: a set of all the lists of children that cover ``span`` and that match ``rhs``. :rtype: list(list(ProbabilisticTree or token) :type rhs: list(Nonterminal or any) :param rhs: The list specifying what kinds of children need to cover ``span``. Each nonterminal in ``rhs`` specifies that the corresponding child should be a tree whose node value is that nonterminal's symbol. Each terminal in ``rhs`` specifies that the corresponding child should be a token whose type is that terminal. :type span: tuple(int, int) :param span: The section of the text for which we are trying to find child lists. The span is specified as a pair of integers, where the first integer is the index of the first token that should be covered by the child list; and the second integer is the index of the first token that should not be covered by the child list. :type constituents: dict(tuple(int,int,Nonterminal) -> ProbabilisticToken or ProbabilisticTree) :param constituents: The most likely constituents table. This table records the most probable tree representation for any given span and node value. See the module documentation for more information. rrN)r!r%rK) rrLr,r(r&r:rOsplitlrightsrs rrKViterbiParser._match_rhss4  rrs r__repr__ViterbiParser.__repr__@s'$--77rr N)r)r^)__name__ __module__ __qualname____firstlineno____doc__rrrr.r$r;rKr?r#re__static_attributes__rSrrrrs?5n  'RB#H@*X. 8rrchSSKnSSKnSSKJn SSKJn SSKJn URS5nURS5nSU4S U4/n[5 [[U55H9n[US -S S XxS35 [S XxS -5 [5 M; [SS [U54-SS9 [URR5R55S - n XyupU R!5n U"U 5n 0n[SU SU SU 35 U R#S5 UR5nU R%U 5nUR5U- nU(a['SUS5[U5- OSn[U5nUHnS UUR)5'M [5 [S5 [S5 [SUUU4-5 UR+5nU(a['SUS5[U5- nOSn[S5 [SS[U5U4-5 [5 [SSS9 URR5R5R-5R/S5(aSS KJn [S!5 U"U6 [5 [S"SS9 URR5R5R-5R/S5(aUHn[U5 M gg! [S5 g=f)#z A demonstration of the probabilistic parsers. The user is prompted to select which demo to run, and how many parses should be found; and then each parser is run on the same demo, and a summary of the results are displayed. rN)tokenize)PCFG)ra7 S -> NP VP [1.0] NP -> Det N [0.5] | NP PP [0.25] | 'John' [0.1] | 'I' [0.15] Det -> 'the' [0.8] | 'my' [0.2] N -> 'man' [0.5] | 'telescope' [0.5] VP -> VP PP [0.1] | V NP [0.7] | V [0.2] V -> 'ate' [0.35] | 'saw' [0.65] PP -> P NP [1.0] P -> 'with' [0.61] | 'under' [0.39] a S -> NP VP [1.0] VP -> V NP [.59] VP -> V [.40] VP -> VP PP [.01] NP -> Det N [.41] NP -> Name [.28] NP -> NP PP [.31] PP -> P NP [1.0] V -> 'saw' [.21] V -> 'ate' [.51] V -> 'ran' [.28] N -> 'boy' [.11] N -> 'cookie' [.12] N -> 'table' [.13] N -> 'telescope' [.14] N -> 'hill' [.5] Name -> 'Jack' [.52] Name -> 'Bob' [.48] P -> 'with' [.61] P -> 'under' [.39] Det -> 'the' [.41] Det -> 'a' [.31] Det -> 'my' [.28] zI saw the man with my telescopez:the boy saw Jack with Bob under the table with a telescoperz>3z: z %rzWhich demo (%d-%d)? r8r9zBad sentence numberz sent: z parser: z grammar: c&XR5-$rr2abs rr6demo..s ALrz)Time (secs) # Parses Average P(parse)z)-----------------------------------------z%11.4f%11d%19.14fc&XR5-$rr2rrs rr6rus FFH rz*------------------------------------------z%11s%11d%19.14fzn/azDraw parses (y/n)? y) draw_treesz please wait...zPrint parses (y/n)? )systimenltkrn nltk.grammarro nltk.parser fromstringr r!r"intstdinreadlinestriprTr parse_allrfreezekeyslower startswithnltk.draw.treerx)ryrzrnror toy_pcfg1 toy_pcfg2demosisnumsentrr'parser all_parsesr5parsesaverage num_parsesrFrxr.s rdemorIs!(  I I< +I6 EyQ E  G 3u:  Qr "UXa[M*+ i%(1+%&  As5z? 2<399%%'--/014  ZZ\F 7 #FJ HTF*VHKy AB LLO A   f %F 99;?DFL(&!4s6{BRS VJ !" 188: G 56 56 z7 ; ;< __ F ,fa 83v; F  67 uc&k15 56 G S) yy!!#))+66s;;-  !F G c* yy!!#))+66s;;E %L<_ #$s 9:L"" L1__main__N) functoolsrnltk.parse.apir nltk.treerrrrrgrSrrrs:"-n8Gn8l xv zFr