2h*\SSKr"SS5r"SS5r"SS5r"SS 5rg) Nc.\rSrSrSrSSjrSrSrSrg) RegexBuilderaABuilds regex using arguments passed into a pattern template. Builds a regex object for which the pattern is made from an argument passed into a template. If more than one argument is passed (iterable), each pattern is joined by "|" (regex alternation 'or') to create a single pattern. Args: pattern_args (iterable): String element(s) to be each passed to ``pattern_func`` to create a regex pattern. Each element is ``re.escape``'d before being passed. pattern_func (callable): A 'template' function that should take a string and return a string. It should take an element of ``pattern_args`` and return a valid regex pattern group string. flags: ``re`` flag(s) to compile with the regex. Example: To create a simple regex that matches on the characters "a", "b", or "c", followed by a period:: >>> rb = RegexBuilder('abc', lambda x: "{}\.".format(x)) Looking at ``rb.regex`` we get the following compiled regex:: >>> print(rb.regex) 'a\.|b\.|c\.' The above is fairly simple, but this class can help in writing more complex repetitive regex, making them more readable and easier to create by using existing data structures. Example: To match the character following the words "lorem", "ipsum", "meili" or "koda":: >>> words = ['lorem', 'ipsum', 'meili', 'koda'] >>> rb = RegexBuilder(words, lambda x: "(?<={}).".format(x)) Looking at ``rb.regex`` we get the following compiled regex:: >>> print(rb.regex) '(?<=lorem).|(?<=ipsum).|(?<=meili).|(?<=koda).' cRXlX lX0lUR5UlgN) pattern_args pattern_funcflags_compileregex)selfrr r s K/var/www/auris/envauris/lib/python3.13/site-packages/gtts/tokenizer/core.py__init__RegexBuilder.__init__3s"(( ]]_ c/nURH;n[R"U5nURU5nUR U5 M= SR U5n[R "X@R5$)N|)rreescaper appendjoincompiler )r altsargaltpatterns rr RegexBuilder._compile;sb$$C))C.C##C(C KK % ((4.zz'::..rc,[UR5$r)strr r s r__repr__RegexBuilder.__repr__Es4::r)r rr r Nr) __name__ __module__ __qualname____firstlineno____doc__rr r!__static_attributes__rrrrs+Z%/rrc.\rSrSrSrSSjrSrSrSrg) PreProcessorRegexIajRegex-based substitution text pre-processor. Runs a series of regex substitutions (``re.sub``) from each ``regex`` of a :class:`gtts.tokenizer.core.RegexBuilder` with an extra ``repl`` replacement parameter. Args: search_args (iterable): String element(s) to be each passed to ``search_func`` to create a regex pattern. Each element is ``re.escape``'d before being passed. search_func (callable): A 'template' function that should take a string and return a string. It should take an element of ``search_args`` and return a valid regex search pattern string. repl (string): The common replacement passed to the ``sub`` method for each ``regex``. Can be a raw string (the case of a regex backreference, for example) flags: ``re`` flag(s) to compile with each `regex`. Example: Add "!" after the words "lorem" or "ipsum", while ignoring case:: >>> import re >>> words = ['lorem', 'ipsum'] >>> pp = PreProcessorRegex(words, ... lambda x: "({})".format(x), r'\\1!', ... re.IGNORECASE) In this case, the regex is a group and the replacement uses its backreference ``\\1`` (as a raw string). Looking at ``pp`` we get the following list of search/replacement pairs:: >>> print(pp) (re.compile('(lorem)', re.IGNORECASE), repl='\1!'), (re.compile('(ipsum)', re.IGNORECASE), repl='\1!') It can then be run on any string of text:: >>> pp.run("LOREM ipSuM") "LOREM! ipSuM!" See :mod:`gtts.tokenizer.pre_processors` for more examples. cX0l/UlUH5n[U/X$5nURRUR5 M7 gr)replregexesrrr )r search_args search_funcr/r rrbs rrPreProcessorRegex.__init__vs>  Cse[8B LL   )rcdURHnURURU5nM! U$)zRun each regex substitution on ``text``. Args: text (string): the input text. Returns: string: text after all substitutions have been sequentially applied. )r0subr/)r textr s rrunPreProcessorRegex.runs+\\E99TYY-D" rc/nURH-nURSRX R55 M/ SR U5$)Nz({}, repl='{}'), )r0rformatr/r)r subs_strsrs rr!PreProcessorRegex.__repr__sB A   .55aC Dyy##r)r0r/Nr# r$r%r&r'r(rr8r!r)r*rrr,r,Is*X* $rr,c.\rSrSrSrSSjrSrSrSrg) PreProcessorSubaSimple substitution text preprocessor. Performs string-for-string substitution from list a find/replace pairs. It abstracts :class:`gtts.tokenizer.core.PreProcessorRegex` with a default simple substitution regex. Args: sub_pairs (list): A list of tuples of the style ``(, )`` ignore_case (bool): Ignore case during search. Defaults to ``True``. Example: Replace all occurrences of "Mac" to "PC" and "Firefox" to "Chrome":: >>> sub_pairs = [('Mac', 'PC'), ('Firefox', 'Chrome')] >>> pp = PreProcessorSub(sub_pairs) Looking at the ``pp``, we get the following list of search (regex)/replacement pairs:: >>> print(pp) (re.compile('Mac', re.IGNORECASE), repl='PC'), (re.compile('Firefox', re.IGNORECASE), repl='Chrome') It can then be run on any string of text:: >>> pp.run("I use firefox on my mac") "I use Chrome on my PC" See :mod:`gtts.tokenizer.pre_processors` for more examples. cSnU(a[ROSn/UlUH0nUupg[U/X7U5nURR U5 M2 g)Nc$SRU5$)Nz{})r<)xs rr2-PreProcessorSub.__init__..search_funcs<<? "rr)rIpre_processorsr,r) r sub_pairs ignore_caser2r sub_pairrr/pps rrPreProcessorSub.__init__sU #$!!H$MG"G9kGB    & &r *"rcNURHnURU5nM U$)zRun each substitution on ``text``. Args: text (string): the input text. Returns: string: text after all substitutions have been sequentially applied. )rIr8)r r7rMs rr8PreProcessorSub.runs&%%B66$>> import re, RegexBuilder >>> >>> def case1(): ... return re.compile("\,") >>> >>> def case2(): ... return RegexBuilder('abc', lambda x: "{}\.".format(x)).regex >>> >>> t = Tokenizer([case1, case2]) Looking at ``case1().pattern``, we get:: >>> print(case1().pattern) '\\,' Looking at ``case2().pattern``, we get:: >>> print(case2().pattern) 'a\\.|b\\.|c\\.' Finally, looking at ``t``, we get them combined:: >>> print(t) 're.compile('\\,|a\\.|b\\.|c\\.', re.IGNORECASE) from: [, ]' It can then be run on any string of text:: >>> t.run("Hello, my name is Linda a. Call me Lin, b. I'm your friend") ['Hello', ' my name is Linda ', ' Call me Lin', ' ', " I'm your friend"] cXlX lUR5Ulg![[ 4an[ S[ U5-5eSnAff=f)Nz`Tokenizer() expects a list of functions returning regular expression objects (i.e. re.compile). ) regex_funcsr _combine_regex total_regex TypeErrorAttributeErrorr)r rWr es rrTokenizer.__init__"sX&  #224D >* ACFq6J  s$AA  Ac/nURHnURU"55 M SRSU55n[R"X0R 5$)Nrc38# UHoRv M g7fr)r).0rs r +Tokenizer._combine_regex..4s7$3;;$s)rWrrrrr )r rfuncrs rrXTokenizer._combine_regex/sN$$D KK %((7$77zz'::..rc8URRU5$)zTokenize `text`. Args: text (string): the input text to tokenize. Returns: list: A list of strings (token) split according to the tokenizer cases. )rYsplit)r r7s rr8 Tokenizer.run7s%%d++rc^[UR5S-[UR5-$)Nz from: )rrYrWr s rr!Tokenizer.__repr__Cs(4##$y03t7G7G3HHHr)r rWrYN) r$r%r&r'r(r IGNORECASErrXr8r!r)r*rrrTrTs'HT+--- / ,IrrT)rrr,rBrTr*rrrks= AAHI$I$X?B?BDmImIr