KiafddlmZmZmZddlZddlZddlZddlZddlZej j dkre Z e Z ddlmZddlmZneZddlmZejeZdZdZ dZ dZ Gd d Zd ZGd d eZdS))print_functionabsolute_importunicode_literalsN)MappingProxyType)getfullargspec) getargspecc,ddlm}t|S)aL Return a dictionary that contains the default collection of known LaTeX escape sequences for unicode characters. The keys of the dictionary are integers that correspond to unicode code points (i.e., `ord(char)`). The values are the corresponding LaTeX replacement strings. The returned dictionary may not be modified. To alter the behavior of :py:func:`unicode_to_latex()`, you should specify custom rules to a new instance of :py:class:`UnicodeToLatexEncoder`. .. versionadded:: 2.0 This function was introduced in `pylatexenc 2.0`. ) uni2latex) _uni2latexmapr _MappingProxyType) _uni2latexs /var/www/development/aibuddy-work/election-extract/venv/lib/python3.11/site-packages/pylatexenc/latexencode/_unicode_to_latex_encoder.pyget_builtin_uni2latex_dictr6s$"766666 Z ( ((r c$eZdZdZ ddZdZdS)UnicodeToLatexConversionRulea* Specify a rule how to convert unicode characters into LaTeX escapes. .. py:attribute:: rule_type One of :py:data:`RULE_DICT`, :py:data:`RULE_REGEX`, or :py:data:`RULE_CALLABLE`. .. py:attribute:: rule A specification of the rule itself. The `rule` attribute is an object that depends on what `rule_type` is set to. See below. .. py:attribute:: replacement_latex_protection If non-`None`, then the setting here will override any `replacement_latex_protection` set on :py:class:`UnicodeToLatexConversionRule` objects. By default the value is `None`, and you can set a replacement_latex_protection globally for all rules on the :py:class:`UnicodeToLatexEncoder` object. The use of this attribute is mainly in case you have a fancy rule in which you already guarantee that whatever you output is valid LaTeX even if concatenated with the remainder of the string; in this case you can set `replacement_latex_protection='none'` to avoid unnecessary or unwanted braces around the generated code. .. versionadded:: 2.10 The `replacement_latex_protection` attribute was introduced in `pylatexenc 2.10`. Constructor syntax:: UnicodeToLatexConversionRule(RULE_XXX, <...>) UnicodeToLatexConversionRule(rule_type=RULE_XXX, rule=<...>) UnicodeToLatexConversionRule(..., replacement_latex_protection='none') Note that you can get some built-in rules via the :py:func:`get_builtin_conversion_rules()` function:: conversion_rules = get_builtin_conversion_rules('defaults') # all defaults Rules types: - `RULE_DICT`: If `rule_type` is `RULE_DICT`, then `rule` should be a dictionary whose keys are integers representing unicode code points (e.g., `0x210F`), and whose values are corresponding replacement strings (e.g., ``r'\hbar'``). See :py:func:`get_builtin_uni2latex_dict()` for an example. - `RULE_REGEX`: If `rule_type` is `RULE_REGEX`, then `rule` should be an iterable of tuple pairs `(compiled_regular_expression, replacement_string)` where `compiled_regular_expression` was obtained with `re.compile(...)` and `replacement_string` is anything that can be specified as the second (`repl`) argument of `re.sub(...)`. This can be a replacement string that includes escapes (like ``\1, \2, \g``) for captured sub-expressions or a callable that takes a match object as argument. .. note:: The replacement string is parsed like the second argument to `re.sub()` and backslashes have a special meaning because they can refer to captured sub-expressions. For a literal backslash, use two backslashes ``\\`` in raw strings, four backslashes in normal strings. Example:: regex_conversion_rule = UnicodeToLatexConversionRule( rule_type=RULE_REGEX, rule=[ # protect acronyms of capital letters with braces, # e.g.: ABC -> {ABC} (re.compile(r'[A-Z]{2,}'), r'{\1}'), # Additional rules, e.g., "..." -> "\ldots" (re.compile(r'...'), r'\\ldots'), # note double \\ ] ) - `RULE_CALLABLE`: If `rule_type` is `RULE_CALLABLE`, then `rule` should be a callable that accepts two arguments, the unicode string and the position in the string (an integer). The callable will be called with the original unicode string as argument and the position of the character that needs to be encoded. If this rule can encode the given character at the given position, it should return a tuple `(consumed_length, replacement_string)` where `consumed_length` is the number of characters in the unicode string that `replacement_string` represents. If the character(s) at the given position can't be encoded by this rule, the callable should return `None` to indicate that further rules should be attempted. If the callable accepts an additional argument called `u2lobj`, then the :py:class:`UnicodeToLatexEncoder` instance is provided to that argument. For example, the following callable should achieve the same effect as the previous example with regexes:: def convert_stuff(s, pos): m = re.match(r'[A-Z]{2,}', s, pos) if m is not None: return (m.end()-m.start(), '{'+m.group()+'}') if s.startswith('...', pos): # or s[pos:pos+3] == '...' return (3, r'\ldots') return None .. versionadded:: 2.0 This class was introduced in `pylatexenc 2.0`. Nc0||_||_||_dSN) rule_typerulereplacement_latex_protection)selfrrrs r__init__z%UnicodeToLatexConversionRule.__init__s # ,H)))rcd|jj|jt |jjt |jS)Nz>{}(rule_type={!r}, rule=<{}>, replacement_latex_protection={}))format __class____name__rtyperreprr)rs r__repr__z%UnicodeToLatexConversionRule.__repr__sAOVV N #T^T$)__5M 2 3 3   r)NN)r __module__ __qualname____doc__rr"rrrrlsPrrf(,.2IIII     rrc|dkr#tttgS|dkr"ddlm}tt|jgSt d|)a> Return a built-in set of conversion rules specified by a given name `builtin_name`. There are two builtin conversion rules, with the following names: - `'defaults'`: the default conversion rules, a custom-curated list of unicode chars to LaTeX escapes. - `'unicode-xml'`: the conversion rules derived from the `unicode.xml` file maintained at https://www.w3.org/TR/xml-entity-names/#source by David Carlisle. The return value is a list of :py:class:`UnicodeToLatexConversionRule` objects that can be either directly specified to the `conversion_rules=` argument of :py:class:`UnicodeToLatexEncoder`, or included in a larger list that can be provided to that argument. .. versionadded:: 2.0 This function was introduced in `pylatexenc 2.0`. defaults)rrz unicode-xmlr )_uni2latexmap_xmlzUnknown builtin rule set: {})r RULE_DICTrr)r ValueErrorr) builtin_namer)s rget_builtin_conversion_rulesr.s.z!!- 3M3O3OQQQS S}$$''''''- 3D3NPPPR R 3::<HH I IIrceZdZdZfdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZdZdZdZdZdZxZS)UnicodeToLatexEncoderu Encode a string with unicode characters into a LaTeX snippet. The following general attributes can be specified as keyword arguments to the constructor. Note: These attributes must be specified to the constructor and may NOT be subsequently modified. This is because in the constructor we pre-compile some rules and flags to optimize calls to :py:meth:`unicode_to_text()`. .. py:attribute:: non_ascii_only Whether we should convert only non-ascii characters into LaTeX sequences, or also all known ascii characters with special LaTeX meaning such as '\\\\', '$', '&', etc. If `non_ascii_only` is set to `True` (the default is `False`), then conversion rules are not applied at positions in the string where an ASCII character is encountered. .. py:attribute:: conversion_rules The conversion rules, specified as a list of :py:class:`UnicodeToLatexConversionRule` objects. For each position in the string, the rules will be applied in the given sequence until a replacement string is found. Instead of a :py:class:`UnicodeToLatexConversionRule` object you may also specify a string specifying a built-in rule (e.g., 'defaults'), which will be expanded to the corresponding rules according to :py:func:`get_builtin_conversion_rules()`. If you specify your own list of rules using this argument, you will probably want to include presumably at the end of your list the element 'defaults' to include all built-in default conversion rules. To override built-in rules, simply add your custom rules earlier in the list. Example:: conversion_rules = [ # our custom rules UnicodeToLatexConversionRule(RULE_REGEX, [ # double \\ needed, see UnicodeToLatexConversionRule ( re.compile(r'...'), r'\\ldots' ), ( re.compile(r'î'), r'\\^i' ), ]), # plus all the default rules 'defaults' ] u = UnicodeToLatexEncoder(conversion_rules=conversion_rules) .. py:attribute:: replacement_latex_protection How to "protect" LaTeX replacement text that looks like it could be interpreted differently if concatenated to arbitrary strings before and after. Currently in the default scheme only one situation is recognized: if the replacement string ends with a latex macro invocation with a non-symbol macro name, e.g. ``\textemdash`` or ``\^\i``. Indeed, if we naively replace these texts in an arbitrary string (like ``maître``), we might get an invalid macro invocation (like ``ma\^\itre`` which causes un known macro name ``\itre``). Possible protection schemes are: - 'braces' (the default): Any suspicious replacement text (that might look fragile) is placed in curly braces ``{...}``. - 'braces-all': All replacement latex escapes are surrounded in protective curly braces ``{...}``, regardless of whether or not they might be deemed "fragile" or "unsafe". - 'braces-almost-all': Almost all replacement latex escapes are surrounded in protective curly braces ``{...}``. This option emulates closely the behavior of `brackets=True` of the function `utf8tolatex()` in `pylatexenc 1.x`, though I'm not sure it is really useful. [Specifically, all those replacement strings that start with a backslash are surrounded by curly braces]. - 'braces-after-macro': In the situation where the replacement latex code ends with a string-named macro, then a pair of empty braces is added at the end of the replacement text to protect the macro. - 'none': No protection is applied, even in "unsafe" cases. This is not recommended, as this will likely result in invalid LaTeX code. (Note this is the string 'none', not Python's built-in `None`.) - any callable object: The callable should take a single argument, the replacement latex string associated with a piece of the input (maybe a special character) that has been encoded; it should return the actual string to append to the output string. .. versionadded:: 2.10 You can specify a callable object to `replacement_latex_protection` since `pylatexenc 2.10`. .. py:attribute:: unknown_char_policy What to do when a non-ascii character is encountered without any known substitution macro. The attribute `unknown_char_policy` can be set to one of: - 'keep': keep the character as is; - 'replace': replace the character by a boldface question mark; - 'ignore': ignore the character from the input entirely and don't output anything for it; - 'fail': raise a `ValueError` exception; - 'unihex': output the unicode hexadecimal code (U+XXXX) of the character in typewriter font; - a Python callable --- will be called with argument the character that could not be encoded. (If the callable accepts a second argument called 'u2lobj', then the `UnicodeToLatexEncoder` instance is provided to that argument.) The return value of the callable is used as LaTeX replacement code. .. py:attribute:: unknown_char_warning In addition to the `unknown_char_policy`, this attribute indicates whether or not (`True` or `False`) one should generate a warning when a nonascii character without any known latex representation is encountered. (Default: True) .. py:attribute:: latex_string_class The return type of :py:meth:`unicode_to_latex()`. Normally this is a simple unicode string (`str` on `Python 3` or `unicode` on `Python 2`). But you can specify your custom string type via the `latex_string_class` argument. The `latex_string_class` will be invoked with no arguments to construct an empty object (so `latex_string_class` can be either an object that can be constructed with no arguments or it can be a function with no arguments that return a fresh object instance). The object must support the operation "+=", i.e., you should overload the ``__iadd__()`` method. For instance, you can record the chunks that would have been appended into a single string as follows:: class LatexChunkList: def __init__(self): self.chunks = [] def __iadd__(self, s): self.chunks.append(s) return self u = UnicodeToLatexEncoder(latex_string_class=LatexChunkList, replacement_latex_protection='none') result = u.unicode_to_latex("é → α") # result.chunks == [ r"\'e", ' ', r'\textrightarrow', ' ', # r'\ensuremath{\alpha}' ] .. warning:: None of the above attributes should be modified after constructing the object. The values specified to the class constructor are final and cannot be changed. [Indeed, the class constructor "compiles" these attribute values into a data structure that makes :py:meth:`unicode_to_text()` slightly more efficient.] .. versionadded:: 2.0 This class was introduced in `pylatexenc 2.0`. c |dd|_|ddg|_|dd|_|dd|_|d d |_|d t |_|r@t d d | tt|jdi|tjd|jD}g|_|D]8}|jt(kr9|jt-j|j|j|L|jt4kr9|jt-j|j|j||jt8krn|j}dt;|dvrt-j|j|}|jt-j|j||t?d |jtC|jtDr#|#d|jd|_$ntK|jrL|j}dt;|dvr!t-j|j||_$n4|j|_$n't?d |j|jsd|_&|jr |j'|_(nd|_(|)|j|_*dS)Nnon_ascii_onlyFconversion_rulesr(rbracesunknown_char_policykeepunknown_char_warningTlatex_string_classz&Ignoring unknown keyword arguments: %s,c3dK|]+}t|trt|n|gV,dSr) isinstance basestringr.).0rs r z1UnicodeToLatexEncoder.__init__..s^B B 1;1j0I0I T )! , , ,QRuB B B B B B ru2lobjr)r@zInvalid rule type: {}do_unknown_charwhatz.Invalid argument for unknown_char_policy: {!r}cdSrr&)chs rz0UnicodeToLatexEncoder.__init__..sDrcdS)NFr&)sps rrFz0UnicodeToLatexEncoder.__init__..s%rr&)+popr2r3rr5r7unicoder8loggerwarningjoinkeyssuperr0r itertoolschain from_iterable_compiled_rulesrr*append functoolspartial_apply_rule_dictr RULE_REGEX_apply_rule_regex RULE_CALLABLEr_apply_rule_callable TypeErrorrr;r<_get_method_fn_do_unknown_charcallable_do_warn_unknown_char_check_do_skip_ascii_maybe_skip_ascii_get_replacement_latex_fn_apply_protection)rkwargsexpanded_conversion_rulesr thecallablefnrs rrzUnicodeToLatexEncoder.__init__s$jj)95AA & += | L L,2JJ7UW_,`,`)#)::.CV#L#L $*JJ/Et$L$L!"(**-A7"K"K  ^ NNCSXXfkkmmE\E\ ] ] ]3#T**3==f===%.O$A$AB B *B B B % % ! "- P PD~**$++%d&;TYMM:--$++%d& >t~ N NOOO d. ; ; ?$($7$7!(*%8%%D ! ! d. / / ?)B>"--a000(1(9$:R[_(`(`(`%%(,(@%%L#VD$<==?? ?( 9)8D &   8%)%>D " "%7%7D ""&!?!?  -" " rcd|zdz|ddz}t||s#td||t ||S)N_-zInvalid {}: {})replacehasattrr,rgetattr)rbasenamerC selfmethnames rr^z$UnicodeToLatexEncoder._get_method_fn seTzC'$,,sC*@*@@ t\** B-44T4@@AA At\***rcTt|r|S|d|dS)Napply_protectionrrB)r`r^)rrs rrdz/UnicodeToLatexEncoder._get_replacement_latex_fns? 0 1 1 0/ /""  (/#   rct|}tjd|}Gdd}|}||_d|_|jt |kr|||r/|jD]}|||rn||j}t|}|dkr|dks|dvr!|xj|z c_|xjdz c_nH| ||xj| |z c_|xjdz c_|jt |k|jS) z Convert unicode characters in the string `s` into latex escape sequences, according to the rules and options given to the constructor. NFCceZdZdS)3UnicodeToLatexEncoder.unicode_to_latex.._NSN)rr#r$r&rr_NSrx!srryr z r ) rK unicodedata normalizer8latexposlenrcrTordrar_)rrHryrI compiledrulerEos runicode_to_latexz&UnicodeToLatexEncoder.unicode_to_latexsa AJJ  !% + + CEE))++ec!ffnn%%a++  $ 4   <1%%E quXGGGGSbHnnGGrMGGEEQJEEE..r222GGt44R888GGEEQJEE)ec!ffnn,wrct||jdkr-|xj||jz c_|xjdz c_dSdS)Nr{r TF)rrr~)rrHrIs rrbz*UnicodeToLatexEncoder._check_do_skip_ascii?sH qx==3   GGqx GG EEQJEE4urct||j}||vr ||||d|dSdS)Nr T)rr_apply_replacement)rruledictrrHrIrs rrXz&UnicodeToLatexEncoder._apply_rule_dictHsD !%MM ==  # #Ax{At < < <4trc4|D]\}}|||j}|rt|r ||}n||}|||||z |dSdSNT)matchrr`expandrendstart) r ruleregexesrrHrIregexreplmreplstrs rrZz'UnicodeToLatexEncoder._apply_rule_regexNs&  KE4 Aqu%%A}D>>-"d1ggGGhhtnnG''7AEEGGaggii4GNNNtt trcj|||j}|dS|\}}|||||dSr)rr)r rulecallablerrHrIresconsumedrs rr\z*UnicodeToLatexEncoder._apply_rule_callableYsHl1ae$$ ;44 44888trc|j}|j||j}||}|xj|z c_|xj|z c_dSr)rerrdr~r)rrIrnumcharsruleobj protect_fns rrz(UnicodeToLatexEncoder._apply_replacementasf+   / ;774Jz$ 4 rc|Srr&rrs r_apply_protection_nonez,UnicodeToLatexEncoder._apply_protection_noneps rc|d}|dkr'||dzdrd|zdzS|S)N\rr {}rfindisalpharrks r_apply_protection_bracesz.UnicodeToLatexEncoder._apply_protection_bracesssL JJt   66d1Q344j((**6:# # rc2|dddkrd|zdzS|S)Nrr rrrr&rs r#_apply_protection_braces_almost_allz9UnicodeToLatexEncoder._apply_protection_braces_almost_allys) !9  :# # rcd|zdzS)Nrrr&rs r_apply_protection_braces_allz2UnicodeToLatexEncoder._apply_protection_braces_all}sTzCrc|d}|dkr$||dzdr|dzS|S)Nrrr z{}rrs r$_apply_protection_braces_after_macroz:UnicodeToLatexEncoder._apply_protection_braces_after_macrosG JJt   66d1Q344j((**6$;  rc|Srr&rrEs r_do_unknown_char_keepz+UnicodeToLatexEncoder._do_unknown_char_keeps rcdS)Nz {\bfseries ?}r&rs r_do_unknown_char_replacez.UnicodeToLatexEncoder._do_unknown_char_replacesrcdS)Nr+r&rs r_do_unknown_char_ignorez-UnicodeToLatexEncoder._do_unknown_char_ignoresrrcDtdt||fzNu>No known latex representation for character: U+%04X - ‘%s’)r,rrs r_do_unknown_char_failz+UnicodeToLatexEncoder._do_unknown_char_fails)Yr77B-()) )rc&dt|zS)Nz7\ensuremath{\langle}\texttt{U+%04X}\ensuremath{\rangle})rrs r_do_unknown_char_unihexz-UnicodeToLatexEncoder._do_unknown_char_unihexsI3r77SSrcXtdt||dSr)rLrMrrs rraz+UnicodeToLatexEncoder._do_warn_unknown_chars0W2ww $ $ $ $ $r)rr#r$r%rr^rdrrbrXrZr\rrrrrrrrrrrra __classcell__)rs@rr0r0slggPK K K K K Z+++    $$$N             )))TTT$$$$$$$rr0) __future__rrrr|loggingsysrVrQ version_infomajorstrrKr<typesrrinspectrdictr getLoggerrrLrr*rYr[rr.objectr0r&rrrs}4IHHHHHHHHH AGJ;;;;;;&&&&&&&444444  8 $ $))),            HJJJFF$F$F$F$F$FF$F$F$F$F$r