# representation (parsing and conversion) **What it is.** Everything that turns a response string into a [[math-tree|MathTree]] and back. A `Parser` (representation/parser.py) has `parse_to_math_tree(str)` and `parse_to_representation(tree)`. Module-level singletons live in representation/parser_utils.py: `mathml_parser`, `pres_mathml_parser`, `latex_parser`, `html_parser`. ## Representation detection `get_math_representation(str | candidateResponse dict)` (utils/representation_utils.py) sniffs the content (first gap's content for multi-gap responses) — the item's `mathRepresentation` is **not** used to pick the parser: | check (in order) | result (`Representation` enum, models/models.py) | |---|---| | contains ``, `` or `

` | `HTML` = `"mathPlusText"` → [[math-plus-text]] | | starts with `

` or any `MmlTag` and contains a presentation token tag (` `, ``, ``, `\frac{`, \dots) | `PRES_MATHML` = `"presMathml"` \to [[mathml]] |
| starts with `}{` / an `MmlTag` | `MATHML` = `"mathml"` \to [[mathml]] (content) |
| anything else | `LATEX` = `"latex"` \to [[latex]] |

`parser_utils.get()` maps MATHML \to `MathMLParser`, PRES_MATHML \to `PresentationMathMLParser`, everything else \to
`LaTeXParser`. HTML never reaches `get()` in scoring: `score_with_validation` first converts it (below). A 5th enum
value `HUMAN_READABLE` ("humanReadable") is an *output* type only (`tree.__repr__()`), used by `/ws/convert`.

## Parsers
| parser | input | how |
|---|---|---|
| `MathMLParser` | content MathML (` `, ` `, ` `, ` `) | lxml `etree.fromstring` after `MmlUtils.remove_namespace`; recursive `_gen_tree`; tag\toclass table in mathml/utils/mathml_tree_mapping.py (`get_tree_element`); ` \begin{matrix} ` \to `Multiline`; ` \\ box/input` \to `Empty` (keeps `id`) |
| `LaTeXParser` | LaTeX | `latex_tokenizer` \to `_ignore_and_promote` \to recursive-descent `_parse_tokens` (`LaTeXCursor`) \to `handle_implicit_times(distinguish_implicit_and_explicit=True)`; command table in latex/definitions/latex_symbol.py and latex/utils/latex_tree_mapping.py; raises `InvalidLaTeXException` |
| `PresentationMathMLParser` (subclass of `LaTeXParser`) | presentation MathML | `pres_mml_to_latex` (syntactic translation) \to LaTeX pipeline \to `apply_replacements` for unknown units / non-ASCII ` ` \to `strip_grouping_symbols`. `parse_to_representation` returns `""` (not implemented) |
| `HtmlParser` | Math+Text HTML (`` paragraphs mixing text and ` `) | BeautifulSoup `html.parser`; per `` tokenises text (`tokenize_text`) and MathML (`tokenize_mathml`), merges implicit mixed numbers across fragments, splits prose from math (`split_tokens_by_math_expressions`), precedence-climbing parse (`_parse_relation` \to additive \to multiplicative \to unary \to implicit times \to atom); several `` \to `Multiline` |

Output: `tree.mathml` (content MathML), `tree.latex`, `repr(tree)`; `MathMLParser.parse_to_representation` = `tree.mathml`.

## MathML validation
`validate_candidate_response` (mathml/utils/mml_validation.py) runs in [[scoring-server]] before parsing: content must be
a string; MathML content is validated with `XmlValidator` against utils/schema/mathex.xsd (initialised once at server
start, server.py `XmlValidator.initialize_parser()`), retried after a series of regex fix-ups (add namespace, replace
empty `csymbol`/box with ` x `, strip `mathvariant="normal"`), then namespace and `mathvariant` are stripped.
Invalid \to `InvalidMathMLException` \to status 8.

## Math+Text \to MathML conversion
`MathMLConverter.convert_candidate_response` (html/mathml_converter.py) parses each HTML content with `HtmlParser` and
replaces it with namespaced content MathML (`tree.mathml`). A single-content response is moved into gap `a` and
`content`/`presentation` are blanked. Called from `score_with_validation` before Score As so templates see MathML.

## Score As
`ScoreAsBuilderFactory.get_builder(item, cr)` (representation/score_as.py) picks `MathML` (for MATHML and HTML items)
or `LaTeX` builder by the item's `mathRepresentation`; `build_score_as_gaps()` fills each `scoreAs` template with the
candidate's gap contents and prepends the derived gaps. See [[score-as]].

## Gotchas
- Representation is sniffed per request, not read from the item. `_validate_item_and_response_consistency` in
  utils/scoring_utils.py only checks HTML-vs-not-HTML against the item's `mathRepresentation`.
- `MathMLConverter._convert_mpt_to_mathml` and the whole `representation/html/text_parser/` path are **dead**: the
  method is `@deprecated` ("Delete once MRE-367 is complete") and nothing calls it (grep); only tests exercise it.
- `MathMLParser` (`etree.fromstring`, representation/mathml/mathml_parser.py:56) and `XmlValidator`
  (xml_validator.py:41) parse untrusted candidate input with lxml's default parser and no explicit hardening; only
  `pres_mml_to_latex` builds a hardened `XMLParser(resolve_entities=False, no_network=True)`. With the pinned
  lxml==6.1.0 the default parser no longer loads external entities (lxml >= 5 defaults `resolve_entities='internal'`,
  `no_network=True`), so classic XXE file reads depend on that pin; internal DTD entities still expand. Score As
  templates use stdlib `xml.etree.ElementTree` (representation/score_as.py:146,155).
  Excess nesting surfaces as an lxml `XMLSyntaxError` that `XmlValidator` rethrows as `XMLDepthException`.
- `PresentationMathMLParser` can't round-trip (no `parse_to_representation`).

## Surfaces
[[scoring-api]] `/ws/convert` (parse any representation, return `mathml` / `latex` / `humanReadable`, server.py
`convert`), all scoring endpoints via `score_with_validation`.

## Related
[[mathml]], [[latex]], [[math-plus-text]], [[math-expression-tree]], [[mre-utils]]. \end{matrix}}