# MRE (ktmath-scoring) knowledge graph

The MRE (Math Response Engine) is the **KT math scoring engine**: a Python 3 / Flask service built on SymPy that
scores constructed math responses. Responses can be content or presentation MathML, LaTeX, or Math+Text HTML. Each
item is scored against an authored configuration made of a pattern, preprocessors and constraints. The repo is
`ktmath-scoring`, and items, regressions and clients are authored in the separate `ktmath-authoring` repo.

**Who calls it**
- **AD** (`ad:mre`) calls `POST /ws/stateless` with the item definition inline. This is the live path today; see
  [[stateless-score-from-ad]].
- **Legacy drivers** (PANext/ATE, and the TestNav8 QTI custom operator) call `POST /ws/rest` with stored item ids.
- **ktmath-authoring** ([[authoring-tool]]) has three links to the MRE. Its item browser and regression runs call
  `/ws/rest` as `secretClientId`. Its GenerateBuildpack job produces the [[buildpack]]. Its transfer service and
  Review Tool read the [[request-archive]].

**What it is not.** It is not the IEA / CF / ktscoringservices essay engine that oscar calls. Both come from the KT
lineage (pearsonkt.com hosts, and private packages such as `hash_pipe` come from `pypi.ktscoringservices.com`), but the
MRE is a separate service that only scores math. It is also not AD's older `pymathengine`.

## Start here
1. [[scoring-api]]: every public route, with its auth, request and response bodies, and callers.
2. [[score-response]]: the end-to-end scoring trace. Read it before changing anything in the pipeline.
3. [[scoring-item]]: the item configuration shape (pattern, preprocessors, Score As, overrides, PPP).
4. [[scoring-pipeline]]: `score_with_validation` and the triage code in utils/scoring_utils.py.
5. [[select-pattern]]: how the code decides whether a pattern runs in-process, as a parent, or on the internal
   server, and which class runs.
6. [[core-patterns]] and [[scoring-pattern]]: the pattern classes. [[pattern-catalog]] lists every accepted name.
7. [[math-tree]], [[representation]] and [[preprocessors]]: how a response becomes a normalised tree.
8. [[client-auth]] and `permissions.md`: the only access control.

## Main flows
- [[score-response]]: `/ws/rest`, `/ws/stateless` and `/v1/scoreWithInstructions`.
- [[stateless-score-from-ad]]: the AD contract, with the exact payload.
- [[select-pattern]]: the in-process, threaded-parent or `/internal-scoring` choice, and customized versus standard
  classes.
- [[score-dependent-response]]: carry-forward (dependent) scoring through `/internal-dependent-scoring`.
- [[apply-manual-override]]: `/ws/hash` and the item `override` map.
- [[convert-representation]]: `/ws/convert`.
- [[release-regression-gate]]: CI, buildpack import, regression replay, and the cutover to the inactive stack.

Product features are `capability` nodes (for example [[stateless-scoring]], [[dependent-scoring]],
[[tolerance-scoring]] and [[item-regression-testing]]). The glossary is the set of `concept` nodes (for example
[[rubric-name]], [[parent-pattern]], [[transaction-id]], [[scoring-status-code]], [[invalid-sympy-value]] and
[[score-as]]).

## Layout
The container runs two gunicorn programs under supervisor ([[deploy-runtime]]):
- [[scoring-server]] (`server.py`, port 5000) is the public API. It parses, authenticates, loads the item, runs
  preprocessing, and runs parent patterns in a thread.
- [[internal-scoring-server]] (`server_internal.py`, port 5001, 21 s worker timeout, `RLIMIT_AS` memory cap) scores
  one pattern or one dependent logic per request, so that runaway SymPy work can be killed.
- In `MODE=DEV`, everything runs in-process.

Items and clients come from [[mre-mongo]] (DocumentDB; `items`, `scoring.client`) in dev and QA. In staging and prod
they come from S3 [[buildpack]] snapshots loaded at boot (`USE_BUILDPACK`). Every qualifying request is archived in
`log.request` ([[request-archive]]).

Code layout:
- `representation/` parses input into `math_tree/` ([[math-tree]]).
- `preprocessors/` rewrites the tree.
- `patterns/` holds the scoring patterns ([[core-patterns]], [[dependent-patterns]], [[customized-patterns]], and
  the deprecated [[custom-patterns]]).
- `constraints/` and `forms/` hold [[scoring-constraints]] and [[scoring-forms]].
- `utils/sympy_utils.py` holds [[symbolic-equivalence]].
- [[sympy]] 1.14.0 does the algebra.

## Conventions
- **Prefer a standard pattern.** A configuration of the standard patterns is the default. If an item needs code, write
  a **customized pattern**: a subclass of a standard pattern in
  `patterns/customized_pattern/custom_files/<item _id>.py`, with the class named `<item _id>`, selected by
  `isCustomizedPattern: true`. The README says it's fine to refactor the parent pattern to isolate the hook. Do not
  add new `CustomScoringPattern` classes in `patterns/custom/`; they are deprecated.
- **Tests.** Write unit tests for all shared scoring logic (patterns, preprocessors, forms, constraints). Item-specific
  logic, including custom and customized patterns, may rely on the item's regression tests. Run `python tests.py`
  for the unit tests, and `python tests.py -r` to also replay the buildpack regressions locally. Local regressions
  need `buildpack_*` files or a local copy of the regressions database.
- **Buildpacks.** An item change reaches staging and prod only through a new buildpack plus a release.
  `BUILDPACK_VERSION` pins it, and [[release-regression-gate]] replays every accepted regression before cutover.
- **Private PyPI.** `hash_pipe` and other KT packages come from `pypi.ktscoringservices.com`. Configure pip with
  `extra-index-url` and `trusted-host` as shown in README.md and `pip.conf`. dependency_workarounds.md records
  pinned-library patches, but its SymPy version (1.11.1) is out of date.
- **Logging.** `utils/ktlogging.py` writes to stdout and `/var/log/math/scoring/pyfull.log`, so the directory must
  exist locally.
- **Docs.** `docs/` (endpoints, patterns, preprocessors, reference) is served at `/docs` outside PROD. Keep it in step
  with code changes.

## Known risks
All of these were checked in the code at the time of writing. Paths are in the repo; no secret values are reproduced.
1. **`secretClientId` backdoor.** In `verify_client`, any request whose `clientUUID` is the literal `secretClientId`
   skips authentication on every environment (patterns/helper.py:63-65). Regression tooling and the authoring item
   browser depend on it ([[secret-client-id]]).
2. **Internal auth bypass when `INTERNAL_SECRET` is unset.** `INTERNAL_SECRET` defaults to `None` (config.py:18).
   `_verify_internal_credentials` compares with `==` (server_internal.py:197), so `{"clientUUID":"INTERNAL_SCORING"}`
   with no secret passes. The internal server binds `0.0.0.0:5001` (prodconf/supervisor-gunicorn.conf:22). No deploy
   file in the repo sets `INTERNAL_SECRET`.
3. **XML parsing of untrusted MathML without explicit hardening.** `MathMLParser` (representation/mathml/mathml_parser.py:56)
   and `XmlValidator` (representation/mathml/utils/xml_validator.py:41) call `etree.fromstring` with lxml's default
   parser. Only `pres_mml_to_latex` passes `resolve_entities=False, no_network=True`. The pinned lxml 6.1.0 does not
   load external entities by default (lxml 5+), so file-read XXE is currently blocked by that version pin, not by the
   code. Internal entities still expand. Treat it as latent: downgrading lxml or adding a custom parser re-opens it.
4. **Static salt and a non-constant-time compare for client secrets.** The salt is a hard-coded literal in
   utils/hash_utils.py:4, the hash is a single SHA-256 round, and `validate_hash` compares with `==`
   (utils/hash_utils.py:17-22).
5. **Unauthenticated `/ws/convert`, `/ws/hash` and `/regressions`.** The handlers are at server.py:380-383, 412-464
   and 467-500. `/ws/convert` parses arbitrary XML or LaTeX in the public worker. `GET /regressions` replays every
   bundled regression against `REGRESSION_URL` on each hit (apis/regression_api.py:17-61), which is a cheap DoS, and
   it writes one `log.request` document per regression. `GET /rest/regressions/bundled` returns the whole regression
   file.
6. **CORS `*`.** `@cross_origin()` with default settings is on `/ws/rest`, `/ws/stateless`, `/rest/regressions/bundled`,
   `/ws/convert` and `/ws/hash` (server.py:181, 273, 394, 413, 468).
7. **Plaintext client secrets on disk, and TLS verification off.** scripts/client_secrets.py appends
   `clientId -- secret` to a local file (lines 26-41 and 67-68). It and two other scripts connect to DocumentDB with
   `tlsAllowInvalidCertificates=True` (scripts/client_secrets.py:91,
   scripts/data_management_scripts/reprocess_item_in_review_tool.py:114,
   scripts/story_scripts/story_script_template.py:50). Separately, `/v1/scoreWithInstructions` archives the caller's
   plaintext `clientSECRET` in `log.request` (server.py:140, see [[request-archive]]).
8. **Candidate responses logged at INFO.** The root logger is at INFO (utils/ktlogging.py:35). Every scoring run logs
   the full candidate response, and the `itemDefinition` on `/ws/stateless`
   (`to_log(..., include_candidate_response=True)`, utils/scoring_utils.py:62; models/models.py:361-368, 439-443).
   scoreWithInstructions logs the instructions and response too (server.py:130-134). Logs go to Splunk.
9. **Unbounded SymPy `simplify`.** `equiv` and the helpers call `simplify()` with no time limit
   (utils/sympy_utils.py:172-186, 271, 419, 491). The only bounds are the internal worker's 21 s timeout and
   `RLIMIT_AS`, but parent patterns and `/v1/scoreWithInstructions` run in the public worker (200 s gunicorn
   timeout). A timed-out parent thread is never killed.

Also visible in the repo: the S3 buildpack bucket name, including the AWS account id (utils/build_pack_loader.py:31).

## Notable bugs
- `requestUUID` without `testUUID` on `/ws/stateless` raises `NoTestIdException`, which no handler catches, so the
  caller gets status 2 and AD retries ([[stateless-score-from-ad]]). Status 11 (`NO_ITEM_DEFINITION`) is unreachable
  because a missing `itemDefinition` becomes `{"rubricName": ""}` and fails as status 6.
- Parent-pattern timeout: `score_with_timeout` reads `result[0]` from an empty list, which raises `IndexError` and
  gives status 1 (utils/scoring_utils.py:207).
- The internal-request retry catches the builtin `ConnectionError`, which `requests` does not raise, so connection
  failures take the `RequestException` branch and return status 2 (utils/scoring_utils.py:263-280).
- A failure while hashing an override calls `.to_log` on a dict (utils/scoring_utils.py:376). That raises
  `AttributeError`, so the response becomes unscorable instead of falling through to normal scoring
  ([[apply-manual-override]]).
- The archived `transactionId` is a fresh uuid from `copy_for_archive`, not the tid in the logs ([[transaction-id]]).
- `scripts/client_secrets.py --rotate` updates by `{'_id': clientId}`, which matches nothing, but still writes the "new"
  secret to the output file (lines 33-41).
- The Jenkinsfile references an undefined `email_distro` in its notification step (Jenkinsfile:176).
- `Score.__str__` is decorated `@property` ([[score]]). `XMLDepthException.__int__` is a typo for `__init__`.
- Dead code: the MPT text-parser path (`MathMLConverter._convert_mpt_to_mathml`, MRE-367), `copyToLocalMongo`, and
  the uwsgi/nginx configs.