# PSAES knowledge graph

**PSAES** (Pearson Scan and Assessment Engineering System) is where Pearson's scanning team engineers the paper answer documents that get scanned. People use it to:

- design scannable forms: [[document]]s, the OMR [[field]]s drawn on each page, and the character sets ([[content]]) those fields use;
- generate test data for those forms;
- have the VIPP print server render the test data as filled-in test decks, and batch the decks into weekly print runs;
- run each form of a customer [[scan-pgm]] through the SPS and PV scan-test stages, which include building and locking its SRL (scan record layout);
- serve those SRL layouts to PASS.

There are two repos:

- `esuite-psaes-backend`: Spring Boot 4 / Java 17. It owns the PostgreSQL schema through Flyway V1..V35.
- `esuite-psaes-gridfe`: a React 18 + Vite SPA, the "PSAES Command Center", which signs users in with Azure AD.

## Start here
1. [[psaes-backend]]: package layout and every controller.
2. [[scan-test-ui]]: the hub screen most users live in. Every stage button is listed there.
3. [[scan-workflow-service]] and [[task]]: the stage engine and the seeded SPS/PV stages.
4. [[srl]]: the scan record layout engine (anchors, generation, lock).
5. [[vipp-service]] and [[vipp-server]]: how PSAES talks to VIPP through S3 drop folders.
6. [[test-case-generation]]: how test data is built.
7. [[form-design]] and [[field]]: the documents, fields and skip patterns behind PETE.
8. [[gridfe-route-access]] and `permissions.md`: the client-side role model. The backend enforces none of it.
9. [[psaes-postgres]] and [[flyway-migrations]]: the schema and how it changes.

## Main flows
- Getting in: [[sign-in]].
- Program setup: [[create-scan-program]], then [[add-document-to-scan-program]].
- Form design: [[upload-document-pdf]], then [[design-fields]] (PETE and clips). [[generate-litho]] and [[import-arl]] are part of the same stage.
- SRL: [[define-and-lock-srl]]. [[unlock-srl]] reverts the stages.
- Test data: [[generate-test-cases]], [[import-test-case-file]] or [[csv-test-data-round-trip]].
- Print: [[request-vipp-test-deck]], then [[tuesday-print-run]].
- Stage moves used by every step: [[advance-scan-test-stage]].
- Outbound to PASS: [[pass-srl-extract]].
- Operations: [[run-vipp-health-check]] and [[delete-program-document]] (API only).

Typical SPS journey for one form: PDF ([[upload-document-pdf]]), then PETE ([[design-fields]]), then SRL ([[define-and-lock-srl]], which also moves PV to VIPP), then VIPP ([[generate-test-cases]] and [[request-vipp-test-deck]]), then TUES ([[tuesday-print-run]]), then VERI, then APRV.

The product features are the `capability` nodes, such as [[scan-test-workflow]], [[visual-field-design]] and [[vipp-test-decks]]. The glossary is in the `concept` nodes, such as [[srl-term]], [[arl-term]], [[litho]], [[test-admin-term]] and [[sps-pv]].

## Module layout
**Backend** (`src/main/java/com/pearson/psaes/`):

- `controller/` holds about 20 REST controllers. All are under `/api`, except `/api/wfm`. They are grouped into the `*-api` interface nodes.
- `services/` holds the business logic, grouped into the module nodes:
  - [[form-design]], [[field-clip]] and [[litho-automation]];
  - [[test-case-generation]], [[csv-test-data]] and [[test-data-validation]];
  - [[vipp-service]] and [[print-run-service]];
  - [[srl]] and [[arl-import]];
  - [[scan-program-service]], [[scan-workflow-service]] and [[test-admin-service]];
  - [[user-service]], [[user-alert-service]], [[pass-extract]] and [[admin-service]].
- `job/` holds the heavy builders: `TestCaseGenerator`, `CsvTestDataProcessor`, `VippFileExtract`, `PrintListLabelExtract`, `TuesdayZipExtract`, `SrlGenerator`, `SrlUnformatted`, `ArlImporter` and `VippHealthCheck`. Despite the package name these are not scheduled jobs. Several are built with `new` per request and run synchronously inside the HTTP call.
- The only schedules are [[vipp-health-check]] (hourly) and [[exception-log-purge]] (daily).
- `extract/PassExtract` builds the PASS payloads.
- `exception/` holds the `PsaesBaseException` hierarchy and `GlobalExceptionHandler` ([[exception-handling]]).
- `config/CloudConfig` hand-builds the Hikari datasource and the JPA setup ([[psaes-app-config]]).

**Flyway.** SQL files `flyway/db/V1..V35` are run by a separate `psaes-flyway` container, never by the app (`spring.flyway.enabled=false`, `ddl-auto=none`). See [[flyway-migrations]] and [[psaes-deploy]].

**gridfe** (`esuite-psaes-gridfe/src/`):

- `pages/` holds the screens: [[scan-test-ui]], [[pete-ui]], [[srl-ui]], [[document-design-ui]], [[vipp-print-ui]], [[user-management-ui]] and [[app-shell-ui]]. [[admin-ui]], [[lakeside-ui]], [[preview-ui]] and [[prototype-pages-ui]] are legacy.
- `services/` holds the axios wrappers per area.
- `auth/` holds MSAL and the user-record checks.
- `utils/routeAccess.js` and `utils/roleUtils.js` hold the role model.

**VIPP S3 drop-folder protocol.** PSAES never calls VIPP. It writes CSV data files into [[psaes-s3]] under:

- `toVipp/documents/` for test decks and the health check;
- `toVipp/labels/` and `toVipp/printlist/` for print-run sheets.

Each file is named `<prefix>_S<vippRequestId>_<timestamp>…`. The [[vipp-csv-submitted]] event covers this side. VIPP writes the rendered PDF back to S3. Something outside both repos then posts the S3 key to `POST /api/postVippPdfLogs` ([[vipp-pdf-returned]]). PSAES parses the [[vipp-request]] id from the second `_` token and flips the request from `P` to `C`. Print-run zips go to `printRun/`. Uploaded documents, ARLs and test files go under `archive/`.

**PASS.** [[pass]] pulls data and PSAES pushes nothing. It calls `POST /api/passSrlExtract` and `/api/passDocCodesExtract` ([[extract-api]], [[pass-extract]]). It identifies the form by `accountCode` + `adminCode` + `docCode`.

**Cross-app identifiers.**

- `account.account_code` maps to `adam:client.customer_code`.
- `test_admin_system_code.code`, for system `ADAM`, maps to `adam:admin.admin_code`.

Both are only stored. PSAES makes no calls to ADAM, MOT or AD, and it does not use ADAM's org or barcode lookup APIs.

`HomeRepository.getMasterAOItems` queries MOT-style tables (`config.test_administrations`, `orders.distribution`). It is dead code copied from esuite-mot: nothing injects the repository, the app has a single `psaes` datasource, and there is no cross-database link.

## Conventions an implementer must follow
These come from `docs/dev-standards.md`, `docs/entity-flyway-schema-sync.md` and `CLAUDE.md` in the backend.

- **Layering.** The chain is Controller → Service → Repository. Never inject a `*Repository` into a controller. This applies even to the "excluded flows".
- **Injection and complexity.** Use constructor injection with `private final` fields and no field `@Autowired`. Keep the cognitive complexity of each method below 15.
- **Exceptions.**
  - Every custom exception extends `PsaesBaseException extends RuntimeException`. Each one maps to an HTTP status in `GlobalExceptionHandler`.
  - A catch-all block needs `catch (PsaesBaseException e) { throw e; }` before `catch (Exception e)`.
  - Controllers do not log before throwing, because the handler is the single log point.
  - Never add throwing to the excluded flows (`uploadTestCaseFile`, `uploadTestData`, `uploadTestDataDoc`, the CSV/validation pipeline). They return row-level errors in the response body by design.
- **Schema changes.**
  - Change the entity `@Column` first, with exact length, precision and nullability.
  - Write a `flyway/db/V{next}__description.sql` that produces exactly that column.
  - Put both halves in the same PR.
  - `ddl-auto` stays `none` or `validate` outside local dev. A smoke incident on 2026-07-14 came from `ddl-auto=update`.
- **Other rules.**
  - The S3 bucket comes from `env.getProperty("psaesBucket.path")`.
  - Temp files use `FileUtilities.ROOT_PATH` with a UUID.
  - `ModelMapper` is a bean and `ObjectMapper` is not.
- **Tests.**
  - Backend: `mvn -o test`. These are JUnit 5 + Mockito + AssertJ unit tests only, with no Spring context. JaCoCo enforces 95% instruction and branch coverage on the bundle. Verify bean wiring by reasoning about it or by starting the app.
  - gridfe: `npm test` (vitest).
- **Branches.** Branches are named after the Jira key: `PSAES-<n>`, `PSAES-<n>-<desc>` or `feature/PSAES-<n>`. The base branch is `master`.
- **Stage moves.** Always go through `ScanWorkflowService.completeStatus` / `removeStatus`, which take the `scan_pgm` row lock. Never insert [[task-approval]] rows directly. `DocumentService.uploadDocumentPdf` does insert one directly, hard-coding task id 2.

## Known risks
Each risk below was checked in the code.

- **No backend auth, and userId is trusted.**
  - There is no Spring Security in `pom.xml` and no token check anywhere in `src/main/java`.
  - Any caller can add or deactivate users, lock or unlock SRLs, start print runs, or read the PASS extracts.
  - `userId`, `insertUser` and `updateUser` come from request params or bodies, so audit columns can be forged.
  - `GET /api/getUserByUserEmail` returns any user's record for any email.
  - CORS (`app.cors.allowed-origins`) is the only boundary.
  - See `permissions.md`.
- **DB passwords printed to logs.**
  - The backend's `start.sh` runs `cat /opt/app/application.properties` after confd renders it. That file contains `psaes.datasource.password`.
  - `flyway-entrypoint.sh` runs `cat /opt/psql/pre-migrate/db-setup.sql`, which contains the app user and read-only user passwords in `CREATE/ALTER USER … WITH PASSWORD`.
  - Both end up in container logs.
- **Old credentials in source.** The confd template keeps a commented-out Oracle datasource with a plaintext username and password.
- **Stack traces in every error response.** `ExceptionResponse` always serialises the first 15 stack frames (`stackTrace`) along with the exception class name. `spring.web.error.include-stacktrace=never` only affects Spring's default error page.
- **`npm start` runs the Vite dev server in production.** The gridfe Dockerfile runs `npm run build`, but `start.sh` (the ENTRYPOINT) ends with `npm start`, which is `vite --host 0.0.0.0`. It serves the source through the dev server, not the built `dist`.
- **Dead MySQL credentials.** `esuite-psaes-gridfe/src/dbUtils/database.xjs` holds hard-coded local MySQL connection settings (user `cps` and others). Nothing imports it.
- **Main bugs.** Details are on the linked nodes.
  - Deleting a program document fails on the FK once an SRL or a linked test case exists ([[delete-program-document]]).
  - The VIPP health check needs a hand-seeded `vipp_request` id 1 ([[run-vipp-health-check]]). `callApiEveryTwentyMinutes` actually runs hourly.
  - `maybeTriggerPrintRunZip` NPEs on an unmatched label or print-list PDF ([[tuesday-print-run]]).
  - "Print All Owatonna" is not atomic ([[tuesday-print-run]]).
  - `removeStatus` NPEs when a document has no approval ([[advance-scan-test-stage]]).
  - Project-cost upload: the admin page posts to `/api/upload-csv`, which does not exist. `uploadRptProjCost` parses dates with `DD` (day of year), inserts the header line as a data row, and truncates the table outside a transaction ([[rpt-proj-cost]], [[admin-ui]]).
  - The WFM `@Async` handler is a self-call, so it never runs asynchronously, and it does nothing anyway ([[wfm-service]], dead).
  - The default `psaes.srl-lock.require-all-core-fields=false` exists only because baseline record type 17 lacks a batch-pass entry ([[psaes-app-config]]).
  - The fallback scan program `999999` is not seeded by any migration ([[fallback-program]]).
  - PDF uploads report failure (500) after the upload and stage move have already been saved, if the RMS copy fails ([[upload-document-pdf]]).
  - `scan_pgm_task` is a view entity whose `@Id` is not unique ([[scan-pgm-task]]).