# CF (KT Continuous Flow) — knowledge graph, `external_client_api` only

**What CF is.** Continuous Flow (repo `ktcf-continuous-flow`, Go module `git.pearsonkt.com/111/111-continuous-flow`) is Pearson Knowledge
Technologies' automated constructed-response scoring platform (`*.ktscoringservices.com`). Clients send student responses, CF scores them with
IEA/IEA2 models, runs learner-in-danger (LID) screening, and pushes outcomes back. OSCAR calls it **IEA** (also CF, KT, ktscoringservices).

**Scope of this graph.** Only the **External Client API** binary (`external_client_api/`), plus the imported packages exactly as far as that binary
uses them (`api_contexts/{client_common,api_common,binding,lidlite}`, `service/*`, `datatypes/*`, `common/{config,db,messaging,ktsecrets,ktxray,ktmetrics}`).
Everything else in the repo (scoring consumers, admin app, system_integration, drivers, modeling) is **not mapped**. Two boundary nodes,
[[system-integration]] and [[external-client-driver]], are summaries that exist so the OSCAR contract can be traced. Their own nodes say they are out of scope.

## Start here
1. [[external-client-api]] — the module: wiring, handlers, dead code, gotchas.
2. [[client-api]] — full route table, swagger cross-check, **OSCAR → CF contract table**.
3. [[oscar]] — both directions of the OSCAR contract, including the CF → OSCAR `/iea/v1/*` callbacks.
4. [[response-intake-and-scoring]] — the main round trip.
5. [[client-common]] — the shared intake logic (de-aliasing, placeholder prompts, queue dispatch, LID kickoff).
6. [[cf-response]] — the central record; then [[cf-score]], [[cf-prompt]], [[cf-admin]].
7. `permissions.md` — auth model (client key/secret → `X-Auth-Token`, no authorization layer).

## Main flows
- [[response-intake-and-scoring]] — `POST /v1/response` → store → `SCORE_RESPONSE` → scoring → outcome to `tags.outcomeEndpoint`.
- [[human-outcome-intake]] — `POST`/`PATCH /v1/outcome`: human scores, QC/true scores, invalidations.
- [[provision-tenant-project-item]] — explicit provisioning lives in system_integration; this API auto-creates placeholder prompts.
- [[lid-review]] — LID workflow, `/v1/outcome/lid`, LID Lite.
- [[rms-media-response-intake]] — RMS image URL policy.
- [[rangefinding-score-intake]] — pre-range-finding / rangefinding-true scores. TOD modeling is not in this API.
- [[plagiarism-check]], [[response-export-release]].

## Layout
`external_client_api/main.go` boots config → logging → Mongo → Prometheus → workflow queue sender → gin router (`api_common.GetKTGinRouter`), mounts
LID Lite (`api_contexts/lidlite`) and then `AppContextImpl.Init` (`api.go`). Handlers in `api.go` are thin: they validate `datatypes`/`service` request
structs, call `client_common.ClientAPIContext` for alias resolution and scoring dispatch, use `service.*Service` for Mongo, and push SQS events. Storage:
a common Mongo DB (`<env>_scorearchive`) plus per-prompt or per-customer DBs for `responses`/`scores` ([[cf-mongo]]). Queues: [[sqs-queues]]. Config:
[[external-client-api-config]] (`default.json` + `CF_*` env, `__` nests).

## Conventions for implementers
- Add a route in `AppContextImpl.Init` (api.go:91-139) inside the `v1`/`v2` group so `AuthMiddleware` applies. Add swag annotations and run
  `make swag` in `external_client_api/` (needs `swag` v1.8.12, `make install-swag`). The checked-in swagger is stale (2024-10-30).
- Request types live in `service/api_datamodel.go` or `datatypes/*types`. Implement `Validate()` (`IsValidatable`) and, for test support, `MockResponse`
  (`IsMockable`), and bind with `binding.DecodeAndMockRequest` so the `mock` convention works ([[mock-field]]).
- Reply with [[ack-response]] (`acktypes.NewAckDetail(...).LogAndReturnJSON`) or `handleErrWithLog` for error-to-status mapping.
- Always de-alias client `adminName`/`itemId` through `ClientAPIContext.GetPromptNameFromItemID` (be aware that it creates prompts) or
  `PromptService.DealiasInboundRequest` (does not).
- Tests: `make test` in `external_client_api/` (gotestsum, needs Mongo and AWS or `BYPASS_SECRETS`). RMS policy: `post_response_rms_test.go`.

## OSCAR ↔ CF contract (summary)
- OSCAR → this API: `/v1/auth`, `/v1/response` (RequestScore), `/v1/outcome` POST (ReportHumanScore, SmartSortCodeReviewed) and PATCH
  (NotifyScoreInvalidated), `/v1/outcome/lid` (LidReviewed), `/v1/response/release` (NotifyResponseExported).
- OSCAR → system_integration (out of scope): CreateTenant, Create/UpsertProject, CreateItem, TOD / smart-sort / field_test items, modeling, ABBI.
- CF → OSCAR: `/iea/v1/auth`, `/iea/v1/outcome`, `/iea/v1/outcome/lid`, `/iea/v1/score/request`, `/iea/v1/workflow/state`, via [[external-client-driver]].
- Mismatches: see [[client-api]] › OSCAR contract and Known risks 9-13.
- `scoredResponseText` (AD-6322) comes from CF **scoring** (IEA2 score explanations), not from this API ([[scored-response-text]]).

## Known risks (verified in code; paths only)
1. **No authorization after authentication.** `AuthMiddleware` checks only that the session exists. It ignores `client.active`, `client.roles` and any
   tenant/admin scoping, so any CF client can read test data for, post to, invalidate, release or delete responses of any admin/item, and bust the alias cache
   (external_client_api/api.go:142-189, service/client_service.go:220-245).
2. **Hardcoded credential in source** for the Swagger UI basic auth (external_client_api/api.go:104-106).
3. **Secrets/PII in logs.** Session tokens are logged on auth failure (api.go:154-163). Raw request bodies, including student response text, are logged on bind/validation failures
   (api.go:412-415, 423-427, 932-937, 1497-1500).
4. **Open endpoints when `API_AUTH_ENABLED` is unset** (it is not in external_client_api/default.json): `/v1/lid_lite/*` (api_contexts/lidlite/lidlite.go:63-68, 77-83)
   and `/internal/:appName/config`, which returns the key-name-redacted config **plus the whole process environment** (api_contexts/api_common/api_common.go:1202-1206,
   common/config/config.go:765-790). Whether the deployment sets it is not visible in this repo.
5. **Silent failures after a 200.** `PostResponse` replies before scoring dispatch and discards its error (api.go:622-624). `PatchOutcome`/`PostOutcomeLid` return
   an empty 200 when alias resolution fails (api.go:1379-1383, 1508-1512).
6. **Crash paths.** `NewAppContextImpl` returns nil if the LID-review or additional-score queue cannot be opened, and main then dereferences it (api.go:250-265, main.go:77-83).
   `GetTestResponses` dereferences a nil `frontDoor` (api.go:854), keeps going after writing a 400 (api.go:791-795), and uses `promptStats` before checking its error (api.go:799-817).
7. **Data race:** `PostBustNameCaches` swaps the alias map without the mutex used elsewhere (api_contexts/client_common/client_common.go:172-177 vs 877-890).
8. **Any client can create prompts.** Unknown `itemId`s create DISABLED placeholder prompts (and send workflow events) under any adminName
   (api_contexts/client_common/client_common.go:357-487).
9. **OSCAR QC true-score invalidation always fails.** OSCAR sends `outcomeId:""` and no `scoreType`, and CF requires `outcomeId` (oscar src/models/iea_models.go:919-938;
   datatypes/responsetypes/external_client_helpers.go:219-248).
10. **OSCAR smart-sort reviews may be rejected.** `SmartSortCodeReviewed` posts `/v1/outcome` with no trait scores. CF requires one per prompt trait when the prompt defines
    any (datatypes/responsetypes/external_client_helpers.go:301-380).
11. **OSCAR calls routes CF does not have:** `/v1/tenant/{t}/project|item/{id}` and `…/workflow_state` (oscar src/services/iea/iea_project_service.go:214-291,
    iea_item_service.go:467-552, iea_tenant_service.go:369-399). There is no match in this repo. `/v1/test_responses` is an unimplemented OSCAR stub.
12. **RMS default-deny vs OSCAR `send_images`.** CF rejects RMS clip URL text unless `allowRmsImageUrls` is set on the prompt/admin (service/cf_datamodel.go:1001-1013,
    datatypes/matchtypes/validation_filters.go:243-268).
13. **OSCAR host routing unverifiable.** OSCAR uses `oscar.<env>.ktscoringservices.com` for every call (oscar src/options/options.go:144-145). In this repo that host is
    the system_integration binary, which does not serve `/v1/auth`, `/v1/response` or `/v1/outcome` (api_contexts/system_integration/system_integration.go:101-187).
    The ingress rules live outside the repo.
14. **Re-posting a response replaces `tags` wholesale**, which drops CF-added tags such as `scoredResponseText` (service/response_service.go:737-830).
15. **Pre-range-finding partial write.** The score is saved, then a failed notification push returns 500. A client retry gets 400 "duplicate" (api.go:1071-1093).
16. **Callback target chosen by the client.** `tags.outcomeEndpoint` overrides the prompt's notification target. Credentials are only attached when a matching
    `CFClientApiConfig_<endpoint>` secret exists (datatypes/responsetypes/cf_response.go:182-245, service/cf_datamodel.go:2362-2420).
17. **Stale/misleading swagger.** It lists `/v2/sync/*` (not served) and omits `/v2/auth`, `/v1/bustCache`, `/v1/plagiarism/*` and `/v1/lid_lite/*`
    (external_client_api/swagger.yaml, external_client_api/Makefile:21-27).