# Knowledge graph index

_Generated by `kg.mjs index` — do not edit. Start with [README](README.md)._

328 nodes.

## entity (68)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [test-attempt](nodes/test-attempt.md) | A student's attempt at one test, i.e. an ADAM session (hub_testattempt + sat_testattempt_adam/_accommodation/_sections/_wida); the central fact of the warehouse. | 93 | 15 | warehouse, data-vault, core |
| [document](nodes/document.md) | A Doc Depot document (documentUUID/docId): metadata item DOC#{uuid} in ReportingTable plus the uploaded file in the orders bucket, optionally linked to a reporting order and tags. | 79 | 5 | reporting, doc-depot |
| [extract-config](nodes/extract-config.md) | An extract configuration (extract_name + JSON data of admin codes, layouts, file-name conventions, split/delivery/ADAM defaults) that users order from; the same table also holds customer_groups and customer-label records. | 62 | 1 | extract-interface, config |
| [extract-order](nodes/extract-order.md) | An extract order placed from the Extract Interface/DepotUI: chosen extract config, resolved extractKeys (the PWS payload keys), status and delivery history; shares its OrderUuid with the PWS reporting order. | 58 | 4 | extract-interface, order |
| [reporting-order](nodes/reporting-order.md) | A Reporting as a Service order (orderUuid): ORDER# item in ReportingTable that records who asked and which Workflow-as-a-Service workflow (uniqueExternalKey = orderUuid) executes it; its status lives in the workflow. | 48 | 4 | reporting, orders |
| [organization](nodes/organization.md) | A state/district/school organization (hub_organization + sat_organization, sat_organization_metadata) with a parent hierarchy via link_organization_parent_organization. | 46 | 6 | warehouse, data-vault, roster |
| [person](nodes/person.md) | A person (student, teacher, proctor) in the warehouse - hub_person + sat_person, sat_student_demographic, sat_person_metadata, sat_demographic_metadata. | 45 | 5 | warehouse, data-vault, roster |
| [workflow](nodes/workflow.md) | A Workflow as a Service workflow — a JSON document of ordered worker events, moved between S3 state folders (submitted/processing/waiting/completed/failed/cancelled) with a status record in WorkflowTable. | 39 | 4 | workflow, core |
| [item](nodes/item.md) | A test item (question) as delivered on a form section (hub_item, keyed item_id+form_identifier+section_id); per-attempt item data lives on sat_link_testattempt_item. | 38 | 4 | warehouse, data-vault, items |
| [abbi-email-notification](nodes/abbi-email-notification.md) | An ABBI email-notification subscription (an email + ABBI environment/admin code/test code/extract filter + frequency) telling the reporting service whom to email when ABBI extract documents arrive. | 36 | 2 | extract-interface, abbi, notification |
| [tenant](nodes/tenant.md) | A PWS tenant - one customer landing zone (REST + WebSocket API, authorizer, ECS cluster, VPC/subnet ids) stored as one PWSTenantMapper row and fanned out to one stack per tenant by every cow-* service pipeline. | 36 | 3 | platform, tenancy |
| [battery-attempt](nodes/battery-attempt.md) | A student's attempt at a battery (a group of tests reported together) - hub_battattempt + sat_battattempt_adam, sat_battattempt_metadata, with battery-level results on sat_link_battattempt_result. | 35 | 3 | warehouse, data-vault |
| [item-outcome](nodes/item-outcome.md) | An item outcome/trait identifier (hub_item_outcome); the per-attempt scored values live on sat_link_testattempt_item_outcome. | 33 | 4 | warehouse, data-vault, items, scoring |
| [admin-job](nodes/admin-job.md) | A tracked operation started from the Admin SPA (JOBS_TABLE, keyed by job_id) — ADAM/STM ingest, MV refresh, ES objective / PR growth / scoring category load — with polled status and monitoring_data. | 30 | 2 | admin-spa, jobs |
| [abbi-extract](nodes/abbi-extract.md) | An ABBI extract file record in the Reporting stack's ABBI extracts DynamoDB table (Doc_id, environment, adminCode, formId, extractName, isoDate, active) that DepotUI/Extract Interface users search, download and soft-delete. | 29 | 2 | abbi, reporting, dynamodb |
| [scheduled-extract](nodes/scheduled-extract.md) | A recurring extract schedule (daily/weekly/monthly/custom, IANA timezone) that owns a frozen clone of its base config and whose runs are ordinary extract orders launched by the in-process scheduler. | 29 | 3 | extract-interface, schedule |
| [admin-client](nodes/admin-client.md) | A COW customer/datasource registry entry (CLIENTS_TABLE, keyed by datasource_id) — ADAM credentials env, prd/pv admin codes, STM previewer instances/tenants; the input to every ingest, refresh and load. | 27 | 1 | admin-spa, registry |
| [warehouse-reference-table](nodes/warehouse-reference-table.md) | Redshift ref_* reference tables - scoring categories/objective keys, accountability codes, percentile objectives, state abbreviation mapping, admin-specific MV definitions, updated-attempts queue, PR growth prior-year data. | 25 | 1 | warehouse, reference |
| [external-score](nodes/external-score.md) | A hand/AI (external) score with up to five rater reads plus a final score (hub_external_score + sat_external_score), linked to attempts, items and item outcomes. | 23 | 4 | warehouse, data-vault, scoring |
| [data-vault-link](nodes/data-vault-link.md) | The Data Vault link tables (link_*) and their satellites (sat_link_*) relating warehouse hubs - attempt↔person/org/item/outcome/result/battery/pdf/video, org hierarchy, enrollments. | 20 | 16 | warehouse, data-vault |
| [document-definition](nodes/document-definition.md) | A per-tenant document type (docCode) in ReportingTable declaring name, MIME type, required keys and tags; its reserved:* tags decide what happens when a document of that type is uploaded. | 20 | 3 | reporting, doc-depot |
| [result-category](nodes/result-category.md) | A score/objective category (hub_resultcategory, e.g. SCALE_SCORE, *_RAW_SCORE, ZH, claim keys) used for session and battery outcomes; values on sat_link_testattempt_result / sat_link_battattempt_result. | 20 | 3 | warehouse, data-vault, scoring |
| [student-pdf](nodes/student-pdf.md) | A student/organization report PDF ingested into the warehouse (hub_pdf + sat_pdf) linked to persons, organizations and attempts. | 20 | 4 | warehouse, data-vault, reports |
| [stm](nodes/stm.md) | Scoring test map (STM) data in the warehouse - formerly hub_stm/sat_stm/hub_stmmetadata tables (dropped V11/V15), now derived from session data by mv_all_stm / mv_all_stm_metadata. | 19 | 3 | warehouse, stm |
| [tag](nodes/tag.md) | A per-tenant tag (TAG#{NAME}) in ReportingTable with a reference count, attached to document definitions and documents via paired link items; drives routing and workflow lookup. | 19 | 1 | reporting, doc-depot |
| [mv-sdf-extract](nodes/mv-sdf-extract.md) | Student Data File (SDF) extract materialized views — one header row per test attempt (mv_sdf_extract) plus its test-level, battery-level, item-outcome and evidence-statement score rows. | 18 | 10 | warehouse, extract-mv, sdf |
| [depotui-user](nodes/depotui-user.md) | A DepotUI user/role row in DepotGatewayIdToken — email (client_id) plus a comma-separated userRole string that drives both API admin checks and SPA screen/extract-catalogue access. | 17 | 1 | depotui, legacy, auth |
| [service-mapping](nodes/service-mapping.md) | A PWSServiceMapper row - one registered cow-* service (service_name to its CodePipeline name, plus optional per-tenant pinned lib versions) used to fan deployments out to tenants. | 17 | 2 | platform, deployment |
| [extract-user](nodes/extract-user.md) | An Extract Interface user PROFILE row in extract-int-token (email key, SSO roles seen at last login, and the per-customer/tier permissions block written by cow-admin-spa). | 16 | 1 | extract-interface, auth, authz |
| [customer-group](nodes/customer-group.md) | The customer_groups record in extract-int-config — maps customer codes (MD, PR, WIDA…) to Extract Interface label prefixes; the unit of Extract Interface customer grants. | 15 | 1 | extract-interface, authz, admin-spa |
| [class](nodes/class.md) | A OneRoster class/section (hub_class + sat_class) with student enrollments (link_student_class + sat_link_student_class), org and attempt links. | 14 | 4 | warehouse, data-vault, roster |
| [interaction](nodes/interaction.md) | A QTI interaction (response slot) within an item (hub_interaction); the student's response per attempt is on sat_link_testattempt_interaction. | 13 | 3 | warehouse, data-vault, items |
| [service-manifest](nodes/service-manifest.md) | The COW release manifest JSON (manifest.json / wida-manifest.json in cow-manifest) pinning each CDK service version and container image URN for one pipeline/environment line. | 12 | 15 | deploy, release |
| [client-credential](nodes/client-credential.md) | A machine client_id/client_secret pair (CLIENT_CREDENTIALS table) that POST /client/oauth exchanges for an admin-role JWT. | 11 | 1 | admin-spa, auth, security |
| [forensics-unit](nodes/forensics-unit.md) | The forensics review unit - one school x test, composed at backend startup from the school-level ABR/RCA/LNG(/L1) rows; carries per-signal flags, convergence (count of flags) and severity (peak adverse z). | 11 | 3 | forensics, entity |
| [pdf-manifest](nodes/pdf-manifest.md) | The CSV manifest that describes a batch of student/org report PDFs (RSRD zip manifest.csv + companion.txt, or Spotlight success CSV), and its rebuilt form with DocDepotId columns that pdfToRedshift loads. | 11 | 3 | data-service, pdf |
| [forensics-signal-report](nodes/forensics-signal-report.md) | Aggregated (district/school x test) forensic signal tables - ABR aberrant response, RCA response change, LNG longitudinal, and derived L1 timing - stored as abr_agg / rca_agg / lng / l1 parquet. | 10 | 2 | forensics, entity |
| [mv-all-extract](nodes/mv-all-extract.md) | Shared per-attempt extract MVs used by every extract family — accommodations, student demographics, school hierarchy, test sections, person/demographic key-value metadata and tools-analytics usage. | 10 | 5 | warehouse, extract-mv |
| [mv-extract-organizations](nodes/mv-extract-organizations.md) | Organisation MVs for extracts — reporting and enrollment school/district per attempt (pipe-joined lists), reporting-org ancestor tree, C2C and summary variants, school→district identifiers and org model country/state metadata. | 10 | 3 | warehouse, extract-mv, organization |
| [mv-extract-responses](nodes/mv-extract-responses.md) | Long-format item/interaction response MVs (mv_sdf_extract_responses, mv_catirf_extract_responses) — one row per attempt × item × interaction with score, attempt status, timing, raw and pre-normalisation responses. | 10 | 4 | warehouse, extract-mv, sdf, irf |
| [result-type](nodes/result-type.md) | The type of a session result (hub_resulttype), the third key of link_testattempt_result alongside attempt and result category. | 10 | 1 | warehouse, data-vault, scoring |
| [stm-tracking-entry](nodes/stm-tracking-entry.md) | One tracked scoring test map in STMIngestTableNew - which datasource/admin/test/form to re-pull from TestNav Previewer nightly, from which instance+tenant, and when it was last pulled. | 10 | 2 | data-service, stm |
| [admin-user](nodes/admin-user.md) | A COW Admin SPA operator account (DynamoDB USERS_TABLE, keyed by email) — local bcrypt login or JIT-provisioned from Entra SSO; carries the admin/readonly role. | 9 | 1 | admin-spa, auth |
| [proctor-group](nodes/proctor-group.md) | An ADAM proctor group (session group) and its proctor (hub_proctor_group + sat_proctor_group), linked to attempts. | 9 | 2 | warehouse, data-vault |
| [video](nodes/video.md) | A family-portal video page link attached to attempts (hub_video + sat_video, link_testattempt_video). | 9 | 2 | warehouse, data-vault, reports |
| [wida-student-uuid](nodes/wida-student-uuid.md) | Persistent Pearson UUID assigned to a WIDA student keyed by (sourced_id, normalized family name), so WIDA extracts emit a stable wida_pearson_uuid across runs. | 9 | 2 | wida, etl, dynamodb |
| [adam-manifest-row](nodes/adam-manifest-row.md) | One row of an ADAM ingest manifest CSV (hard_id, identifier, scope, report type, S3 location, school year) describing a staged extract/report file ADAM should ingest. | 8 | 2 | adam, manifest, extract-interface |
| [datasource-config](nodes/datasource-config.md) | One OneRoster ingest client in DataConfigTable - datasource_id, landing bucket, Secrets Manager credential name and per-entity last_update watermarks. | 8 | 1 | data-service, oneroster |
| [mv-all-stm](nodes/mv-all-stm.md) | Form/STM lookup MVs derived from delivered attempts — one row per form (mv_all_stm — test number, format, admin name, product code, subject) and per form item status (mv_all_stm_metadata), plus the mv_data_checks integrity report. | 8 | 5 | warehouse, extract-mv, stm |
| [mv-percentile-scores](nodes/mv-percentile-scores.md) | Percentile-rank MVs — per-attempt scale-score and claim percentile ranks within school, district and state (mv_extract_percentile_scores) and within the state (mv_scale_score_percentile_scores); quartile, Puerto Rico and per-level variants are legacy or dead. | 8 | 6 | warehouse, extract-mv, scoring |
| [mv-summative-extract](nodes/mv-summative-extract.md) | Summative extract MVs — per-attempt header with student, accountability, proctor and score-report PDF / video links (mv_summative_extract), plus class and filtered score outcomes. | 8 | 8 | warehouse, extract-mv, summative |
| [tools-analytics](nodes/tools-analytics.md) | Per-attempt tool usage/analytics payloads (hub_tools_analytics + sat_tools_analytics SUPER blob) linked to attempts; exposed by mv_tools_analytics. | 8 | 2 | warehouse, data-vault |
| [agent-conversation](nodes/agent-conversation.md) | One turn of an Admin SPA AI-agent conversation (DynamoDB cow-admin-agent-history-<env>, pk conversation_id + timestamp, 7-day TTL). | 7 | 1 | admin-spa, ai |
| [forensics-student-report](nodes/forensics-student-report.md) | Student-test-grain forensics tables (abr_student, rca_student, summative, and derived l1_student / tools_student / tools_item) keyed by hashed student_test_uuid; pseudonymized names. | 7 | 4 | forensics, entity, pii |
| [mv-catirf-extract](nodes/mv-catirf-extract.md) | CAT / Item Response File (IRF) extract MVs — per-attempt header (mv_catirf_extract), class, test-level outcomes and item-outcome scores; the adaptive (CAT theta) view is no longer created. | 7 | 9 | warehouse, extract-mv, irf |
| [mv-extract-external-scores](nodes/mv-extract-external-scores.md) | Hand-scoring (external score) MVs for SDF and CAT/IRF extracts — up to five rater scores plus the final score/condition per item or per item-outcome of an attempt. | 6 | 5 | warehouse, extract-mv, sdf, irf, hand-scoring |
| [mv-sdf-extract-items](nodes/mv-sdf-extract-items.md) | Wide pivoted SDF item and trait MVs — one row per test attempt with ItemUIN/Score/Status/Time-NN columns for up to 115 items (mv_sdf_extract_items, NYC variant) and 12×5 trait scores (mv_sdf_extract_traits, admin-specific _ad variant). | 6 | 7 | warehouse, extract-mv, sdf |
| [pdf-staging-item](nodes/pdf-staging-item.md) | An item in the RptIncomingStaging DynamoDB table tracking one file of a C2C PDF batch (manifest row, published PDF with its DocDepotId, companion datasource, and the manifest.csv tally/expected_count counter). | 6 | 2 | reporting, c2c, pdf, dynamodb |
| [forensics-evidence-statement](nodes/forensics-evidence-statement.md) | Evidence Statement content profile - per org x test x evidence statement percent-earned vs state (es_state, es_org parquet), unfolded from a variable-width CSV array. | 5 | 2 | forensics, entity |
| [validation-rules](nodes/validation-rules.md) | Validation as a Service rules document (rules.json) — UniqueKeyColumns plus an ordered list of rules (Name, Columns, RuleType Matches/Cross Column/Lookup, Expression, Severity, Required/EmptyOk/BlankOk, Verbose, Profile, Locations). | 5 | 1 | validation |
| [forensics-item-response](nodes/forensics-item-response.md) | Item-grain forensics tables from the IRF (Item Response File) - irf_item (student x item time/score), irf_pval (item difficulty + OP/FT status), irf_response (coded answer tokens). | 4 | 3 | forensics, entity |
| [lineage-spec](nodes/lineage-spec.md) | A Sleuth data-lineage spec (specs/lineage/<rs_\|adam_><job>.json) — the code-grounded, per-column provenance map of one ETL extract job (41 Redshift-sourced + 16 ADAM-direct). | 4 | 3 | sleuth, lineage, extract-interface |
| [mv-latest](nodes/mv-latest.md) | mv_sat_*_latest helper MVs (auto-refresh) giving the newest ingest_dts per satellite key, used to pick the current satellite row in the legacy dev-schema MVs and ddl snapshots; not created by the current migrations chain. | 4 | 6 | warehouse, extract-mv, legacy |
| [split-zip-definition](nodes/split-zip-definition.md) | The split-zip-upload definition — a JSON reporting document listing the fields (CSV column names, or fixed-width name/offset/length) whose values group rows into separate output files. | 4 | 2 | batch, split-zip |
| [data-oneroster-payload](nodes/data-oneroster-payload.md) | Shape of the ADAM OneRoster/sessions API pages that DataIngestApp lands as ingest/*.json - a {request_id, total_count, <entity>[]} envelope per entity type. | 2 | 6 | data-service, oneroster, adam, payload |
| [mv-summary-extract-performance](nodes/mv-summary-extract-performance.md) | Pre-aggregated summary (performance) report MVs — counts, average scale scores, proficiency-level distributions and reporting-category distributions per state/district/school × demographic aggregation group; defined only as ddl files and never created by a migration. | 2 | 4 | warehouse, extract-mv, summary |
| [mv-vsr-extract](nodes/mv-vsr-extract.md) | Dropped VSR extract MVs (mv_vsr_extract header, mv_vsr_extract_outcomes scores); kept only as _DEPRECATED ddl files and still named in refresh exclusion lists. | 2 | 2 | warehouse, extract-mv, dead |
| [act-summative-last-run](nodes/act-summative-last-run.md) | Per-admin_code watermark of the last successful ACT summative-from-ADAM extract run, so the next run only exports sessions updated since then. | 1 | 1 | act, etl, dynamodb |

## module (95)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [admin-spa-backend](nodes/admin-spa-backend.md) | Go (Gin) backend of the COW Admin SPA — ingest control plane (Step Functions/Batch/Glue/ECS/EventBridge Scheduler), Redshift Data API operations and CSV loaders, users/clients/jobs in DynamoDB, Extract Interface grants, Bedrock AI agent. | 66 | 16 | admin-spa, go, control-plane |
| [extract-interface-backend](nodes/extract-interface-backend.md) | Go (Chi) backend of the Extract Interface (new DepotUI): extract ordering, deliveries, schedules, status, documents/zip, ABBI search, ADAM manifest, Entra/Cognito auth and per-customer authz. | 58 | 11 | extract-interface, go, backend |
| [glue-redshift-extract-jobs](nodes/glue-redshift-extract-jobs.md) | 42 Glue PySpark jobs (jobs/glue_redshift-extract) that read the Redshift extract MVs, filter by admin/test/org, pivot outcomes, apply a layout file and write customer extract CSV/fixed-width files to the ETL bucket (SDF, IRF, CAT, Summative, Summary, RCR/RCS, DPLS, CSR, Growth, C2C, forensics, Evidence Statement, WIDA, Utah, PR, ND, NY, Detroit, Tools). | 54 | 25 | etl, extract, glue, redshift |
| [reporting-workflow-lambdas](nodes/reporting-workflow-lambdas.md) | Python 3.11 file-processing lambdas of aaas-reporting (Unzip, PdfExtract, UnzipPdf, extractLambda, zip, RCRrezip, filterExclude, deleteDoc, startSplitZip, startPdfExport) that reshape Doc Depot documents for an order and hand off to AWS Batch. | 53 | 15 | reporting, lambda, python, workflow |
| [glue-ingest-jobs](nodes/glue-ingest-jobs.md) | Glue PySpark jobs that load ADAM data entity-by-entity into the Redshift Data Vault - ADAMDBtoRedshift (JDBC), ADAM_S3_REDSHIFT (ECS-staged CSVs) and the scoped variants ByELKs, session_related and selective-ingest. | 49 | 20 | etl, ingest, glue, adam, data-vault |
| [mv-refresh-task](nodes/mv-refresh-task.md) | Java 17 ECS Fargate task (refresh-views) that runs CALL refresh_mvs('all') on a warehouse database and drop/recreates every materialized view Redshift reports as "up to date" (handling the mv_data_checks dependency). | 48 | 12 | etl, ecs, redshift, materialized-views |
| [reporting-api-lambdas](nodes/reporting-api-lambdas.md) | Java 11 lambdas behind /reporting/v1 for Doc Depot CRUD (DocumentDefinitions, Documents, Orders, Tags) plus DocumentsS3Lambda, the orders-bucket upload handler that routes files to unzip/catalog lambdas or acknowledges waiting workflows. | 44 | 19 | reporting, java, lambda |
| [redshift-flyway](nodes/redshift-flyway.md) | etl-flyway Docker image (Flyway 11.8.2 + Redshift JDBC) that applies redshift/migrations V1-V63 (Data Vault tables, refs, helper views, procedures, extract MVs) to every admin_code database listed for the env. | 43 | 31 | warehouse, migrations, deploy |
| [wh-stored-procedures](nodes/wh-stored-procedures.md) | Redshift stored procedures and helper views in each warehouse db - refresh_mvs, generate_unique_categories, percentile_categories, delete_testattempt/delete_updated_attempts, merge_link_tables/clear_stopped_tables, update_growth (PR growth), v_helper_* catalog views. | 43 | 5 | warehouse, procedures |
| [workflow-service](nodes/workflow-service.md) | Workflow as a Service (cow-workflow-as-a-service) — four Java 11 lambdas per tenant (WorkflowLambda, ConductorS3Lambda, AckWorkerLambda, NotifyLambda) that accept workflow JSON, store it in S3/DynamoDB and drive its worker events through an S3-event state machine. | 43 | 13 | workflow, lambda, java, cdk |
| [cdklibs](nodes/cdklibs.md) | cow-sacoreservlibs (com.pearson.cdklibs, Maven sacoreservlibs 1.18.x) - Java CDK library every cow-* service pipeline uses - ServicePipelineProvider builds a self-mutating CodePipeline that reads the tenant/service registry at synth and deploys one stack per tenant. | 42 | 7 | platform, cdk, deployment, library |
| [etl-infrastructure](nodes/etl-infrastructure.md) | ETL as a Service CDK app (ETLStack): per tenant it provisions the Redshift Serverless warehouse, 69 Glue jobs, the two ingest Step Functions, ECS extract/refresh tasks, ETL bucket, DynamoDB tables and IAM. | 41 | 19 | etl, cdk, infrastructure, glue |
| [reporting-workers](nodes/reporting-workers.md) | Java worker lambdas that execute Workflow-as-a-Service steps for reporting — WorkersLambda (HTTP POST /reporting/v1/notifications) and WorkersQueueLambda (SNS WorkflowTopic → SQS WorkflowQueue) — dispatching by notifyType to Glue jobs, lambdas, HTTP APIs or SendGrid email. | 36 | 13 | reporting, java, lambda, workflow |
| [etl-ingest-orchestrator](nodes/etl-ingest-orchestrator.md) | The two ADAM→warehouse ingest Step Functions in ETLStack (Glue-direct ETLStateMachine and the ECS-based ecs-etl-state-machine) plus their helper lambdas (clear schema, refresh schema, entity-list validator, Glue run monitor, Teams notifier, ADAM warmup) and the DoDEA nightly schedule. | 35 | 15 | etl, ingest, step-functions, lambda |
| [jenkins-dsl-pipelines](nodes/jenkins-dsl-pipelines.md) | Jenkins Job DSL (jaws-jenkins) generating per-service Development/Test/Release/WIDA-Development deploy pipelines - manifest job, per-environment deploy jobs and ServiceNow approval gates - for the CDK services (via S3 artifact drop + CodePipeline), DepotUI (cdk deploy + Amplify) and ETL-Flyway (terragrunt). | 34 | 21 | deploy, ci, jenkins |
| [tenant-infrastructure](nodes/tenant-infrastructure.md) | cow-sacoreservtenant - self-mutating CDK pipeline PWSTenant-<env> that, per tenant row, deploys TenantStack (REST API + WebSocket API + Lambda authorizer + ECS cluster + Route53 + alarms) and a WAF association, then postbuild.js writes the resource ids back to PWSTenantMapper and restarts every registered service pipeline. | 32 | 9 | platform, cdk, tenancy |
| [release-manifest](nodes/release-manifest.md) | cow-manifest repo — manifest.json/wida-manifest.json plus GitHub workflows (tst-manifest, release-manifest) that update one manifest key on the test/release branch and auto-dispatch the cow-deployment tst/uat deploy. | 31 | 5 | deploy, release, ci |
| [extract-layouts](nodes/extract-layouts.md) | jobs/resources - ~200 CSV/JSON layout and config files (CSV column layouts, fixed-width txt layouts, scoring-category maps, Evidence Statement configs, test/RC name lookups) zipped as pws-resources.zip and deployed to s3://<etlbucket>/instructions/ for the extract Glue jobs. | 29 | 3 | etl, extract, layout, config |
| [pws-job-lib](nodes/pws-job-lib.md) | jobs/pws shared Python library shipped to every Glue job as pws-common.zip — Glue arg parsing, warehouse/ADAM connection resolution, Redshift read/write, S3 output writers, layout application, ADAM extract SQL helpers, ADAM ingest SQL, TestNav client, WIDA UUIDs, reporting publish and validation. | 29 | 12 | etl, glue, python, library |
| [reporting-infrastructure](nodes/reporting-infrastructure.md) | Reporting as a Service CDK app (AaasReportingApp → AaasReportingServicePipeline → one AaasReportingStack per tenant) provisioning /reporting/v1 routes on the tenant API Gateway, 8 Java + 21 Python lambdas, buckets, DynamoDB tables + streams, WorkflowTopic/Queue, PublishStateMachine, cross-account IAM and alarms. | 28 | 25 | reporting, cdk, infrastructure |
| [split-zip-job](nodes/split-zip-job.md) | SplitZipUpload — Java 17 container run on AWS Batch that downloads a split definition and a source document from the reporting API, splits the CSV/fixed-width file into one file per key combination, zips them with metadata.json, uploads the zip as a new reporting document and POSTs success/failure to a callback URL. | 28 | 7 | batch, java, split-zip |
| [admin-data-loads](nodes/admin-data-loads.md) | Admin SPA reference-data loaders run as tracked async jobs against an admin code's Redshift database — ES objective keys (ref_scoring_category_unique), PR prior-year growth (Glue PR_PriorYear_CSVToRedshift + embedded flyway growth SQL), scoring categories (ref_scoring_category rebuild + generate_unique_categories + refresh_mvs). | 27 | 12 | admin-spa, redshift, loaders |
| [etl-file-ingest-jobs](nodes/etl-file-ingest-jobs.md) | Glue jobs loading non-ADAM files into the Redshift Data Vault - STM JSON (stm_to_redshift), RSRD/Spotlight/Battery PDF manifests (glue_pdf_to_redshift), ZH score CSVs, ND battery attemptedness CSVs and PR prior-year/growth CSV reference tables. | 27 | 13 | etl, ingest, glue, stm, pdf |
| [lambda-authorizer](nodes/lambda-authorizer.md) | Java 11 Lambda REQUEST authorizer (PWSAuthorizer-<env>-<tenant>) that verifies a Cognito ID token's RS256 signature and allows the call only if its custom restApiId claim equals the API being called; passes sub/tenantId/subtenantId/authorizedServices to backends. | 27 | 3 | platform, security, auth |
| [glue-adam-extract-jobs](nodes/glue-adam-extract-jobs.md) | ADAM-direct Glue extract jobs that query ADAM Postgres (not Redshift) and write customer extract CSV/TXT files (SDF, Summative, C2C, IRF, SRF, ESEA, WIDA MOAR/License Usage, Survey, Tools) to S3 using layouts from the ETL bucket. | 26 | 7 | etl, extract, glue, adam |
| [deploy-gha-workflows](nodes/deploy-gha-workflows.md) | cow-deployment GitHub Actions — tst/uat/prd deploy workflows (reusable-env-deploy: manifest → general.tfvars → terragrunt run --all apply), ServiceNow RFC approval/scheduling, /plan PR comments, Slack/SNS notifications. | 25 | 9 | deploy, ci, github-actions |
| [data-infrastructure](nodes/data-infrastructure.md) | Data as a Service CDK app (AaasDataStack) deploying, per tenant, the data ingest bucket, two DynamoDB tables, four S3-trigger lambdas, four Fargate Batch jobs and two nightly EventBridge rules. | 24 | 11 | data-service, infrastructure, cdk |
| [reporting-delivery-lambdas](nodes/reporting-delivery-lambdas.md) | Python lambdas that deliver an order's documents out of COW - s3CopyLambda (stage + cross-account S3 copy to Family Portal, ACT, Sandbox/PV, ADAM), createAdamManifestLambda (ADAM manifest CSV) and sendToS3Lambda (SFTP to Spotlight/CDQ/Cognia). | 24 | 13 | reporting, lambda, python, delivery, cross-account |
| [depotui-lambdas](nodes/depotui-lambdas.md) | Legacy DepotUI backend — CDK (Python) stack DepotUIStack with 14 Python 3.11 VPC lambdas (one per API route + OrdersScheduled) that proxy to Reporting/Workflow-as-a-Service and keep order, user, extract-config and ABBI-subscription records in DynamoDB. | 23 | 27 | depotui, legacy, lambda, cdk, python |
| [oneroster-ingest-app](nodes/oneroster-ingest-app.md) | Nightly Java Batch app (DataIngestApp / radarIngestBatchJob) that pages the ADAM OneRoster+sessions API per DataConfigTable client, drops JSON batches in the data ingest bucket, and DataTriggerLambda starts Glue ADAMToRedshift per file - a Glue job that ETLStack no longer defines. | 23 | 15 | data-service, ingest, oneroster, adam |
| [xi-order-service](nodes/xi-order-service.md) | Extract Interface order pipeline: OrderService plus ExtractAssembler/ExtractDeliveryAssembler, which turn an extract config + user selections (or a completed order + delivery action) into a PWS reporting-order payload whose wfTag selects the workflow, then book the extract order in DynamoDB. | 23 | 9 | extract-interface, orders, cross-repo |
| [admin-ingest-control](nodes/admin-ingest-control.md) | Admin SPA ingest control plane — starts ADAM ingests (ETLStateMachine Glue path / ECSETLStateMachine ECS path), STM ingests (AWS Batch stmIngestBatchJob → STMToRedshift Glue), ADAM validation Glue job, ECS MV-refresh task, EventBridge Scheduler schedules, and polls them into admin-job records. | 22 | 11 | admin-spa, ingest, orchestration |
| [cow-terraform-modules](nodes/cow-terraform-modules.md) | Terraform modules in cow-deployment/modules (ALB, ECS cluster/service/task, flyway, DynamoDB, Cognito, KMS, SSM params, WAF/Shield, backup, ECR, IAM/OIDC, Glue/Batch→SNS→Slack alerting, metrics poller) composed per env by terragrunt live-infrastructure. | 22 | 16 | deploy, terraform |
| [extract-interface-frontend](nodes/extract-interface-frontend.md) | Vue 3 + TypeScript + PrimeVue SPA of the Extract Interface (new DepotUI), served by nginx; talks only to the extract-interface Go API with an Entra ID / Cognito bearer token. | 22 | 19 | extract-interface, frontend, vue |
| [xi-authz](nodes/xi-authz.md) | Extract Interface authorization engine: four SSO roles with action boundaries, per-customer/tier grants from the user's permissions block, customer groups that map extract names to customer codes, and RequireAction route guards with body/order/schedule resource resolvers. | 22 | 5 | extract-interface, authz, permissions |
| [data-pdf-ingest](nodes/data-pdf-ingest.md) | Student PDF ingest - DataPdfZipTriggerLambda submits pdfIngestBatchJob (pdfIngest.py) for an RSRD zip or Spotlight manifest; the job writes each PDF into Doc Depot (reporting bucket + ReportingTable), optionally wraps them in a reporting order, and drops a rebuilt manifest whose arrival makes PDFTriggerLambda start Glue pdfToRedshift. | 21 | 12 | data-service, pdf, doc-depot |
| [data-stm-ingest](nodes/data-stm-ingest.md) | STM ingest - stmApp (STMIngestApp, stmIngestBatchJob) pulls scoring test maps from TestNav Previewer into testmaps/*.json, and DataStmTriggerLambda (DataStmIngestLambda S3Trigger) starts Glue STMToRedshift per file. | 20 | 9 | data-service, ingest, stm |
| [deploy-job-alerting](nodes/deploy-job-alerting.md) | Terraform modules glue-event-notifier, batch-event-notifier, sns and sns-to-slack — EventBridge rules for Glue failures and Batch success/failure → <env>-success/failure-alerts SNS → Python Lambda <env>-sns-to-slack → Slack workflow webhook. | 20 | 6 | deploy, terraform, alerting |
| [reporting-notification-lambdas](nodes/reporting-notification-lambdas.md) | triggerEmailNotifications (hourly/daily ABBI digest e-mail to subscribers, sent by creating an e-mail reporting order) and createFEOrderLambda (registers a workflow-generated order in the DepotUI/Extract Interface order list). | 19 | 10 | reporting, lambda, python, notifications, abbi |
| [admin-auth](nodes/admin-auth.md) | Admin SPA authentication/authorization — HS256 app JWT (2 h) issued by local bcrypt login, Entra OIDC+PKCE SSO, or client-credential login; two roles admin/readonly enforced by RequireAdmin. | 18 | 6 | admin-spa, auth, security |
| [deploy-flyway-task](nodes/deploy-flyway-task.md) | Terraform module modules/flyway registering the <env>-etl-flyway Fargate task and running it once per new flyway_image_urn during apply to migrate Redshift. | 18 | 5 | deploy, terraform, ecs, redshift |
| [etl-ecs-worker](nodes/etl-ecs-worker.md) | Java 21 ECS/Fargate ADAM extractor (etl-in-a-container) — a DRIVER task fans out one WORKER task per ingest entity (10 per batched entity by UUID range) that streams an ADAM SQL result as CSV to the data ingest bucket for the ECSS3AdamDBIngestJob Glue load. | 18 | 3 | etl, ecs, java, ingest |
| [batch-service](nodes/batch-service.md) | RADAR Batch as a Service (cow-batch-split-zip-and-upload) — CDK app + Java 11 API Gateway lambda (APIGatewayHandler) that thinly wraps AWS Batch, and provisions the per-tenant split-zip compute env, queue, job definition, EFS and IAM. | 17 | 8 | batch, lambda, java, cdk |
| [cow-live-infrastructure](nodes/cow-live-infrastructure.md) | Terragrunt live tree in cow-deployment that instantiates the terraform modules per AWS account (nonprd/prd) and environment (cow-dev, cow-smk, cow-tst, cow-uat, cow-prd, cow-wida-uat, shared) - ECS cluster/services/tasks, ALB, SSM params, DynamoDB, Cognito, KMS and account-wide shared stacks. | 17 | 17 | deploy, infrastructure, terraform |
| [depotui-frontend](nodes/depotui-frontend.md) | Legacy DepotUI Vue 3 SPA (vueFrontend, Vite + Vuex + Bootstrap) hosted on AWS Amplify — order Core Warehouse extracts, track/approve/stop their workflows, download zipped results, manage extract configs, user roles and ABBI extract search/subscriptions. | 17 | 6 | depotui, legacy, vue, spa |
| [reporting-c2c-pdf-lambdas](nodes/reporting-c2c-pdf-lambdas.md) | C2C PDF intake - c2cPublishLambda (S3-event; publishes each PDF to Doc Depot and tallies it in RptIncomingStaging) and createC2CIngestLambda (DynamoDB-stream; when tally == expected_count writes a pdf_manifest JSON for warehouse ingest). | 17 | 8 | reporting, lambda, python, c2c, pdf |
| [sleuth-lineage](nodes/sleuth-lineage.md) | Sleuth Data Lineage tool (stdlib Python bundled in the Extract Interface backend image) that statically traces every extract output column back through extract job → Redshift MV → Data Vault sat/hub/link → ADAM ingest → ADAM SQL and renders an HTML report. | 16 | 11 | sleuth, lineage, extract-interface, python |
| [deploy-ecs-service](nodes/deploy-ecs-service.md) | Terraform module modules/ecs-service deploying one Fargate ECS service (admin-spa-backend/-frontend, extract-int-backend/-frontend) with task def, IAM, SG, ALB target group + host rule and Route53 alias. | 15 | 18 | deploy, terraform, ecs |
| [forensics-ingest](nodes/forensics-ingest.md) | Forensics ingest CLI run on a trusted laptop - unpacks the nested-zip CSV drop, guards each report's header, and converts to pseudonymized Parquet with DuckDB (salted-hash IDs, fake names). | 15 | 7 | forensics, ingest, pii |
| [platform-build-management](nodes/platform-build-management.md) | PWSBuildManagement-<env> Node 14 lambda invoked by CodeBuild synth steps (aws lambda invoke) to scan PWSTenantMapper/PWSServiceMapper into tenant_config.json/services.json and to upsert a service's pipeline_name. | 15 | 3 | platform, lambda, deployment |
| [shared-infrastructure](nodes/shared-infrastructure.md) | cow-sacoreservshared - self-mutating CDK pipeline PWSShared-<env> deploying the per-environment platform control plane - PWSTenantMapper/PWSServiceMapper tables, Cognito pool PWSAuth, the tenant-management REST API and its three Node lambdas, a VPN security group, WAF and alarms. | 15 | 8 | platform, cdk |
| [xi-scheduler](nodes/xi-scheduler.md) | Extract Interface scheduled extracts: schedule CRUD that clones a frozen backing config, an in-process 1-minute launcher that atomically claims due schedules and starts runs via StartExtract, and a 5-minute watcher that polls run status and emails outcomes over SendGrid SMTP. | 15 | 11 | extract-interface, schedule, background-job |
| [admin-spa-frontend](nodes/admin-spa-frontend.md) | Vue 3 + PrimeVue 4 + Pinia SPA ("COW Admin") that operators use to run ADAM/STM ingests, watch Step Functions jobs, schedule EventBridge ingests, run Redshift maintenance loads, manage clients and Extract Interface permissions, and chat with the data agent. | 14 | 4 | admin-spa, frontend, vue |
| [forensics-backend](nodes/forensics-backend.md) | FastAPI + DuckDB backend that composes forensics signal reports into school x test units at startup and serves pre-serialized, org-scoped dashboard JSON. | 14 | 9 | forensics, backend |
| [admin-redshift-ops](nodes/admin-redshift-ops.md) | Admin SPA direct Redshift Data API operations — ad-hoc SQL console (no guardrails), org tree lookup, test-attempt lookup by external_login_key, legacy synchronous refresh_mvs, CSV test-attempt delete. | 13 | 5 | admin-spa, redshift, security |
| [data-pdf-export](nodes/data-pdf-export.md) | exportPDFBatchJob (exportPDF.py) - for one reporting order, resolves its Doc Depot documents (including docIds listed in the order's zip), copies PDFs/CSVs from the reporting orders bucket into the staging bucket, then assumes the Family Portal role and copies them to the Family Portal bucket. | 13 | 8 | data-service, pdf, export, family-portal |
| [xi-adam-manifest](nodes/xi-adam-manifest.md) | Extract Interface ADAM manifest generator: for a completed order, lists the members of its packaged zips via HTTP range reads, ports the reporting createAdamManifest/s3Copy file-selection logic to build the ADAM ingest CSV, and persists it as a PWS CLIENT_MANIFEST document on the order. | 13 | 8 | extract-interface, adam, cross-repo |
| [admin-ai-agent](nodes/admin-ai-agent.md) | Admin SPA "data investigation" AI agent — Bedrock Converse tool loop (list_tables, describe_table, execute_sql) over Redshift with read-only SQL guardrails, 500-row limit and regex PII redaction; history in DynamoDB. | 12 | 6 | admin-spa, ai, redshift, security |
| [xi-status-service](nodes/xi-status-service.md) | Extract Interface status polling: asks the PWS workflow API for an order's workflow state, loads the workflow JSON from the workflow S3 bucket, looks up the latest Glue job run, builds S3 console links, and can stop Glue job runs. | 12 | 7 | extract-interface, status, cross-repo |
| [etl-manual-sql](nodes/etl-manual-sql.md) | cow-etl-as-a-service/sql runbook scripts - preingest_delete_<entity>.sql to empty Data Vault tables before a re-ingest, an ELK-scoped delete, refresh scripts for *_latest and extract materialized views, and an Evidence Statement objective-key loader; run by hand, referenced by no code. | 11 | 8 | sql, runbook, redshift, etl |
| [platform-tenant-management](nodes/platform-tenant-management.md) | Node 14 lambdas PWSTenantManagement-<env> (POST /tenants creates a tenant row and starts the tenant pipeline; POST /services registers a service) and PWSIDTokenGenerator-<env> (POST /idToken exchanges client_id/secret for a Cognito ID token). | 11 | 5 | platform, lambda, tenancy |
| [reporting-publish-lambdas](nodes/reporting-publish-lambdas.md) | Publish-to-warehouse lambdas - callPublishStepFunction starts a Step Function that polls a Glue extract run (checkGlueLambda) and then publishLambda concatenates the Glue output files into one Doc Depot document; plus wakeUpRedshiftLambda. | 11 | 10 | reporting, lambda, python, glue, publish |
| [deploy-ecr-repos](nodes/deploy-ecr-repos.md) | Terraform module modules/ecr creating the COW ECR repositories in the prd account with lifecycle rules and cross-account (nonprd/prd) pull/push policy. | 10 | 1 | deploy, terraform, ecr |
| [deploy-kms-key](nodes/deploy-kms-key.md) | Terraform module modules/kms creating the per-environment customer-managed KMS key (alias/<env>) with deletion denied and bunker-account access. | 10 | 1 | deploy, terraform, security |
| [xi-auth](nodes/xi-auth.md) | Extract Interface request authentication: multi-issuer JWT validation (Entra ID, Cognito user pool, offline e2e OIDC), role parsing, login provisioning of the user PROFILE row, and UserContext/policy construction. | 10 | 7 | extract-interface, auth |
| [deploy-ecs-cluster](nodes/deploy-ecs-cluster.md) | Terraform modules ecs-cluster (one Fargate/Fargate Spot ECS cluster per env with KMS-encrypted exec logging) and network (data-only VPC/subnet/NAT lookups by Name tag). | 9 | 2 | deploy, terraform, ecs, network |
| [etl-evidence-statement-extracts](nodes/etl-evidence-statement-extracts.md) | Evidence Statement (content-profile) extracts - per state/district/school x test x evidence statement average percent of points earned, driven by per-admin "Evidence Statement CoW config" CSVs and writing-field CSVs; ELA/Math (EvidenceStatement.py) and Science (EvidenceStatementScience.py). | 9 | 12 | etl, extract, evidence-statement |
| [forensics-auth](nodes/forensics-auth.md) | Forensics dashboard auth - Launchpad OIDC login with signed session cookie, tenant->customer gate, and Launchpad OneRoster role/org lookup mapping users to all / district / school data scope (or optional basic auth when OIDC is off). | 9 | 2 | forensics, auth, permissions |
| [deploy-ecs-task](nodes/deploy-ecs-task.md) | Terraform module modules/ecs-task registering a run-on-demand Fargate task definition with Redshift access; used for <env>-etl-refresh-views (MV refresh). | 8 | 5 | deploy, terraform, ecs |
| [deploy-iam-access](nodes/deploy-iam-access.md) | Terraform modules github-oidc (GitHub Actions OIDC deploy/build roles), github-resources (github_sre_notification SNS topic) and dev-iam-role (AzureAD SAML developer role cow-prd-developer). | 8 | 1 | deploy, terraform, iam |
| [etl-wida-extracts](nodes/etl-wida-extracts.md) | WIDA (ELP battery - Listening/Reading/Speaking/Writing units) Redshift extracts - MOAR battery file, MODEL, STATEDISTRICT and ROSTER (state/district/school) variants, and the Student Proficiency Level Comparison (PLCR) across two administrations - stamping a persistent Pearson student UUID from DynamoDB. | 8 | 15 | etl, extract, wida |
| [validation-service](nodes/validation-service.md) | Validation as a Service (vaas) — 2023 sandbox prototype: a Python API lambda that uploads a file + JSON rules to S3 and starts an out-of-band ECS Fargate task (vaas.py) which validates CSV/fixed-width/JSON records and writes summary/detail JSON + a zip; not deployed by any COW pipeline and not called by any COW code. | 8 | 3 | validation, python, legacy-prototype |
| [deploy-waf-shield](nodes/deploy-waf-shield.md) | Terraform modules waf/regional, waf/global, shield, cloudfront and route53 — WAFv2 ACLs (PCM sanctions + AWS managed rules) and Shield Advanced protections/SRT role/DDoS alarm per account. | 7 | 1 | deploy, terraform, security |
| [etl-legacy-adam-json-ingest](nodes/etl-legacy-adam-json-ingest.md) | Original S3-JSON ADAM/OneRoster ingest (glue_adam_to_redshift.py and its one-off/archived variants) that loaded OneRoster JSON dropped in S3 into the Data Vault; unregistered from ETLStack on 2026-08-31, superseded by ADAMDBtoRedshift. | 7 | 6 | etl, ingest, legacy, dead |
| [xi-document-service](nodes/xi-document-service.md) | Extract Interface documents: lists an order's PWS docIds, returns presigned download URLs for documents verified to belong to the order, builds a ZIP of several documents server-side and re-uploads it as a new PWS document, and creates ABBI download documents. | 7 | 5 | extract-interface, documents, cross-repo |
| [deploy-alb](nodes/deploy-alb.md) | Terraform module modules/alb creating each environment's internet-facing ALB (HTTPS 443 listener, SG limited to NAT IPs) with Shield protection and the regional WAF attached. | 6 | 2 | deploy, terraform, network |
| [etl-forensics-extracts](nodes/etl-forensics-extracts.md) | Test-security (forensics) signal extracts built from Redshift - Aberrant Response Analysis (Zh person-fit flags), Response Change (points gained/lost on answer changes) and Longitudinal Performance Monitoring (current vs prior-year school/district residuals) - student and district/school files with state z-scores. | 6 | 14 | etl, extract, forensics, test-security |
| [etl-validation-framework](nodes/etl-validation-framework.md) | jobs/pws/validation + jobs/tools - Glue Data Quality (DQDL) wrappers and rule sets for extract files (SDF, IRF, summative, RCR, growth), ADAM source tables and mv_catirf_extract_responses; imported by three extract jobs but never invoked, tool scripts not deployed. | 6 | 5 | etl, validation, glue, data-quality |
| [reporting-publish-state-machine](nodes/reporting-publish-state-machine.md) | Step Functions PublishStateMachine defined in AaasReportingStack — Wait 60s → CheckGlueHandler, looping until $.GlueComplete is set → PublishHandler — publishing extract results once the Glue run finishes. | 6 | 2 | reporting, step-functions, publish |
| [xi-ci-pipeline](nodes/xi-ci-pipeline.md) | GitHub Actions for cow-extract-interface — PR gate (Go vet/lint/test/build + vitest/type-check/build) and post-merge build that pushes backend/frontend images to ECR, Snyk-scans on main, and dispatches the cow-manifest tst/release workflow. | 6 | 5 | ci, github-actions, extract-interface |
| [deploy-cloudwatch-poller](nodes/deploy-cloudwatch-poller.md) | Terraform module modules/custom-cloudwatch-poller — a python3.7 Lambda run every 5 minutes that copies selected CloudWatch metrics to the I&O StatsAgg/Graphite endpoint. | 5 | 2 | deploy, terraform, monitoring |
| [deploy-dynamodb-table](nodes/deploy-dynamodb-table.md) | Terraform module modules/dynamodb creating one KMS-encrypted DynamoDB table (<env>-<name>) per terragrunt instance; provisions the extract-int-* and admin-spa-* tables. | 5 | 4 | deploy, terraform, dynamodb |
| [forensics-infra](nodes/forensics-infra.md) | Terraform + laptop deploy scripts for the forensics dashboard in sacoreserv-nonprd - ECS Fargate (ARM64) backend/frontend behind a CIDR-restricted ALB, ECR, CloudWatch logs, SSM secrets, Route53 forensics-dashboard.cow.pearsondev.tech. | 5 | 3 | forensics, infra, terraform |
| [wh-dev-schema-history](nodes/wh-dev-schema-history.md) | Frozen legacy Flyway history (redshift/sql, V1–V347) for the old single `dev` database with one schema per datasource; superseded by the rebaselined redshift/migrations chain (V1–V62) that runs per admin_code database. | 5 | 3 | warehouse, flyway, legacy |
| [xi-e2e-suite](nodes/xi-e2e-suite.md) | Playwright end-to-end suite + fully offline stack for the Extract Interface (Moto for AWS, mock PWS/workflow API, mock OIDC issuer, real Go backend and Vite frontend) pinning auth/RBAC, extract ordering, delivery, documents, ADAM manifest, ABBI, schedules and history journeys. | 5 | 10 | e2e, testing, extract-interface |
| [deploy-backup-plan](nodes/deploy-backup-plan.md) | Terraform module modules/backup — AWS Backup vault/plan (daily, weekly, monthly, yearly) selecting resources tagged t_awsbackup_default=true, with copies to the PCM bunker account vault. | 4 | 1 | deploy, terraform, backup |
| [deploy-cognito-user-pool](nodes/deploy-cognito-user-pool.md) | Terraform module modules/cognito creating the per-environment Cognito user pool (admin-created users only) and app clients used by the Extract Interface, with the regional WAF attached. | 4 | 3 | deploy, terraform, auth |
| [forensics-frontend](nodes/forensics-frontend.md) | Vite + React 19 + TypeScript SPA (TanStack Query, React Router, Tailwind 4, Pearson Nebula tokens) rendering the test-security screens, served by nginx. | 4 | 3 | forensics, frontend |
| [reporting-wip-copy-lambdas](nodes/reporting-wip-copy-lambdas.md) | Python lambdas present in py-lambda/ but not defined in AaasReportingStack, so never deployed - s3BatchCopyLambda and s3InventoryLambda (S3 Batch Operations copy to Family Portal/ADAM) and RezipPdfLambda (stub). | 3 | 2 | reporting, lambda, python, wip |
| [deploy-s3-bucket-module](nodes/deploy-s3-bucket-module.md) | Terraform module modules/s3-bucket (generic bucket with versioning, lifecycle, SSE, logging, CORS, public-access block) — not instantiated anywhere in live-infrastructure. | 2 | 1 | deploy, terraform, dead |
| [wh-ops-scripts](nodes/wh-ops-scripts.md) | Hand-run warehouse maintenance scripts in the flyway repo — clone a dev schema, export/import one student's Data Vault rows, load ref_scoring_category from CSV, migrate every dev schema, generate MV refresh filters, wipe a datasource, Windows docker helpers. | 2 | 5 | ops, warehouse, scripts |
| [etl-one-off-jobs](nodes/etl-one-off-jobs.md) | jobs/glue_one-off - ad-hoc and experimental Glue scripts (CAT/IRF/SDF JSON-output extracts, Ed-Fi ETL prototypes, MD emergency summative, PAN CDF, summary/summative ETL prototypes); only Summary_mv_perf.py is registered in ETLStack. | 1 | 2 | etl, legacy, dead, one-off, archive |
| [wh-redshift-cost-report](nodes/wh-redshift-cost-report.md) | Local Python CLI that breaks Redshift Serverless spend into compute / storage / snapshots per workgroup and day, and profiles the last ~7 days of queries per database, rendering an offline HTML dashboard plus report.json. | 1 | 2 | ops, cost, warehouse |
| [wh-rds-oneroster-flyway](nodes/wh-rds-oneroster-flyway.md) | Deprecated confd + Flyway container that migrated a PostgreSQL (RDS) `oneroster` schema — OneRoster 1.2 tables plus an ADAM accommodations extension. | 0 | 1 | legacy, oneroster, postgres |
| [xi-standalone-test-stack](nodes/xi-standalone-test-stack.md) | App-owned CloudFormation stack (infra/cloudformation/standalone-test-stack.yaml) that stood up a corporate-CIDR-restricted Extract Interface test env — ECR, ECS Fargate backend+frontend, dedicated ALB with /api path routing, Route53, logs, IAM, and the scheduled-extract table. | 0 | 9 | infra, cloudformation, extract-interface |

## interface (17)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [reporting-api](nodes/reporting-api.md) | Reporting as a Service REST API mounted at /reporting/v1 on each tenant's API Gateway — document definitions, documents (+ elk lookup), orders and order status, tags, workflow worker notifications. | 60 | 10 | reporting, api |
| [admin-spa-api](nodes/admin-spa-api.md) | COW Admin SPA REST API (Gin) — public auth routes, read routes for any authenticated user, and /api write routes gated by RequireAdmin (JWT role claim "admin"). | 40 | 9 | admin-spa, rest |
| [extract-interface-api](nodes/extract-interface-api.md) | Extract Interface Go REST API (Chi, /api/*) consumed by its Vue SPA; every non-health route needs a bearer JWT and a per-route authz action. | 40 | 13 | extract-interface, api, authz |
| [extract-interface-ui](nodes/extract-interface-ui.md) | Extract Interface (new DepotUI) SPA screens — start/schedule extracts, order history and details with delivery actions, status, data lineage, ABBI search and notifications, admin extract-config editor. | 38 | 10 | extract-interface, ui, frontend |
| [admin-spa-ui](nodes/admin-spa-ui.md) | COW Admin SPA screens (Vue router) — Dashboard, Clients, Job History, ADAM Ingest, Ingest Schedules, Database tools (refresh MVs, create DB, delete test attempts, scoring category / ES objective / PR growth loads, org tree), Extract UI Permissions, Run Schema Validation, Data Debug, plus the Data Agent chat drawer. | 36 | 2 | admin-spa, ui |
| [workflow-api](nodes/workflow-api.md) | Workflow as a Service REST API under /workflow/v1 on each tenant API Gateway — submit a workflow, poll status by workflowUUID or uniqueExternalKey, acknowledge a waiting event, plus the ackWorkers and notifications worker endpoints. | 34 | 3 | workflow, api |
| [platform-tenant-api](nodes/platform-tenant-api.md) | Per-tenant REST API PWS-<env>-<tenant> at https://<tenant>-<env>.<domain_name> (plus WebSocket API PWSWebsocket-<env>-<tenant>) that every cow-* service attaches its routes to, guarded by the tenant Lambda authorizer, WAF and (nonprd) an IP allowlist. | 26 | 1 | platform, api, tenancy |
| [depotui-ui](nodes/depotui-ui.md) | Legacy DepotUI screens (Vue router routes) with their client-side role guards — Extracts, Archive, History, All History, Extract Config, Extract Details, User Roles, ABBI Extracts/Delete/Subscriptions. | 18 | 6 | depotui, legacy, ui |
| [depotui-api](nodes/depotui-api.md) | Legacy DepotUI REST API Gateway "COW-api-{env}" (live-api.cow.pearsondev.tech / prd-api.cow.pearsonprd.tech) — 13 POST routes, one Python lambda each, no gateway authorizer; auth is a Cognito access token checked inside each lambda. | 17 | 10 | depotui, legacy, rest, api-gateway |
| [forensics-api](nodes/forensics-api.md) | Forensics dashboard JSON API (/api/health, /api/me, /api/meta, /api/summary, /api/units, unit detail, student item/tool drills) plus the /auth/* Launchpad OIDC routes. | 11 | 5 | forensics, api |
| [platform-tenant-management-api](nodes/platform-tenant-management-api.md) | Shared REST API PWSTenantManagement-<env> at https://<env>.<domain_name> - POST /tenants, POST /services, POST /idToken; no authorizer, WAF only. | 11 | 2 | platform, api |
| [batch-api](nodes/batch-api.md) | Batch as a Service REST API under /batch/v1 on each tenant API Gateway — list job definitions and queues, list/submit jobs, describe/cancel a job (thin AWS Batch wrapper). | 7 | 1 | batch, api |
| [etl-ingest-state-machine](nodes/etl-ingest-state-machine.md) | StartExecution input contract of the two ADAM ingest Step Functions (ETLStateMachine-{env}-{tenant} and {env}-{tenant}-ecs-etl-state-machine) used by the Admin SPA and the nightly EventBridge rule. | 7 | 0 | etl, ingest, step-functions |
| [forensics-ui](nodes/forensics-ui.md) | Forensics dashboard screens - Administration summary, Triage queue, Unit drill-down (with per-student signal detail and PDF export), printable Security report, plus later-phase placeholders. | 6 | 2 | forensics, ui |
| [deploy-env-deploy-dispatch](nodes/deploy-env-deploy-dispatch.md) | workflow_dispatch entry points of cow-deployment (tst-deploy.yml, uat-deploy.yml, prd-deploy.yml, servicenow-approval.yml) taking MANIFEST_RUN_NUM, called by cow-manifest and by ServiceNow scheduling. | 5 | 0 | deploy, ci |
| [val-api](nodes/val-api.md) | Validation as a Service REST API — POST /validation/v1/processFile (multipart input_file, rules_file, job_name, input_format, csv_to_json, lookup_file) returning a presigned URL for the result zip; unauthenticated prototype. | 4 | 1 | validation, api |
| [data-records-api](nodes/data-records-api.md) | Never-implemented "Data as a Service" records API (POST /datainges/v1/records, PUT /dataingest/v1/records/{objectId}) described only by a static OpenAPI file; plus the leftover lambda/DataIngestLambda folder. | 2 | 0 | data-service, dead |

## integration (23)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [adam](nodes/adam.md) | ADAM, Pearson's assessment delivery platform and COW's primary source system; COW reads it via read-only Postgres replicas (JDBC from Glue and the ECS worker), legacy OneRoster API JSON pushed to S3, and indirectly via TestNav exports keyed by ADAM external_login_key. | 38 | 1 | adam, ingest, source-system |
| [amazon-cognito](nodes/amazon-cognito.md) | Amazon Cognito user pool PWSAuth-<env> (one per environment, shared by all tenants) whose users are API clients; its ID tokens carry custom tenantId/restApiId/subtenantID/authorizedServices claims checked by the tenant Lambda authorizer. | 38 | 0 | platform, auth, aws |
| [aws-glue](nodes/aws-glue.md) | AWS Glue (4.0, PySpark): runs the 69 per-tenant ingest/extract jobs ETLStack defines, through two JDBC connections (Redshift warehouse, ADAM Postgres); started on demand by reporting workers, Admin SPA, data-ingest trigger lambdas and the ingest state machines. | 34 | 0 | aws, etl, glue |
| [family-portal](nodes/family-portal.md) | Family Portal - Pearson's parent/family-facing report site; COW delivers student report PDFs and upload files into its S3 bucket by cross-account copy (STS AssumeRole) from the reporting staging bucket. | 23 | 0 | external, delivery, cross-account |
| [microsoft-entra-id](nodes/microsoft-entra-id.md) | Microsoft Entra ID (Azure AD) — Pearson's OIDC identity provider for internal COW tools; the Admin SPA backend is a confidential OIDC client (auth code + PKCE), the Extract Interface uses MSAL in its SPA. | 20 | 0 | auth, external |
| [aws-batch](nodes/aws-batch.md) | AWS Batch on Fargate — runs the per-tenant split-zip job (pws-batch-split-zip-job-<env>-<tenant> on queue pws-batch-splitzip-queue-<env>-<tenant>) and the data service's PDF ingest/export and STM ingest jobs. | 19 | 0 | aws, batch |
| [data-testnav-previewer](nodes/data-testnav-previewer.md) | TestNav Previewer API (tn8/parcc/abbi/aspire instances) - source of Pearson scoring test maps (STMs) per tenant/test/form, authenticated with a ticket from api/ticket. | 16 | 0 | data-service, stm, external |
| [slack](nodes/slack.md) | Slack incoming/workflow webhooks receiving COW Glue/Batch job alerts (cow-bell app via the sns-to-slack Lambda) and CI deploy/manifest success-failure notices. | 15 | 0 | deploy, alerting |
| [act](nodes/act.md) | ACT Inc. - external testing partner; COW copies order files staged under Pearson/<orderUUID>/ into ACT's S3 bucket by cross-account AssumeRole with an ExternalId (plus an ACT sandbox and PV variant). | 13 | 0 | external, delivery, cross-account |
| [deploy-servicenow](nodes/deploy-servicenow.md) | Pearson ServiceNow change management (RFCs) gating COW production deploys via pearson-dnt-sre/servicenow-action-api. | 12 | 0 | deploy, change-management |
| [deploy-statsagg](nodes/deploy-statsagg.md) | Pearson StatsAgg / I&O Graphite monitoring: receives CloudWatch metrics from the poller Lambda and has its alerting suspended during prd deploys. | 11 | 0 | deploy, monitoring |
| [xi-sendgrid-smtp](nodes/xi-sendgrid-smtp.md) | SendGrid SMTP relay (host from secret tokenUrl, port 587, STARTTLS, PLAIN auth) used by the Extract Interface scheduler for schedule notices and run-outcome emails, and by reporting lambdas; credentials in Secrets Manager. | 11 | 0 | email, integration, extract-interface |
| [platform-pagerduty](nodes/platform-pagerduty.md) | PagerDuty, reached through an SNS topic (pagerduty_sns_topic_arn, uat/prd) that PWS CloudWatch alarms built with CloudwatchAlarmBuilder publish to. | 9 | 0 | platform, monitoring |
| [deploy-aws-codepipeline](nodes/deploy-aws-codepipeline.md) | AWS CodePipeline pipelines PWS<Service>-<stage> (one per CDK service per stage) that Jenkins triggers by dropping an artifact zip in S3 and then polls until success or failure. | 6 | 2 | deploy, aws |
| [etl-teams-webhook](nodes/etl-teams-webhook.md) | Microsoft Teams incoming webhook receiving ADAM-ingest Adaptive Cards (STARTED/SUCCESS/FAILED + failed stage) from the ingest-notification lambda. | 6 | 0 | external, notifications, etl |
| [reporting-spotlight](nodes/reporting-spotlight.md) | Spotlight (Compass "sl-assets" buckets in another AWS account) — pushes files into CoW's ReportingIncomingStagingBucket by assuming S3CopyToCoWRole. | 6 | 0 | reporting, external, s3 |
| [amazon-bedrock](nodes/amazon-bedrock.md) | Amazon Bedrock Converse API (Claude Sonnet 4 inference profile by default) powering the Admin SPA's text-to-SQL data investigation agent. | 5 | 0 | ai, external, admin-spa |
| [reporting-rsrd](nodes/reporting-rsrd.md) | RSRD (bridgeftws IAM users in RSRD accounts) — uploads PV/PRD files into the data-as-a-service DataIngestBucket under uploads/rsrd-user/ via roles the reporting stack creates. | 5 | 0 | reporting, external, s3, rsrd |
| [reporting-sftp-destinations](nodes/reporting-sftp-destinations.md) | External SFTP servers that sendToS3Lambda pushes order files to - Spotlight (DPLS/RCS/RCR CSVs combined per order) and other named clients such as CDQ and Cognia - with credentials in Secrets Manager per target/env/tenant. | 5 | 0 | external, delivery, sftp |
| [deploy-aws-amplify](nodes/deploy-aws-amplify.md) | AWS Amplify Hosting apps serving the DepotUI frontend; Jenkins DepotUI jobs push the manifest-pinned DepotUI-Frontend zip via create-deployment / start-deployment. | 4 | 0 | deploy, aws, frontend |
| [deploy-sre6-shared-actions](nodes/deploy-sre6-shared-actions.md) | pearson-aq-school/sre6-shared reusable GitHub actions/workflows used by COW CI (update-manifest-vars, download-manifest, update-deploy-vars, terragrunt-plan, statsagg suspension, slack notifications). | 4 | 0 | deploy, ci |
| [etl-testnav-api](nodes/etl-testnav-api.md) | TestNav 8 Response Data Export API (api-<customer>.testnav.com) - OAuth2 client-credentials, item states by external login key; called only by the SRF-from-ADAM Glue extract. | 3 | 0 | external, testnav, etl |
| [forensics-launchpad](nodes/forensics-launchpad.md) | Pearson Launchpad identity provider as used by the forensics dashboard - OIDC login (launchpad-dev issuer) and the OneRoster v1p2 rostering API (client_credentials) for user roles and orgs. | 3 | 1 | forensics, external, auth |

## datastore (21)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [redshift-warehouse](nodes/redshift-warehouse.md) | Redshift Serverless Data Vault warehouse, one database per admin_code (state/customer administration), holding hubs, satellites, links, refs, helper views, procedures and extract materialized views. | 97 | 0 | warehouse, datastore |
| [reporting-buckets](nodes/reporting-buckets.md) | The three Reporting S3 buckets from PersistenceConstruct — ReportingOrdersBucket (Doc Depot document bytes; OBJECT_CREATED → DocumentsS3Handler), ReportingStagingBucket (outbound staging read by Family Portal/ACT/sandbox) and ReportingIncomingStagingBucket (inbound from Spotlight). | 37 | 0 | reporting, s3 |
| [reporting-table](nodes/reporting-table.md) | DynamoDB single table (ReportingTable, "Doc Depot") keyed tenant + SK holding document definitions, documents, orders, tags, their link items and per-tenant service URLs. | 37 | 0 | reporting, dynamodb |
| [extract-interface-tables](nodes/extract-interface-tables.md) | DynamoDB tables used by the Extract Interface (new DepotUI) backend, provisioned by terragrunt: extract-int-config, -order, -scheduled-extract, -token, -abbi-email-notif. | 35 | 0 | extract-interface, dynamodb |
| [data-ingest-bucket](nodes/data-ingest-bucket.md) | Per-tenant S3 bucket dataingestbucket-<env>-<tenant>; its ingest/, testmaps/, pdfs/ and uploads/ prefixes each fire a trigger lambda that starts a Glue job or Batch job. | 33 | 0 | data-service, s3 |
| [etl-bucket](nodes/etl-bucket.md) | Per-tenant ETL S3 bucket etlbucket-{env}-{tenant} (versioned, KMS, SSL-only) holding extract layouts under instructions/, Redshift UNLOAD/COPY temp, extract outputs under output/, ingest inputs and the shared Redshift JDBC jar (prd-lilith). | 31 | 0 | s3, etl |
| [admin-spa-tables](nodes/admin-spa-tables.md) | DynamoDB control tables of the COW Admin SPA per env — cow-<env>-admin-spa-users, -admin-spa-clients, -admin-spa-jobs, -admin-client-credentials, plus cow-admin-agent-history-<env> (CloudFormation, TTL 7 d). | 23 | 0 | admin-spa, dynamodb |
| [depotui-tables](nodes/depotui-tables.md) | DynamoDB tables of the legacy DepotUI stack — CoreWarehouseTable (orders), DepotGatewayIdToken (users/roles), EmailNotification (ABBI subscriptions), depotUI-<env>-ExtractConfig (extract catalogue) — table names published via SSM. | 15 | 4 | depotui, legacy, dynamodb |
| [workflow-bucket](nodes/workflow-bucket.md) | Per-tenant versioned, KMS-encrypted S3 WorkflowBucket whose key prefix encodes workflow state (<tenant>#<subtenant>/workflow/<state>/<uuid>.json); ObjectCreated events drive ConductorS3Lambda. | 15 | 0 | workflow, s3 |
| [reporting-abbi-extracts-table](nodes/reporting-abbi-extracts-table.md) | DynamoDB ABBI_Extracts table (PK Doc_id, stream NEW_IMAGE) cataloguing ABBI extract files; its stream plus hourly/daily EventBridge rules drive TriggerEmailNotificationsHandler digests. | 14 | 0 | reporting, dynamodb, abbi |
| [tenant-mapper-tables](nodes/tenant-mapper-tables.md) | DynamoDB platform registry - PWSTenantMapper-<env> (one row per tenant) and PWSServiceMapper-<env> (one row per service pipeline), scanned at CDK synth time by every cow-* service pipeline. | 12 | 0 | platform, dynamodb |
| [forensics-parquet-store](nodes/forensics-parquet-store.md) | Pseudonymized Parquet files (one per report table) baked into the forensics backend image at /data/parquet and queried in-process through DuckDB views; no database, no S3. | 11 | 0 | forensics, datastore |
| [deploy-artifact-buckets](nodes/deploy-artifact-buckets.md) | S3 deployment-artifact buckets (nonprd sacoreservnonprdcodebase, prd pws-prd-artifacts) holding versioned service zips and the per-stage <stage>-deployment-artifacts/<Service>.zip objects that trigger each service's CodePipeline. | 9 | 0 | deploy, s3 |
| [etl-dynamodb-tables](nodes/etl-dynamodb-tables.md) | Per-tenant DynamoDB tables created by ETLStack for Glue jobs - act-summative-last-run-time-{env}-{tenant} (ACT incremental watermark) and cow-wida-student-uuid-v2-{env}-{tenant} (stable WIDA student UUIDs). | 8 | 0 | dynamodb, etl |
| [reporting-incoming-staging-table](nodes/reporting-incoming-staging-table.md) | DynamoDB RptIncomingStaging table (PK/SK, stream NEW_IMAGE) tallying incoming C2C/PDF files; stream records trigger CreateC2CIngestHandler once tally reaches expected_count. | 7 | 0 | reporting, dynamodb, c2c |
| [data-dynamodb](nodes/data-dynamodb.md) | Two provisioned, PITR-enabled DynamoDB tables owned by Data as a Service - DataConfigTable (OneRoster ingest clients + watermarks) and STMIngestTableNew (STM tracking per datasource/admin/test/form). | 6 | 0 | data-service, dynamodb |
| [reporting-workflow-queue](nodes/reporting-workflow-queue.md) | Reporting's SNS WorkflowTopic-{env}-{tenant} fanning out to SQS WorkflowQueue (+WorkflowDeadLetterQueue) — the asynchronous path by which Workflow-as-a-Service hands steps to WorkersQueueLambda. | 6 | 0 | reporting, sns, sqs, workflow |
| [workflow-table](nodes/workflow-table.md) | Per-tenant DynamoDB WorkflowTable (PK/SK strings, provisioned) holding workflow status items (WF#uuid) and uniqueExternalKey index items (UEKEY_WORKFLOW#key#WF#uuid) under PK tenant#subtenant. | 6 | 0 | workflow, dynamodb |
| [reporting-pdf-staging-bucket](nodes/reporting-pdf-staging-bucket.md) | The S3 bucket literally named pdfstagingbucket-dev, hardcoded as STAGING_BUCKET for PdfExtract and the C2C lambdas in every environment; not created by any COW CDK stack. | 5 | 0 | reporting, s3, legacy |
| [val-bucket](nodes/val-bucket.md) | S3 bucket validation-service-stack-bucket holding per-job folders (<job_name>/rules.json, input_file.<fmt>, lookup_file.<fmt>, detail-*.json, summary.json, <job_name>.zip) for the dead Validation as a Service prototype. | 4 | 0 | validation, s3 |
| [batch-efs](nodes/batch-efs.md) | Encrypted EFS file system (+ root access point) created per tenant by the BatchJob construct and mounted at /mount/efs in the split-zip job — but the job code never writes there. | 3 | 0 | batch, efs |

## event (7)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [wf-worker-event](nodes/wf-worker-event.md) | The worker request the workflow conductor sends for each event — an HTTP POST to event.url (2xx=done, 202=wait, else fail) or an SNS publish to event.topicARN (always wait) — carrying the workflow's top-level keys plus workflowUUID, event, tenant, subtenant. | 10 | 3 | workflow, messaging |
| [reporting-document-uploaded](nodes/reporting-document-uploaded.md) | S3 OBJECT_CREATED notification on ReportingOrdersBucket, delivered to DocumentsS3Handler; marks a Doc Depot document uploaded and triggers tag-based routing. | 9 | 1 | reporting, s3, event |
| [admin-ingest-schedule](nodes/admin-ingest-schedule.md) | A recurring EventBridge Scheduler schedule created from the Admin SPA that fires an ADAM ingest state machine, an STM Batch job, or the refresh-DB lambda for one client. | 8 | 1 | admin-spa, schedule, ingest |
| [deploy-job-state-change](nodes/deploy-job-state-change.md) | AWS EventBridge Glue/Batch "Job State Change" events (Glue FAILED/TIMEOUT, Batch SUCCEEDED/FAILED) routed to the <env>-failure/success-alerts SNS topics for Slack notification. | 7 | 0 | deploy, alerting |
| [wf-status-callback](nodes/wf-status-callback.md) | Best-effort HTTP POST of {workflowUUID, uniqueExternalKey, state, event, updated} to a workflow's workflowStatusCallbackUrl on every state update. | 7 | 1 | workflow, messaging |
| [reporting-abbi-digest-schedule](nodes/reporting-abbi-digest-schedule.md) | Two EventBridge cron rules in AaasReportingStack - runHourly-abbi-digest (minute 0 every hour) and runDaily-abbi-digest (05:00 UTC) - that invoke TriggerEmailNotificationsHandler to e-mail ABBI extract digests. | 4 | 1 | reporting, schedule, abbi |
| [wf-state-transition](nodes/wf-state-transition.md) | S3 ObjectCreated notification on WorkflowBucket for <tenant>#<subtenant>/workflow/<state>/<uuid>.json that invokes ConductorS3Lambda — the tick of the workflow state machine. | 4 | 1 | workflow, s3, messaging |

## config (23)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [admin-config](nodes/admin-config.md) | Environment variables of the Admin SPA backend (config.Load) — DynamoDB table names, Redshift host/secret/db, Glue/Step Functions/ECS/Lambda/Scheduler ARNs, JWT secret, Entra SSO, Bedrock — set per env through SSM parameters in cow-deployment. | 25 | 0 | admin-spa, config |
| [etl-env-config](nodes/etl-env-config.md) | cow-etl-as-a-service env/{dev,tst,smk,uat,prd,sandbox,wida-uat}.json read by EnvironmentController at CDK synth - accounts, subnets, Redshift namespace, per-job-family Glue sizing, ADAM secret names, ingest bucket, feature opt-ins. | 17 | 0 | config, etl, cdk |
| [xi-backend-config](nodes/xi-backend-config.md) | Environment variables of the Extract Interface backend (one value per deployment, injected by SRE via cow-manifest/SSM): tier, IdPs, DynamoDB table names, PWS credentials, buckets, CORS, scheduler/email, sleuth. | 15 | 0 | config, extract-interface |
| [deploy-ssm-env-params](nodes/deploy-ssm-env-params.md) | SSM Parameter Store tree /<env>/<service>/<KEY> written by modules/ssm-parameters from tfvars and injected as container secrets into every COW ECS service/task. | 14 | 7 | deploy, config |
| [etl-glue-job-args](nodes/etl-glue-job-args.md) | The Glue job argument contract - default arguments ETLStack bakes into each job family (DB connection, output bucket/folder, placeholder filters) that callers override per run, and that jobs resolve with pws.utils.get_glue_args. | 14 | 0 | config, glue, etl |
| [wf-service-credentials](nodes/wf-service-credentials.md) | Secrets Manager client-credential secrets named <service>/<env>/<tenant>/<subtenant> ({client_id, client_secret, tokenUrl}) exchanged for a PWS idToken by the workflow conductor, reporting lambdas and the split-zip job. | 14 | 0 | security, config, workflow |
| [deploy-env-image-pins](nodes/deploy-env-image-pins.md) | Per-environment general.tfvars in live-infrastructure pinning the container image URNs (admin-spa, extract-interface, etl refresh-views, etl-flyway), ACM cert domain and the manifest pipeline version that produced them. | 13 | 1 | deploy, config, versions |
| [etl-adam-db-connection](nodes/etl-adam-db-connection.md) | How ADAM-reading Glue jobs pick an ADAM read replica - the ADAMDBEnv job argument mapped to hardcoded Secrets Manager ARNs and RDS hosts, plus the adamDBConnection env keys that build the Glue JDBC connection. | 12 | 0 | adam, config, secrets |
| [xi-env-setup](nodes/xi-env-setup.md) | Embedded env_setup.json — per-ENVIRONMENT (PRD/UAT/TST/DEV) PWS tenant/env and the wfTag_* workflow tags the extract and delivery assemblers choose from. | 12 | 0 | config, extract-interface, workflow |
| [reporting-env-config](nodes/reporting-env-config.md) | Reporting as a Service per-environment CDK config (env/{dev,tst,smk,uat,prd,sandbox}.json) — accounts, VPC, tenant, base_url, and every cross-account role/bucket for Family Portal, ACT, ADAM, sandbox/PV, Spotlight and RSRD. | 10 | 0 | reporting, config, cross-account |
| [data-config](nodes/data-config.md) | Configuration of Data as a Service - env/<env>.json CDK context, Batch/Lambda environment variables, CLI args of the Java batch apps, Secrets Manager names, SSM parameters and hard-coded per-env Redshift endpoints. | 9 | 0 | data-service, config |
| [forensics-config](nodes/forensics-config.md) | Forensics dashboard runtime config - backend env vars (PARQUET_DIR, DUCKDB_MEMORY_LIMIT, LP_OIDC_*, SESSION_SECRET, APP_URL, DATA_CUSTOMER, DEV_ALLOW_ROLES, BASIC_AUTH_CREDS), ingest ANON_SALT, nginx AUTH_BASIC/BACKEND_UPSTREAM, SSM secrets, and code-constant thresholds. | 9 | 0 | forensics, config |
| [depotui-extract-catalog](nodes/depotui-extract-catalog.md) | The DepotUI extract catalogue — ~435 active / ~200 archived named extract definitions (Glue job, extract type, filters, delivery targets, file-naming conventions) seeded from vueFrontend *Setup.json files into the depotUI-<env>-ExtractConfig table. | 8 | 3 | depotui, legacy, config, extracts |
| [reporting-py-lambda-config](nodes/reporting-py-lambda-config.md) | Runtime configuration of the aaas-reporting python lambdas - CDK env vars per function, the Secrets Manager naming scheme for PWS client credentials, partner ExternalIds and SFTP logins, SSM table-name parameters, and the values hardcoded in source instead. | 8 | 0 | reporting, config, secrets |
| [platform-env-config](nodes/platform-env-config.md) | env/<env>.json files (sandbox, dev, tst, smk, uat, prd) loaded by sacoreservlibs EnvironmentController for the tenant, shared and every service CDK app; -c env=<env> selects the file and suffixes resource names with -<env>. | 7 | 0 | platform, config |
| [wh-flyway-config](nodes/wh-flyway-config.md) | Environment variables and list files that drive the etl-flyway container (SECRET_NAME, DB_HOST/PORT/ENGINE/NAME, TARGET_ENV, MIGRATE_DB_LIST, DB_LIST_FILE, MAX_PARALLEL, MAX_RETRIES, CONTINUE_ON_FAILURE, JDBC_PARAMS, PROBE_DB...). | 6 | 0 | warehouse, config, deploy |
| [reporting-runtime-config](nodes/reporting-runtime-config.md) | Runtime configuration the reporting Java lambdas read — env vars (REPORTING_TABLE, ORDERS_BUCKET, ENVIRONMENT_NAME, UNZIP_FUNCTION, CATALOG_FUNCTION…), ReportingTable ENV#WORKFLOW/ENV#REPORTING url items, and Secrets Manager secrets workflow/, reporting/, sendgrid/{env}/{tenant}/{subtenant}. | 5 | 0 | reporting, config, secrets |
| [xi-frontend-runtime-config](nodes/xi-frontend-runtime-config.md) | Extract Interface SPA runtime config — API_BASE_URL, Azure AD / Cognito client ids, E2E_AUTH_URL — injected as window.__APP_CONFIG__ (/config.js) at nginx container start, with VITE_* build-time fallbacks. | 5 | 0 | extract-interface, config, frontend |
| [batch-config](nodes/batch-config.md) | Batch as a Service configuration — env/*.json (account, VPC/subnets, artifact bucket/path, tags, PagerDuty), LOCAL_DEV switch, Powertools env on the API lambda, split-zip job parameters and MAX_OPEN_FILES. | 4 | 0 | batch, config |
| [depotui-env-config](nodes/depotui-env-config.md) | DepotUI environment wiring — cdk.json per-env VPC/subnets/stage, lambda env vars and SSM parameter names, Secrets Manager reporting/<env>/<tenant>/<subtenant> PWS credentials, and the SPA's envSetup.json (API URLs, PWS base URLs per UI env, workflow tags). | 4 | 0 | depotui, legacy, config |
| [xi-sleuth-config](nodes/xi-sleuth-config.md) | Env vars locating and bounding the bundled Sleuth lineage tool (SLEUTH_TOOL_DIR, SLEUTH_PYTHON_BIN, SLEUTH_TIMEOUT_SECONDS) plus build-time COW_ETL_REPO / COW_FLYWAY_REPO. | 3 | 0 | sleuth, config, extract-interface |
| [navvy-db-connection](nodes/navvy-db-connection.md) | How COW Glue jobs reach the Navvy PostgreSQL reader (new data source, COW-4550, Oct 2026) - one Secrets Manager secret per COW AWS account picked by account id at runtime, plus a temporary connectivity-check Glue job; no extract or ingest reads Navvy yet. | 2 | 0 | navvy, config, secrets, etl |
| [wf-config](nodes/wf-config.md) | Workflow as a Service configuration — lambda env vars (WORKFLOW_BUCKET, WORKFLOW_TABLE, ENVIRONMENT_NAME, POWERTOOLS_*) and per-env env/*.json keys (account, VPC/subnets, artifact bucket/path, workflow_lambda.memory, PagerDuty topic). | 2 | 0 | workflow, config |

## flow (24)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [xi-deliver-completed-order](nodes/xi-deliver-completed-order.md) | Post-completion delivery action on an extract order (SFTP repost, Family Portal stage/copy, Stage for ADAM, email) — backend rebuilds a follow-on PWS order from the original order + Default Delivery config and records it in the original order's extractSent. | 13 | 6 | flow, extract-interface, delivery |
| [xi-start-extract-order](nodes/xi-start-extract-order.md) | User starts an extract from the Extract Interface — config in DynamoDB is merged with shared Split_Zip configs, assembled by ExtractAssembler into a PWS order payload (wfTag, file names, split rules), posted to PWS reporting, and saved as an extract-order row. | 11 | 8 | flow, extract-interface, orders |
| [run-extract-order](nodes/run-extract-order.md) | End-to-end extract order across repos — Extract Interface (or legacy DepotUI) posts a reporting order, OrdersLambda submits the tagged workflow, the WaaS conductor walks its events (GLUE extract → PUBLISH_TO_WAREHOUSE → split-zip on Batch → DELETE_FROM_ORDER → SEND_ZIP/S3 copy → EMAIL), each worker acking back through /orders/{id}/status, while the UI polls workflow status. | 9 | 21 | flow, cross-repo, extract, orders, workflow |
| [xi-download-documents](nodes/xi-download-documents.md) | Downloading extract output from the Extract Interface — list an order's PWS docIds, direct presigned download for one doc, or server-side in-memory ZIP re-uploaded to PWS (COW_DOWNLOAD / ABBI_DOWNLOAD) and polled until readable. | 9 | 10 | flow, extract-interface, documents |
| [platform-deploy-service-to-tenants](nodes/platform-deploy-service-to-tenants.md) | Deploy a cow-* service to all tenants - Jenkins uploads the service zip to S3, the service's self-mutating CodePipeline (ServicePipelineProvider) dumps the tenant/service registry, synthesises one stack per tenant, registers itself, deploys, copies lib zips and redeploys each tenant API stage. | 8 | 7 | platform, flow, deployment |
| [ingest-pdfs-to-doc-depot](nodes/ingest-pdfs-to-doc-depot.md) | Externally produced student/school report PDFs into Doc Depot and the warehouse - RSRD drops a zip (manifest.csv + companion.txt + PDFs) into dataingestbucket uploads/, DataPdfZipTriggerLambda submits pdfIngestBatchJob, which writes each PDF as a Doc Depot document (optionally wrapped in a reporting order) and a rebuilt manifest under pdfs/; PDFTriggerLambda then starts Glue pdfToRedshift loading hub_pdf/sat_pdf and links. | 7 | 12 | flow, pdf, doc-depot, ingest, data-service |
| [route-uploaded-document](nodes/route-uploaded-document.md) | What happens when a file lands in ReportingOrdersBucket — DocumentsS3Lambda stamps the document uploaded, acks a waiting workflow if the definition is tagged reserved:workflow:acknowledgeWaiting, and async-invokes Unzip / PdfExtract / extractLambda (catalog) by the definition's reserved:* tags; outputs become new documents (re-entering the flow) or ABBI catalog rows. | 7 | 8 | flow, reporting, doc-depot, s3 |
| [xi-scheduled-extract-run](nodes/xi-scheduled-extract-run.md) | In-process scheduler of the Extract Interface backend — every minute claims due scheduled-extracts and replays StartExtract on the frozen config clone; every ~5 min polls workflow status and emails per notifyMode via SendGrid SMTP. | 7 | 6 | flow, extract-interface, schedules |
| [xi-spa-sign-in](nodes/xi-spa-sign-in.md) | How a user signs in to the Extract Interface SPA — Entra ID popup, Cognito password or offline e2e issuer — and how the IdP token becomes an app user with roles via POST /api/auth/token. | 7 | 6 | extract-interface, auth, frontend |
| [platform-onboard-tenant](nodes/platform-onboard-tenant.md) | Onboard a PWS tenant - POST /tenants creates the PWSTenantMapper row, the PWSTenant pipeline builds the tenant API/authorizer/cluster, postbuild records the ids and restarts every registered service pipeline so each service deploys a stack for the new tenant. | 6 | 7 | platform, flow, tenancy |
| [export-pdfs-to-family-portal](nodes/export-pdfs-to-family-portal.md) | Batch export of an order's Doc Depot PDFs to Family Portal - a reporting workflow LAMBDA step invokes startPdfExportLambda, which submits exportPDFBatchJob (hard-coded prd-lilith queue); the job resolves the order's docIds (plus docIds listed inside the order zip), stages PDFs/CSVs in the reporting staging bucket and copies them cross-account into the Family Portal bucket. | 5 | 9 | flow, pdf, export, family-portal, doc-depot |
| [ingest-adam-to-warehouse](nodes/ingest-adam-to-warehouse.md) | ADAM client data into the Redshift Data Vault - Admin SPA (or EventBridge schedule / DoDEA nightly rule) starts one of two Step Functions; optional clearDB wipe; ECS path streams ADAM SQL to S3 CSVs by UUID buckets then Glue ECSS3AdamDBIngestJob loads them, Glue-direct path reads ADAM over JDBC in DirectADAMDBToRedshift; per-entity hub/link/sat inserts; optional refresh_mvs('all') via the ECS refresh task. | 5 | 16 | flow, etl, ingest, adam, data-vault |
| [load-es-objectives](nodes/load-es-objectives.md) | Admin uploads an Evidence Statement objective CSV; an ES_OBJECTIVE_LOAD job inserts derived <KEY>_RAW_SCORE / <KEY>_MAX_RAW_SCORE keys into ref_scoring_category_unique (one idempotent BatchExecuteStatement) and refreshes mv_evidence_statement_outcomes, which the Evidence Statement extract Glue jobs read. | 5 | 7 | flow, admin-spa, redshift, loaders, evidence-statement |
| [load-scoring-categories](nodes/load-scoring-categories.md) | Admin uploads a scoring-category CSV in the Admin SPA; a SCORING_CATEGORY_LOAD job drops and rebuilds ref_scoring_category in the admin code's Redshift database in one BatchExecuteStatement, then CALLs generate_unique_categories() and refresh_mvs('all') — percentile_categories() is left manual. | 5 | 7 | flow, admin-spa, redshift, loaders |
| [release-container-service](nodes/release-container-service.md) | Release a COW container service (Admin SPA, Extract Interface, refresh-views) via GitHub Actions - service CI pushes images to prd ECR, dispatches cow-manifest (tst-manifest / release-manifest), which dispatches cow-deployment tst/uat deploy (manifest → general.tfvars → terragrunt apply of ECS services/tasks); prd is a separate ServiceNow-gated dispatch. | 5 | 11 | flow, deploy, github-actions, release |
| [roll-out-redshift-migration](nodes/roll-out-redshift-migration.md) | Roll a Redshift schema change out - a new V<n>__*.sql in cow-etl-as-a-service-flyway is built into the pws/etl-flyway image by Jenkins, pinned in the manifest (DockerImages.flyway_image_urn), applied per env by the Jenkins ETL-Flyway job (flyway_deploy.sh → terragrunt ecs-tasks/etl-flyway), whose apply runs the one-shot ECS task; migrate.sh baselines + migrates every admin_code database in the env's db list. | 5 | 9 | flow, deploy, warehouse, flyway, jenkins |
| [xi-adam-manifest-download](nodes/xi-adam-manifest-download.md) | Download / Regenerate ADAM manifest CSV for a completed non-PDF extract directly from its packaged zips (Go ports of s3CopyLambda + createAdamManifest.py), persisted as a PWS CLIENT_MANIFEST document referenced by the order's adamManifestDocId. | 5 | 7 | flow, extract-interface, adam |
| [load-pr-growth](nodes/load-pr-growth.md) | Puerto Rico prior-year growth load — Admin SPA streams a prior-year CSV to etl-bucket input/PR Prior Year/, a PR_GROWTH_LOAD job truncates ref_growth_<crece\|alterna>, runs Glue PR_PriorYear_CSVToRedshift, verifies the row count, then runs the embedded growth SQL (copy of flyway ddl/pr_growth_process) that rebuilds v_prior_year_growth, sat_testattempt_prior_year_growth and mv_prior_year_growth for the PR extracts. | 4 | 8 | flow, admin-spa, redshift, loaders, puerto-rico, growth |
| [promote-cdk-service](nodes/promote-cdk-service.md) | Promote a COW CDK service (ETL, Reporting, Workflow, Batch, Data, Tenant, Shared, DepotUI) through Jenkins - tagger_and_deployer tags + zips the repo to the S3 artifact buckets and calls the Development/Test/Release manifest jobs, whose per-env deploy jobs copy the pinned zip to <stage>-deployment-artifacts and wait on CodePipeline PWS<Service>-<stage>; prd gated by ServiceNow. | 4 | 9 | flow, deploy, jenkins, cdk, release |
| [send-abbi-digest](nodes/send-abbi-digest.md) | ABBI catalog-to-inbox — ABBI files catalogued into ABBI_Extracts by extractLambda are matched hourly/daily by triggerEmailNotifications against EmailNotification subscriptions (written from Extract Interface or DepotUI), which creates one e-mail reporting order whose workflow EMAIL worker sends the digest through SendGrid SMTP. | 4 | 9 | flow, abbi, notifications, email |
| [forensics-data-refresh](nodes/forensics-data-refresh.md) | Manual forensics data refresh - COW ETL extract CSVs -> OneDrive nested-zip bundle -> laptop unpack + pseudonymizing convert -> parquet baked into backend image -> ECR push -> terraform apply -> units recomposed at container start. | 3 | 5 | forensics, flow |
| [ingest-stm](nodes/ingest-stm.md) | Scoring test map (STM) ingest - Admin SPA (or the 07:00 UTC stmIngestRule / an EventBridge schedule) submits Batch stmIngestBatchJob; STMIngestApp pulls each form/test map from TestNav Previewer, tracks it in STMIngestTableNew and drops testmaps/*.json in the data ingest bucket; DataStmTriggerLambda starts Glue STMToRedshift per file. Extract STM data actually comes from mv_all_stm, derived from sessions. | 3 | 12 | flow, ingest, stm, data-service |
| [alert-job-failure-to-slack](nodes/alert-job-failure-to-slack.md) | Prd-account job alerting - an AWS Glue (FAILED/TIMEOUT) or AWS Batch (SUCCEEDED/FAILED) Job State Change event matches an EventBridge rule, is published to SNS cow-prd-failure-alerts / -success-alerts, and the cow-prd-sns-to-slack Lambda formats it and posts to the cow-bell Slack workflow webhook. | 2 | 5 | flow, alerting, monitoring |
| [grant-extract-permission](nodes/grant-extract-permission.md) | Admin SPA admin grants an Extract Interface user per-customer / PRD-tier access — writes the extract-user permissions block and the customer_groups record in DynamoDB; the Extract Interface compiles them per request (customer groups via a 60 s cache) into a Policy that RequireAction checks against <customer>/<tier>/<adminCode> resources. | 2 | 7 | flow, admin-spa, extract-interface, authz, permissions |

## concept (25)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [wh-admin-code-database](nodes/wh-admin-code-database.md) | The per-admin_code Redshift database model - one database per administration (e.g. ilspr26, ilspr26pv), listed in meta-data/<env>-db-list.txt and mapped in ref_state_abbreviation_mapping; replaces the legacy one-schema-per-tenant dev database. | 17 | 2 | warehouse, glossary, multi-tenant |
| [wh-data-vault-conventions](nodes/wh-data-vault-conventions.md) | COW warehouse Data Vault conventions - md5 hash keys over business key + datasource_id, ingest_dts load timestamp, hash_diff change detection, datasource_id as record source, hub/link/sat/ref naming. | 17 | 5 | warehouse, data-vault, glossary |
| [etl-extract-catalog](nodes/etl-extract-catalog.md) | Glossary/catalogue of COW customer extract types (SDF, IRF, CAT, Summative, Summary, DPLS, RCS, RCR, CSR, Growth, C2C, Battery, forensics ABR/RCA/LNG, Evidence Statement, WIDA MOAR/PLCR, Utah events/scores, PR r600/r650, ND battery, NY SHSAT, Detroit, Tools) and the Redshift Glue job, MVs, layout and output folder behind each. | 14 | 4 | etl, extract, glossary |
| [deploy-environments](nodes/deploy-environments.md) | COW deployment environments and their naming - stage ids dev/smk/tst/uat/prd/wida-uat, terragrunt env names cow-<stage>, AWS accounts nonprd 509332739732 and prd 558626028390, and the PWS tenant behind each. | 13 | 3 | deploy, glossary |
| [xi-permission-resource](nodes/xi-permission-resource.md) | Extract Interface authz resource string `<customerCode>/<tier>/<adminCode>` — customer code from customer_groups name prefixes (or UNMAPPED), tier pv/prd from the admin code — against which per-user customer grants are matched. | 13 | 2 | glossary, authz |
| [reporting-reserved-tags](nodes/reporting-reserved-tags.md) | Tag names starting with reserved: on a document definition that make Doc Depot act on upload — reserved:workflow:acknowledgeWaiting, reserved:lambdaUnzip, reserved:lambdaUnzipPdf, reserved:lambdaCatalog, reserved:lambda. | 12 | 2 | reporting, glossary |
| [etl-datasource-id](nodes/etl-datasource-id.md) | datasource_id - the ADAM client id (clients.id UUID) identifying one COW customer; part of every Data Vault hash key and, in the shared dev DB, the schema name _<uuid>. | 7 | 1 | glossary, adam, warehouse |
| [etl-external-login-key](nodes/etl-external-login-key.md) | ELK (external_login_key) - ADAM sessions.external_login_key, the TestNav StudentTestUUID of one test attempt; COW's join key for PDFs, ZH scores and TestNav item-state exports. | 7 | 1 | glossary, adam, testnav |
| [xi-wf-tag](nodes/xi-wf-tag.md) | wfTag — the tag name sent as `tag` on a PWS reporting order that selects which registered workflow template (extract+email, SFTP, split, ADAM staging, S3 copy...) the order runs. | 7 | 2 | glossary, workflow |
| [xi-config-form-convention](nodes/xi-config-form-convention.md) | The implicit schema by which an extract config's `data` JSON is rendered as the Start Extract / Schedule form in the Extract Interface SPA (arrays → selects, "multi" keys → multiselects, default_* prefill, admin-scoped defaults). | 6 | 1 | extract-interface, frontend, glossary |
| [customer](nodes/customer.md) | "Customer" is overloaded in COW - an ADAM datasource/client (datasource_id, Admin SPA "Clients"), an Extract Interface customer group code (MD, PR, WIDA…) matched by extract-name prefix, a DepotUI customer prefix in userRole, the forensics DATA_CUSTOMER, and never a PWS tenant. | 5 | 4 | glossary, customer |
| [abbi](nodes/abbi.md) | ABBI - an extract-producing system outside COW whose files are parked in COW's Doc Depot (docCode ABBI_EXTRACT), indexed in the ABBI_Extracts table and searched/subscribed to by ABBI-role users; expanded as "Automated Batch Business Intelligence" only in DepotUI docs. Also the name of a TestNav Previewer instance. | 3 | 4 | glossary, abbi |
| [doc-depot](nodes/doc-depot.md) | Doc Depot - Reporting as a Service's document store (ReportingTable + orders bucket) in which every COW file lives as a document (docId/documentUUID) of a document type (docCode) with tags; "document" in COW means one of these, and DepotUI is merely the legacy UI named after it. | 3 | 5 | glossary, doc-depot, reporting |
| [pv-administration](nodes/pv-administration.md) | PV - an admin code containing "pv" (e.g. ilspr26pv) is the preview/pre-validation twin of a production administration - its own warehouse database, the Extract Interface "pv" tier (vs prd), and different delivery buckets (two-hop via TST staging in prd, sand/ACT PV targets); not the "sandbox" environment and not TestNav Previewer. | 3 | 4 | glossary, pv, tier |
| [subtenant](nodes/subtenant.md) | tenant vs subtenant in PWS - a tenant is a landing zone (own API Gateway, authorizer, stacks, e.g. lilith); a subtenant is only a Cognito claim (custom:subtenantID) that partitions data inside a tenant (DynamoDB keys tenantId#subtenantId, secret names .../<tenant>/<subtenant>, usually "none"). | 3 | 5 | glossary, platform, multi-tenant |
| [unit](nodes/unit.md) | "Unit" has two meanings - in the forensics dashboard a review unit is one school x test (the thing flagged and triaged); in batteries (WIDA, ND ELA) a unit is one component test of the battery. | 3 | 2 | glossary, forensics, battery |
| [battery](nodes/battery.md) | Battery - a group of unit tests reported together under one score (WIDA's four ELP domains, North Dakota's ELA Reading+Writing units, Utah/Aspire); stored as battery attempts with battery-level results and extracted by Battery SDF / Battery C2C / ND Battery Summary / WIDA MOAR; nothing to do with AWS Batch. | 2 | 3 | glossary, battery |
| [etl-extract-year-prefix](nodes/etl-extract-year-prefix.md) | The 25_/26_/27_ (and v26_/v27_) prefix on extract scripts, Glue job names and layout folders = the school-year layout spec generation (26 = SY2025-26, 27 = SY2026-27); unprefixed jobs are evergreen or pre-convention. | 2 | 2 | etl, extract, glossary |
| [pws](nodes/pws.md) | PWS (Pearson Web Services) - the multi-tenant AWS platform COW's backend services run on (per-tenant API Gateway + Cognito authorizer); its services are the "as a Service" family (ETL, Reporting, Data, Workflow, Batch, Validation as a Service = "AaaS"), and RADAR is the older name of Reporting as a Service. | 2 | 8 | glossary, platform |
| [scoring-test-map](nodes/scoring-test-map.md) | STM = scoring test map - Pearson's per-form description of a test (form, test number, items, item status, subject/grade/format) pulled from TestNav Previewer; in the current warehouse it is derived from session data (mv_all_stm), and "STM ingest" is the legacy loader. | 2 | 4 | glossary, stm |
| [wida](nodes/wida.md) | WIDA - the English-language-proficiency (ELP) customer whose results are a battery of four unit tests (Listening, Reading, Speaking, Writing); in COW it is a customer group, a set of extracts (MOAR, Model, Roster, State/District, PLCR) with persistent student UUIDs, and its own deployment line (wida-uat environment, wida-manifest.json). | 2 | 4 | glossary, wida |
| [c2c](nodes/c2c.md) | C2C - two COW uses with no expansion anywhere in code - the "C2C Summative" / "Battery C2C" extract format handed to an internal C2C team, and the legacy C2C PDF intake (c2cPublishLambda + RptIncomingStaging) that publishes report PDFs to Doc Depot and builds a manifest for warehouse ingest. | 1 | 3 | glossary, c2c, extract, pdf |
| [cow](nodes/cow.md) | COW / CoW = "Core Warehouse" - Pearson's assessment-results data platform (ADAM ingest → Redshift Data Vault → customer extracts and deliveries) and the umbrella name of the cow-* repos, AWS accounts and resources; not copy-on-write, not a single service. | 1 | 4 | glossary, platform |
| [puerto-rico-crece-alterna](nodes/puerto-rico-crece-alterna.md) | Puerto Rico "crece" vs "alterna" - the two PR assessment programs as seen in COW admin codes (prcrece*, pralterna*/pvalterna*): crece is the general assessment (extract jobs named "meta"), alterna the alternate assessment; each has its own growth reference table and r600/r650/SDF extract jobs. | 1 | 3 | glossary, puerto-rico |
| [order](nodes/order.md) | "Order" is overloaded - a PWS reporting order (Doc Depot ORDER# that runs a workflow), an extract order (Extract Interface/DepotUI bookkeeping row sharing the same OrderUuid), a delivery child order, an ABBI digest email order and an internal PDF-ingest order; "ordering" an extract creates both the first two. | 0 | 3 | glossary, order, extract |

## capability (25)

| node | summary | in | out | tags |
|---|---|---|---|---|
| [xi-data-lineage-viewer](nodes/xi-data-lineage-viewer.md) | "Sleuth Data Lineage" page in the Extract Interface — pick an extract and see, field by field, where each output column comes from (extract job → Redshift MV → Data Vault → ADAM), optionally narrowed by an uploaded layout. | 5 | 5 | sleuth, lineage, capability, extract-interface |
| [etl-customer-extracts](nodes/etl-customer-extracts.md) | Customer data-file extracts - on request, COW produces state/district/school result files (student data files, item responses, score summaries, rosters, growth, test-security signals, WIDA files) in each customer's agreed layout and delivers them. | 4 | 8 | extract, capability |
| [deliver-extract](nodes/deliver-extract.md) | Deliver a completed extract onward - re-email it, push it to a customer SFTP server, copy it to Family Portal / ACT / sandbox (PV) buckets, or stage it (with a manifest) for ADAM, from the order's details page. | 3 | 9 | capability, extract, delivery |
| [schedule-extract](nodes/schedule-extract.md) | Recurring extracts - a user schedules an extract configuration to run daily/weekly/monthly (or on custom rules, in a chosen timezone) and gets email notices about each run's outcome. | 3 | 6 | capability, extract, extract-interface, schedule |
| [stm-ingest](nodes/stm-ingest.md) | STM ingest - pull scoring test maps (form/test/item metadata) for a client's admin code from TestNav Previewer into the warehouse, on demand from the Admin SPA or nightly; legacy because the warehouse tables it loads were dropped and STM data is now derived from sessions. | 3 | 10 | capability, ingest, stm |
| [user-access-management](nodes/user-access-management.md) | Who can do what - granting Extract Interface users per-customer and PRD/PV access (Admin SPA "Extract UI Permissions"), SSO roles for the Admin SPA and Extract Interface, legacy DepotUI user roles, and machine/API client credentials for the PWS APIs. | 3 | 14 | capability, auth, permissions |
| [abbi-notifications](nodes/abbi-notifications.md) | ABBI email notifications - a user subscribes to hourly or daily digest emails listing new ABBI extract files that match an ABBI environment / admin code / extract / test filter. | 2 | 7 | capability, abbi, email |
| [customer-registry](nodes/customer-registry.md) | Customer (client) registry in the Admin SPA - the list of ADAM customers COW serves, each with its production and PV administration codes, ADAM database environment and TestNav Previewer instances, which every ingest and database tool picks from. | 2 | 6 | capability, admin-spa, customer |
| [order-extract](nodes/order-extract.md) | Order a customer extract - an internal Pearson user picks a customer and a named extract configuration in the Extract Interface (or legacy DepotUI), fills in the generated form and COW builds the data file through a reporting workflow. | 2 | 17 | capability, extract, extract-interface |
| [reference-data-loads](nodes/reference-data-loads.md) | Load scoring and growth reference data into an administration's warehouse database - scoring categories, Evidence Statement objective keys and Puerto Rico prior-year growth files - from the Admin SPA, as tracked jobs. | 2 | 9 | capability, admin-spa, warehouse, reference-data |
| [student-report-pdfs](nodes/student-report-pdfs.md) | Student and school report PDFs - batches of score-report PDFs produced outside COW (RSRD, Spotlight, C2C) are taken in, stored in Doc Depot, linked to students' test attempts in the warehouse and copied to Family Portal (or staged for ADAM) on order. | 2 | 13 | capability, pdf, family-portal, doc-depot |
| [abbi-extract-search](nodes/abbi-extract-search.md) | ABBI extract search - ABBI users find ABBI-produced extract files by ABBI environment and admin code (plus date, extract, test, form), download them as a zip, or delete them. | 1 | 7 | capability, abbi |
| [adam-ingest](nodes/adam-ingest.md) | Load a customer's ADAM test data into the warehouse - an operator (or a schedule) ingests a client's sessions, students, organizations, items and scores from ADAM into the per-administration Redshift database, optionally wiping it first and refreshing the extract views after. | 1 | 12 | capability, ingest, admin-spa, warehouse |
| [data-agent](nodes/data-agent.md) | AI data assistant in the Admin SPA - an operator asks questions about warehouse data in plain English and an Amazon Bedrock (Claude) agent explores tables and runs read-only SQL to answer, showing the queries and result tables. | 1 | 6 | capability, admin-spa, ai |
| [edit-extract-configs](nodes/edit-extract-configs.md) | Maintain the extract catalogue - admins create, edit, clone and archive the named extract configurations (JSON) that users order from, per customer, in the Extract Interface (or legacy DepotUI Extract Config). | 1 | 8 | capability, extract, config |
| [refresh-materialized-views](nodes/refresh-materialized-views.md) | Refresh the reporting views - rebuild every materialized view in one administration's warehouse database so extracts see newly ingested data, on demand from the Admin SPA, at the end of an ingest, or on a schedule. | 1 | 7 | capability, warehouse, mv |
| [tenant-onboarding](nodes/tenant-onboarding.md) | Multi-tenant onboarding - a new PWS tenant (customer landing zone) is registered with one API call, after which the platform builds its API, authorizer and cluster and every COW service pipeline deploys a stack for it. | 1 | 8 | capability, platform, multi-tenant |
| [test-security-dashboard](nodes/test-security-dashboard.md) | Test-security (forensics) dashboard - state, district and school users review schools x tests flagged by aberrant-response, answer-change, year-over-year and timing signals, triage them and print a security report; pilot in the nonprd account on pseudonymized data. | 1 | 10 | capability, forensics, test-security |
| [xi-order-history-time-index](nodes/xi-order-history-time-index.md) | Planned fix so My/Recent History reliably shows the newest extract orders — add a time-keyed (and per-user) DynamoDB index instead of paging env-extract-index, which is sorted by extract name. | 1 | 2 | planned, history, extract-interface |
| [download-extract-output](nodes/download-extract-output.md) | Download an extract's output files (single file, or several zipped server-side) and download or regenerate the ADAM manifest CSV for a completed order. | 0 | 8 | capability, extract, download |
| [file-validation](nodes/file-validation.md) | File validation - checking extract files and incoming ADAM tables against rule sets; the Validation-as-a-Service API and the Glue Data Quality framework were built but never deployed or invoked, and the Admin SPA "Run Schema Validation" button starts a Glue job that no COW repo defines. | 0 | 6 | capability, validation |
| [job-monitoring](nodes/job-monitoring.md) | Job monitoring and alerting - operators watch ingests, view refreshes and data loads in the Admin SPA Job History (stages, Glue runs, rerun), extract users watch order workflow status, and failures are pushed to Slack, Teams and PagerDuty. | 0 | 10 | capability, monitoring, alerting |
| [release-deploy](nodes/release-deploy.md) | Release and deploy pipelines - how a COW change reaches dev/tst/uat/prd - container services via GitHub Actions and the cow-manifest bill of materials into terragrunt, CDK services via Jenkins and self-mutating CodePipelines per tenant, Redshift schema migrations via the flyway task, with ServiceNow approval for production. | 0 | 15 | capability, deploy, release |
| [split-extract-files](nodes/split-extract-files.md) | Split and zip large extracts - a state-wide extract file is cut into one file per district/school (or any key combination) and packed into a single zip, as a step of the extract workflow. | 0 | 7 | capability, extract, batch |
| [warehouse-admin-tools](nodes/warehouse-admin-tools.md) | Operator tools on the warehouse in the Admin SPA - look up one test attempt's raw data by ELK (Data Debug), create a database, delete test attempts, retrieve an organization tree, and an API-only ad-hoc SQL endpoint; several are placeholders or miswired. | 0 | 6 | capability, admin-spa, warehouse |