Learning outcomes assessment where every number in the report opens the student papers behind it
Assessment reports built from scored student work, where every number opens its papers.
See one case, screen by screen ↓Learning outcomes assessment is how a college shows what its students learned against stated outcomes, by scoring a sample of student work against a faculty rubric, and what it changed because of the results. Learning Outcomes Assessment puts seven agents in front of that cycle: they draw a sample anyone can re-draw, score every paper twice and blind, measure their agreement against a hidden faculty set, and draft the report, while faculty decide the hard papers and the Director of Assessment signs.
AI in education: a buyer's guide to agentic AI for colleges, universities and school districts →
Updated
Learning outcomes assessment is how every regionally accredited college shows what its students learned against its stated outcomes, and what it changed because of the results. At most colleges the evidence for a general education outcome is assembled once a year: the assessment office asks faculty to send "a sample" of papers, a group of faculty scores them on a reading day, someone types the scores into a spreadsheet, and the director writes the report from that sheet.
The sample over-represents willing faculty and in-person day sections, and nobody can draw it again when an accreditor asks. Scorer agreement is rarely measured. Student names sit in file headers while scoring sheets travel by email. And the use of results, the part accreditors ask about most, is written from memory, with no owner, date or follow-up. Learning Outcomes Assessment puts seven agents in front of that work, so faculty judge the hard papers and the director signs a report whose every count is a stack of papers.
Estimated split of one general education outcome's cycle, by hand and with the agents. The 'after' column follows the working solution's fall 2026 cycle, Nov 30 to Dec 10, and totals 10 days, the design goal; the 'before' column is our estimate, kept modest.
Seven agents sample, hide names, score every paper twice blind and draft the report; faculty settle gaps and the director signs.
Written Communication, fall 2026, at Brandvale Community College: the faculty's rubric v3, 2,946 signature-assignment submissions in 128 sections, and a report due before the spring term. Here is the cycle, screen by screen, in the working solution.
The Sampler read the frame from the learning management system and drew 312 papers from 47 sections, 10.6% of each stratum: Central, North and West campuses by modality, and dual enrollment from five partner high schools. Six blank, unreadable or duplicate files were replaced from the same stratum in seed order, and the Redaction agent hid 44 student names before anyone scored.
Re-draw from seed 7731905: 312 of 312 papers match, in the same order.
Faculty scored and sealed a 31-paper calibration set first. Then two Scorer agents on different models scored all 312 papers against the rubric, never seeing each other's levels, the student's name or the faculty's scores, and cited the passage behind each level. Of 1,560 criterion scores, 1,229 matched exactly (78.8%) and 1,503 were within one level (96.3%).
57 scores in 21 papers were more than one level apart, so the Agreement agent sent them to the faculty panel.
Against the 31 papers the faculty sealed on Dec 1, the agents' final levels agree at a weighted kappa of 0.74, the same as faculty against faculty; 124 of 155 criterion scores match exactly. Use of evidence is lowest at 0.68, within 0.03 of the 0.65 threshold, so it is marked "watch".
Below the threshold, agent scoring stops for that criterion and it goes wholly to faculty.
The panel, chaired by Prof. Nadia Kessler with Dr. Tom Albrecht, Prof. Lena Ortiz and Prof. Sam Whitford, resolved 57 of 57 scores. Two panelists scored each packet before they saw the agents' levels. The panel matched Scorer A on 12, Scorer B on 17 and fell between them on 28.
WC-286, Use of evidence: "Two sources are quoted and cited, but neither is explained or connected to the claim."
The Reporter drafted method, results, what they show and use of results. At 8:30 a.m. Prof. Nadia Kessler attached the English department's action plan: retire the 2019 prompt by Jan 19, 2027, add the library's evidence-integration module by Jan 25, hold an adjunct workshop on Jan 14 with 14 stipends of $300, and re-assess Use of evidence in fall 2027.
Use of evidence over three cycles: 46%, 49%, 52% at Proficient or above.
Gwen checks the draft against the results. The rubric is one grid: five criteria by four levels, and each cell is a stack of the papers scored there. Context and purpose (71%), Organization (69%) and Syntax and mechanics (65%) meet the faculty's benchmark; Content development (62%) and Use of evidence (52%) do not. In Use of evidence, 119 papers, 38%, sit at Developing.
Gwen clicks a paper in the Developing cell. WC-065, a North Campus online ENGL 101 paper, opens beside the grid with the passage that earned its level: an argument about bike lanes that quotes the news without analysis. Scorer A said Developing at 0.74 confidence, Scorer B Developing at 0.81.
Lighting North Campus online marks its 44 papers in every cell, each showing "x of 44". On Use of evidence they average 2.18 against 2.58 for the other 268, and 22 of the 44 sit at Developing. Central Campus online averages 2.56, so modality alone does not explain it.
The Reporter found that 6 of the 7 North online sections still use the 2019 assignment prompt, which does not require sources.
Gwen confirms the results were checked against the rubric grid and the panel's decisions, and that the faculty lead reviewed the action plan, then signs. She can edit a paragraph or return the draft with a reason code, but she cannot change a score or the rubric, and no agent signs.
The Evidence agent maps each element of the regional standard to its records and builds a redacted pack from the signed report: the sample record with its seed, 312 scored papers, 21 panel decisions and the agreement report. It checks every page for names, finds 0, seals the pack and logs it.
One outcome's cycle, seen by the five people who carry it: what their year looked like, and what it looks like now.
Seven agents sample, hide names, score every paper twice blind and draft the report; faculty settle gaps and the director signs.
Builds the frame from the learning management system and the student information system, draws a stratified random sample by campus, modality and dual enrollment, and records the seed and the replacement order.
Hides student names and IDs, and for scorers the instructor and section, before anyone scores.
Scores each paper against the faculty's rubric, one level per criterion, citing the passage and the rubric descriptor it meets.
Scores the same papers independently, on a different model and in a separate session, so one model's bias does not become the result.
Compares the two Scorers, applies the faculty's rules (adjacent scores keep the lower level; more than one level apart goes to the panel), builds panel packets and measures agreement against the sealed calibration set.
Drafts results, results by group of sections, the explanation of a gap and use of results, ties them to program review and the action plan, and schedules the follow-up re-assessment.
Maps the signed report and its records to the regional accreditor's standard and builds a redacted, sealed evidence pack.
Settle, sign and fund. The agents propose; a named person decides.
The assistant answers from the cycle's own records, cites each one, and opens the screen that shows it.
Why does North Campus online score lower on use of evidence?
Its 44 papers average 2.18 on Use of evidence, against 2.58 for the other 268; 22 of the 44 sit at Developing, against 38% overall. Central Campus online averages 2.56, so it is not online teaching as such. Six of the seven North online sections still use the 2019 prompt, which asks for an opinion "supported by examples" and requires no sources.
Can the accreditor reproduce this sample?
Yes. The Sampler recorded the frame (2,946 submissions, Nov 30 snapshot), the seven strata, seed 7731905 and the replacement order. Drawing again from the seed gives the same 312 papers in the same order.
Which scores did the faculty panel change, and why?
All 21 papers (57 scores) more than one level apart went to the panel on Dec 9. Its level matched Scorer A on 12, Scorer B on 17 and fell between them on 28. For WC-286 the panel wrote: "Two sources are quoted and cited, but neither is explained or connected to the claim."
Build the evidence pack for the regional accreditor
The pack takes only signed reports. Once the report is signed, it holds the signed report and action plan, the sample record with seed 7731905, all 312 scored papers with both Scorers' levels and passages, the 21 panel decisions and the agreement report, with names hidden, sealed and logged.
13 screens from the working solution, on its sample data. Pick one to see it large.
Sampling, scoring and privacy rules are versioned settings: rules v7 was changed by the systems administrator and approved by the Director of Assessment.
| Setting | Default | Choose from |
|---|---|---|
| Sample rate per stratum | 10.6% | 5% to 25%, with a minimum of 5 papers per stratum so small groups are represented |
| Scores one level apart | Keep the lower level | Keep the higher level · Send to the panel |
| Scores more than one level apart | Always go to the faculty panel | Locked on |
| Calibration set | 10% of the sample, scored first and sealed | 5% to 20% |
| Agreement threshold (weighted kappa) | 0.65 | 0.50 to 0.85; within 0.03 it is marked "watch"; below it, agent scoring stops for that criterion |
| Default benchmark | 65% at Proficient or above | Faculty may set their own per outcome |
| Scorer confidence floor | 0.70 | 0.60 · 0.80 · 0.90; below it the score is "unsure" and goes to the panel |
| Keep papers | 3 years after the cycle closes | 5 years · Until the next accreditation visit; scores, reports and the audit trail 10 years |
Every figure is labelled: a target the solution is built to, an estimate, a typical published result, or a proven one.
Sources: Association of American Colleges and Universities, On Solid Ground (national VALUE rubric scoring: weighted percent agreement of about 84% to 94% by dimension) · "target" = design goal, measured in the live solution · "demo" = seen in the working solution, on its sample data · "typical" = published figures · "estimated" = our arithmetic, kept modest. Brandvale Community College, its courses and the people named on this page are fictional: characters and sample data in the working solution.
Learning outcomes assessment is how a college shows what its students learned against stated outcomes, by scoring a sample of student work against a faculty rubric, and what it changed because of the results. Learning Outcomes Assessment puts seven agents in front of that cycle: they draw a sample anyone can re-draw, score every paper twice and blind, measure their agreement against a hidden faculty set, and draft the report, while faculty decide the hard papers and the Director of Assessment signs.
Collect student work from the courses that teach an outcome, score a sample of it against a rubric the faculty own, compare the results with a benchmark, and record what will change. Here the Sampler draws a stratified sample by campus, modality and dual enrollment, two Scorer agents score each paper with the passage cited, and faculty resolve any gap of more than one level.
The method accreditors see most is scoring a sample of student work, such as a signature assignment, against a rubric the faculty own, then comparing the results with a benchmark. This solution runs that method end to end: a seeded stratified sample, two blind Scorer agents, a faculty panel for disagreements, a hidden calibration set, and results by campus, modality and dual enrollment.
Faculty score a hidden calibration set, 10% of the sample, and seal it before the agents begin. When both Scorers finish, the Agreement agent reports exact agreement, agreement within one level and weighted kappa per criterion, agents against faculty and faculty against faculty. In the working solution that is 0.74 overall, and any criterion below the 0.65 threshold goes wholly to faculty.
Yes. The Sampler records the frame, the strata, the seed and the replacement order, and "Re-draw from seed" reproduces the same papers in the same order: in the working solution, 312 of 312.
No. The agents sample, hide names, score, compare and draft. They never sign, approve or settle a disagreement; a score changes only through the faculty panel with a reason code, only the faculty outcome lead approves a rubric change, and only the Director of Assessment signs the report.
Student names are hidden before anyone scores, and instructor and section are hidden from scorers. Every view of a paper is logged, student work is never used to train models, and papers are kept for a set period after the cycle closes, 3 years by default, then deleted with a record of deletion.
The Agentic Solution Engine builds and deploys it from your requirements and documents, such as your outcomes, rubrics, assessment plan and a sample of past signature assignments, and it goes live once every quality gate has passed. We will walk you through it on your own students' work first.
Every course file fixed before April 2027, and heard right by a screen-reader test.
1-min demoAcademic Integrity Case ManagementReferrals screened on how the work was written, with both sides in one packet.
1-min demoCourse Evaluation Comment AnalysisFixable teaching issues by course in minutes, and no biased comment in a faculty file.
1-min demoWe'll run Learning Outcomes Assessment on a sample of your own signature assignments and rubrics.