All projects

Independent exploratory pilot · 2026

Human–AI collaboration audit

In a fictional healthcare-governance scenario, how does a one-shot response compare with a response that is followed by structured review?

Two sheets of paper on a desk, one marked with a pencil line and a teal note.

This pilot makes no effectiveness claim. There was no independent second reviewer. The scenario is fictional. The scores describe this small exercise only.

What was compared

The exercise set a one-shot response beside a response that then went through structured review. Scoring used four dimensions: accuracy, evidence fidelity, completeness, and traceability. Errors were also placed in categories, so a mistake could be described rather than only counted.

What the scores did

Records19in the file
Scored154 missing
Recorded score2every scored record
Second reviewerNoneno agreement check

IBM SPSS was used for a descriptive analysis. Every recorded score was 2. That is a ceiling: the scale, as applied here, did not separate the records. A ceiling stops a comparison. It does not show that review improved the response, and it does not show that the first response was already sufficient.

What the pilot is useful for

Governance work depends on being able to trace a statement back to its evidence. The four dimensions are an attempt to make that requirement explicit when a draft is produced with an automated tool. The result of this first pass is that the instrument needs more range, a second reviewer, and a scenario that can actually produce different scores. Reporting the ceiling is the finding.