Help a small team create useful evaluation labels without silently replacing disagreement with majority vote. Import independently labeled records, blind annotator identities during review, group disagreements and capture an explicit final decision with its evidence. Export a local review page and versioned JSONL that preserves both original labels and adjudication history.
Proposed milestones
- Import independent labels and preserve record provenance
- Measure agreement and build a blinded review queue
- Capture evidence-backed adjudication decisions
- Export versioned labels with unresolved cases visible
This is an upstream project brief; an implementation and completion tests are not supplied.