Thomas Piacentini
← Workshop

Project brief

Metamorphic Evaluation Workbench

Original text · English

Help evaluators catch inconsistent behavior across related inputs. Author transformations such as unit conversion, record reordering and equivalent formatting, define the output relationship each should preserve, and shrink violating examples. Export a property-based evaluation runner and small failure bundles that can be consumed by CI.

Proposed milestones

  1. Define input transformations and expected relations
  2. Run paired cases with recorded response provenance
  3. Check relations and shrink violations
  4. Export reproducible property failures for CI

This is an upstream project brief; an implementation and completion tests are not supplied.

Code and artifacts 1
What you take away

Saved in your notebook on this device.

Sources and limits

Reading checkpoints are self-reported. Test results are recorded separately and apply to the executed cases.

AI Engineering from Scratch · 3f9715c · MIT

Download this file ↓