Quality validator prototype

Can the internal checks find real defects before a proofreader does? We seeded 400 defects into 1,000 correct translations in German, French, Chinese, Arabic and Thai.

DoneIllustrative data

Step 5 of 24 · Presenter note

The internal checks are tested against seeded defects. Deterministic checks must catch 100%. Semantic checks help the proofreader but do not replace review in Phase 1.

93%Seeded defects found372 of 400, all checks combined

100%Deterministic defects foundNumbers, units, tags, protected tokens

7.2%False positives on clean unitsTarget: 10% or lower

1.8 sMedian time per unitAll checks

Detection rate by defect type

Number changed100%

Unit changed100%

Tag or placeholder broken100%

Protected token translated100%

Prohibited or missing term96%

Omission88%

Negation error86%

Unsupported addition81%

Wrong condition or sequence74%

Wrong locale variant69%

050100%

Required: 100% for hard-fail defect types

What this means for Phase 1

Deterministic checks gate release
Numbers, units, tags, placeholders and protected tokens. Every miss is a blocker.
Semantic checks assist, they do not approve
Omission, negation and condition errors are caught 74 to 88% of the time. That is why every new or fuzzy unit is still reviewed.
Reviewer edits become calibration data
Tagging each edit with an error category builds the data needed to calibrate a quality score later, in Phase 3.

A structured finding, as the proofreader and the correction loop receive it

§17.5

{
  "translation_unit_id": "SEG-12345",
  "evaluator_version": "gtl-qe-0.3",
  "issues": [{
    "category": "accuracy_omission",
    "severity": "major",
    "source_span": "before releasing the brake",
    "target_span": null,
    "explanation": "The temporal safety condition is missing.",
    "detected_by": "semantic_evaluator"
  }],
  "hard_fail": false,
  "recommended_action": "HUMAN_CORRECTION"
}