dsh-eval-regression: Deterministic Output Regression Plugin for DeepSeek Harness

`aryswisnu/dsh-eval-regression` is a plugin for DeepSeek Harness (DSH) that provides a deterministic, CI-safe golden output evaluation tool. It registers the `evaluate_golden_output` tool to specifically check whether the candidate output contains required fragments or forbidden fragments. This plugin does not evaluate semantic correctness or model quality; it only generates repeatable pass/fail evidence. Its core functions include fragment-level checks, a deterministic report outputting a total score and case-by-case missing or forbidden fragments, and a CLI tool that supports JSON suite input, prints reports via standard output, and returns different exit codes based on results, facilitating...

Read More