dsh-eval-regression: Deterministic Output Regression Plugin for DeepSeek Harness
2026-09-01
DeepSeek Harness plugin
DeepSeek
dsh plugin
dsh-eval-regression
CI regression testing
Deterministic assessment
`aryswisnu/dsh-eval-regression` is a plugin for DeepSeek Harness (DSH) that provides a deterministic, CI-safe golden output evaluation tool. It registers the `evaluate_golden_output` tool to specifically check whether the candidate output contains required fragments or forbidden fragments. This plugin does not evaluate semantic correctness or model quality; it only generates repeatable pass/fail evidence. Its core functions include fragment-level checks, a deterministic report outputting a total score and case-by-case missing or forbidden fragments, and a CLI tool that supports JSON suite input, prints reports via standard output, and returns different exit codes based on results, facilitating...
Read More